Fork me on GitHub
#re-frame
<
2020-07-26
>
genekim19:07:32

I’m wondering if anyone else has the problem of sprawling events.cljs files, and a flow of events that’s difficult to keep track of and keep in one’s head. I have an SPA which I use for Trello card management, which I’ve built up over 3 years — it’s actually two SPAs, one intended for use in desktop browser (supporting keyboard accelerators) and the other in mobile interface . The two SPAs share the same db.cljs and events.cljs. The total is about about 3K LoC, of which the largest file is the events.cljs, which is 1.2K LoC. There are 105 events, and I often have trouble keeping track of them — they can be as high level as “load list of cards” or “load board,” to “set active card position” to “move card to list” to “set browser focus on description text field” — to internal private events: “callback-load-list,” “embed-youtube-urls.” I’m ridiculously proud of myself for writing a Clojure program to read the events.cljs file and extract the “call graph,” pulling out the re-frame/dispatch calls and the reg-event-fx dispatch maps. (OMG. This is my first visceral experience of awe at the value of homoiconicity!) I’ve learned that I have at least 65 events calling other events, putatively making the remainder leaf events, which don’t dispatch any other events. Even before this exercise, it occurred to me that I don’t know what the ideal event flow and shape of the call graph should even look like! What does one look like that allows for re-use, versus what feels like spaghetti right now? Are there naming schemes for the events to support better organization? And what should the naming scheme be for events, to impose an order/organization to it? Many thanks in advance!

genekim19:07:00

FWIW, here’s the “call graph” that I extracted — any quick reactions to it or critiques of anything I’m doing wrong? (Not posting as a reply, to allow easier, wide reading.) Each vector is the name of the event, followed by all the other events that it dispatches.) (I’ll post the repo in the next couple of days, despite all its flaws/inadequacies. I was playing around with rendering it with graphviz, or maybe vega arc-diagrams, but finding the text display surprisingly compact and useful.)

genekim19:07:00

([:graph/load-initial-state-success :graph/server-load-hotkeys]
 [:graph/select-board :graph/save-and-clear-searchbox]
 [:graph/select-board :graph/load-board-lists]
 [:graph/select-board :graph/select-board-next-leftpane-state]
 [:graph/callback-load-board-lists :graph/load-list-cards]
 [:graph/callback-load-board-lists :graph/load-list-card-counts]
 [:graph/load-list-cards :graph/reset-card]
 [:graph/load-list-cards :graph/generate-materialized-cards]
 [:graph/reload-all-lists-and-cards :graph/load-list-cards]
 [:graph/reload-all-lists-and-cards :graph/load-board-lists]
 [:graph/reload-all-lists-and-cards :graph/load-list-card-counts]
 [:graph/change-list-sort-mode :graph/generate-materialized-cards]
 [:graph/generate-materialized-cards :graph/iphone-materialize-view]
 [:graph/load-card-comments-attachments :graph/reset-card]
 [:graph/load-card-comments-attachments :graph/load-card-comments]
 [:graph/load-card-comments-attachments :graph/load-card-attachments]
 [:graph/iphone-materialize-view :graph/iphone-load-card-attachments]
 [:graph/select-list :graph/save-and-clear-searchbox]
 [:graph/select-list :graph/load-list-cards]
 [:graph/select-board-or-list :graph/save-and-clear-searchbox]
 [:graph/callback-load-card-comments :graph/rewrite-desc-twitter-call-oembed]
 [:graph/iphone-rewrite-desc-twitter-call-oembed-all-cards :graph/rewrite-desc-twitter-call-oembed]
 [:graph/rewrite-desc-twitter-call-oembed :graph/callback-handle-twitter-oembed]
 [:graph/next-card :graph/scroll-to-top]
 [:graph/previous-card :graph/scroll-to-top]
 [:graph/goto-top-card :graph/reset-card]
 [:graph/goto-top-card :graph/load-card-comments-attachments]
 [:graph/goto-bottom-card :graph/reset-card]
 [:graph/goto-bottom-card :graph/load-card-comments-attachments]
 [:graph/archive-card :graph/next-card]
 [:graph/callback-archive-card :graph/generate-materialized-cards]
 [:graph/callback-archive-card :graph/load-card-comments-attachments]
 [:graph/callback-archive-card :graph/load-list-card-counts]
 [:graph/left-pane-set-form-text :graph/generate-materialized-cards]
 [:graph/start-move-card :graph/focus-searchbox]
 [:graph/move-card-to-list nil]
 [:graph/move-card-to-list remove]
 [:graph/move-card-success :graph/reset-card]
 [:graph/move-card-success :graph/load-card-comments-attachments]
 [:graph/move-card-success :graph/generate-materialized-cards]
 [:graph/move-opt-number-key :graph/move-card-to-list]
 [:graph/opt-letter-key :graph/execute-hotkey-move]
 [:graph/move-current-card-to-top :graph/move-card-to-top-or-bottom-of-list]
 [:graph/move-current-card-to-bottom :graph/move-card-to-top-or-bottom-of-list]
 [:graph/move-card-to-top-or-bottom-of-list [:graph/move-card-to-list [dest-list-id board-id pos]]]
 [:graph/move-card-to-top-or-bottom-of-list if]
 [:graph/iphone-set-cardpos-for-move :graph/execute-hotkey-move]
 [:graph/iphone-set-cardpos-for-move :graph/iphone-move-card]
 [:graph/iphone-set-cardpos-for-opt-number-key :graph/move-opt-number-key]
 [:graph/iphone-set-cardpos-for-opt-number-key :graph/iphone-move-card]
 [:graph/edit-card-text :graph/focus-edit-card-name]
 [:graph/handle-keyboard-next-prev-list :graph/load-list-cards]
 [:graph/handle-keyboard-move-to-hotkey :graph/focus-hotkeyform]
 [:graph/handle-keyboard-set-move-hotkey :graph/focus-hotkeyform]
 [:graph/handle-keyboard-delete-move-hotkey :graph/focus-hotkeyform]
 [:graph/handle-keyboard-goto-hotkey :graph/focus-hotkeyform]
 [:graph/create-list-success :graph/load-board-lists]
 [:graph/submit-save-hotkey-form :graph/server-save-hotkeys]
 [:graph/submit-delete-hotkey-form :graph/server-save-hotkeys]
 [:graph/submit-goto-hotkey-form :graph/select-board]
 [:graph/submit-goto-hotkey-form :graph/load-list-cards]
 [:graph/execute-hotkey-move :graph/move-card-to-list]
 [:graph/handle-keyboard-repeat-last-command :graph/move-card-to-list]
 [:graph/route-goto-card-id :graph/scroll-to-top]
 [:graph/route-goto-card-id :graph/reset-card])

👏 1
👍 1
genekim14:07:40

This has helped me zero in on where the complexity/messiness is — not surprisingly, it typically is around operations that have multiple asynchronous steps. Like this one:

[:graph/archive-card :graph/next-card]
 [:graph/callback-archive-card :graph/generate-materialized-cards]
 [:graph/callback-archive-card :graph/load-card-comments-attachments]
 [:graph/callback-archive-card :graph/load-list-card-counts]
I.e., to archive a card, it’s got to advance the view to the next card, call the backed to archive it, upon the callback, pessimistically update the list of card, load the next card contents/comments… I’m wondering aloud how better to do these multi-step sequence of events, where it’s more obvious what the steps are — documenting this way is good, but steps seem currently splattered across too many events, which linkages not obvious enough.

rberger05:07:40

We ended up mostly breaking our event.cljs up into files under events that matched our views using namespaced keywords for the events. We kept the subs in a global subs file at least for now.

👍 2
genekim14:07:26

Whoa! What a super, super idea!!! Thank you!!!

genekim14:07:03

Thanks so much for the suggestion, @U07GPFFAS. PS: as I’ve been studying what has been tripping me up in the events, one of them is the lack of “information hiding.” In the global db, I have the raw list of Trello cards, and a “materialized view”, where filters are applied and other things I don’t remember. I had some events inappropriately accessing the raw list, instead of the materialized list… I’m now pondering how to keep certain parts of code from seeing the raw card list, lest I make the same mistake again… (Maybe I nest certain elements in the global db into an area called, :private-do-not-look-or-touch` — as they are the “model,” not for the view?)

David Pham05:07:04

I support the idea of namespaced events. I usually partition my app by views/usage with an additional general “components” folder for generic components. I think the challenge is to make a trade off between generic events and reasonability/locality of your logic. Moreover, you need the discipline to avoid calling events from other views, except the general one.

genekim16:07:48

@UEQGQ6XH7 This sounds super interesting, as well — so to make sure I’m processing this all correctly: Option 1: @U07GPFFAS suggests breaking up the events namespace, which I’m guessing are all required into the main events.cljs. This has the benefit of organizing the events, modularizing, information hiding, etc. Option2 : @UEQGQ6XH7 suggests going beyond just splitting up the events namespace, but actually creating separate dbs/events/subs, as per: http://day8.github.io/re-frame/App-Structure/ I think in my case, I’m going to try Option 1, because I’m not quite ready to distangle everything yet — but making separate piles of events is something I can manage. 🙂 thank you, all!

David Pham18:07:42

The advantage of having fully qualified namespace is also to know where they are defined. But good luck :)

🙏 1
rberger18:07:16

@U6VPZS1EK We don’t require the events/*even.cljs in the top level / global events.cljs. We just require them as needed in the usually parallel views/*view.cljs files. Our current project did evolve from a global events.cljs to this model and I susepct we may eventually evolve to something like what @UEQGQ6XH7 suggests. But evolution is good. Learn as you go. One thing good about having them all in a global events though, is no worry about circular dependencies. It hasn’t bit us hard, but it has bit us a few times. Usually a bit of refactoring/rethinking fixes that.

🙏 1
Jose Varela00:07:26

I’ve seen this problem in Redux too. I think this is a problem with event-driven architectures. Maybe these could help make sense of the dependencies: 1. Logging: you see what event happens before/after another event. Even while running tests or playing around with the app (checkout https://github.com/day8/re-frame-10x) 2. Make the other end visible: similar to what you did but make subscriptions visible - what view listens to what change? (view -> sub -> event) could complement your event list

uosl10:08:30

I second using re-frame-10x. Our app has an event call graph at least as convoluted as yours, but I find it very manageable with 10x. Instead of trying to discern the flow of events or the contents of app-db from the code, I simply run the application and observe its behavior using 10x. To make event flows more explicit in the code, you can use https://github.com/day8/re-frame-async-flow-fx but it has a lot of overhead so it's only recommended for the initial boot of your app, after which it gets disabled. The call graph program you created sounds very cool! I think 10x could be made even more powerful if we had a visual representation of that (I know there's a similar issue for visualizing subscription graphs).

Jose Varela20:08:27

Love the syntax https://github.com/day8/re-frame-async-flow-fx#flow-as-data @UCW9TUDNK This makes event-chains easy to read. The docs say the main purpose is for sequential app-boot logic. Why isn’t this flow-syntax used for chaining events in any stage of the app’s life-cycle? My hunch is it’s too much overhead for chaining 1-2 events together, and if you have long chains you’re probably doing something wrong by structuring your flow of data that way?

uosl09:08:24

@U016Y9K11H9 I think there are scenarios where you are forced to structure long event chains of HTTP requests during an app's lifecycle, maybe because of a badly designed backend API. The overhead of async-flow-fx comes from it using forward-events-fx to forward all events to the async-flow event handler, in effect causing everything to be dispatched double. I wonder if you can make something similar using an interceptor instead, and then injecting that into all the participating event handlers. I think there are some places in our app that could benefit from this, as currently we're manually checking some part of app-db in every event handler in the chain, to be able to tell which parallel events have completed and which have not, and when it's ready to move on to the next step. It's something I'll try exploring when I find an excuse to do it during worktime. 😼

genekim16:08:25

Holy cow! I LOVE the idea of `re-frame-async-flow-fx`!!! Thanks for all your help on this — I’ve had a bit of a breakthrough that has allowed me to build what I want to build… but it’s made it pretty clear that I need to clean my mess up somehow. But I’ve been thinking about all y’all’s advice on how to think about reorganizing — I’ll keep you posted on my plans. (After I finish building this one feature, that’s got me so excited, I woke up early this morning to tackle! 🎉)

💪 1
genekim01:08:20

And here’s the graphviz rendering of my horrible event dispatch graph. :)

uosl18:08:27

Thanks for sharing the graph! That is very nice, I imagine it would be useful to have for every re-frame project.

1
genekim07:08:15

For y’all’s consideration and comment — I wrote up how this graph helped me get something done. If y’all think it’s solid enough, I’d love to post to this channel and @ mention Mike Thompson.

genekim07:08:24

Many thanks in advance!

Jose Varela18:08:32

@U6VPZS1EK in this case, what did you end up doing with the output? Did you rearrange some of these relations?

Jose Varela18:08:36

Got it, “Countermeasure” explains what you ended up doing

genekim19:08:37

Great catch — new title. :)

Jose Varela19:08:59

It’d be awesome to show a chart like that and highlight the bubbles as the events are ocurring. Imagine running your app at ‘slow speed’ and seeing the event flow go through the graph.

1
Jose Varela19:08:54

Or a debugger step-through.. click, click, click and the graph moves

genekim19:08:54

That blows my mind — that’d be amazing! (Maybe motivation to get a wrapper around MermaidJS… could dynamically render the graph…)

genekim19:08:27

@U016Y9K11H9 Just added screenshot of your idea to the deck. 🙂

Jose Varela19:08:14

haha thanks! what’s this deck are you giving a talk?

Jose Varela19:08:28

hmmm maybe this is a great excuse for me to get started with clojure

genekim19:08:12

Oh, just the Google Slides you already saw — no intent to present it. It’s just easier to write in bullet points than actual full prose. 🙂 (PowerPoint/Slides is the tool for the lazy, which isn’t always a bad thing — it’s so efficient. 🙂

Jose Varela19:08:54

Agree! Been doing something similar too: when there’s something I want to think more about, I schedule a talk at a local meetup (nothing fancy) to force myself to come up with a coherent story hehe

David Pham16:08:46

I would look at react-cytoscapejs for rendering networks with a functional interface. I am using it at work and it seems decent.

Jose Varela16:08:24

Looks awesome! Thanks for the tip

uosl11:08:55

I had no idea re-frisk had accumulated all these features! I thought it was just a app-db explorer.