Clojurians Log v2

To clarify, which is best practice in a rf/reg-event-fx? 🅰️

{:db (assoc db :loading? true)
 :http-xhrio (...)}

{:fx [:dispatch [::loading? true]]
 :http-xhrio (...)}

> Should my event handlers update the db directly Most likely, yes. > What if the http response handlers may trigger other http calls? Use a state machine.

Thanks for your quick response! I’ll continue updating the db directly, like 🅰️.

Yes 🅰️ is the typical choice. If you did a deferred nested :dispatch that actually may not happen “soon enough” and your UI could have a flicker state or something of that form. When you start to get into “chained” async flow of this type of events happening, I like https://github.com/day8/re-frame-async-flow-fx We’ve used it extensively in some pretty large apps. However, we did fork it so that it had cljc support since that was convenient for some of our testing pipeline. Need to push a PR to the main repo to see if they’ll take that change at some point.

Nice. For now we just have a single chain of 2 potential calls. But if that ever expands, I’ll be looking much more closely at async-flow-fx. Thanks!

We often have an async flow of events - perhaps doing XHR in the process. We keep track of all this and have a loading state to go with it. We use signals there to know when the whole operation is done.

Note that it also common here to specifically recommend not using re-frame-async-flow-fx.

I didn’t use async flow originally for smaller cases. It started to get carried away without some proper infrastructure (like this lib) at a certain point.

I don’t know why you’d recommend against it. The idea at least is quite sound. The impl is fairly basic. Would be good to see the critiques.

We’ve certainly used it in very complex flows in some large apps that have quite a few devs involved over time (so coordination is involved too). We also are sensitive to performance etc.

> Would be good to see the critiques. Just search in this channel. ;)

I did. I’ve mostly just read “there have been problems” with no clear rationale. Then there was this older one https://clojurians.slack.com/archives/C073DKH9P/p1631039809068800?thread_ts=1631015086.050100&cid=C073DKH9P but again, it just says “it is bad” and doesn’t say why.

I wouldn’t recommend using it where it isn’t needed. But that tautological anyways.

If you have to chain async events together into one coherent piece though - you really need some sort of state machine implemented one way or another. Doing it manually with a bunch of coupled event handlers is a huge pain.

But I have done it other ways before. async-flow just really simplified that logic. If another lib wanted to do it an alt way, that’d be interesting - but haven’t seen anything of note.

I also found https://gist.github.com/olivergeorge/edc572eab653cc228ec5ecbbcc8271a2 which is the most concrete thing I found. I don’t see how it addresses dependencies among async actions though. Commented there as such. Either way, I remain unconvinced here. I think async-flow can be abused, but so can anything else.

Hmm, it might be harder to find than I thought. Could easily find only this: https://clojurians.slack.com/archives/C073DKH9P/p1644175506330019?thread_ts=1643901164.589709&cid=C073DKH9P And that second item kinda hints at a bigger picture given that events themselves are async by nature. When you have stuff like {:when :seen? :events [...]}, all of those events must be used with the async flow in mind. You can't reuse them in other contexts at all. With that in mind, it becomes trivial to implement race conditions. It becomes trivial for you or someone else to trip over your code just because it implicitly monitors an event that's now used elsewhere. Apart from that, it's not composable. If you have an existing workflow and then decide that it should be a part of another workflow, you aren't gonna have a good time.

We have done this in large complex apps with quite a few devs - even new and old cycling in and out. I think it can be quite scalable. You do have to have some level of convention to avoid pitfalls. All of this gets complicated either way in large apps.

We know the events that are for flows. You could even organize it via conventions in a way that makes it clear.

I don't see any points that would make re-frame-async-flow-fx better than a state machine. > We have done this in large complex apps with quite a few devs It doesn't mean that your experience is generalizable. JavaEE has worked for many people as well. Or C++, or whatever - it doesn't mean that if somebody could do it with C++ then C++ should be recommended to anyone who wants to do a similar thing.

I think you can easily get into race conditions and a mess doing it with no structured support too. I think its worse even when ad hoc. We have apps to compare X vs Y with this stuff really. Our older apps didn’t use async-flow, then we tried to retrofit a few to use it and use it in newer apps. We get far more problems in our apps without it and makes us want to spend time to refactor to use async-flow there too.

I think we’ve had a lot of success with it is my point. I see pretty weak theoretical arguments that it “could be bad”. So I don’t think general community discouragement makes sense.

I think the JavaEE analogy is not really related either. async-flow is a tiny library to implement a small state machine.

They are not theoretical. :) Perhaps @emccue could chime in.

If I were to implement it from scratch, I may do it slightly differently on implementation choices - but conceptually it works quite nice.

Oh great

If you do just end up implementing a state machine ad hoc locally, that’s fine too. But seems like you’d then have to copy/paste that around or pull out some lib for it if you had multi apps. Also docs etc.

yeah I had an extremely bad experience there

whatever clarification you want I can give

It’s just vague. I’ve had a very good experience. And with quite a bit of scale.

> async-flow is a tiny library to implement a small state machine. But it's not. It's not a state machine at all. It doesn't have states. If anything, it's like a reactive flow but imposed externally.

the codebase where i had issues is also 100% dead now, so i can probably get away with sharing direct snippets

> If you do just end up implementing a state machine ad hoc locally, that’s fine too There are also libraries for this, including re-frame-aware ones.

Let me introduce the audience to our emblematic bastard

(rf/reg-event-fx
  ::init-tracking
  (fn [cofx _]
    {:dispatch [::db-util/xhr-with-loading ::load-tracking-data
                {:method     :get
                 :uri        "/api/tracking-info"
                 :on-success [::process-tracking]}]}))

It’s more of an async DAG - but this is what the typical problem is. Quite similar to core.async or JS await/async stuff etc.

(rf/reg-event-fx
  ::init
  (fn [cofx [_ initial-db have-auth have-creator root]]
    {:db (-> (merge (:db cofx) initial-db)
             (init-db-loading-status)
             (bootup/set-app-context bootup/app-context:advertiser))
     :async-flow
     {:id             ::lumanuapp-async
      :db-path        [::lumanuapp-async]
      ;; APP BOOT UP SEQUENCE
      ;; 1. Trigger the first rules:
      :first-dispatch [(if have-auth ::begin-booting-auth ::begin-booting)]
      :rules          [{:when       :seen-any-of?
                        :events     [::begin-booting ::begin-booting-auth]
                        :dispatch-n [[::init-tracking]
                                     [::bootup/render-app root]
                                     [::link-helpers/register-non-app-link-click-listener]]}
                       ;; END APP BOOTUP SEQUENCE

                       ;; MAYBE.
                       ;; Initial data loading rules
                       {:when       :seen-all-of?
                        :id         :base-rule ; undocumented feature of async-flow
                        :events     [(fn [& args]
                                       (and (apply (supported-for-user-type-matcher have-creator) args)
                                            (apply (routing/legacy-advertiser-route-changed-matcher smartboost-routes) args)))]
                        :dispatch-n [[::advertiser/fetch-team-creators]
                                     [::smartboost/fetch-smartboost-channels]
                                     [::sb-settings/fetch-brands]]}
                       {:when     :seen?
                        :events   ::advertiser/completed-fetching-creators
                        :dispatch [::smartboost/creators-loading-complete]}
                       {:when       :seen-all-of?
                        :events     [(fn [& args]
                                       (and (apply (supported-for-user-type-matcher have-creator) args)
                                            (apply (routing/legacy-advertiser-route-changed-matcher collab-routes) args)))]
                        :dispatch-n [[::collabs/fetch-collaborations]
                                     [::collabs/fetch-collaborating-advertising-links]
                                     [::collabs/fetch-collab-participants]
                                     [::advertiser/fetch-team-creators]
                                     [::collabs/fetch-terms]
                                     [::collabs/completed-fetching-collabs]]}

                       {:when       :seen-all-of?
                        :events     [(fn [& args]
                                       (and (apply (supported-for-user-type-matcher have-creator) args)
                                            (apply (routing/legacy-advertiser-route-changed-matcher payment-routes) args)))]
                        :dispatch-n [[::payments/fetch-all-payments-info]
                                     [::advertiser/fetch-team-creators]
                                     (bulk-payments-events/init)
                                     (bulk-pay-det-events/init)]}

                       {:when     :seen-all-of?
                        :events   [::channel-db/set-creator-id->advertising-links
                                   ::advertiser/completed-fetching-creators
                                   ::collabs/completed-fetching-collabs]
                        :dispatch [::collabs/set-collabs-loading-completed]}]}}))

This was the async-flow init event that started up our app

the root problems we had were

I really don’t mind the above flow. I’d say we do not try to make really generic event handlers as sort of utilities for other things though. Like this ::db-util/xhr-with-loading - we’d just set loading statuses in a specific area of our app-db and have utils around updates to that.

Basically, we don’t do “higher order event handlers” - just helpers that can update the cofx/fx map directly for reused patterns.

• This gives zero shits if any of the events "fail", just that they happened • Everything only runs once. To load all the data you need "for a page" you need to hack in and reset/refire some of this nonsense • It gets huge. This is extremely trimmed down from what it was • It is very difficult to reason what state the app is in because events work exactly opposite to regular functions

Happy to hear your details though. It’s nice to see concrete stuff in general (most people can’t share things really).

We never end up with flows as big as above. So not sure what the disconnect is there.

> Basically, we don’t do “higher order event handlers” - just helpers that can update the cofx/fx map directly for reused patterns. And that is the problem, essentially

We always handle failure cases.

If you have events lead directly dispatch other events, which is both extremely tempting and lightly encouraged by the official docs, it becomes very hard to reason about things

I’d really like to see an example of how you manage a DAG of async events in other ways really. It’s not an easy thing to reason about conceptually or debug in general as it gets complex.

and async-flow-fx encourages that + adds a very brittle caching step on top of it

> If you have events lead directly dispatch other events, which is both extremely tempting and lightly encouraged by the official docs, it becomes very hard to reason about things I don’t see how this is really even practically avoidable.

You have to sometimes have dependencies where things must “wait” for other stuff to be done.

well think about it, when you are in an event you have the db and you have the ability to put any number of effects

So there is never a mechanical reason to use an event as a helper

Yeah, you could role your own continuation etc. if that is what is being suggested

no

just

sorry i got a lot to type for this to make sense

hold with me

I agree it doesn’t need to be event dispatch specifically. Something like a callback continuation could also work. It’s basically just CSP stuff.

no, not that either

Ok, sorry I’ll just hold back for a bit to let you “cook” 😉

lets go through a few scenarios

first, our app just tracks a number

{:n 0}

this is the app db

we have two buttons, one which should multiply by 3 and add 1, another which should divide by 2

(rf/reg-event-db
  :mul-3-add-1
  (fn [db _]
    (assoc db :n (inc (* 3 (:n db))))))

(rf/reg-event-db
  :divide-2
  (fn [db _]
    (assoc db :n (/ (:n db) 2))))

so far so good?

okay so now lets move adding one to its own event

> It’s more of an async DAG With every node being a start node, yes. Saying "but we're very careful, so it's not a problem" is not a fix to this problem. The analogy with C++ still stands. I used to know how to write C++ correctly and effectively, a decade ago. And after realizing how much thought has to go into it to make sure that it all works properly, I would never recommend C++ to anyone. If a thing makes it way, way too easy to trip up people when alternatives that are strictly better exist, that thing is not a good thing. > Quite similar to core.async or JS await/async stuff etc. Both of those have a very explicit DAG imposed from within a system. You cannot tell JS to run some function X whenever a function Y is being run, unless you deliberately monkey-patch the hell out of the function Y.

(rf/reg-event-fx
  :mul-3
  (fn [db _]
    {:db (assoc db :n (* 3 (:n db)))
     :dispatch [:add-1]}))

(rf/reg-event-db
  :add-1
  (fn [db _]
    (assoc db :n (inc (* 3 (:n db))))))

(rf/reg-event-db
  :divide-2
  (fn [db _]
    (assoc db :n (/ (:n db) 2))))

pausing here: give some initial thoughts

I figure your are going to now try to put this in a particular order and want to ensure that order. Other than that, that’s my only current thought. I think perhaps that :add-1 is wrong its impl too in that regard - shouild just be (inc (:n db)) if I understand the intent.

With every node being a start node, yes.Doesn’t make sense to me. > Saying “but we’re very careful, so it’s not a problem” is not a fix to this problem. I think conventions are a very important thing within a re-frame app. It’s very open ended and can quickly become chaotic. The same is true for basically any web framework. You have to care about organization. > If a thing makes it way, way too easy to trip up people when alternatives that are strictly better exist, that thing is not a good thing. I’ve yet to see the alternatives that are “strictly better”. This is a key problem I have with the counter argument here. > Both of those have a very explicit DAG imposed from within a system. > You cannot tell JS to run some function X whenever a function Y is being run, unless you deliberately monkey-patch the hell out of the function Y. Doing the “bad re-frame” thing is the same as the “bad JS” thing that is alluded to here as absurd and no one would/should do it.

So i'm pretty sure re-frame will make the dispatch here run before any other events are run, so as-is there are no particular issues. Just a strange way to break up the code. But pay attention to there being no mechanical impediment to just running the logic to update the db in that first event.

add another wrinkle, lets log out the db

(rf/reg-event-fx
  :mul-3
  (fn [{:keys [db]} _]
    {:db (assoc db :n (* 3 (:n db)))
     :dispatch-n [[:log-db] [:add-1]]}))

(rf/reg-event-db
  :log-db
  (fn [db _]
    (println db)
    db)

(rf/reg-event-db
  :add-1
  (fn [db _]
    (assoc db :n (inc (* 3 (:n db))))))

(rf/reg-event-db
  :divide-2
  (fn [db _]
    (assoc db :n (/ (:n db) 2))))

> Doesn’t make sense to me. The events in :seen are being monitored globally. Not just within a context of a workflow. If an event is used elsewhere while the workflow is running (between :setup and :halt? true), the :dispatch parts will also be triggered. It's trivial to have a fragmented workflow that you have no way of easily debugging. Even with re-frame-10x, you can see what event resulted in which effects but you cannot see from an event why it has been dispatched. > I’ve yet to see the alternatives that are “strictly better” Have you tried e.g. this? https://lucywang000.github.io/clj-statecharts/docs/integration/re-frame/

I think the example would be better to see if it wasn’t as easy to “just update the db” - like if an HTTP req had to be made to the server to do each math op. Perhaps you are just having an assumption that it is like that here?

so pop quiz - which of these will observe the db in a consistent state

@mikerod 🧑‍🍳

(rf/reg-event-fx
  :mul-3
  (fn [{:keys [db]} _]
    {:db (assoc db :n (* 3 (:n db)))
     :dispatch-n [[:log-db] [:add-1]]}))

(rf/reg-event-fx
  :mul-3
  (fn [{:keys [db]} _]
    {:db (assoc db :n (* 3 (:n db)))
     :dispatch-n [[:add-1] [:log-db]]}))

I don’t know what “consistent state” means here. Which one will log the state of db after the math ops have happened?

after both the *3 and the +1

Second

But you’d have to know that :add-1 is not going to need to do async steps to finish the op.

okay now lets change how log-db works

(rf/reg-event-db
  :info
  (fn [db _]
    (println db)
    db)

(rf/reg-event-fx
  :log-db
  (fn [{:keys [db]} _]
    {:db db
     :dispatch [:info]))

now which one will see both changes

wait

there

They both will?

try it

@james.luke.johnson Sorry for hijacking the thread BTW.

🙂 It’s okay, you answered my question immediately. The rest is just gravy.

> The events in :seen are being monitored globally. Not just within a context of a workflow. > If an event is used elsewhere while the workflow is running (between :setup and :halt? true), the :dispatch parts will also be triggered. > It’s trivial to have a fragmented workflow that you have no way of easily debugging. Even with re-frame-10x, you can see what event resulted in which effects but you cannot see from an event why it has been dispatched. I am not sure I’m interpreting your concerns here correctly, but it sounds to me like “synthetic event” names in async-flow to trigger each step - we don’t refer to event handlers specifically. eg. :seen? :events [[::async-flow/notify ::something maybe-args-here]] We found decoupling the “notify” signals helped quite a bit. Perhaps that is what you are alluding to as a problem when not done. > Have you tried e.g. this? https://lucywang000.github.io/clj-statecharts/docs/integration/re-frame/ No, I will take a look. I hope it actually handles async flow dependencies though. A lot of examples seem to ignore that there can be ordered deps to deal with.

Oh i just got pinged. @james.luke.johnson I will probably be giving you a different answer by the end of this

sorry if my examples are sloppy, but i'm kinda making them up

say instead you wanted to observe the time between *3 and +1.

I don’t see how the example above could go wrong, but I’m also starting to not really see the purpose in this exercise. Even if this somehow goes out of order in a way I don’t see (in modern re-frame where :db always goes first). I’d never have just implicit assumptions like this. Event handlers know to wait for what they need to wait for. It’s actually a big reason async flow is so useful - it actually makes them wait in the correct DAG order.

Your event order will go from :mul-3 :log-db :add-one to :mul-3 :log-db :add-one :info

I think the example may be able to show a point - only maybe - better if all math ops were delayed as well. That at least shows how you can get lost in a tangle of async dispatches hanging out.

im building to the full set of problems

Ok, yes, I can see your interleaving example indeed.

We’d only ever assume you can’t expect an event to be “done” in a sequence though

As in if we wanted :log-db to happen before :add-one, we’d not rely on :dispatch-n and just assume dispatches have their effects fully done in that “cycle”. Again, this is actually where async-flow comes in. So sounds like a good demonstration for it’s usefulness! 😆

the first one here is just that when you turn what could otherwise be helper functions into dispatches you get potentially hard to understand behavior on refactors. Its not safe to turn an event into a chain of dispatches or to add more dispatches to any given event without understanding all the events that rely on it

no

so the clojure codebase started here. Then eventually someone came up with ::db-util/xhr-with-loading. It set some generic loading flag and made an http request over a chain of a few events

which was neat because then there was a one stop place for • Make http request • Get something in the db you can subscribe to for when that finishes

> Perhaps that is what you are alluding to as a problem when not done. Not really. An example from the README: {:when :seen? :events :success-X :dispatch [:do-Y]}. Let's assume that :success-X is not just a "marker event" that can be handled with ::async-flow/notify but rather a proper event that does something, e.g.

(rf/reg-event-db :success-X
  (fn [db [_ success-key]]
    (update db :success-markers (fnil conj #{}) success-key)))

I think what you are getting at is you want some way to have it where you do not have to care at all if an event handler does something sync/immediately vs later when you are in some other fn/handler and calling out sequences of dispatches. But you will still have dependencies - so they have to be dealt with. So your handlers will just have to be checking and waiting for something one way or another. I’ve done this before with things like “busy loops” in a handler even. Where a handler checks if some app-db state is “loaded yet”, if it is “loading”, it just dispatches itself later after some time has passed. This sort of decoupled the need for one handler to notify another handler directly - more of an “bus” sort of decoupling. This just was clunky too and async-flow made it clearer and more explicitly managed.

ugh

please stop saying its name

it made stuff so bad and I want you to understand exactly how

I’m just really lost in what is being proposed. I do appreciate the discussion though.

We use :async-flow/notify intentionally to make it clear how signaling works at a broader level than perhaps individual events. It’s quite useful to not tie it to actions directly. We may not want to do an action if it has already happened, but we can just send the notify as an indicator that nothing needs to be done.

> assume dispatches have their effects fully done in that “cycle”. Alas, it's not something you can assume with re-frame's events. But it is something you can assume with core.async or JS promises. There's a very explicit "end" to every go block, to every function producing a channel, to every promise.

I can jump straight to where we ended on this but i have a feeling you'll blow it off if i don't go through the full problems list

If you start cherry picking an event that has dependencies of a flow out of the flow - perhaps you could get trouble. I just don’t see how that is a practical scenario where you are using event handlers in random places without understanding its dependencies.

okay

anyways

I’m not blowing things off, I’m just not seeing the “light” basically. I do think there are alternative ways to model async flow dependencies than using event dispatches. I think async-flow chose that route and it hasn’t came up as a problem in quite a few real world cases for us. We haven’t seen it ever get carried away and unmanageable either. I think conventions can help perhaps problems like people trying to use handlers with dependencies without wiring up their dependencies first. I guess it’d be interesting if instead there was some concept of more explicit handler deps declared on a handler itself in some way that would be a framework to trigger loading what it may need if those things haven’t been loaded. This is an interesting takeaway to me. something like [:x :depends-on :y] where if :x is triggered directly at some point, there is some intelligence that knows to ensure :y has been done first. - Maybe more of a “flow” that instead goes the opposite direction and I have not thought at all of how that could be represented impl wise.

So as we've established, long chains of dispatches can have an adverse effect on the refactorability of code and, in the codebase I was working in, every http request meant like ~3 dispatches

Now lets introduce some more aspects

1. Every page of the app wanted different data 2. Each bit of data had its own route. In general they didn't want to make "route for page a" and instead made routes per "thing"

you can see this in the async flow I shared above

{:when       :seen-all-of?
                        :events     [(fn [& args]
                                       (and (apply (supported-for-user-type-matcher have-creator) args)
                                            (apply (routing/legacy-advertiser-route-changed-matcher collab-routes) args)))]
                        :dispatch-n [[::collabs/fetch-collaborations]
                                     [::collabs/fetch-collaborating-advertising-links]
                                     [::collabs/fetch-collab-participants]
                                     [::advertiser/fetch-team-creators]
                                     [::collabs/fetch-terms]
                                     [::collabs/completed-fetching-collabs]]}

when we hit a page that is a "legacy-advertiser-route", dispatch these 6 events

each of those 6 events would make an http request, set a loading flag in the db, and dispatch maybe more events

So obvious question - how do you know that the whole process was done? We have 6 or more independent things loading

basically every view ended up with

(when (and @(rf/subscribe [::stuff])
           @(rf/subscribe [::other-stuff])
           ...)
  [:p "hello"])

"if the data is non-nil, we got it. good to go"

problems this created • Those requests can fail and then the page just loads forever in the best case, white screens in the worst case • Sometimes you would check that one request of the N required for a page was done and assume that all of them were done. In dev you have no issues then boom, random crash in prod

then also

> Have you tried e.g. this? https://lucywang000.github.io/clj-statecharts/docs/integration/re-frame/ @p-himik read this. Surface level seems nearly the exact same as how we think of async-flows using notify events for the “state names”. My guess as to why this is preferred is: 1) statecharts is a broader and popular way of organizing this sort of thing. 2) I’m going to guess it doesn’t manage the actual actions and transitions as re-frame dispatches? I don’t see anything saying that it does or doesn’t though.

Because the logic for "boot up a page" was in async-flow it would only run once. Once. Even if you changed pages and came back we wouldn't refetch any data by default

We’d just have all of those dispatches dispatch a notify event of what they were done - then a new async-flow step that looks for all success - for “loaded case” or any failure for “something failed” case and we’d organize a failure response if needed etc.

there were a few places where someone found how to re-run the event cascade again, but it was always a mess and a terrible substitute for normal control flow

and really doesn't work for "initialize page state"

Ok, at this point I’ll say I can agree that a bunch of “nested dispatches” can have pitfalls and confusion as far as understanding when things happen etc. I’d like to see alternatives.

which is, coincidentally, the only thing that it seemed appropriate for at first

well, and this isn't perfect either, the alternative we came up with was :fx

You can just have a handler that deals with reload/refresh and uses another async flow or modifies the existing structure. I don’t see the big problem there. I still agree that understanding a bunch of event dispatches on the queue can eventually become difficult to conceptualize correctly.

its not highlighted too well in the documentation (or at least it wasn't), but you can have a :fx key in the return value of a reg-event-fx

this directly triggers side-effects without going through the event queue

A more recent re-frame addition. We now (basically) only use :db and :fx for our top-level effect map keys.

If your side effect is an async HTTP fetch though - how does that work out?

so instead of some events doing

{:db ...
 :dispatch ...}

{:db ...}

{:db ...
 :async-flow ...}

banning :dispatch from being in :fx

It’s an interesting rule, but I feel like conceptually you still end up with and “async flow” just not done via dispatches - just something else instead. Async callbacks or something?

for http requests we did something like this

(defevent get-line-items:success
  [response]
  :handler
  (fn [{:keys [db]}]
    (let [line-items response]
      {:db (-> db
               (assoc-in db:quickbooks-line-items line-items)
               (assoc-in db:quickbooks-line-items-loading false))})))

(defevent get-line-items:failure
  [_response]
  :handler
  (fn [{:keys [db]}]
    {:db (assoc-in db db:quickbooks-line-items-loading false)
     :fx (micro-toast-fx/open-danger-toast {:message "Could not get QuickBooks line items."})}))

(defn get-line-items-request
  [{:keys [generic-team-id on-success on-failure]}]
  {:method     :get
   :uri        (str "/api/generic-team/" generic-team-id "/quickbooks/line-items")
   :on-success on-success
   :on-failure on-failure})

(defevent get-line-items
  []
  :handler
  (fn [{:keys [db]}]
    (let [generic-team-id (-> db :team :generic-team-id)
          qb-integration  (boolean (get-in db db:quickbooks-integration))]
      {:db db
       :fx (when qb-integration
             (http-fx/transit-request
               (get-line-items-request {:on-success      get-line-items:success
                                        :on-failure      get-line-items:failure
                                        :generic-team-id generic-team-id})))})))

ignore defevent for now, pretend it said reg-event-fx

We would give functions to call with the result of a request that return an event vector to dispatch

the goal was to make every re-frame event real

This forms a coupling where you have to explicitly chain things right?

meaning if an event happened it was because of some outside force. An http request finished, a user clicked a button, etc

get-line-items:success may actually need to now trigger more events that need to know when it is done.

not because of a synthetic code structure choice

> get-line-items:success may actually need to now trigger more events that need to know when it is done. No it does not

for the same reason that splitting (inc (* n 3)) is unneccesary

How do you represent having an handler that is going to rely on the on-success of get-line-items:success having already been done?

thats not a real event

you would roll up all the state changes and side effects you want to trigger into get-line-items:success. If you need to, breaking them into helper functions. Because :db and :fx are composable

yeah, we had this type of things before though and felt it still got confusing and hard to scale.

we would do this for our route changed handler

(rf/reg-event-fx
  ::route-changed
  (fn [{:keys [db]} event]
    (let [change-route-key (fn [{:keys [db]} [_ next-route]]
                             (let [current-route (route db)]
                               {:db (cond-> db
                                      current-route (update-in [route-state-key :previous-routes]
                                                               (fn [history]
                                                                 (cons current-route history)))
                                      :always       (assoc-in [route-state-key :current-route] next-route))}))
          composed-handler (fx/compose-effect-fns #(change-route-key {:db %} event)
                                                  #(route-change-handler {:db %} event))]
      (composed-handler db))))

We’d then have perhaps 4 layers of things that need to happen in sequence, but have to wait on each other

So each of the 4 had to be explicitly linked together. Step 1 had to know about Step 2. Step 2 had to know about Step 3. Step 3 had to know about Step 4. You can do this without dispatches or with it, same overall issue though.

where every time the route changed we would do some common db changes and side effects but then also include a custom function for that route

Yeah, I think we have some similar code to this in a few places.

oh almost forgot how "helper events" made 10x useless

We compose fx, we add fx as args to the initial dispatch to know what to “do next” etc. I think this is what I was getting at with a “continuation” sort of feel.

with or without async-flow, I never had really high value in 10x event logging. Could be just naive there. I mostly just like it for the app-db view.

the idea was • If a re-frame event happens, it should be in response to something in "the real world" happening, otherwise its horrible to reason about • If we use async-flow for page state, we have problems managing both "replaying" those events when someone gets back on the page and predicting the results of those events. So just have each page maintain its own state with normal events + a route changed handler to set up a valid "zero" state besides nil

It seems to me though that the real crux of the problem here is that async-flow uses events to coordinate the flow. You can instead just chain :fx order in a particular way to capture something a bit similar. You can have “waiting effects” in a sense with that sort of “continuation passing style”. This may be better than using events for coordination because perhaps the re-frame event queue is just too opaque for good reasoning.

seperately, defevent was nice too

when you dispatch a reframe event you need to give the right args in the vector which you have little/no ide support for

and events can't really be documented in the same way

It’s been interesting conversation for me. Oddly enough, we really have no issues using async-flow as I already said and it just was an experiment we used early on a while back and everyone was happy with the simplification of things. I’m quite curious to see how https://lucywang000.github.io/clj-statecharts/docs/integration/re-frame/ is implemented if it isn’t through a bunch of dispatches and I’d like to see how it works in complex flows.

Yeah, I like the idea of the defevent. I’ve heard ideas like this thrown around. I think nothing is open sourced? We have a few internal wrapper helpers, but not some that I’d like to see.

let me check exactly how defevent worked

I mentioned in some other thread here a bit back that we have done quite a bit over “subscriptions” that has helped us simplify a lot of flows and also took performance into consideration. Was here https://clojurians.slack.com/archives/C073DKH9P/p1723137501762649?thread_ts=1722864507.061879&cid=C073DKH9P

I know we’ve blown this thread way out of proportion. Can discuss these topics in a separate one if that helps. (like defevent)

here was the docstring

"Defines an event. Expects input in one of these forms

  (defevent event-name
    [arg-1 arg-2]
    :handler
    (fn [cofx]
      {:db ... :fx ...}))

  (defevent event-name
    \"docstring\"
    [arg-1 arg-2]
    :handler
    (fn [cofx]
      {:db ... :fx ...}))

  (defevent event-name
    [arg-1 arg-2]
    :interceptors
    [interceptor-1 ...]
    :handler
    (fn [cofx]
      {:db ... :fx ...}))

  (defevent event-name
    \"docstring\"
    [arg-1 arg-2]
    :interceptors
    [interceptor-1 ...]
    :handler
    (fn [cofx]
      {:db ... :fx ...}))

  And generates the following
    - a constant named event:event-name which is a namespaced keyword
    - a function named event-name which returns an event vector which can be dispatched with re-frame
    - a function named handler:event-name which takes the coeffects and an event vector and returns :db, :fx
    - the call to register the handler:event-name function to handle events.
      - by default this registration is done globally inside of re-frame
      - if you use the register-with flag, it will instead assoc itself into the given atom
  "
  [& args]

having handler:event-name helped if we needed to compose two events without introducing a dispatch

Nice. I want to think about this one some more. Compelling.

I think we often do that sort of thing by just manually removing the implementation into a common function to use. But I could see it being nicer this way.

its one of the few macros that i've written that actually ended up being a net positive

not net positive enough to save the code base, but 🤷

anyways @james.luke.johnson We had this helper for initial loading states

(defn create
  "Creates an object that can be used to track the status of multiple requests on a page.

  :resources
  a collection of the names of resources that need to load."
  [{:keys [resources]}]
  {:things-still-loading  (set resources)
   :things-that-succeeded #{}
   :things-that-failed    #{}})

(def status:failed ::failed)
(def status:success ::success)
(def status:loading ::loading)

(defn overall-status
  "Returns the overall status of the process.
  Assumes that any failure means the whole process was a failure"
  [initial-loading-process]
  (cond
    (seq (:things-that-failed initial-loading-process))
    status:failed

    (seq (:things-still-loading initial-loading-process))
    status:loading

    :else
    status:success))

(defn status
  "Returns the status of an individual resource."
  [initial-loading-process resource]
  (cond
    (contains? (:things-that-failed initial-loading-process) resource)
    status:failed

    (contains? (:things-still-loading initial-loading-process) resource)
    status:loading

    :else
    status:success))

@mikerod > 1) statecharts is a broader and popular way of organizing this sort of thing. And it's also a proper state machine, without any need for its users to come up with a bespoke convention. > 2) I’m going to guess it doesn’t manage the actual actions and transitions as re-frame dispatches? I don’t see anything saying that it does or doesn’t though. Not sure how to interpret the words "manage the actions and transitions". It does dispatch events. But it doesn't monitor events to see what needs to be done. You specify states and transitions not via "if this event is fired, do this" but via an explicit state description. Dispatched events then become not a management mechanism but a side-effect.

and we would slap it on page state

(defn initial-state
  [{:keys [user-kind collaboration-uuid]}]
  {:initial-loading-process         (initial-loading-process/create
                                      {:resources (cond-> #{thing-to-load:public-info}
                                                    (= user-kind user-kind:creator) (-> (conj thing-to-load:private-note)
                                                                                        (conj thing-to-load:advertising-links)
                                                                                        (conj thing-to-load:collab-info)))})
   :file-upload-status              {}
   ...})

then when requests finish their success/failure events would update their state in the bucket. Once everything succeeded you could move forward with rendering the page.

the way i think about this stuff will make a lot more sense if you try writing a program in Elm

i don't know the directionality of influence, but in elm you are forced to do things more or less the way we ended up doing them near the end

I’ve read a lot on elm but never done anything big.

> Not sure how to interpret the words “manage the actions and transitions”. > It does dispatch events. But it doesn’t monitor events to see what needs to be done. You specify states and transitions not via “if this event is fired, do this” but via an explicit state description. > Dispatched events then become not a management mechanism but a side-effect. If you use notify events it does decouple the “action” from the “transition” - so I think it is somewhat accomplished by async-flow with that pattern. It is not a proper state machine in a sense - it’d be weird to write a flow where a state could transition “backwards to a previous state” for example. I have to think if that really helps anything though. I think with the state machines approach there is an argument for your original point that you don’t want event handlers that may accidentally be called standalone when they were only meant to be part of a flow with dependencies. These state machine actions aren’t explicit handlers so you can’t mistakenly try to use them “standalone”. I think this is what I meant by conventions can accomplish the same thing, but you don’t like that and think it’s too brittle. However, you may actually want to reuse actions where the states/transitions vary, so you’d just have to use different state graphs for different initializations of that I guess.

I’m fine to leave it at this for now though. This is a lot of typing and responding and I appreciate the feedback and takeaways to consider.

@james.luke.johnson, I would suggest that you use a counter instead of a boolean :loading? key. When an http request starts, you inc the counter, and when it ends, you dec it. Whenever it’s pos?, something is loading.

set of things loading, set that finished ok, set that failed

counters are begging for shenanigans

Thanks for a great idea! That seems much better than a boolean. TL;DR if you don’t feel like reading the middle 205 comments in the thread 😅 The answer to my initial question is: Best practice is to use a helper function to operate directly on the db instead of dispatching an event to do the db update. Specifically for keeping track of loading, consider using a set of elements loading instead of a boolean.

Any opinions on this pr? https://github.com/day8/re-frame-async-flow-fx/pull/32 And generally, @emccue, do you think any of these issues could be solved through incremental changes to re-frame-async-flow-fx?

Well aside from the "only run once" stuff which that PR would address: the issues around "unpredictability of synthetic event chains" remain

Long thread which I didn't want to read. My vote is that you would set the state on the db directly when the loading starts, but dispatch when the loading ends. This would be to set the widest range that the UI wouldn't have the right info to show.