Clojurians Log v2

My original vision for flows was quite simple. It was motivated by usecases where: 1. derived data from multiple pieces of state has to be calculated. 2. Each of those pieces of state can be updated by different event handlers. For example, a trivial case might be an app that calculates the area of a rectangle : 1. One input for the length 2. One input for the width When the user changes either, the area (derived data of length and width) in app-db has to be recalculated. That's when I'd use a flow. Less trivial usecases, might involve calculating (derived) error states if there are many possible inputs and collectively they contribute to an overall error state (and if the "ok" button is disabled or not). I'd be cautious about using flows in more complex situations. I really did add flow to support the derived data flowing promise of re-frame.

If anything the current implementation of flow is a bit too capable IMO. It has grown beyond my original vision. I'd hate for it to be used too much. But, for certain usecases, it can be a really simplifying technique.

To put that another way, I regard flows to be a less fragile version of the on-change interceptor (because there is no need to remember to put an interceptor on the right event handlers) https://github.com/day8/re-frame/blob/master/src/re_frame/core.cljc#L907-L927

Summary: I would recommend against "over doing it" with flows. If you are seeing them as "architectural", rather than simply a convenience which makes code less fragile to change, then you might be "over doing it".

> because there is no need to remember to put an interceptor on the right event handlers > There has been no need for that for years though - global interceptors.

In many situations, true. So long as the application is simple enough a static solution like this is fine. More complicated and it starts to feel a little less clean. It feels like something more dynamic is necessary. On the other hand, global interceptors can be added and removed , so that could be enough even in larger, dynamic situations.

But, I think my main point was to say, don't get too enamored with flows. They are a nice convivence in certain, limited cases. Don't orient your architecture around them.

I would only argue that the set of limited cases perhaps should be extended to support dynamic paths. I structure data in my apps in a normalized form. So in essence it's just nested maps of model -> id -> item. And it's a relatively frequent need to: • Do X to any item of model Y that got its field Z changed by anything • Do X when any item of model Y is removed • Do X to an item of model Y that was just added And it gets gnarlier when there are references: • When an item of model X refers to an item of model Y, do Z to the first item when the field A of the second item changes Of course, currently I solve all these with global interceptors. But they're largely the same across all the scenarios, with tiny tweaks. And seems like flows would be a good fit, and I bet I'm not the only person who would benefit from something like this.

(Ah, actually, "do X" definitely means not just "set a value at a particular path in app-db" but also "cause an effect". So I guess my argument is, overall, for powerful flows.)

here are my thoughts (still in progress): • the app which I am developing is unusually complex. because it is not normal website. It is more like real (normally installed on your computer) app, but in web browser. It is perfect for SPA, but at the same time such SPA are rare. • global interceptors can make app a little too slow, check too many fields each time and to too many calculations. I don’t know, but I would expect that. • Flows seems to be perfect, because I can develop code in one place to handle changing values effect on other inputs and outputs. It doesn’t work with dynamic path. • Event interceptors work in every scenario, but logic spread into multiple files and lines. Complexing of developing start to be frustrating. Team can’t miss any event in each input / output. Normally it could be achievable, because more or less apps are about things which we know like finance and things are more or less obvious. Here with such hard physics calculations there is no way software developer can know if it is correct or not. Engineer do calculations and software developer everything else. Keeping things in one place in code is very helpful. Ah I actually forgot to mention it forces each field to have own event only to use different interceptors, which produce a lot of lines. to summarize this for me it is choice between: Flows simplicity (code is in one place and clearly show what it does) vs Events interceptors (always work, but hard to fallow). For now I will probably end up with Flows for static paths + Event Interceptors for dynamic paths. I don’t like it too, because solution is not consistent. We will have some things in Flows and some in Event Interceptors. At least for now I don’t have better idea. Ideally it will be perfect if can use input as: [:foo :bar uuid-1 :baz :uuid-2 :foo] and Flows will check the pattern. uuid-1 and uuid-2 can be whatever. The Flow will be triggered as long as pattern match. This will be perfect. Of course I will need to read uuid-1 and uuid-2 to make correct output path. But the point is to achieve code around multiple inputs which produce outputs in one place in code. It is much easier to develop such app if refer to inputs in app-db instead of abstractive events.

PS inputs -> output in this case it not simple (* a b), calculations are more resources consuming.

> global interceptors can make app a little too slow, check too many fields each time and to too many calculations. I don’t know, but I would expect that. That's subject entirely to how you write them. I haven't delved deep enough into the code of flows, but I don't think that they would be inherently significantly faster. > Event interceptors work in every scenario, but logic spread into multiple files and lines Lines - yes, just because achieving the same thing in flows would require less boilerplate. Files - well, it also depends on how you write them. Nothing stops you from putting all your interceptors into a single file.

> That’s subject entirely to how you write them. I haven’t delved deep enough into the code of flows, but I don’t think that they would be inherently significantly faster. The speed will be the same, but Flow will be triggered only when needed vs global interceptors.

> Files - well, it also depends on how you write them. Nothing stops you from putting all your interceptors into a single file. Code, files is organized around for example wellbore, pipes, casing design etc. If you change dimension of pipe, then everything change: pressure, pressure for stones around etc. No way to keep it in one file.

to be precise this is the app for drilling operations in both onshore and offshore environments.

to extract oil, gas, water etc. this is really complex physic.

The speed will be the same, but Flow will be triggered only when needed vs global interceptors.Then it's irrelevant w.r.t. performance, because in a global interceptor you can do the exact same check that a flow does internally. But it is more code to be written manually, yes. > No way to keep it in one file. If you can't keep relevant code in a single file when using global interceptor, then you also won't be able to do so with flows. They reduce the amount of loc that they can reduce by a constant factor. It's not magic, it's conceptually not different from just some specific boilerplate for global interceptors.

let me give an example: casing design is about pressure in pipe and round pipe. It need to know Pore Pressure Profile and Fracture Gradient Pressure Profile which let’s say is about different pressure in stones around on different measured depth. One of input is gradient in wellbore which is about wellbore and should be there. Pressures are about Casing Design and should be there. With Flows I can make flows.cljs in Casing Design and have all inputs to calculate pressure for gradients in once place. With Event Interceptors I will need at least 1 interceptor in each input event which is multiple files.

it is not magic, but changing logic to base on data changing instead of events calling let to write code in much different way.

> With Event Interceptors I will need at least 1 interceptor in each input event which is multiple files. I thought we were talking about global interceptors?.. I have never attempted to compare flows to regular interceptors in this thread. Only to the global ones.

In global interceptors, you do have access to old app-db and new app-db, and you can determine whether anything has changed at a particular path yourself.

ah ok, misunderstanding

yeah writing own Flows but DynamicFlows (patterns [:foo :bar uuid-1 :baz :uuid-2 :foo]) is probably something to consider

> That would require registering and un-registering flows every time an UUID becomes available or unavailable. I think that's our intended use-case for re-frame's :reg-flow and :clear-flow effects. I do a similar thing in my app.

Right, my point is that it's just a ton of manual work. Plus, AFAICT there's no way for a flow to receive the old data. So if a path is like [:pipes uuid :pressure], the changes to UUIDs cannot be monitored via yet another flow on [:pipes] - you have to use a global interceptor or manually keep track. Which makes me think that maybe letting flows access the old value as well would also be beneficial.

In the current revision, flows can access https://github.com/day8/re-frame/blob/6f8ac8282d4385fedaac70223e8e7330ddb4a94f/src/re_frame/flow/alpha.cljc#L163. They can also have https://github.com/day8/re-frame/blob/6f8ac8282d4385fedaac70223e8e7330ddb4a94f/src/re_frame/flow/alpha.cljc#L179. I'm cautiously trying these things out, and I have a feeling we might not keep the effects. But it sounds like these kind of patterns could be built that way.

I guess that means a flow could have a :reg-flow effect. Sounds dangerous, somehow 🧟

@kwladyka I should caution you that we don't really think about high-performance computing as part of re-frame's mission. On the contrary, UI is often "fast enough," even if theoretically it could be 100x faster. And @p-himik is right: you can think of re-frame as a reference for certain data-driven patterns, rather than a one-of-a-kind power tool.

In the current revision, flows can access old-inputs and old-output. They can also have effects. Oh, I was looking at the docs and not at the impl. > I have a feeling we might not keep the effects. Well, then watching for changes in data under UUIDs would be impossible - every flow would have to be run for any change made to the uuid->data map. > I guess that means a flow could have a :reg-flow effect. Sounds dangerous, somehow 🧟 Yeah... But maybe just put a large "caveat emptor" in there and that's it. Everybody can already do anything by just calling plain functions. Or creating an effect that calls reg-event-fx, or something else.

Even if it's not as stable, I would probably still use it in a similar use case just because re-frame is tiny and the impl is very straightforward. And even if all hell somehow breaks loose, you can retain the current syntax of flows and implement them on your own on top of global interceptors. Also, just in case - what you're describing sounds like a perfect fit for a rules engine.

Yes, we probably won't delete or break anything in there.

Let me know how it goes. I recently added the capability for a flow to dispatch an event, similar to re-frame-async-flow-fx.

I am experimenting. It seems to work exactly like I expected. Although I have realized I can’t use it for dynamic paths which refer to set of data by uuid. I mean here path looks like this: [:pipes uuid :pressure] with many possible UUIDs. I can’t make Flows based on :pressure. I would have to check :pipes and calculate all data, because I don’t know which UUID have changed. This is probably wrong approach. But besides of dynamic paths should be good experience.

Perhaps it makes sense to transpose the data? So the paths become [:pipes :pressure uuid]. Then you can have a single flow for [:pipes :pressure] that gets a map of uuid->pressure. Perhaps it's also possible to extend the flows from the outside so you can register something that ends up receiving a single pair of uuid and pressure when the pressure is changed for that UUID. I do something similar, only my approach is based on global interceptors.

In reality data structure is more complex, than my example. It will almost impossible to understand data if all fields will have paths like [:pipes :pressure uuid]. Now I can use re-frame-10x with filter [:foo :bar uuid :baz] to debug what I am doing. If I could get event parameters which changed the db, then I will be able to get the UUID.

Consider using a template function:

(defn make-pipe-pressure-flow [id]
  {:id [:pipe-pressure-flow id]
   :inputs {:pipe [:pipes :pressure id]}
   :output {...}})
(reg-flow (make-pipe-pressure-flow #uuid 1))
(reg-flow (make-pipe-pressure-flow #uuid 2))

> It will almost impossible to understand data if all fields will have paths like [:pipes :pressure uuid]. Depends on how you approach it in general. Such a transposition is very reminiscent of ECS - Entity Component System. A common approach in some areas. If the data is stored that way, nothing prevents you from having transposed views of that data that you inspect. But I get that it can definitely be cumbersome if no other part of your app benefits from such an arrangement of the data. @kimo741 That would require registering and un-registering flows every time an UUID becomes available or unavailable. Maybe some more generic approach to flows is possible... E.g. where flows can accept a function as :inputs that receives app-db and returns a potentially empty set of paths that should be monitored. The function would be called every time the flow should be triggered. If the function returns some new paths, it's equivalent to reg-flow. If the function doesn't return some particular paths anymore, it's equivalent to clear-flow. On the surface, it sounds like a feasible non-breaking change that can make flows much more flexible.

if no other part of your app benefits from such an arrangement of the data.Even worst, this make it harder, because data is grouped in more or less reasonable way to have inputs which you will need in leaf of tree if you take any from there for calculations. So you get :foo :bar uuid and you get batch of data which you need for calculations and are about “bar” calculations.

> That would require registering and un-registering flows every time an UUID becomes available or unavailable. indeed