Fork me on GitHub
#fulcro
<
2020-03-28
>
Jakub Holý (HolyJak)08:03:57

I am getting > Attempt to get an ASM path [:com.fulcrologic.fulcro.ui-state-machines/local-storage :pending-path-segment] for a state machine that is not in Fulcro state. ASM ID: :minbedrift.ui/SubscribersListRouter and do not understand why the router's UISM is not in the state. I have just rendered the component that includes it (via (if some-cond? (ui-bill-run props)) , ie. it hasn't been on the screen originally, if that matters. The parent component:

(defsc BillRun [_ {:ui/keys [subscribers-list-router]}]
  {:ident :bill-run/id
   :query [:bill-run/id {:ui/subscribers-list-router (comp/get-query SubscribersListRouter)}]
   :initial-state {:ui/subscribers-list-router {}}}
  (ui-subscribers-list-router subscribers-list-router))
What do I do wrong? (The router's target component is rendered, though.) Or is it nothing to worry about? Hm, it actually is a problem. When I try to dr/change-route to its second target, I get an error > ERR You are routing to a router :minbedrift.ui/SubscribersListRouter whose state was not composed into the app from root. Please check your :initial-state. yet its default target is still showing?!

currentoor19:03:36

is BillRun composed in some parent component’s :query and :initial-state?

Jakub Holý (HolyJak)22:03:13

Good point, it is in the query but not initial state.

Jakub Holý (HolyJak)07:03:22

Spot on! The parent component was missing from the Root's initial-state. Thank you!

Alex H08:03:40

Something I really enjoy about re-frame is the subscription model, how it's trivial to chain subscriptions that e.g. extract something from the database, and then further refine that (think filters, rearranging data to fit the UI, etc). That way with the single source of truth of the database, you can trivially have several views, if you will, of that data, for different parts of the UI. And all of it updates only when any of the underlying data changes. In Fulcro, it seems no such thing exists, so the options are either to do it in the individual UI components (doesn't seem great if some of those computations are reasonably expensive), or to do it as some form of post-mutation after the underlying data arrives/changes, that writes it into some different place of the database. That second approach seems workable, though much less ergonomic than re-frame's subscriptions. Am I missing some other option?

Alex H09:03:04

That is literally what I described as the "second approach". Just seems much less ergonomic having to write it into the DB, and having to have several hooks on the various mutations that happen to update the underlying data, rather than it automatically triggering a refresh whenever the data changes (regardless of which mutation caused it - which is what re-frame does with its subscriptions)

Alex H09:03:40

To me, that just seems like you have much closer coupling of things which really ought to be decoupled.

Jakub Holý (HolyJak)09:03:44

Yes, I know. But I have not mentioned the 3rd approach

Alex H09:03:12

If I have e.g. 3 mutations which can cause various different updates of the underlying data, I'm not sure I want each of those 3 to have to know about a specific post-mutation of sorts to run so that various bits of derived state are updated.

Alex H09:03:26

It'd seem much saner to me if there was a way of saying: ok - this data changed, so I'm triggering the refresh of derived data

Jakub Holý (HolyJak)09:03:16

It might be easier to discuss if you gave an example of the concrete data and relationships? Normally 1 component corresponds to 1 data entity, which is stored normalized in the DB. If you want to derive more data for that component, it is perfectly fine to do it in the component's :pre-merge b/c that is the only place involved in getting its data in the DB.

Jakub Holý (HolyJak)09:03:31

It is possible you might need to model your data a little differently to fit better with Fulcro

Alex H09:03:53

hm - pre merge might be what I'm looking for, actually - it's post-mutations that are specifically what I didn't want.

Alex H09:03:14

I'll experiment a bit with pre-merge and see if that's more along the lines of what I was looking for, thanks for the hint.

Alex H09:03:37

(as an example of the sort of thing I'm looking for, you could imagine e.g. loading a list of cars - and I'd then want to derive multiple views of that data: one that is just a list of model names, for e.g. some autocomplete of sorts; one that is a view of the cars grouped by brand, and that view might be derived itself from a filtered view of the cars). Not actually doing anything with cars, but that seems to roughly reflect what I'm looking for 🙂 In re-frame, I'd have the list of cars, and then a subscription that does the filtering, another subscription based on that filtered subscription that does the grouping, another subscription based on the original car list that extracts model names, etc.

Alex H09:03:51

it sounds as if pre-merge might be able to handle that, by checking if the car list has changed, and if it has, re-derive the relevant other views of data

Alex H09:03:08

still feels slightly less ergonomic, but at least it's sounding workable

Jakub Holý (HolyJak)10:03:56

You can get a better answer when @U0CKQ19AQ, @U09FEH8GN etc come online. I'm sure that making multiple (cached) derived views of a source data (list of car names, cars grouped by maker,..) is well supported.

currentoor16:03:30

How large is the data? @alex340

currentoor17:03:36

If it’s less than 10,000-50,000 cars you can make a list of all the cars in app state, using idents, query for that and do your filtering in render. This approach often works without perceptible performance slow downs.

currentoor17:03:35

All the components have shouldComponentUpdate optimizations on them

currentoor17:03:40

If you can’t do that, then yeah you have to maintain the derived/filtered data in app-state. But that usually isn’t as cumbersome as you’d think, write a function that places all the filtered data where you want it and then call that whenever you make a relevant state change, like loading a new record

currentoor17:03:49

I can understand how coming from re-frame this sounds more finicky, but it comes down to the fact that the source of truth in re-frame is denormalized trees of data and in fulcro the source of truth is a normalized database

currentoor17:03:11

But you get used to it quickly and it becomes second nature, and there are soo many advantages to using a normalized data store

markaddleman18:03:19

I'm a fulro newbie so forgive the naive question: Couldn't this be semi-automated by an atom that watches the appdb and recomputes the derived state on relevant changes.

markaddleman18:03:54

It would be nice if this is integrated into the fulcro world through a component-like thing that issues the query and inserts the result into the app db only when relevant components are displayed.

currentoor18:03:32

@U2845S9KL so app state is already stored in an atom and you can already add a watch to it

markaddleman18:03:05

yeah. i realize i wrote my message badly. i meant exactly that 🙂

currentoor18:03:10

but the problem with that approach is then your watch runs every time any change happens to app state

markaddleman18:03:39

right. the watch would have to introduce optimization logic that (i imagine) duplicates the logic in queries

currentoor18:03:40

typically you only need to re-compute derived data when a car is added/removed

markaddleman19:03:23

further, you only need to compute derived data if a relevant component is mounted

currentoor19:03:03

think of fulcro’s app-state the way you think of a postgres database in the backend

currentoor19:03:55

single source of truth, normalized, get what you want directly out of there, keep it simple

currentoor19:03:34

you could add things like subscriptions, watches, but you’re usually better off not doing that

Alex H19:03:04

well, my re-frame db is also fully normalized, so I don't think that distinction really holds; the denormalization happens in the subscriptions

Alex H19:03:36

handling it in the component isn't really the way to go, imo, so I think I'll give that pre-merge fun a try (haven't yet) - which does sound like it ought to do the trick

Alex H19:03:36

I really don't like the idea of the post-mutation - that's just clunky

Alex H19:03:04

and couples things closer than they should - the parts that e.g. add a car shouldn't really care about the various bits of derived state

currentoor19:03:39

what are you using for the data store in re-frame?

currentoor19:03:43

why do you think post-mutation is clunky?

currentoor19:03:55

(defmutation fill-schedule-cells
  "Mutation: For every employee in the global list of all employees: create a row on the work schedule whose cells
   reflect the currently-scheduled shifts, or placeholder nodes if nothing is scheduled in that cell.

   params can include:

   `:gc` - A list of old schedule IDs to GC before filling out the current schedule."
  [params]
  (action [{:keys [state]}]
    (let [query         [{:ucv.models.employee/all-employees
                          [:employee/id :employee/first-name :employee/last-name :employee/hourly-wage]}]
          all-employees (-> (fdn/db->tree query @state @state)
                          :ucv.models.employee/all-employees)
          schedule-id   (ns/get-in-graph @state [:component/id ::work-schedule-editor :work-schedule/current-week :schedule/id])]
      (swap!-> state
        (fill-schedule-cells* {:employees   all-employees
                               :schedule/id schedule-id})
        (add-materialized-metrics* {:schedule/id schedule-id})))))

(defmutation goto-schedule
  "Mutation: Load the schedule for the week that contains the inst `of`, GC old schedules, and fill in the cells for the
  work schedule editor."
  [{:keys [of]}]
  (action [{:keys [app state]}]
    (let [old-schedule-ids (-> state deref :schedule/id keys)]
      (swap!-> state
        (cond-> (seq old-schedule-ids) (gc-schedules* old-schedule-ids)))
      (df/load! app :ucv.models.schedule/for-week WorkScheduleEditor
        {:target        [:component/id ::work-schedule-editor :work-schedule/current-week]
         :marker        ::schedule
         :params        {:of of}
         :post-mutation `fill-schedule-cells}))))

currentoor19:03:31

it’s made for this exact type of situation

currentoor19:03:43

^ that’s an example from one of our apps

Alex H19:03:02

what I feel is clunky is that it's tying together multiple concerns - it means the mutation that adds the car needs to know about various bits of denormalized state that get derived from the car list, even though it has nothing to do with adding a car in itself

Alex H19:03:43

(as for re-frame - I currently have my own load functionality that handles the normalization of incoming data into the database - which is why the database is normalized, and the subscriptions handle denormalization)

Alex H19:03:08

your example also ties things together in a way that, imo, shouldn't be necessary; but maybe a question here based on your example - why a post-mutation instead of pre-merge?

currentoor19:03:53

IMO the mutation that creates a car should also update any references that need to point to it, just like you would in a server DB no?

currentoor19:03:35

on the server, the logic that creates a car also updates any foreign keys that need to point to it no?

Alex H19:03:11

hm. kinda, though I'm not sure that's entirely comparable.

currentoor19:03:03

well why not?

currentoor19:03:33

> I currently have my own load functionality that handles the normalization of incoming data into the database so what kind of database is that?

Alex H19:03:35

from my original example - does it really make sense that the mutation that know about adding a car knows about (A) adding a car (B) some auto-complete component for brand names, and (C) some component that shows cars by brand (or manufacturing year, or whatever)?

Alex H19:03:42

it's re-frame's app-db

Alex H19:03:54

(just an atom)

currentoor19:03:56

i see, so you’re (A) normalizing the data yourself and (B) de-normalizing it yourself? for every entity/relation you add to the database you’ll have to update A and B?

currentoor19:03:09

so in your car example, the logic of (B) and (C) is encapsulated in a function call it (D), then (A) just needs to know to call that function, not the details of (D) that seems totally reasonable to me and it’s easy to test, you could even stub (D) in your tests if you really cared

Alex H19:03:11

well, the normalization happens automatically via some additional layer I have built

Alex H19:03:22

the de-normalization is handled in each subscription, though, yes.

Alex H19:03:42

the logic for B and C are two orthogonal subscriptions, of which the logic for A knows nothing.

Alex H19:03:30

fair, though, that it doesn't need to know the details of what B/C do even if it knows to call something as a post-mutation

Alex H19:03:10

so - a related question, then - when do you use pre-merge vs post-mutation, in practice?

currentoor19:03:44

these docs explain it better than i could

currentoor19:03:01

> Post-mutations are a little limiting in this case because they are not component-centric: they are not co-located with the components, and may have to deal with an entire sub-tree all at once. Pre-hooks decompose this logic to the component level making things a bit simpler in many cases.

Alex H19:03:28

maybe I lack imagination, but I do at times struggle with the examples in the fulcro doc - for something as prescriptive as fulcro, I'd have hoped to have a bit more realistic examples highlighting the practical differences than some counter values incrementing/decrementing

Alex H19:03:22

your earlier example about schedules/schedule cells would've struck me right on in the pre-merge camp rather than the post-mutation, for example

Alex H19:03:35

so it's not entirely clear to me why it's one and not the other

currentoor19:03:25

examples are tough to get right, you have to balance several concerns

currentoor19:03:19

so my earlier example was neither pre-merge nor post-mutation it was just inside client side mutations

currentoor19:03:57

but i suppose it’s similar to post-mutation

currentoor19:03:40

so i think the most important thing to keep in mind is, when you want to reason about trees and the logic makes sense to co-location in the UI use :pre-merge

currentoor19:03:51

also :pre-merge only happens when data is loaded from the server using that particular UI component

currentoor19:03:17

for all other situations you need to use post-mutation and mutation helper functions

currentoor19:03:29

like in my schedule example

Alex H19:03:53

thanks, I'll spend a bit of time trying things out and might come back with better questions 🙂

Alex H19:03:56

appreciate all the insight

currentoor20:03:29

my pleasure!

tony.kay21:03:25

@alex340 I’ll throw in a short 2 cents: If you concentrate on any one feature, and implements things like normalization yourself, then you get to a point of diminishing differences. So, saying “I’ve implemented half of Fulcro on top of Re-frame, so why use Fulcro?“…well, perhaps you shouldn’t 🙂 But I can tell you why I leaned the way I did in designing it this way: First, I dislike event-based/subscription models. They lead to very hard to trace, disconnected logic. Yeah, you get a certain level of ease in one direction (data flows from db through transforms via events), but pain in another (you’re later trying to remember where the heck you put X or Y and there’s no direct line of logic to follow..you do primitive and frsutrating searches on your code to figure it out). Fulcro aims to be as transparent as possible in the data flow: There is a query, there is data. The query can be used to talk to the server, and pull data from the db. It’s very very simple to trace from db -> UI. There’s no mystery ever (beginner’s excluded…if you don’t understand it yet, lots of things are mysteries). Now, the points you bring up are valid ones, but they fall into a few categories: personal preference of API (I don’t care to even talk about those), performance (worthy of consideration, at least to the point of answering can you do it cleanly), and what I consider to be “ease of reasoning”, which is hard to quantify, but very very important in large-scale applications that have many data needs. Experience has shown the optimization is very easy to do. So, ultimately we’re talking about this last one: ease of reasoning. Your complaint about things “feeling clunky” is one I definitely am sensitive to, and have spent a lot of time working on. Om Next put this kind of logic in the parser. That was a nightmare to maintain, and a very poor choice on this criteria. I did not consider the event-based or subscription model to be ideal simply because it decouples the query from the hard data, and I very much value that simplicity. There is a reason re-frame 10x exists, and before it did, I bet a lot of people spent a lot of time cursing the lack of traceability. Post mutations were actually the first attempt at solving the problem. They work very well for many cases, because many of the cases don’t have multiple views of data. There is one use-case, and putting the logic right where you can nav directly to it is perfectly fine. It’s also common for multiple views of the same data that shares the screen to really just be two different displays of it…why not have one component with one query that loads the data, and then dispatches that data to two difference rendering functions (or stateless components with shouldComponentUpdate)? That works quite well, and optimizations are still reachable from regular mutations, post-mutations, and component-local state-based memoizations. Pre-merge was an improvement based on experience: often your post-mutation is just munging the tree slightly, and doing that to the tree before you merge it is perfectly fine. More complex systems that have even more complex needs often need a lot more than just view logic. That’s where UI State machines came in. They solve the issue of reasoning about the UI over time.

tony.kay21:03:31

Now, all of the above is just considering the pros/cons of the data model and derived views. You have not even begun to talk about the advantages of having the query, idents, and an open/extensible map of options on the component. These small things, which many ppl dismiss as “boilerplate” are anything but. Not only do the simplify and unify the entire I/O story, they also provide valuable introspection and abilities that have proven quite useful in many many places. Take UI State Machines. The actor model in that namespace is based on UI actors. Those actors can be swapped out, and can do full-stack interactions within a re-usable state machine, and the state machine needs to know almost nothing about the ui component itself. The CRUD form support in RAD is a single bit of logic that leverages these facts to obtain very compact and reusable systems of code. Then look at dynamic routers. They leverage the queries to dynamically construct and find at runtime the routing tree of the application, which will auto-heal during code refactoring. The list goes on and on. That said, I see the appeal of Reagent and Re-frame. They are very easy to get started with and use for small projects. Proper discipline and good helper libraries will make them just as successful as anything else.

tony.kay21:03:52

Fulcro is my personal set of preferences, turned into open-source 🙂

tony.kay21:03:51

And it evolves all the time. For example, one very valid complaint for much of the history of Fulcro is that you have to reify certain edges in the database to get the query to work: the ui query has to compose to root, meaning the graph db has to be complete as well. This is made much simpler by the initial-state option on components, but it is both a source of confusion and a true source of some boilerplate (though as I said, this composition is not without benefits). So, recently I added support for “floating roots”: The ability to embed a subtree in the app that is stateful, but is not connected to the root by query/ident/state….the edge in the db is not needed. This turns out to be quite handy and necessary in some circumstances (using vanilla React routing libraries, for example), so I added it. That said, I don’t use it often, because the loss of that connection has costs. Then there are React “hooks”. Those were pretty easy to add support for, and are now present. The most important result-to-date, however, is the fact that I’m able to build pretty interesting reusable things in RAD because all of these things exist in Fulcro: standard normalization, unified I/O story, UI state machines…if I was cobbling things together in my web apps like most ppl do, there is no way such a unification would be possible.

tony.kay21:03:38

My ultimate goal has always been this: I’m building software for my businesses, and I want a unified full-stack story that is well-documented, performant, easy to reason about, is understood by tooling (things like code navigation), and is still as flexible as possible. I can’t tell you how many times this design has surprised me with “Oh, that was easier than I thought it would be”. I’m happy with it, and have used it for years now.

tony.kay22:03:26

The transaction processing system of Fulcro is intended to be pluggable. It would not be terribly difficult, if you wanted to, to put a middleware layer into Fulcro that would run any manner of things against the database at predictable times. I did such a demo showing how you could use Clara Rules engine to run rules against your Fulcro db (though I cheated and used an atom watch). The normalized database actually makes it pretty darn easy to build a diff that can be used to selectively trigger updates (run this thing when that thing changes), which is what that demo did. I don’t personally need it yet, but I might write a better-designed such thing in the future.

Jakub Holý (HolyJak)09:03:02

When I route to a component, it fails with a Spec error, I guess because the query only has a :ui/ component. But when I look at the target's query, it has more than that. Why the difference and thus error? The query shown in the spec error:

:children [... ... {:type :join,
                               :dispatch-key :alt0,
                               :key :alt0,
                               :query [[:ui.fulcro.client.data-fetch.load-markers/by-id _]],
                               :component minbedrift.ui/SubscribersList,
                               :params nil}],
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
the query when I fetch it manually:
(comp/get-query SubscribersList) ;=>
[{:bill-run/subscribers [:br-subscriber/subscriber-id ..]}
 [:ui.fulcro.client.data-fetch.load-markers/by-id _]]
(the component is a RAD report, if it matters)

4
currentoor19:03:31

so that spec error is an AST, not query

currentoor19:03:11

what does the rest of the spec error say?

Jakub Holý (HolyJak)22:03:14

Hi, thank you! Yes you are right.. But it is an AST of a query, not? I will post it tomorrow, I can't now.

Jakub Holý (HolyJak)07:03:44

Hm, I see something suspicious. According to the stack trace, this happens in my mutation set-selected-org , name in this load t runs:

(df/load! app ident query-component
                        {:without #{:bill-run/subscribers}
                         :marker :ui/selected-org
                         :target [:ui/selected-org]})
I did not expect the mutation to run at this point; it has bee run before and now I expected just a load. It explicitly excludes the key :bill-run/subscribers that the SubscribersList needs, so that would likely explain the AST error. Ah, I see, the error is indeed happening during the organization load, while I believed it is happening in the later stage, when I click the next button.

Jakub Holý (HolyJak)08:03:27

I have fixed it by replacing the :withouton the load with renaming the key of the SubscribersList's router in the parent component to :ui/... so that the whole thing is excluded from the query sent to the backend.

currentoor19:03:52

@alex340 here’s the same example but with better formatting

grischoun19:03:23

Hello. Fulcro newbie here. What are the options if I want to deploy a Fulcro app on my own server (not Heroku)? I assume I have to build an uberjar. If I take the fulcro-template (Fulcro 3) as an example, how should I proceed?

currentoor19:03:53

have you looked at this?

currentoor19:03:08

at the bottom of the README

currentoor19:03:43

our deploy process is usually use shadow-cljs to make release builds of all our JS assets

currentoor19:03:51

and any other static assets we need

currentoor19:03:02

then use depstar to make the uberjar

grischoun19:03:07

I see! Sorry I missed the part about deploying at the end of the readme. Thanks for the extra info about your own process, I’ll try that.

currentoor19:03:20

no worries, happy to help simple_smile

currentoor19:03:39

@alex340 here’s what our app-state looks like, we have several dozen entities that depend on each other in complex ways, writing the denormalization logic for every situation would be a lot of work, in fulcro you have a query engine that does this for, that’s what i meant by fulcro comes with a normalized data store, perhaps i should have said it comes with a normalized data store and query engine that can denormalize anything for you

👍 4
roklenarcic21:03:46

I am trying to use com.fulcrologic.semantic-ui.collections.form.ui-form-input/ui-form-input with a custom input component in a form but whenever I transact, the component gets rerendered and loses focus on each keypress. Here’s the general render code:

(ui-form {}
    (ui-form-input {:control #(comp/with-parent-context this (dom/input %1))
                    :onChange #(m/set-string! this ::sepa/full-name :event %)
                    :value full-name})
.....
Imagine here that dom/input is some other input control (StringBufferedInput..), if you have this setup it’s impossible to type because the control gets defocused every key press… is there some sort of workaround for this?

roklenarcic21:03:26

this setup works otherwise when I don’t specify :control

tony.kay22:03:31

@roklenarcic This is a react-ism that all libraries like ours have to work around. Async updates on form fields cause this. The inputs in Fulcro are actually wrapped with components that use component-local state to “buffer” the value so things don’t screw up.

tony.kay22:03:15

don’t use ui-form-input….it’s doesn’t do anything active does it? Just format the thing with divs as shown in semantic UI docs.

tony.kay22:03:35

Or, study the code of wrapped inputs and figure out how to get ui-form-input to work 😕

roklenarcic22:03:27

it has a neat error popup and a couple of other nice things, which is why I’m using it