Clojurians Log v2

I think that extra layer of indirection, storing multiple things in the state, was what I was groping for. I used to work on a framework with a common interface over storage, and that was the main locus of acquiring data to drive the UI. the net effect was very similar to RAD, though with an entity-based Smalltalk feel rather than attribute-based Clojure + Datomic feel. I think that has deeply trained my thinking into "subset of the database" terms rather than "ephemeral state of the app" terms, which is a mismatch with Fulcro. so I think the path forward is decorate my Instant queries with extra information; if I'm fetching a subset of some list it should be a separate key, either top-level or nested on individual maps: • top-level :ui/selected-todos, or • a filtered inner list: {:todo/id "1234", :todo/tags [[:tag/id 9] [:tag/id 12]], :todo/favorite-tags [[:tag/id 12]], ...} then the mutation (fired each time Instant calls back my subscription) it knows where to put the latest data when normalizing, and the UI is already querying for the right lists without having to know or care what :where clause is driving the current contents of :ui/selected-todos.

🎉

I'm successfully subscribing to an Instant collection and normalizing it into the Fulcro state. needs a bit of work to get the UI noticing that things have been updated.

it's powered by a lightly augmented InstaQL query attached to the Fulcro components. it allows overriding the join keys, both at the top level and nested inside things.

I'm trying to build a draft of this that's all hand-rolled (ie. no component middleware yet) and see how it goes. I've realized that I'm not sure how to handle dynamic Instant queries, that is those with some input parameter (eg. a :where value) coming from the Fulcro DB.

I'm struggling to mate together the filtering available on InstantDB queries with Fulcro's approach. both in RAD (eg. ro/parameters) and in base Fulcro, there's not really a way to express an inner filter, only outer ones. to illustrate, suppose I have a schema of high-level goals which reference a list of todos via [:goal/id :goal/title {:goal/steps [:todo/id :todo/title :todo/done]}]. now I can filter the goals by selecting some named list from my remotes: [{:goals/high-priority [... the above query ...]}] and I can use EQL parameters, mutation parameters or ro/parameters to send my remote some inner filter conditions (eg. only fetch the :todo/done false ones), and my Pathom parser can understand that and return all the goals with only some of their :goal/steps. but then, IIUC, the only options I have for normalizing that subset of the :goal/steps are to union or replace the single normalized :goal/steps list for each goal. it feels like my options are: 1. have inconsistent filtering 👎 2. throw away data 😢 3. let Fulcro fetch everything and implement the filtering in the components 4. introduce further indirection eg. pouring Instant's data into DataScript and letting a client-side Pathom parser do the filtering. then throwing away the inner lists is cheap. are there any helpers for this? any good hooks for wrangling the props on their way into a component? I'm imagining using either EQL parameters or extra defsc settings like ro/parameters to define my filters, and applying them automatically to the props. perhaps I'm completely off track. I'm seeking to treat Fulcro's state atom as an authoritative client-side database, populated asynchronously by subscriptions to Instant queries. I think that this filtering is the only missing link. I'm happy to write the code to compile the filtering from the parameters, but I don't know what hooks are available to implement them.

I think you’re expecting the client-side database to be something it is not, and perhaps also thinking Fulcro is different than it is. You can certainly implement the behavior you are looking for, but you need to adjust your thinking as follows. Fulcro is, as much as possible, a state management system with the following very simple structure: View = Render(Q(state)) where Q is the query (which literally traverses the reified graph in the state map, and Render is whatever you want (defaulting to React with Fulcro DOM). The state is meant to be an immutable picture the app’s entire state at a point in time. ALL evolutions of that state happen in mutations. This removes all logic, magic, mystery and mess from Q. Q is intentionally trivially stupid. Walk the (possibly dynamic, whose dynamism is also reified in the state) query and gather what you asked for. So, your logic for manipulating the state can be a set of referentially transparent functions that take state -> state’, just like swap! expects to do. The time-based sequence of “frames” seen in the UI is just a sequence of applications of functions (mutations). Additional side-effects and asynchrony are purposely moved to the concept of “remotes” (names because they usually talk to something async like a network server). I’ve written remotes that compile solidity…doesn’t matter what they do, but the communication layer with remotes is EQL. You can ask for data, or augment mutations with the remote section to indicate effects that should happen “elsewhere”. Load is technically implemented as a mutation, simplifying the internals even further. Everything that side-effects (loads pull in data that will be merged to state) is a mutation. Look at the data-fetch ns for details. So, if you want a complex behavior you compose it from these primitives. If you want to lazy fetch, then you create a UI state machine or a state chart so that you can more easily combine logic into a coherent unit, but you’re still writing pure functions against state, and running side-effect mutations. So, for example in RAD reports using the server-side pagination machine: I have the raw results stored in one key, where I can accumulate them over time (e.g. run loads and append the results using data targeting), and I cascade that data into a different key (e.g. :ui/current-page) as a list of idents derived as a subvec of the primary list. So if you’re trying to treat fulcro’s database as a full cache for your results, then do so…just load them to some key that you have named for that purpose (not in view), then write mutations that can do the filtering and place the results at a different location in the view. That’s exactly what RAD reports do. The reasoning is all quite clear and simple, and gives you ultimate flexibility and testability. So, perhaps gain some inspiration from RAD report ns and state machine?

still making progress on this. I think the subscription scheme above is pretty plausible. I realized that the Fulcro components' :query nesting has nothing to do with what I get back from Instant, so only the top-level join keys matter - ie. I got this list of entities from Instant and converted them to (possibly deeply nested) CLJS maps, where does that list land in the DB? the returned data, though, is not based on an EQL query built from my UI, it's always a subset of the Instant schema. so I'm now leveraging RAD to define my test app's data model in RAD, generate the Instant schema from that, and also generate EQL queries for each entity in the schema that can drive normalization.