Fork me on GitHub
Jakub Holý (HolyJak)10:01:13

Hello! I have been told that some people were unsure how to apply troubleshooting steps recommended by my Fulcro Troubleshooting Decision Tree and were unsure about how to interpret the results. I have therefore added with many screenshots and refer to it from everywhere in the guide where it is relevant. I would appreciate your feedback 🙏 Is it clear? Any improvements to make? ... For the curious, here is the diff Also, I am thinking about perhaps making ☝️ into a screencast but I am not sure whether the value added w.r.t. the existing text & pics justifies the effort. Thoughts?

🎉 3
❤️ 6

Wonderful work, @U0522TWDA! I read the whole thing, and I found it incredibly instructive, and I think it wonderfully captures the process I’ve seen you go through. It is very clear, and describes very well the Fulcro machinery at work in a way I certainly would have found helpful — and I laughed at the resolution of the problem, as it’s something I’ve omitted by accident many times! IMHO, a short screencast would be helpful, as it shows not just the problem-solving discipline, but also how one uses Fulcro Inspector (which is incredible, @U0CKQ19AQ!), navigating to the relevant areas of the resolvers, etc. Anything that shows how all the small pieces comprise the whole is incredibly illuminating.

☝️ 3

I think it’s a nice improvement. beginners definitely need a lot more “look here” and “click on that”. The biggest problem I’ve seen with the screencasts is that a lot of people won’t take the time to watch them. They are in too much of a rush and don’t make the time. The written form allows them to be more efficient and quickly scan for something that looks “close” to the question they have at the moment. It’s why I do both. Some people will make the time to watch, and the videos can be really enlightening. Others will never watch and just want the quick scan of text for an approximate answer that they can immediately fiddle with.

👍 3

My 2 cents on this. You're totally right; I was heavily in a rush when starting out with fulcro, so I skimmed the docs, cloned RAD and started coding. That obviously didn't work out well, so I finally took the time to watch your YT series multiple times and read the book. The book and videos are highly complementary to each other, but what I underestimated the most was the amount of information in the podcast series you did. There's a lot of background info in there and I also appreciate that you explained a lot of your reasoning behind fulcro and its internals. I also started a similar collection of notes to @U0522TWDA (in german right now though). Simple stuff like "This exception is caused by that" + example code. Also, some rookie thoughts and aha moments. So I guess what I want to say is, you're right about people being in a rush, but most of all thank you for all the materials and fulcro of course! Oh and as you said multiple times, just cloning the entire fulcro source and looking things up was key to figure things out for me

❤️ 6
Christopher Genovese17:01:13

I'm puzzled about how in practice resolvers for attributes stored remotely should be constructed so as to query the remote a reasonable number of times. I've gone through the documentation and the code in fulcro for automaticall generating resolvers for sql and datomic. I suspect I'm missing something simple. For now, suppose we are hand writing resolvers for simple entities/attributes [:ent/id :ent/a :ent/b :ent/c] to be read in from a remote data source. An EntsList component will be displaying all the ents with their attributes. So, along with the attributes above we define :ent/all-ents. In the RAD tutorial, the all-* queries only get ids from the remote, though I'm not sure why we wouldn't get it all at once, like this:

(defattr all-ents :ent/all-ents :ref
  {::pc/output  [{:ent/all-ents [:ent/id :ent/a :ent/b :ent/c]}]
   ::pc/resolve (fn [env _]
                     (if-let [db (get-in env [:mydata/datasource :db])]
                       {:ent/all-ents (mydata/get-ents db)}
                       (log/error "Cannot find data source!"))))})
Either way, though: Key Question. What should the resolvers for :ent/a etc. look like? If I have all-ents like the above, I could do ::pc/input #{:ent/all-ents}, but then it can only return a list of :ent/a's, which is an output that is not :ent/a but something else, right? This doesn't seem quite right. If I don't use all-ents (or if all-ents has only ids), then where does the :ent/a resolver get its data? Something like this?
(defattr all-ents :ent/a :string
  {::pc/input #{:ent/id}
   ::pc/resolve (fn [env {:keys [ent/id]}]
                     (if-let [db (get-in env [:mydata/datasource :db])]
                       {:ent/a (:a (mydata/get-ent-by-id db id))}
                       (log/error "Cannot find data source!"))))})
But then I'm querying the remote data source for each attribute of each row, perhaps many thousands of queries for a simple report. Or do I use a ::pc/batch? true in the a,b,c attribute resolvers and only do a query when given non-sequential input? If so, what would the query need to look like to engage the batch case? On the graph-database side, like the examples in the fulcro book with the in-memory normalized database, the ideas are clear, but it's at the boundary with the remote that I'm fuzzy. Thanks for any insights or concrete examples you can offer!

Jakub Holý (HolyJak)19:01:38

I think batch? true is indeed a good solution. Pathom is smart to leverage it automatically so if you query for [{:all-entities [:prop1 :prop2..]}] it will make 2 queries - 1 to fetch the ids, 1 to fetch all the "bodies". My all-* resolvers for SQL fetch the full data, not just idents. A system with separate resolver for ident lists and (batched) for ident -> props is simpler and more flexible. You can leverage the same ID -> some props resolver at a number of contexts instead of writing more complicated resolvers for each. It is your call to make, simplicity vs performance.

Jakub Holý (HolyJak)19:01:22

Not sure whether it is clear but Pathom is smart enough to find out which resolvers it has to call, ie it can combine a number of resolvers to answer a single query

Jakub Holý (HolyJak)19:01:13

Not sure what you mean by "resolver for ent/a". If you have an "all at once" all-ents resolver that returns all to props, not just idents, then you don't need it? Otherwise you will make something akin the auto-generated resolvers, taking :ent/id and returning all / the requested properties of the entity.

Christopher Genovese21:01:49

Thanks for the response. That's good to know about batch and the trade-offs between the ids only and the all at once. Regarding the resolver for ent/a, I had thought that pathom would choose for the resolver at the closest context so that it would be useful to be able to get smaller sets depending on whether all-ents was in the context. But I gather from what you said that if I have all-ents then it can figure out how to answer a query like [{[:ent/id id] [:ent/a]] automatically using all-ents. Is that right?

Christopher Genovese21:01:34

So to follow up, if the remote returns a data structure that looks like {:ent/all-ents [{:ent/id 1 :ent/a 1 :ent/b 1 :ent/c 1} {:ent/id 2 :ent/a 2 :ent/b 2 :ent/c 2} ...]} then that will be stored and normalized into the database automatically and any more specific query related to ents will use that client side automatically? So I need not define any other resolvers on the individual pieces because ents is read only. Do I have that right? Is the data structure properly formed in that case? If I do add a form, say, to write/update individual ents, do I then need additional resolvers or does it all still flow through all-ents?

Christopher Genovese21:01:44

Thanks again for your help

Jakub Holý (HolyJak)09:01:33

Sorr,y I am not sure I manage to follow your thinking. Thus I am not able to provide a good answer. You either define an all-in-one resolver like this

(pc/defresolver all-ents [env _]
  {::pc/input  #{}
   ::pc/output [{:all-ents [:ent-id :ent/a :ent/b :ent/c]}]}
  {:all-ents [{:ent-id "id" :ent/a 1, :ent/b 2, :ent/c 3} #_...]})
or you define it to only return IDs and define another one to resolve the properties based on the ID:
(pc/defresolver all-ents [env _]
  {::pc/input  #{}
   ::pc/output [{:all-ents [:ent/id]}]}
  {:all-ents [{:ent/id "id1"} {:ent/id "id2"} #_...]})

(pc/defresolver ent-id [env inputs]
  {::pc/input  #{:ent/id}
   ::pc/output [:ent/a :ent/b :ent/c]
   ::pc/transform pc/transform-batch-resolver}
    (fn [{id :ent/id}] {:ent/id id :ent/a 1, :ent/b 2, :ent/c 3})
Or leverage RAD to generate the id -> props resolver for you, if the data is stored in Datomic/SQL:
(pc/defresolver all-ents [env _] ..)
(defattr id :ent/id :string
  {ao/identity? true
   ao/schema    :production})
(defattr a :ent/a :int
  {ao/identities #{:ent/id}
   ao/schema    :production})
(defattr b :ent/b :int ...)
(defattr c :ent/c :int ...)

Jakub Holý (HolyJak)09:01:24

> So to follow up, if the remote returns a data structure that looks like `{:ent/all-ents [{:ent/id 1 :ent/a 1 :ent/b 1 :ent/c 1} {:ent/id 2 :ent/a 2 :ent/b 2 :ent/c 2} ...]}` then that will be stored and normalized into the database automatically and any more specific query related to ents will use that client side automatically? Yes, provided you have defined components with correct idents, i.e. the query is something like [{:ent/all-ents (comp/get-query Entity)}] where Entity has :ident :ent/id . Given that and having loaded the data above, you will get :entity/id "table" in the client DB and be able to resolve any request for a particular entity from there. Of course if you manually issue (df/load! app [:entity/id 123] Entity) then this would require a resolver able to resolve it. But if you never issue a load like that, you are good.

Christopher Genovese18:01:19

P.S. For the case with the all-ents resolver above that gets everything at once, I suppose one could do

(pc/defresolver ents-resolver [env {:ent/keys [id all-ents]}]
  {::pc/input  #{:ent/all-ents :ent/id}
   ::pc/output [:ent/id :ent/c :ent/b :ent/c]}
  (some #(and (= id (:id %)) %) all-ents))
in addition to the resolver that queries for an individual [:ent/id :ent/a], say. In that case, once all-ents is available, will pathom always choose ents-resolver versus the individual attribute resolvers? Is that a reasonable approach?

Jakub Holý (HolyJak)19:01:16

I think this doesn't make sense,at least to me. Just make a single #{:ent/id} -> [:ent/a...] resolver and make it batched. (Notice pathom has a helpful transform for that)

Christopher Genovese21:01:53

What transform is that? So if all-ents exists, will pathom draw from that (sequential case) and do queries otherwise (scalar case)? This is getting close to the center of my confusion. Thanks!

Jakub Holý (HolyJak)22:01:57 Not ure what all-ents is in this context. If, within the processing of a single query a resolver returns a sequence of idents (inside a map, of course) and the query needs some properties other than id from those entities and there is a resolver from id -> those props, it will be called for each ident. If it is batched, it will only be called once with all the IDs. See pathom docs about batching. If still unclear, you might get a. Better answer in #pathom

Christopher Genovese23:01:05

Thanks for your patience and sorry to belabor this, but what you just said hits on the one key question if you don't mind. The question is what the resolver(s?) for id -> other props looks like specifically in this context?? Suppose that I have the the following like you said:

(defattr all-ents :ent/all-ents :string
  {::pc/output [{:ent/all-ents [:ent/id]}]
   ::pc/resolve (fn [env _]
                     (if-let [db (get-in env [:mydata/datasource :db])]
                       {:ent/all-ents (mydata/get-ent-ids db)}  ;; returns vector of ent/id's
                       (log/error "Cannot find data source!"))))})
Then do I have a resolver for each other prop like this
(defattr a :ent/a :string
  {::pc/input #{:ent/id}
   ::pc/output [:end/id :ent/a]
   ::pc/batch? true
   ::pc/resolve (fn [env input]
                     (if-let [db (get-in env [:mydata/datasource :db])]
                           (if (sequential? input)
                             (mydata/get-ents-in-id-set db (mapv :id input))
                             (mydata/get-ent-by-id db id)) 
                           (log/error "Cannot find data source!"))))})
or one intermediate ent/id -> [all props]? I think that's the piece I'm missing. In any case, thanks for bearing with me. (I had seen the batch transform, thanks. And I will check out #pathom too.)

Jakub Holý (HolyJak)22:01:06

You can do either. I would do the ent/id to all props, or let RAD auto-generate it from the attributes.

Jakub Holý (HolyJak)22:01:11

Notice the resolver can be smart, look at the actual query and only select the attributes requested from the DB. In pseudo code: (str "select " (map column-name (-> env :query)) " from someTable")

Jakub Holý (HolyJak)19:01:52 is alive! And awesome-fulcro has been moved there. Anyone who should be made a collaborator?

❤️ 39
👏 12

Moving the demos repo under the name FulcroDemos . Intent is to have 2 primary branches - first for all workspaces (demo state, form state, merge, etc.), and second a string of branches [tags?] progressively building up a full app with a remote as simply as possible. LMK if you can think of a better name. I plan to dump a bunch of my initial example code into the repo and write a blog post referencing it.

❤️ 3