are there any resources on how to debug data not getting normalized properly post fetch from a remote? pesky return from a union query isn’t getting normalized and I’d like to figure out why. looking for something a little bit lower level than tree->db if that exists

fwiw, the components I’m working with look something like the following:

(defsc Assertion [this props]
  {:query [:db/id
           :assertion/text]
   :ident [:assertions/by-id :db/id]})


(defsc Event [this props]
  {:query [:db/id
           :event/text]
   :ident [:events/by-id :db/id]})

(defsc TestStep [this props]
  {:query (fn []
            {:assertions/by-id (prim/get-query Assertion)
             :events/by-id     (prim/get-query Event)})
   :ident (fn []
            (cond
              (and (contains? props :event/text)
                   (not= (:event/text props) ::prim/not-found))
              [:events/by-id (:db/id props)]
              (and (contains? props :assertion/text)
                   (not= (:assertion/text props) ::prim/not-found))
              [:assertions/by-id (:db/id props)]))})

(defsc Test [this props]
  {:query [:db/id
           :test/name
           {:test/steps (prim/get-query TestStep)}]
   :ident [:test/by-id :db/id]})

[{:db/id            1
                 :test-suite/name  "My Test Suite"
                 :test-suite/tests [{:db/id      2
                                     :test/name  "First test"
                                     :test-steps [{:db/id                     3
                                                   :assertion/text "assertion text"}
                                                  {:db/id                 4
                                                   :event/text "event text"}]}]}])

I’m guessing it has something to do with the TestStep and Event/`Assertion` components declaring functionally the same idents. I don’t know if that’s actually an issue though

@mss idents on all components used in the union. Query and data must match. Other than that there isn’t much to it. I realize it is a little heady because of the alternation…but if you just look at it carefully you’ll see the mismatch

The merge system is very well-tested. There’s nothing lower level to see

You should test your ident function against real data for one

it looks reasonable to me, but you have not show the actual load you’re using compare to the data being returned, so I can be of no further help. Unless you mean to claim that Test is the proper thing to query that with

because it doesn’t match

:test-suite != :test

so your join on Test cannot possibly match (or normalize) that data

you’re right, sorry. was trying to reduce it to the simplest possible branch of my state tree and accidentally left the test-suite part in

what is your load?

if I run

(prim/tree->db Test
               {:db/id      2
                :test/name  "First test"
                :test-steps [{:db/id          3
                              :assertion/text "assertion text"}
                             {:db/id      4
                              :event/text "event text"}]})

{:db/id 2,
 :test/name "First test",
 :test-steps
 [{:db/id 3, :assertion/text "assertion text"}
  {:db/id 4, :event/text "event text"}]}

nope

well, hold on

where I would expect to see something like:

{:db/id 2,
 :test/name "First test",
 :test-steps
 [[assertions/by-id 3]
  [events/by-id 4]]}

and the prop names actually match in your real code?

show me (prim/get-query Test)

[:db/id
 :test/name
 {:test/steps
  {:assertions/by-id [:db/id :assertion/text],
   :events/by-id [:db/id :event/text]}}]

test-steps vs test/steps

😵😵😵😵 my goodness. makes sense

also good to know that next time I see something super weird like this it’s almost certainly a typo

It’s funny…I have the same kinds of trouble sometimes trusting the mechanisms internals…but it is always that simple in my experience.

if you get something you don’t expect, you screwed up a query or ident or data property name. Nothing more to it 🙂

but since they all have to match perfectly, it isn’t any harder to debug than careful reading

yep makes total sense. really appreciate the work you put into this, and thanks for the help

sure. If you think of an idea for giving people warning on this efficiently, glad to hear it 🙂

I guess we could put in some kind of dev time warning during db->tree, but it would likely be too noisy to be useful

If we had schema that said what was required that might help

but that all adds a lot of overhead

I guess a warning in the direction of “your data contains something not in the query” would be best we could do

yeah exactly. I wonder if there’s some way to have a should-be-there type schema (the req’d keys) fall out of a query and only pay that overhead in a dev env. happy to poke around and try something if you have any ideas about the easiest way to do that

I think it can be done without schema

basically just a db->tree that complains if it sees something at the “current” query level in the data that isn’t queried for would probably be sufficient

rather tree->db

Could make two versions: one for dev mode

If you don’t query for it, you don’t get it back. If you query for it, you might not get it back. But if you get something back, you should have definitely queried for it.

and that check is easy to implement and would catch this specific error

tree->db should probably also filter the result with the query, so stuff would disappear completely

that would be a better indicator as well

seeing data non-normalized fools you into thinking you got it part-way right

when in fact it doesn’t match the query

I you wanted to copy the code for tree->db and make a checked-db->tree ppl could use that (less efficient) one to debug, and we could also test it realtively easily since there is a reference implementation that is known to work properly.

absolutely, that makes sense. will take a crack at it over the weekend

Updated documentation to include better detail on additional client options: http://book.fulcrologic.com/#_useful_reconciler_options

this also discusses shared state, which can be useful in some common circumstances.

hi all! I’m trying to reconcile using df/load-action or any other *-action and avoiding circular dependencies. df/load-action lives in the mutations namespace, but to use it, I either have to require some namespace with a component Foo, or I have to declare a new component Foo within the mutations namespace. The latter seems wasteful, but the former will inevitably result in a circular dependency. What am I missing?

The thing I did to get round this was to put components into state on application startup load.

In the :started-callback.

I have a mutation (only called once, in :started-callback) that does this: (swap! state assoc :root/components components).

The components mutation parameter might look like: {:organisation ~bookkeeping/Organisation}

Then whenever a component is needed: (get-in st [:root/components :organisation])

Another possibility is to pass in the components you need as parameters. Even post-mutations take parameters. I think they didn't in the past, which is why I resorted to putting components in state. But having them in state (put there only at application startup) kind of works as components are static things.

@cjmurphy not a good idea to put components in app state. They are not serializable.

But they don't ever go over the wire where I use them - just put them in a link.

until you decide you'd like to use history for something like a support viewer.

Right.

so, couple of thoughts:

But no one would care about history for those components which are in a link. Just saying 🙂

Right I get it - the state would be no good in the target environment - but if started app from beginning they would be overwritten.

@levitanong Mutations are just symbols. You don't need to require them in your UI

there is no need for circular referencing..they'll work. The only thing you lose is ns aliasing

@cjmurphy If they are there at all, then serialization will crash

The other thing you could do that is OK is use :shared

that I documented last night

oh right, crash 😝

but ultimately, calling a mutation does not require you to require their ns in the UI. Ever. The ns aliasing is just a convenience.

Just became aware of Fulcro's book release! http://book.fulcrologic.com Congrats @tony.kay, and a huge thank you for open sourcing your inspiring work!