Clojurians Log v2

hi @smniel, thanks for all the digging so far, and again sorry my delay. first, are you still having the issue? its being a long time already, but if so, have you tried unbatching the resolvers to see if the problem is related to batching?

I have been focusing on some other things, so I haven’t had to worry about it, but I will be back on it soonish, like in the next week or so.

I haven’t tried to unbatch it. The resolvers are dynamically generated to query a sql database, so it won’t be too straightforward, but I can try and make sometime to try it this week.

I feel pretty confident that the issue has to do with combination of returning nested attributes from inside of a batch, but I could not create a simple example to show this.

yeah, its a tricky place (nested + batching), which makes it super cool if we can unveil another edge case to be fixed on it

hi folks, I was on vacation, sorry the absense, will have a look in the thread later today

can you share what the query and any inputs look like? or at least the relevant parts of them.

Yeah, I would love to. This is the query query and the data, very simple, but the resolvers are not necessarily.

(p.eql/process
  indexes
  {:account/id ""}
  [{:account/field-instances
    [:field-instance/rich-value]}]))

account-field-instances
{::pco/input [:account/id]
 ::pco/output [{:account/field-instances
                [:field-instance/id
                 {:field-instance/field [:field/id]}]}]
 ::pco/batch? true}

field-instance-attributes
{::pco/input [:field-instance/id]
 ::pco/output [:field-instance/id
               :field-instance/created-at
               :field-instance/updated-at
               :field-instance/value
               {:field-instance/auto-id [:field-instance-auto-id/id]}
               {:field-instance/field [:field/id]}
               {:field-instance/form-instance [:form-instance/id]}]
 ::pco/batch? true}

field-instance-files
{::pco/input [:field-instance/id]
 ::pco/output [{:field-instance/files
                [:file/id
                 :file/created-at
                 :file/updated-at
                 :file/name
                 :file/mime-type
                 :file/key
                 :file/size
                 {:file/form-instance [:form-instance/id]}
                 {:file/field-instance [:field-instance/id]}]
 ::pco/batch?}



field-attributes
{::pco/input [:field/id]
 ::pco/output [:field/id
               :field/created-at
               :field/updated-at
               :field/deleted-at
               {:field/deleted-by [:user/id]}
               :field/label
               :field/alias-name
               :field/hint
               :field/icon
               :field/type
               :field/sort-order
               {:field/parent [:field/id]}
               :field/max-length
               :field/required
               :field/setting-auto-id]
 ::pco/batch? true}

field-instance-rich-value
{::pco/input [:field-instance/id
              :field-instance/value
              {:field-instance/auto-id
               [:field-instance-auto-id/id]}
              {:field-instance/files
               [:file/id
                :file/key
                :file/name
                :file/mime-type]}
              {:field-instance/field
               [:field/type
                :field/setting-auto-id]}]
 ::pco/output [:field-instance/rich-value]}

I can run the following query and it works as expected.

(p.eql/process
  indexes
  {:field-instance/id "<some-uuid>"}
  [:field-instance/rich-value])

But when I add the account layer like below it returns the error in the original message.

(p.eql/process
  indexes
  {:account/id ""}
  [{:account/field-instances
    [:field-instance/rich-value]}])

what about if you do (p.eql/process indexes {:account/id "etc"} [{:account/field-instances [:field-instance/id]}])?

That works as expected.

okay. I'd try the same shape with each of the top-level inputs to field-instance-rich-value to make they're all reachable.

They are all reachable. I can run (p.eql/process indexes {:field-instance/id "etc"} [:field-instance/rich-value]) and it returns as expected.

sure, but I meant inside the [{:account/field-instances [...]}] box, since that's the part that's not working for the rich-value.

Gotcha. I can give that a try.

Yeah, that also works as expected.

what about selecting [{:account/field-instances [:field-instance/id :field-instance/rich-value]}]?

I edited part of the longer message, realizing that field-instance-files had all of the file attributes as output and the file-attributes resolver wasn’t being used.

Let me double check.

Same error as without the field instance id.

the next thing to check is copying the ::pco/input of field-instance-rich-value into that ... box [{:account/field-instances [...]}], ie. pulling all the inputs.

Yeah, I have done that and it fixes the problem. I didn’t even need to pull all of it in, just the two field attributes, type and setting-auto-id. I would prefer to figure out why pathom is unable to resolve it on it’s own though.

I agree, it was just a test to see if they worked together and there wasn't some lurking conflict.

I think I'm out of ideas, though 😞

So if I change the output of account-field-instances to the following it works:

[{:account/field-instances
  [:field-instance/id
   {:field-instance/field
    [:field/id
     :field/type
     :field/setting-auto-id]}]}]

I appreciate you looking at it.

oh, that might make sense.

wait, no, that shouldn't be necessary. Pathom should for sure be able to resolve that with the other resolver.

Yeah, it even does for many queries.

usually when I've had trouble like this, it's because I was mixing inputs from different scopes on the same resolver. like if you had :account/id, :field-instance/id and :field/id as siblings in a ::pco/input. it doesn't really make sense - they're properties of different entities and you need to show their relationships using EQL joins (maps).

We don’t currently have any “mixed” entities with our resolvers. I have considered it, but given that you can only do that for one to one relationships, I didn’t like the inconsistency.

it's actually even stronger than that - it only works for unique 1-1 relationships. if the same key is used for two different relationships then they collide. definitely better to be explicit and use the joins, unless it's really more like "this is one entity semantically but it's stored as two"

We also have instances where we have multiple one to one relationships from one entity to an entity of the same type. It seemed that it could get very confusing doing that.

I found the beginning of something helpful with a bunch of logging. I am logging the beginning of each pcr/run-graph-impl! using the wrap-run-graph! plugin. I went a put some other logging that wraps each run of a resolver, as well as each run of a batch. The full logs are here. https://gist.github.com/scizo/b1f68dd621a7a250e25ce1f7f69009bd It doesn’t seem to be a problem with the planner. It is correctly calling the needed resolver, but it happens in a batch. Later when the same node is run again to merge the result that was cached with the batch (at least I think that is how it might be working). The node result is nil. The interesting logs start on line 370 when the batch with the needed resolver gets run. The node is then rerun on line 400, but the entity already has all of the expected data. I am wondering if that is why the result comes back as nil. The nil result is probably a red herring. I am still combing through this trying to see if it can give me some hints about where to go next.

Then on line 427, the root node of the middle graph begins to run and it doesn’t have the data from the graph below merged in, even though it has run and been merged into the sub-graph entity.

I thinking because one batch is created before the other is run, the inputs are captured before some of the needed inputs are merged.

@wilkerlucio Any thoughts on this?