pathom

smniel 2025-04-03T19:49:23.490279Z

Hi Everybody! We got a little surprise when using pathom3 today. It appears that the planner merged nodes in a situation where an attribute is requested with two different sets of parameters on the same attribute. The parameters were merged overwriting a key in one. We made a https://gist.github.com/scizo/fed582cf2e215c4725093236d72b590f to demonstrate the problem for us. I am curious if this is expected behavior or if there is a way to not have nodes merged in these types of circumstances. The only difference in our example is that in one case the attribute has no params and in the other it has them. Thanks in advance for any input or guidance on if this is expected or if there are other things we should be considering or understanding!

đź‘€ 1
Thomas Moerman 2025-04-04T08:20:54.706749Z

Interesting, I didn't know you could use a :where in the input section, is this documented somewhere? Can't immediately find it. This appears to work correctly, although it's not an answer to your question.

(pco/defresolver company-problems [env {:company/keys [locations]}]
  {::pco/input [{:company/locations [:location/deleted
                                     :location/problems]}]
   ::pco/output [:company/problems]}
  {:company/problems
   (reduce
     (fn [acc {:location/keys [deleted problems]}]
       (cond-> acc (not deleted) (+ problems)))
     0 locations)})

burbma 2025-04-04T13:16:13.551439Z

I think it's that the :where is just something we chose to put in the params and our resolver can choose to look at the params and do what it wants with them. So it's not about documenting that :where can be in the input section, but rather that you can put whatever you want there and then act on it however you choose in your resolver.

smniel 2025-04-04T16:26:19.844099Z

Yes, the above suggestion of not using params and doing the filtering in the resolver that is aggregating avoids the problem. It doesn’t address the issue that if parameters are meant to effect what a resolver returns, then if two nodes of the same resolver with the same inputs but different parameters are merged, you are changing the output of one or both nodes. That isn’t an optimization, but a mutation of the computation.

smniel 2025-04-11T17:08:56.166119Z

Thanks for taking the time to look at it @wilkerlucio! If there is any way I can be helpful please let me know. I am happy to contribute time and effort to pathom.

🙏 1
Eric Dvorsak 2025-04-10T14:58:57.432369Z

I am also interested in the outcome here, params and caching issues in pathom3 make me nervous

wilkerlucio 2025-04-10T16:44:00.197769Z

hi folks, thanks for bringing it up and the example @smniel, I plan to have a look at it this weekend

smniel 2025-04-05T16:25:24.670399Z

I have been diving into the code more and it appears the run graph and the optimizer are not the problem as much as how the data is stored as the entity-tree is built up. It was pretty straightforward to update the requirements for nodes to be combined to include having the same parameters. However, the results of both calls to resolvers are merged into the entity tree with no indication of which parameters were used. In order to be able to avoid one being merged over the top of the other, the results would need to be stored in a way that separates results calculated with different parameters, and how to then populate resolver inputs and overall results with this being taken into account. I am thinking through how this could be done. @wilkerlucio Is this an idea that you are interested in entertaining in pathom?

wilkerlucio 2025-04-13T15:53:30.963349Z

@smniel hello, I finally got to look, and your analysis is on spot. but it does hit an important heuristic of pathom, that is: for each entity, every attribute can only be resolved once. what you suggesting would imply to resolve :company/locations twice for the same entity, and that's a no-go. this is done to help reducing complexity on both planner and runner. my suggestions is to make the location load an external function that you share across resolvers, instead of trying to share an attribute that might be resolved multiple times

wilkerlucio 2025-04-13T16:08:58.229729Z

that said, I think it as issue that it goes unrepported to the user, maybe we should fail during planning to ensure the user gets aware of the conflict

smniel 2025-04-03T19:50:30.819089Z

Edited to include correct link.