This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-11-11
Channels
- # asami (19)
- # babashka (41)
- # beginners (115)
- # biff (7)
- # calva (78)
- # clj-kondo (29)
- # cljs-dev (9)
- # clojure (39)
- # clojure-europe (17)
- # clojure-gamedev (29)
- # clojure-nl (1)
- # clojure-norway (9)
- # clojure-spec (2)
- # clojure-uk (3)
- # clojurescript (7)
- # core-async (26)
- # cursive (16)
- # datomic (13)
- # emacs (1)
- # events (5)
- # fulcro (2)
- # funcool (4)
- # gratitude (1)
- # helix (1)
- # holy-lambda (1)
- # humbleui (1)
- # introduce-yourself (4)
- # java (1)
- # jobs (2)
- # jobs-discuss (9)
- # lsp (28)
- # matcher-combinators (2)
- # mathematics (1)
- # membrane (1)
- # nbb (12)
- # off-topic (10)
- # pathom (52)
- # polylith (38)
- # portal (32)
- # re-frame (4)
- # reagent (16)
- # reitit (2)
- # remote-jobs (1)
- # reveal (1)
- # rewrite-clj (10)
- # sci (67)
- # shadow-cljs (45)
- # squint (1)
- # tools-build (13)
- # tools-deps (16)
hey all - I have a resolver that describes a deeply nested output structure but I’m unable to join against the keys in that nested output here’s the gist: https://gist.github.com/davidyang/b492acf0024f480139eadf79bc5e214a
Did you specify ::pc/output
using nested output? Ex: {:my/output [:nested/a :nested/b]}
hey @U2U78HT5G - yes I specified the ::pco/output
option
hmmm, this seems to work correctly:
(p.eql/process env [{[:user/id 1] [:user/id
:user/full-name
{'(:user/feed {:start-date 1 :end-date 2})
[{:feed/day
[:feed/date
{:feed/user-days
[:feed/food :user/id :user/full-name]}]}]
}]}])
to have them at the same level of nesting - just the joining doesn’t seem to work
but this doesnt:
(p.eql/process env [{[:user/id 1] [:user/id
:user/full-name
{'(:user/feed {:start-date 1 :end-date 2})
[{:feed/day
[:feed/date
{:feed/user-days
[:feed/food {:user/id [:user/full-name]}]}]}]
}]}])
The problem with your second query is that you're asking for :user/full-name
underneath :user/id
which doesn't make sense since :user/id
is a string.
If instead you made the response look like:
:feed/user-days [{:feed/user {:user/id 2}
:feed/date ....
You could then ask for [:feed/food {:feed/user [:user/id :user/full-name]}]
generally it's better to keep pathom data flatter as it gives your resolvers more input data they can use
thanks for the help - I don’t understand what you mean when you say that :user/id is a string?
my mental model is that the returned attributes can be converted to their references, but it sounds like they need to return idents (the map of {:user/id 2})
so if I have an ident as the value of the map then I can join on that
and when I ask for :user/full-name it seems to be smart enough to use the user/id at it’s sibling level or I guess whatever is lower in the tree
so pathom doesn’t go up the tree looking for idents, it needs it at the sibling level
The only reason you would include :feed/user {:user/id 1}
is if you needed the user attributes to be in an isolated key
For example, the response to the query would be:
:feed/food "food"
:feed/user {:user/id 1
:user/full-name "Rich"}
but if you don't need the user attributes to be in their own key, it's preferable to have all the attributes at the same level which means your response would look like:
:feed/food "food"
:user/full-name "Rich"
ok - so just rely on pathom’s smart resolver to look for a sibling when I join on new attributes
correct, in the example you posted you had :user/id
as a sibling to :feed/food
which is why you could also for :user/full-name
at the same level.
so I join on user/full-name at that level it’ll look for the :user/id at that level (that level only right?) and resolve it that way
that’s very cool
yep! and if it already has that resolver response cached it won't need to do another lookup (say that resolver required a database lookup)
but that means that if I need a date parameter I need to pass it down the tree - I can’t rely on it to find the attribute from higher in tree
Potentially, if you're able to keep your resolvers pretty flat you might not need to pass the data down. It also might make more sense as input depending on how it's being used
is there a performant way to break out these nested resolvers? maybe but a look-through cache in env
so that when the nested resolvers are called they can get the data from the cache instead of having to fetch it again from DB? Can I separate out the resolvers from the data nesting itself without multiplying the calls out?
There are ways to do that. One thing I've done before is have a top-level resolver that does one query to get all needed data and then hydrates pathom's resolver cache so that downstream resolver calls will already have the data accessible. Pathom has an optional parallel parser that will parallelize all the requests it can, which might be enough for what you need.
interesting - so if I do this big resolve it’ll normalize it for subsequent queries?
You could (doesn't do that out of the box). I was able to get it working by creating one resolver that would interface with my db and then I just had to normalize to that db resolver
There's also batch resolvers (https://pathom3.wsscode.com/docs/resolvers/#batch-resolvers)
I guess to clarify - if I have a query that returns a deeply nested structure, but I define only the top level outputs, if I have separate resolvers that access that nested structure pathom could figure it out?
Or are you saying I would have that top level resolver put it’s substructure into the resolver cache somehow
If you return a deeply nested structure, downstream resolvers would be able to access all the nested data. Every "nested level" is a new context though which means attributes above or below wouldn't be accessible (just that level itself)
but your query would also need to closely match the response (which takes away from the value of pathom)
and I’d have to define the output with all the nesting correct?
OK - that makes sense to me - the batching is cooler than I first understood from reading, it sounds like it aggregates many places in the tree (so it can combine multiple queries for comments and then I can group those into one SQL query)
for some reason I thought it was just passing in a sequence of items from one level to another - not all the requests from the different parts of the EQL tree
OK - lots to experiment with. thanks so much for your help! very much appreciate your time/insight
@wilkerlucio From my own experience and perusing the slack history, GitHub issues and documentation, Pathom3 seems to stuggle with solving cyclic problems among others like resolving back up a nested datastructure (maybe also cyclic?). I lack the experience, but could you provide any insight into this? Is there something fundamental wrong with trying to solve these cyclic problems in this paradigm? Perhaps it’s impossible? Or maybe it’s a design choice to avoid solving problems this way? For example Clojure struggles with cyclic dependency resolution and community consensus is that good design shouldn’t need cyclic dependencies. (Apologies for no references). If none of those, maybe this functionality just isn’t here yet? I’m eager to dive into these interesting problems but with my lack of experience in Pathom, I just hope I’m not wasting my time solving the unsolvable. I could put together some use-cases if anything needs clarity.
hello @U3XCG2GBZ, I love if you can provide some use cases so I have a better understanding of the problem
in general, Pathom has cycle detection and that will case a failure in planning, since there is no way to resolve the cycle
@wilkerlucio Thanks for getting back to me on this one. https://github.com/wilkerlucio/pathom3/issues/168 I am running into a stack overflow with the example in that issue. If the cycle detection was working I believe it would log a warning and stop the planner from cycling here. Is there something fundamentally wrong with the approach here in the example? Perhaps the cyclic warning log is a clue of that?
I started looking at it yesterday, I was able make your example work, but it also made a test fail, so Im trying to get all working, its in progress ;)
Oh thanks. Would you mind pushing your WIP changes? I would like to take a look if I could.
Hey - I’m planning on running the results of my EQL queries through normalization - but if I use put references to :user/id in other objects then the normalizer will percieve that as a user object. Is the solution to put something like :feed/user-id in that object and then put an alias in the resolver chain?