Fork me on GitHub

I have a body that can deliver recursive results. I do this by looking at (::pcn/expects (::pcn/node env)) and computing result accordingly. Is it possible to instruct a resolver to expect a recursive return value? I.e., {::pco/output [{:some/thing '...}]} The above doesn't work.


what's the goal here? can you give me an example?


Sure! So in this instance, I'm passing along queries to XTDB. XTDB is capable of resolving recursive queries itself (it even uses EQL to do so), so rather than issuing multiple requests to XTDB, I could pass along the one request, and receive the full response in one go. That is, it would be possible if I could tell that a recursive query has been requested.


if I remember right you can see that by looking into the ast, let me do a quick test here

🎉 1

I believe this is the fragment you are looking for:

(-> env :com.wsscode.pathom3.connect.planner/graph :com.wsscode.pathom3.connect.planner/source-ast :query)


if query is ... (the symbol), its a recursion going on


complete example:

(ns com.wsscode.pathom3.demos.recursive
    [ :as io]
    [com.wsscode.pathom3.connect.indexes :as pci]
    [com.wsscode.pathom3.connect.operation :as pco]
    [com.wsscode.pathom3.interface.eql :as p.eql])
  (:import ( File)))

(pco/defresolver file-resolver
  [{:file/keys [path]}]
  {::pco/output [:file/dir?]}
  (let [f    (io/file path)
        dir? (.isDirectory f)]
    {:file/dir? dir?}))

(pco/defresolver directory-files-resolver
  [env {:file/keys [path dir?]}]
  {::pco/output [{:dir/files [:file/path]}]}
  (tap> (-> env :com.wsscode.pathom3.connect.planner/graph :com.wsscode.pathom3.connect.planner/source-ast :query))
  (if dir?
     (mapv (fn [^File f0] {:file/path (.getPath f0)})
       (.listFiles (io/file path)))}))

(def env

  (p.eql/process env
    {:file/path "src/demos"}
     {:dir/files ...}]))


if that ends up being a bit unreliable, try using the graph -> index-ast -> the output property, that's for sure the right ast (but requires knowing the attr)


Thanks @wilkerlucio! I'll try it out.


The query indicates the recursion and then pathom will continue querying based on the results - the resolver should just return a way to map an id to an entity (which may contain more nested ids) This is from my notes on pathom2 (pretty sure I copied it from this slack someplace in the past) but should translate directly to p3:

(pc/defresolver file-resolver [env {:file/keys [path]}]
  {::pc/input  #{:file/path}
   ::pc/output [:file/type]}
  (let [f    ( path)
        dir? (.isDirectory f)]
    {:file/type (if dir? :dir :file)
     :file/dir? dir?}))

(pc/defresolver directory-files-resolver [env {:file/keys [path dir?]}]
  {::pc/input  #{:file/path :file/dir?}
   ::pc/output [{:dir/files [:file/path]}]}
  (if dir?
     (mapv (fn [^File f0] {:file/path (.getPath f0)})
       (.listFiles ( path)))}))

 (parser {} [{[:file/path "src"]
                {:dir/files '...}]}])

☝️ 1

Oh yes, I know how to use Pathom recursively. In this case, I have an IO target that is capable of resolving recursion, so I'm looking to optimize it down to one query to the resource, rather than multiple queries as would be issued by Pathom.


You can return a tree of data and pathom won't make the recursive resolver calls. I've used this with datalog dbs to issue one a recursive pull query instead of having pathom perform the recursion for a big perf gain


Yes, but I don't want to do a big recursive pull unless it's expected. That would produce a lot of overhead for when we don't want it, or know a specific depth we want in advance. Hence, I want to look at what the user expects, and adjust my pull accordingly. This works great with the expects map outside of recursion, but not when recursion is present. My hope was that there would be some way to inform the parser, statically, that we are capable of returning a recursive result, and that this would in turn deliver an expects, that might look something like:

{:some/thing ...}
Rather than,
{:some/thing {}}
This would be handy, since the expects structure can be trivially turned into a pull query structure. Perhaps it would give good static information to Pathom as well, that wouldn't be present if you return a recursive structure as a surprise (rather than a promise). @wilkerlucio has given me some things to check out here, however, so it's not a lost cause yet:


ah I see - you can simplify things and pass the query depth as a query parameter


I don't love that idea. The depth is already known, since it is passed in the query. If I require it as an additional parameter on top of the query, I'm burdening the user with technicalities that they shouldn't have to worry about.


Ok, I don't understand what problem you're trying to solve, but seems like you figured it out.


@U051V5LLP the idea is to keep the same interface working (the ...), the param idea can work but them its not using the eql feature that was designed for that specific purpose, so if we can keep it consistent its better, and that's what @U06B8J0AJ is going for


I see, so it's fundamentally about a resolver which might be used to resolve both a recursive query and a non-recursive query?


Yeah, since I happen to have some IO resources where I can leverage the recursive bit straight inside the resolver, it's an optimization opportunity. The overhead of Pathom becomes effectively nil for queries that stay within the same resource. If I don't do this, then you could argue that calling XTDB directly is more efficient, since that would be 1 call instead of the N calls that Pathom generates for the same pull expression. Now I'll be able to wave aside the naysayers. 🙂


@wilkerlucio, I found this talk the other day and was struck by the similarity to Pathom. While the implementation is not Clojure, it has similar motivations for the project.

👀 1

yup, conceptually is the same idea, what I've been saying all this time is that by using an attribute driven approach we get a graph system that's much easier to extend over time, without having to keep messing with schemas


Right, exactly. And Pathom is more demand-driven as opposed to explicitly stating which relationships should be traversed in which order, which this seems to do. Anyway, I just found it interesting as a data point that validates both the problem you’re trying to solve as well as the general solution (a graph of data over all the DBs).

👍 1