Fork me on GitHub
#pathom
<
2023-01-25
>
Eric Dvorsak08:01:18

I found two issues related to nested outputs in the regular parser, I'll work on some repro: • if a query asks for nested attributes, the result might return without them (I assume it's because they are already resolved by another but not listed as outputs of the last resolver which drops them), this happens even when lenient mode is false without a warning, if the last resolvers lists those nested attributes as output then they appear in the result • a batch resolver for a nested attribute in a collection only receives one input These issues don't appear with the parallel parser.

Eric Dvorsak08:01:15

In the meatime this is the query I was working with for the second point, where user/latest-data-privacy was returning 2 items, but organization/latest-data-privacy was returning empty for the first, and I could see in the inputs of this resolver that it was only receiving the second one: (pathom/parser {:current-user/id 5} [{[:user/id 16] [{:user/latest-data-privacy [:data-privacy/organization-id {:organization/latest-data-privacy [:data-privacy/id]} :data-privacy/id :data-privacy/version]}]}])

Eric Dvorsak08:01:09

for the first point I had this query: (pathom/parser {:current-user/id 5} [{[:organization/id 2] [{:organization/latest-data-privacy [:data-privacy/organization-id :data-privacy/id]}]}]) where data-privacy/organization-id will not be in the result organization/latest-data-privacy was not listing data-privacy/organizatio-id as output but the resolver that provided its input did

caleb.macdonaldblack17:01:47

From what I can tell. If two resolvers output the same top level key, Pathom3 will treat them as equal regardless of nested params.

caleb.macdonaldblack17:01:32

In my graph design, I’m particularly careful when modeling relationships. I’ll have resolvers with nested inputs/outputs return an ID attribute only. And the other resolvers that return data for that id.

Eric Dvorsak17:01:33

They don't output the same top-level keys, one resolver gets the inputs that the latest-data-privacy wants as well as some attributes requested by the query, normally they should be in the response but the regular parser drops them. The parallel parser outputs them as expected

Eric Dvorsak18:01:49

What you suggest to do is producing the case with the bug (the nested output is only the Id)

caleb.macdonaldblack18:01:21

Ah ok. Not sure then. Interested to see your example.

Eric Dvorsak18:01:16

This would be the repo for the first point

(ns com.wsscode.pathom3.test
  (:require  [clojure.test :as t]
             [com.wsscode.pathom3.format.shape-descriptor :as pfsd]
             [com.wsscode.pathom3.connect.operation :as pco]
             [com.wsscode.pathom3.connect.indexes :as pci]
             [com.wsscode.pathom3.connect.planner :as pcp]
             [edn-query-language.core :as eql]
             [com.wsscode.pathom3.interface.eql :as p.eql]
             [com.wsscode.pathom3.interface.async.eql :as p.a.eql]))

(pco/defresolver toy
  [env input]
  {::pco/input  [:toy/id]
   ::pco/batch? true
   ::pco/output [:toy/name :toy/id :toy/order]}

  (map #(get {1 {:toy/name "Bobby" :toy/id 1 :toy/order 3}
              2 {:toy/name "Alice" :toy/id 2 :toy/order 2}
              3 {:toy/name "Rene" :toy/id 3 :toy/order 1}}
             (:toy/id %))
       input))

(pco/defresolver child-toys
  [env input]
  {::pco/input  [:child/id]
   ::pco/batch? true
   ::pco/output [{:child/toys [:toy/id]}]}

  [{:child/toys [{:toy/id 1}
                 {:toy/id 2}
                 {:toy/id 3}]}])

(pco/defresolver favorite-toy
  [env input]
  {::pco/input  [{:child/toys [:toy/id :toy/order]}]
   ::pco/batch? true
   ::pco/output [{:child/favorite-toy [:toy/id]}]}

  [{:child/favorite-toy (->> (:child/toys (first input))
                             (sort-by :toy/order)
                             first)}])



(def env (-> (pci/register
              [toy
               child-toys
               favorite-toy])))

(defn test-case []
  (let [query [{[:child/id 1] [{:child/favorite-toy [:toy/name]}]}]]
    (println "Regular")
    (println (p.eql/process env query))
    (println "===========")
    (println "Parallel")
    (println @(p.a.eql/process (assoc env ::p.a.eql/parallel? true) query))))

Eric Dvorsak18:01:57

Regular
{[:child/id 1] #:child{:favorite-toy {}}}
===========
Parallel
{[:child/id 1] #:child{:favorite-toy #:toy{:name Rene}}}
This is the output

wilkerlucio13:01:57

thanks for the additional repro on that @U03K8V573EC, this is the next thing I'm planning to work on Pathom 3

wilkerlucio13:01:16

and I think they have a good change of being the same issue (#173 & #177)

Jakub Holý (HolyJak)21:01:04

In P3, how do I make a similar index-explorer defresolver as I did in P2? In P2 I have

(pco/defresolver index-explorer [{::pco/keys [indexes]} _] ; FIXME for P3
  {::pco/input  #{:com.wsscode.pathom.viz.index-explorer/id}
   ::pco/output [:com.wsscode.pathom.viz.index-explorer/index]}
  {:com.wsscode.pathom.viz.index-explorer/index
   (p/transduce-maps
     (remove (comp #{::pc/resolve ::pc/mutate} key))
     indexes)})
I search the docs but found just https://pathom3.wsscode.com/docs/indexes which wasn’t helpful in this regard. Then I found https://pathom3.wsscode.com/docs/debugging/#index-explorer which reads > To learn about this, please https://blog.wsscode.com/pathom/v2/pathom/2.2.0/connect/exploration.html. The behavior of them in Pathom 3 is the same as in Pathom 2. but I am not sure where to find p/transduce-maps or what to replace it with 🙏

2
wilkerlucio13:01:01

the recommend way to make that in Pathom 3 is using the boundary interface: https://pathom3.wsscode.com/docs/eql#boundary-interface

Jakub Holý (HolyJak)13:01:02

I see! And since Fulcro already uses it then I do not need to do anything for index explorer to work 🎉 Thx a lot!

Jakub Holý (HolyJak)22:01:11

Hm, this does not seem to work. Fulcro uses the boundary interface but my in-browser Fulcro Inspect’s Index Explorer tells me > Seems like the index is not available.

wilkerlucio22:01:39

that's a different problem, Fulcro inspect is not updated for Pathom 3 =/

wilkerlucio22:01:24

to get that working we need to update Pathom Viz in Fulcro Inspect, the catch is that Pathom Viz has migrated to Fulcro 3, while Fulcro Inspect didn't

wilkerlucio22:01:07

so we need to upgrade Fulcro Inspect to Fulcro 3 first, and then update Pathom Viz, or do some weird glue to connect Fulcro 2 and 3 there, but this last one seems a bad option IMO

wilkerlucio22:01:05

what you can do now is to use Pathom Viz stand alone app, that properly handles Pathom 3, a bit annoying to have one extra thing, but works

Jakub Holý (HolyJak)22:01:36

I see, thank you! Trying it out now.

Jakub Holý (HolyJak)22:01:51

How does Viz see my Fulcro app? Or rather it does not see it, in my case. so how do I connect it? It happily accepted http://localhost:3009/api as the connection url but that doesn’t really seem to be working as no resolvers are shown 🙏

wilkerlucio22:01:46

it is probably hitting it just fine, but its asking for the Pathom 2 keyword for indexes, and the Pathom 3 doesn't respond to that key

wilkerlucio22:01:54

I guess we may be able to do a dirty fix

wilkerlucio22:01:06

but adding a new resolver ,to expose Pathom 2 key in Pathom 2 index format

Jakub Holý (HolyJak)22:01:10

ah, maybe it does work, I see errors like > ava.lang.RuntimeException: java.lang.Exception: Not supported: class com.fulcrologic.rad.resolvers_common$secure_resolver$fn__39477 > at com.cognitect.transit.impl.WriterFactory$1.write(WriterFactory.java:65)

Jakub Holý (HolyJak)22:01:32

so I guess I need to clean up the resolvers before sending them over, as I remember we needed to do with P2

wilkerlucio22:01:19

there are some key things to do in this scenario: 1. a resolver with the input of com.wsscode.pathom3.connect.indexes/indexes and output of :com.wsscode.pathom.connect/indexes 2. do a similar thing for all the namespaces for the indexes (from ::pci from p3 to ::pc from p2 3. update each resolver and mutation so the keys use pathom 2 names 4. convert input values from EQL to flat sets

wilkerlucio22:01:39

I think I already have this code on Pathom Viz, let me check

Jakub Holý (HolyJak)07:01:06

Thank you, I will try that. However I suspect I have the same problem in both cases - I need to “clean” the indices to remove stuff that cannot be transferred over transit, namely (resolver) functions. How do you do that? Thats how we fix it for P2: https://github.com/fulcrologic/fulcro-rad-demo/blob/c09d1c30482c63ca2b468c9a7789dd5e80604a50/src/sql/com/example/components/parser.clj#L29-L31

Jakub Holý (HolyJak)07:01:29

For integration approach #2, I have added com.wsscode.pathom.viz.ws-connector.pathom3.adapter/env as the starting env of my parser. I assumed that it exposes indices in fulcro inspect compatible way but it does not seem to be fully the case, as it doesn’t produce the :com.wsscode.pathom.viz.index-explorer/index resolver it is looking for. I see it declares the output of :com.wsscode.pathom.connect/indexes , which is not the same.

Jakub Holý (HolyJak)09:01:34

I need your advice 😭 I am trying to get the external Pathom Viz working and thus to remove from indexes stuff that cannot be transferred over transit. I have fixed that but now P.V. does not like the response 😭 First, my code to ensure the data is transit-able:

(defn protect-attributes-wrapper [mse]
  (fn [env source {:keys [key] :as ast}]
    (if (#{:com.wsscode.pathom.connect/indexes
           ::pci/indexes
           ::pci/index-attributes
           ::pci/index-io
           ::pci/index-resolvers
           ::pci/transient-attrs
           :com.wsscode.pathom3.connect.runner/attribute-errors
           :pathom.viz/support-boundary-interface?}
         key)
      (walk/prewalk (fn [form] (if (fn? form) nil form)) source)
      (mse env source ast))))

; create the plugin
(p.plugin/defplugin clean-indices-plugin {:com.wsscode.pathom3.format.eql/wrap-map-select-entry protect-attributes-wrapper})
and use the plugin with the parser. Now the dev tools in P.V. show: > The result-action mutation handler for mutation com.wsscode.pathom.viz.index-explorer/load-indexes* threw an exception. > #error {:message “Resolver com.wsscode.pathom3.connect.indexes->com.wsscode.pathom.connect/index-resolvers--attr-transform exception at path []: No protocol method IOperation.-operation-config defined for type cljs.core/PersistentArrayMap: > {:config {::pco/input [], ::pco/provides {:p/method {}}, ::pco/output [:p/method], ::pco/op-name :p/method-resolver, ::pco/requires {}}, :resolve nil}” So it seems to fail b/c my cleanup code replaced :resolve value, which was a function and thus not transit-able, with nil . So I am in a Catch 22 😞

wilkerlucio10:01:31

@U0522TWDA to use the Pathom Viz I suggest you use via the pathom viz connector: https://github.com/wilkerlucio/pathom-viz-connector/#connecting-env-pathom-3

wilkerlucio10:01:44

using from it will make sure all the setup/encoding stuff is properly handled

wilkerlucio10:01:11

so you dont have to any transit or anything

wilkerlucio10:01:31

but for your server comms anyway, I suggest you can use transito: https://github.com/wilkerlucio/transito

wilkerlucio10:01:48

this is wrap top of transit, but I handle things like ensure everything encodes, no matter what (so nothing breaks from failed encoding, as functions for example)

Jakub Holý (HolyJak)10:01:39

I do use the connector and p.connector/connect-env . How does it work? Does it start a separate server on a different port, or what?

wilkerlucio10:01:18

yes, it makes a server on your side and connects the app with it

Jakub Holý (HolyJak)10:01:49

When I have added connect-env to my parser’s env and start the standalone Viz, it does not detect anything, so I guess I have to manyally add a connection? http://localhost:<some port> I assume?

wilkerlucio10:01:55

it actually depends, on cljs it will make a connection with the app via web sockets

Jakub Holý (HolyJak)10:01:05

no, this is clj / backend

wilkerlucio10:01:08

for clj it uses http servers (one at the client, one at the server)

wilkerlucio10:01:23

it should detect automatically

wilkerlucio10:01:32

you just need to load your source after opening the app

wilkerlucio10:01:40

it should add a tab automatically

Jakub Holý (HolyJak)10:01:22

WDYM by “you just need to load your source after opening the app” ? Like load my parser.clj file into the repl?

wilkerlucio10:01:42

yes, run the part that has the connection code (p.connector/connect-env ...)

wilkerlucio10:01:34

one CLJ side I commonly use a pattern on the end of the threading to make the env, eg:

(def env
  (-> {:some "config"}
      (pci/register ...)
      ((requiring-resolve 'com.wsscode.pathom.viz.ws-connector.pathom3/connect-env)
       "debug")))

wilkerlucio10:01:03

the requiring-resolve here is nice because I can add/remove viz just by changing this line (no need to change anything at the ns declaration)

Jakub Holý (HolyJak)10:01:08

ah, sorry, I did not get that it takes a parser and returns a parser, I though it takes an env and returns an env

wilkerlucio10:01:29

it does take and env and return an env

wilkerlucio10:01:42

that's the env in the example above

Jakub Holý (HolyJak)10:01:51

oh, I am just confused b/c the docstirng on connect-env reads >

Connect a Pathom parser to the Pathom Viz desktop app. The return of this function
> is a new parser,
>

wilkerlucio10:01:07

are you using the correct namespace?

wilkerlucio10:01:13

pathom 3 has a different namespace

wilkerlucio10:01:22

[com.wsscode.pathom.viz.ws-connector.pathom3 :as p.connector]

Jakub Holý (HolyJak)10:01:27

this is com.wsscode.pathom.viz.ws-connector.pathom3/connect-env

wilkerlucio10:01:42

ah, you right, that docstring needs changing

wilkerlucio10:01:25

probably copied over and missed changing it

Jakub Holý (HolyJak)10:01:31

I do not have access to the env b/c I use Fulcro’s P. parser that hides that. Can I use an env plugin? Like this

[(pbip/env-wrap-plugin #(com.wsscode.pathom.viz.ws-connector.pathom3/connect-env % {::pvc/parser-id "jhtest"}))]
but that doesn’t seem to have any effect…

wilkerlucio10:01:27

no, you really need to make that in the env, because it setups up some plugins, it cant be done at an env wrap level

👍 2
wilkerlucio10:01:39

docs updated, will be right once there is a next release

Jakub Holý (HolyJak)10:01:10

It got connected 🎉

Jakub Holý (HolyJak)10:01:10

but somehow it does not see my resolvers and attributes, only foreign-indexes-resolver and 3 pathom’s attributes

Jakub Holý (HolyJak)10:01:29

Is order important? I put it into the env before anything else

wilkerlucio10:01:36

yes, order is important

wilkerlucio10:01:46

if you connect before registering, things wont be there

wilkerlucio10:01:59

the connection needs to be last part, after the env is properly ready

👍 2
Jakub Holý (HolyJak)11:01:04

it works 🎉 Thx a lot for the advice! I will contribute the necessary changes to Fulcro and document this.

🙏 2