Fork me on GitHub
#pathom
<
2021-09-01
>
caleb.macdonaldblack01:09:57

@wilkerlucio What inspired you to create Pathom?

souenzzo02:09:52

I'm also curious to read Wilkers response but the part that I know: Pathom started from om.next ideas (server part) https://github.com/omcljs/om/wiki/Quick-Start-(om.next)#parsing--query-expressions I also recommend to watch one of David Nolen videos on youtube about om.next ideas. om.next was never finished. Then emerged #fulcro as the spiritual successor for the frontend, and #pathom to the backend "Pathom connect" is a original idea from Wilker as far I know.

caleb.macdonaldblack02:09:47

@U2J4FRT2T Same. I’m really excited about Pathom. It seems like such a simple idea of which surely has somewhat been inspired by other ideas. I’m curious about what is original and what isn’t. I’m vaguely understanding that is has some inspiration from Logic Programming as mentioned on the Pathom website but it seems to me it takes it a step further than that. Either that or I need to spend some more time on logic programming

souenzzo13:09:48

After ~3 years working with pathom, programming without it feels like "programming without a GC" We need to take care about if a value is fetched before pass it to another function With pathom, we just require what we want and use it. We don't need to care about the "fetch lifecycle"

wilkerlucio18:09:52

hello @U3XCG2GBZ, as @U2J4FRT2T said, Pathom started as some extensions I made to create parsers for Om.next (so the name: Path Om) early inspirations to go on the idea are @U050B88UR presentations like: https://www.youtube.com/watch?v=fYLM5nCIBQg in the beginning there connect idea didn't exist, I was just trying to make easier to write simple parsers, the main idea was to have a single implementation for every attribute in the system. When I had the connect idea, I was implementing fetching for basic customer data, back then it looked something like this:

(defn read-customer-data [env]
  (let [{:keys [customer/id]} (p/entity env)
        response (cached-read env
                   (str "" id))]
    (get 
      (adapt-customer response) 
      (-> env :ast :key))))

(def customer-readers
  {:customer/name
   read-customer-data
   
   :customer/email
   read-customer-data
   
   :customer/account-id
   read-customer-data
   
   ...})
By the time I wasn't very happy about that, for each request we need to give the full implementation, but the turning point was that on top of getting user by ID, I also needed to pull him by SSN, which made the request fn look like this:
(defn read-customer-data [env]
  (let [{:keys [customer/id customer/ssn]} (p/entity env)
        response (cached-read env
                   (if ssn
                     (str "" id)
                     (str "" id)))]
    (get
      (adapt-customer response) 
      (-> env :ast :key))))
The change might not seem so bad, until you realise the endpoint for SSN doesn't have all the same fields of the endpoint by ID, so now we need 3 fetch implementations, one that supports both ID and SSN, one just for ID, one just for SSN... By that time we were experimenting if the idea would fit, and IMO this wouldn't scale well for the whole organization, I felt it needed to be better, easier to handle Looking at that I though: what if I index the attributes with some fns, and have some code to traverse this index? And that was the beginning of the Connect idea, wasn't something I saw anywhere else, but an emergent solution for the problem I had in front of me. The first versions were very crude (had lots of issues with circle references for example), you can still it, its the reader on Pathom 2 (https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/connect.cljc#L709). These connect readers had a lineage: reader - the first experimental one, very eager, it wasn't used for long since it has a bunch of pitfals (inneficient, cycle errors...) reader2 - this was the first time Pathom did some kind of planning, for a single attribute it traverses the indexes to find a sequence of resolvers to call (or to find the path isn't possible) reader3 - this has a more advanced planning, that generates an execution graph, Pathom 3 uses an evolved version of this algorithm And the Pathom lineage: Pathom 1 - this version depended on om.next, used the basic EQL parser and parser construct directly from the om.next code Pathom 2 - removed dependency with Om.next, EQL was extracted in its own library (so both Pathom and Fulcro could share it), and the parser infra-structure was copied into Pathom Pathom 3 - a complete redesign, abandoning the parser concept from Om.next and making its own processing method, that aligns with the knowledge from Pathom 2 and make connect the center of it (instead of an extension, like it is on Pathom 2) So the connect idea is mostly original, I only learned about RDF much later and was surprise how the modeling was similar, but not by accident, my modeling was inspired by Spec/Datomic, which are inspired in RDF to begin with.

pathom 8
caleb.macdonaldblack06:09:07

Hi @wilkerlucio. Thanks for such a detailed response. I really appreciate it. The David Nolen talk is great and describes similar issues to what I’m experiencing in my software.

JAtkins06:09:31

I think I found a bug with p3 nested input planning:

(defresolver nested-output-1 
  [_ _]
  {::pco/output [{:nested-output-1 [:basis-key]}]}
  {:nested-output-1 {:basis-key 10}})

(defresolver derived-key 
  [{:keys [basis-key]}]
  {:derived-key (inc basis-key)})

(defresolver nested-output-2 
  [_ {{:keys [basis-key derived-key] :as nested-output-1} :nested-output-1}]
  {::pco/input [{:nested-output-1 [:basis-key
                                   :derived-key]}]
   ::pco/output [{:nested-output-2 [:basis-key
                                    :derived-key
                                    :output-2-key]}]}
  {:nested-output-1 (assoc nested-output-1
                           :output-2-key (+ basis-key derived-key))})

(let [env ((requiring-resolve 'com.wsscode.pathom.viz.ws-connector.pathom3/connect-env)
           (pci/register [nested-output-1 nested-output-2 derived-key])
           {:com.wsscode.pathom.viz.ws-connector.core/parser-id "TRACER"}) ]
  (p.eql/process
   env 
   [{:nested-output-2 [:basis-key
                       :derived-key
                       :output-2-key]}]))

;; =>

;; Execution error (ExceptionInfo) at com.wsscode.pathom3.connect.runner/check-entity-requires! (runner.cljc:733).
;; Required attributes missing: [:nested-output-2] at path []
Run on latest 2021.08.14-alpha

donavan08:09:33

Your stated output in nested-output-2 doesn’t match the actual output; :nested-output-2 in ::pco/output but the map key is :nested-output-1

donavan08:09:58

That does seem to be the issue from a glance… There are currently limitations with planning and nested output currently in P3, I’m going to take a stab at implementing Wilker’s suggested solution today but it doesn’t look like it’s related to this.

JAtkins11:09:27

Hmm… I think I was too tired when I wrote that. Good catch :thumbsup:

sheluchin13:09:46

I'd like to contribute a small change to the docs:

diff --git a/docs/index.adoc b/docs/index.adoc
index 9bf035a6..863dd36b 100644
--- a/docs/index.adoc
+++ b/docs/index.adoc
@@ -1969,6 +1969,11 @@ your index out.
 
 Using this you can control what gets out to the explorer.
 
+Note that you may need to take additional steps to make sure the index doesn't trip
+when coming across functions or other values that cannot be serialized for network
+transfer without special handling. See the <<Fixing transit encoding issues,Transit issues>>
+section for a snippet that implements the filtering on the resolver.
+
 === Visualizing your index
 
 Here you will find some ways to visualize your index.
@@ -2033,6 +2038,20 @@ include other things that are not possible to encode with transit by default. We
 you setup a default write handler on Transit so it doesn't break when it encounter a value
 that it doesn't know how to encode.
 
+Another approach is to filter out the `:resolve` and `:mutate` functions when setting up
+the index explorer resolver:
+
+[source,clojure]
+----
+(pc/defresolver index-explorer [{::pc/keys [indexes]} _]
+  {::pc/input  #{:com.wsscode.pathom.viz.index-explorer/id}
+   ::pc/output [:com.wsscode.pathom.viz.index-explorer/index]}
+  {:com.wsscode.pathom.viz.index-explorer/index
+   (p/transduce-maps
+     (remove (comp #{::pc/resolve ::pc/mutate} key))
+     indexes)})
+----
+
 If you are running Pathom in Clojure, then you also need to know there is a bug in the
 current Clojure writer, it doesn't support default handlers (although the docs say it does).
Should this be a PR to master? Also, when I run antora docs-dev.yml, the change set is quite large:
$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   docs/index.adoc
        modified:   docs/v2/404.html
        modified:   docs/v2/index.html
        modified:   docs/v2/pathom/2.2.0/cljs-specs.html
        modified:   docs/v2/pathom/2.2.0/connect.html
        modified:   docs/v2/pathom/2.2.0/connect/basics.html
        modified:   docs/v2/pathom/2.2.0/connect/connect-mutations.html
        modified:   docs/v2/pathom/2.2.0/connect/exploration.html
        modified:   docs/v2/pathom/2.2.0/connect/indexes.html
        modified:   docs/v2/pathom/2.2.0/connect/readers.html
        modified:   docs/v2/pathom/2.2.0/connect/resolvers.html
        modified:   docs/v2/pathom/2.2.0/connect/shared-resolvers.html
        modified:   docs/v2/pathom/2.2.0/connect/thread-pool.html
        modified:   docs/v2/pathom/2.2.0/core.html
        modified:   docs/v2/pathom/2.2.0/core/async.html
        modified:   docs/v2/pathom/2.2.0/core/entities.html
        modified:   docs/v2/pathom/2.2.0/core/error-handling.html
        modified:   docs/v2/pathom/2.2.0/core/errors.html
        modified:   docs/v2/pathom/2.2.0/core/getting-started.html
        modified:   docs/v2/pathom/2.2.0/core/mutations.html
        modified:   docs/v2/pathom/2.2.0/core/parsers.html
        modified:   docs/v2/pathom/2.2.0/core/path-track.html
        modified:   docs/v2/pathom/2.2.0/core/placeholders.html
        modified:   docs/v2/pathom/2.2.0/core/readers.html
        modified:   docs/v2/pathom/2.2.0/core/request-cache.html
        modified:   docs/v2/pathom/2.2.0/core/trace.html
        modified:   docs/v2/pathom/2.2.0/graphql.html
        modified:   docs/v2/pathom/2.2.0/graphql/edn-to-gql.html
        modified:   docs/v2/pathom/2.2.0/graphql/fulcro.html
        modified:   docs/v2/pathom/2.2.0/introduction.html
        modified:   docs/v2/pathom/2.2.0/modeling.html
        modified:   docs/v2/pathom/2.2.0/other-helpers.html
        modified:   docs/v2/pathom/2.2.0/plugins.html
        modified:   docs/v2/pathom/2.2.0/rationale.html
        modified:   docs/v2/pathom/2.2.0/sugar.html
        modified:   docs/v2/pathom/2.2.0/upgrade-guide.html
        modified:   docs/v2/sitemap.xml
Should I only commit the changes to docs/index.adoc in this case?

wilkerlucio13:09:05

yes, please make a pr