Fork me on GitHub
#pathom
<
2022-12-02
>
Tyler Nisonoff00:12:05

Hey! dealing with a perf issue in Pathom2 I have a resolver that has an input :foo/large-map

::pc/input [:foo/large-map]
It seems that pathom is spending a lot of time processing / doing something with the map despite its resolver looking something like
{:foo/large-map (with-meta large-map {::p/final true}) }
I see 2 seconds in between a log right before my return of the large-map above and the first line of the parent resolvers function was trying to follow the discussion https://clojurians-log.clojureverse.org/pathom/2021-02-12 to see if its relevant, but was having trouble following. Should my ::p/final meta here be circuiting pathom?

wilkerlucio13:12:41

hello Tyler, it looks correct to me, and the final should short circuit (meaning pathom shouldn't try to process the items there), but usually a large map isn't an issue, most of the time a large sequence is an issue (because of processing a large N number of items)

wilkerlucio13:12:57

can you make a repro of the issue that we can work on top?

Tyler Nisonoff16:12:44

yeah, let me try to do that today! thanks wilker

Tyler Nisonoff17:12:08

my minimal repro doesnt have this behavior! new theory is perhaps a plugin in my fulcro project is doing something silly? 🤷

wilkerlucio18:12:08

it might be a plugin to remove the special values (`p/elide-special-values` if I can remember correctly) from pathom (`::p/not-found` and ::p/error) from the output, this plugin needs to scan all the data, and might be causing your issue

wilkerlucio18:12:31

if that's the case, you can write a version of the plugin that also looks for ::p/final on meta, and skip when its the case

Tyler Nisonoff20:12:21

was just coming here to say that appears to be whats happening, thank you!

Tyler Nisonoff20:12:47

@U066U8JQJ got an implementation working with a modified transduce-maps:

(defn prewalk-with-final-check
  [f form]
  (if (::p/final (meta form))
    form
    (walk/walk (partial prewalk-with-final-check f) identity (f form))))

(defn transduce-maps-with-final-check
  "Walk the structure and transduce every map with xform."
  [xform input]
  (prewalk-with-final-check
    (fn elide-items-walk [x]
      (if (native-map? x)
        (with-meta (into {} xform x) (meta x))
        x))
    input))
Would you accept a PR for this in pathom2? or is this not something we’d generally want to have?

Brett Rowberry19:12:35

Say I host an EQL endpoint. How do I give clients an OpenAPI/Swagger/GraphiQL experience?

wilkerlucio20:12:42

you can expose the indexes (eg: using boundary interface on pathom3), this is the raw data of the graph, tooling can be built around that, I provide Pathom Viz as a official UI for that purpose

thanks2 4
hadils20:12:04

I have been trying to wrap my head around placeholder queries to no avail. Can someone post some examples/explanations? Thanks in advance!

1
dehli14:12:19

placeholder queries allow you to hydrate pathom input whicn you can then ask for keys underneath. For example, if you had a resolver "full name" that took first name and last name you could query for it like:

[{(:>/user-one {:user/first-name "Jane"
                :user/last-name "Doe"})
  [:user/full-name]}

 {(:>/user-two {:user/first-name "John"
                :user/last-name "Smith"})
  [:user/full-name]}]
It would then return data like:
{:>/user-one {:user/full-name "Jane Doe"}
 :>/user-two {:user/full-name "John Smith"}}
Let me know if that helps explain it for you! There's also https://pathom3.wsscode.com/docs/placeholders/ which goes in more depth

hadils14:12:22

Thank you @U2U78HT5G. So full-name is a parameterized query, from what I gather from your EQL?

dehli14:12:12

full name is a resolver that takes two inputs. something like:

(pco/defresolver full-name
  [{:user/keys [first-name last-name]}]
  {:user/full-name (str first-name last-name)})

dehli14:12:26

you could instead do something like:

[{(:>/user {:user/id "foo" :user/first-name "John"})
  [:user/full-name]}]
and it would take the input :user/id and :user/first-name and figure out how it can get :user/last-name to fulfill the requirements of :user/full-name

dehli14:12:28

so if you had a resolver that took :user/id and spit out :user/last-name it would make that request to get the last name before passing the data to full name

dehli14:12:27

placeholders are more powerful than parameters b/c they can be used as inputs in any other resolver

wilkerlucio16:12:57

to help understanding placeholdes, we can talk about the primary use case, that is to be able to split the view of some entity between multiple components in a tree

wilkerlucio16:12:12

for instance, lets say you have a movie, and wanna break the representation in a few different components: • basic data • movie gallery • movie comments

wilkerlucio16:12:45

and here the queries for each component: • basic: [:movie/id :movie/title :movie/director-name] • gallery: [{:movie/images [:image/url :image/thumbnail-url]}] • comments: [{:movie/comments [:comment/id :comment/message]}]

wilkerlucio16:12:12

now, lets say you have a Fulcro component that uses the :movie/id as identity, and wanna use those components inside

wilkerlucio16:12:51

you can't just merge their queries, that would completly break Fulcro, but even without fulcro, merging queries has its own issues with conflicts (maybe different components what the same thing using different params for example)

wilkerlucio16:12:23

that's where placeholders come, they allow a parent entity, to represent different subparts (the subqueryes) at different levels, but keeping the identity of the parent

hadils16:12:12

Thank you @U066U8JQJ. I am not using Fulcro. I don’t know what a “parent” is in this context.

wilkerlucio16:12:19

here is the solution in this case, a wrapper component using:

[:movie/id
 {:>/movie-basics [:movie/id :movie/title :movie/director-name]}
 {:>/movie-gallery [{:movie/images [:image/url :image/thumbnail-url]}]}
 {:>/movie-comments [{:movie/comments [:comment/id :comment/message]}]}]

wilkerlucio16:12:49

this way, we have each different context (each sub-section requirement) separated in its own query part

wilkerlucio16:12:04

so they have isolation, but still share context

wilkerlucio17:12:07

the params part that @U2U78HT5G described is more a helper that got add in Pathom 3, and I may have to break that interface, because it prevents some custom usages of params with placeholders