pathom 2020-02-13 | Slack Archive

jaihindhreddy10:02:23

Sorry for being so wordy. Feel free to ignore if noise. Say I had a query like this that tries to fetch information about the role hierarchy in an organisation:

[{:org/employees [:emp/id :emp/name {:emp/manager [:emp/id :emp/name]}]}]

I'm requesting for all employees, and for each employee. I want their id, name, and the id, name of their manager (the person they report to) In this kind of a situation, the response will end up containing a lot of duplicate information, because of the fact that managers are employees too. Seems like Netflix's JSON Graph solves this problem using their ref concept. Does Pathom do something similar, can we borrow (steal) this idea from JSON Graph, or is this not that big of an issue?

kszabo11:02:02

this kind of normalization happens in the client-side in Fulcro. There is no current support to reduce the over-the-wire part of this.

👍 4

Chris O’Donnell16:02:15

I believe transit will compress duplicated values, reducing size over the wire if you use it.

jaihindhreddy07:02:23

#Also sent to the channel

Of course, Transit does cache keywords, and that gives us a decent compression, but I'm talking about entire maps that could be repeated dozens (even hundreds) of times, and could be avoided. Is this something Pathom could do in the future? Seems like GraphQL doesn't do this too currently.

souenzzo11:02:17

With this benchmark

(for [i [1e1 1e2 1e3]]
  (let [baos (.ByteArrayOutputStream.)
        zipper (java.util.zip.GZIPOutputStream. baos)
        data (.getBytes
               (cheshire.core/generate-string
                 {:org/employees (for [i (range i)]
                                   {:emp/id      i
                                    :emp/name    (str "emp" i)
                                    :emp/manager (for [i (range i)]
                                                   {:emp/id   i
                                                    :emp/name (str "emp" i)})})}))]
    (.write zipper data)
    (.close zipper)
    [i (.size baos) (count data)]))

You can see that GZip handles the size when you have duplicated elements The json parser/JS engine should also apply reference/reuse optimizations I'm not sure if reimplement it in query language will result in better performance.

wilkerlucio18:02:16

@codonnell it does by default for keywords, if I remebember correctly you can extended to introduce extra compression

jaihindhreddy07:02:58

data.fressian is extensible that way but not transit I believe. I couldn't find anything like that on the transit github.

👍 4

Chris O’Donnell11:02:06

My mistake, thanks for the correction.

souenzzo20:02:06

[ANN] I wrote some docs to eql-as , a library to qualify/unqualified keywords using #pathom and EQL. If you use pathom and need to handle JSON (aka unqualified keys) checkout. https://github.com/souenzzo/eql-as

jaihindhreddy07:02:23

replied to a thread:I believe transit will compress duplicated values, reducing size over the wire if you use it.

2020-02-13

Channels