I want to communicate with Postgresql for fetching and posting data from Pathom in Fulcro application, are there any examples and code for help to get a started? I'm using Honeysql for queries. (Additional question) What migration library do you use for SQL DBs?

Hello, I'm trying to get this example going, but I'm puzzled by the response. Why am I getting channels out? Can anyone shed any light on this? Thanks! Query:

(parser {} [{:t/data [:t/id :t/text :t/test_train :t/label]}])

#:t{:data #:com.wsscode.pathom.parser{:provides #{:t/data},
                                      :response-stream #object[clojure.core.async.impl.channels.ManyToManyChannel
                                                               0x70c6abd9
                                                               "clojure.core.async.impl.channels.ManyToManyChannel@70c6abd9"]}}

(def data [{:t/id         1
            :t/text       "asadad"
            :t/test_train "train"
            :t/label      "a"}
           {:t/id         2
            :t/text       "assdgsd vvdad"
            :t/test_train "train"
            :t/label      "b"}
           {:t/id         3
            :t/text       "asadadw kjonjo"
            :t/test_train "test"
            :t/label      "c"}])

(defn mk-resolver []
  {::pc/sym     (symbol "my-ns" "my-resolver")
   ::pc/output  [{:t/data [:t/id :t/text :t/test_train :t/label]}]
   ::pc/resolve (fn [_ _] {:t/data data})
   })

(defn mk-index []
  (-> {} (pc/register (mk-resolver))))

(def parser
  (p/parser
    {::p/env     {::p/reader               [p/map-reader
                                            pc/parallel-reader
                                            pc/open-ident-reader
                                            p/env-placeholder-reader]
                  ::p/placeholder-prefixes #{">"}}
     ::p/plugins [(pc/connect-plugin {::pc/indexes (atom (mk-index))})
                  p/error-handler-plugin
                  p/request-cache-plugin
                  p/trace-plugin]}))

I'm using the map version of the resolver because I intend to generate them dynamically later on. Is this even a good idea at all, btw?

@hjrnunes hello 🙂 the issue in your code is that you are using a serial parser combined with the pc/parallel-reader, those are not compatible, pc/parallel-reader requires the parallel-parser, if you just want serial parser you can replace the pc/parallel-reader with pc/reader2

@wilkerlucio Ah, I see! Makes sense

thanks

Is there a big advantage in going parallel for simple stuff?

the parallel (as the name says) can run things in parallel, it really depends on how your data is distribuited

some system will be more parallelizable than others, as you grow it usually gets good things, but in some cases the serial can be faster (when there is no room for parallelism)

I see

also, the mk-resolver helper you created, there is one in pathom already, pc/resolver or pc/defresolver

Does it allow me to register any symbol?

yup, whatever you send on the map will be the one it picks

but keep in mind that resolver names doesn't matter much, they just need to don't collide with others

you rarely reference resolvers by symbol

So is there any problem in creating resolvers dynamically at all?

not at all, the graphql integration does a lot of that

Cool!

I'm thinking about using EQL for a very simple data language. I don't actually need the tree aspect of the data, but I could use the expressiveness of the query parameters. Something like a poor man's Walkable, but without all the SQL bells and whistles, and without joins

Like to query one single table

However, I do need global query params, so to speak

Walkable does the :table/all join to get a bunch of rows, and I can apply :limit to that join

is that the best way?

Probably wasn't very clear. EQL params have to be applied to either a property, ident or join, right?

there's no way to apply them to the whole query itself?

Also, in the resolver, how do I get the parameters? Is it supposed to be from the query in env?

@hjrnunes yes, you get from env (-> env :ast :params), about the globally aplication, that can be done with plugins, you can write a plugin that captures the params on read and make it part of env, doing that will propagate that inforamtion to every subchildren, still you can't do at query level, but can be at something that queries everything else

to do query level you can use query meta-data if you want

that's a bit different but works as well

hmm I see. An explicit join is probably clearer. Thanks!

is (-> env :ast) supposed to contain the portion of the AST relevant for the resolver, or do I get-in with the path?

I got it

@wilkerlucio Thanks for your help and work on Pathom and Walkable. All in all, really useful and impressive work!

@wilkerlucio can I manipulate ::pc/processing-sequence to filter out entities? Basically, I want to apply a limit to the results per property value. [{:ents [:e/prop1 e:/prop2 (e/prop3 {:limit 2})]}] would return at most 2 rows for each distinct value of :e/prop3. I'm aware this can be done at the join level, but I like the expressiveness of doing it like this, because I'll be having a typical limit at the join level too.

messing with processing-sequence is a bad idea, thats an internal thing

maybe what you want can be better served as a plugin

so you can parse params and limit results across the whole parsing

yeah, so i'd collect any :limit params at the property level, stick them on the env, and then filter out when resolving :ents?

@hjrnunes hum, I would be careful about using limit on env, sounds weird that a limit in a parent would affect some joins many levels down, usually that is a thing per join

you still can do it, but you would be complecting parts of the query in a way that's not obvious

with a plugin you can implement the limit once and it will work anywhere the user adds the param (in joins returning lists)

thats a quite easy plugin to write

@hjrnunes

(def limit-plugin
  {::p/wrap-read
   (fn [reader]
     (fn [env]
       (if-let [limit (-> env :ast :params ::limit)] ; use qualified keyword so it doesn't clash with other people things
         ; let-chan only required to support async/parallel parsers
         (let-chan [res (reader env)]
           (if (sequential? res)
             (vec (take limit res))
             res))
         (reader env))))})

@wilkerlucio I'll give it a go! Thanks again!

I was wondering something about pathom, since you can use placeholders in queries. Do people actually use it instead of the default fulcro parser?

@mitchelkuijpers I did some tests around it, the problem is that pathom is too slow to work as the client parser, client parser needs to be really fast (and it tries to optimize on that, for example, it doens't even convert the query to AST, parse the query directly to reduce overhead), pathom on the other side is designed for flexibility, so a lot of wrappers and injection points that add overhead

Ah that clears it up, then I wont waste any time on it. I was just curious

@mitchelkuijpers but if you need for other purposes (maybe want to simulate fulcro reader on the server), pathom has that setup ready too: https://github.com/wilkerlucio/pathom/blob/master/src/com/wsscode/pathom/map_db.cljc

Ah nice, No was just wondering since we sometimes use placeholder queries with pathom. Since it makes structuring some UI stuff a bit easier

yeah, the good part is that the client parser doesn't need to know about that, in the end it turns in regular fulcro links 🙂

Yeah doesn't really matter for Fulcro luckily

Btw I tried out the index browser today but it seems to fail on:

::pc/transform pc/transform-batch-resolver

With this error:

[INFO] [talledLocalContainer] Caused by: java.lang.Exception: Not supported: class com.wsscode.pathom.connect$batch_resolver$fn__36671                                                               
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.marshal(AbstractEmitter.java:194) [?:?]                                                                                
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.JsonEmitter.emitMap(JsonEmitter.java:171) [?:?]                                                                                        
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.emitMap(AbstractEmitter.java:85) [?:?]                                                                                 
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.marshal(AbstractEmitter.java:184) [?:?]                                                                                
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.JsonEmitter.emitMap(JsonEmitter.java:171) [?:?]                                                                                        
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.emitMap(AbstractEmitter.java:85) [?:?]                                                                                 
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.marshal(AbstractEmitter.java:184) [?:?]                                                                                
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.JsonEmitter.emitMap(JsonEmitter.java:171) [?:?]                                                                                        
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.emitMap(AbstractEmitter.java:85) [?:?]                                                                                 
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.marshal(AbstractEmitter.java:184) [?:?]                                                                                
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.JsonEmitter.emitMap(JsonEmitter.java:171) [?:?]                                                                                        
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.emitMap(AbstractEmitter.java:85) [?:?]                                                                                 
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.marshal(AbstractEmitter.java:184) [?:?]                                                                                
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.emitArray(AbstractEmitter.java:97) [?:?]                                                                               
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.marshal(AbstractEmitter.java:182) [?:?]                                                                                
[INFO] [talledLocalContainer]   at com.cognitect.transit.impl.AbstractEmitter.emitTagged(AbstractEmitter.java:49) [?:?]                                                                             

But now that I am reading it is probably because it tries to convert a fn to transit

yup

@mitchelkuijpers check this https://clojurians.slack.com/archives/C87NB2CFN/p1555352482010500?thread_ts=1555127499.001300&cid=C87NB2CFN