Fork me on GitHub
#xtdb
<
2020-05-06
>
ouvasam12:05:38

Hi, I have a problem using crux and pathom and i can't find the problem. I have a resolver that call a function with a datalog query in crux. The first time it is executed all his fine, the crux function is called and return the results. If i call it within a few seconds, the resolver is not called anymore or at least no error is seen. If i call the crux function from the ring query directly it works perfectly. So it seems that crux is ok. If i remove the crux call in the resolver it is call normally and return correctly I have no idea what could be wrong here ? If someone has an idea ? Many thanks This is cross posted on both #pathom and #crux since i can't find from where the pb come

ouvasam12:05:26

Here is the code for the parser

(def parser
  (p/async-parser
   {::p/env     {::p/reader               [p/map-reader
                                           pc/async-reader2
                                           pc/ident-reader
                                           p/env-placeholder-reader]
                 ::p/placeholder-prefixes #{">"}}
    ::p/mutate  pc/mutate-async
    ::p/plugins [(pc/connect-plugin {::pc/register app-registry})
                 p/error-handler-plugin
                 p/trace-plugin
                 params-down-pathom-plugin]}))

ouvasam12:05:48

The code for the ring handler

(defn query
  [query]
  (let [params (clojure.walk/keywordize-keys (:params query))
        q (:query params)]
    (println "ring query" q)
    ; (println (crux-api/get-concept-translations-by-jurisdiction 1 (rand-nth (range 730 750))))
    {:status 200
     :body
     (encode-transit-json
      (<!! (parser params (clojure.edn/read-string q))))}))

ouvasam12:05:05

The code for the resolver that only work "sometime"

(pc/defresolver by-jurisdiction-id [{:keys [] :as env} params]
                {::pc/input #{:translation/query}
                 ::pc/output [:translations
                              [:translation/id
                               :translation/language
                               :translation/value
                               :translation/status
                               :jurisidction/id
                               :concept/id
                               :app/type]]}
                (let [query (parse-query-params env)
                      jurisdiction-id (:jurisdiction/id query)
                      concept-id (:concept/id query)
                      results (crux-api/get-concept-translations-by-jurisdiction)]
                    {:translations []}))

ouvasam12:05:37

and the crux function

ouvasam12:05:38

(defn get-concept-translations-by-jurisdiction
  [jurisdiction-id concept-id]
  (let [ids (crux/q
             (crux/db node)
             {:find '[typ]
              :where '[[e :app/type typ]
                       [e :jurisdiction/id jurisdictionid]
                       [e :concept/id conceptid]]
              :args [{'jurisdictionid jurisdiction-id
                      'conceptid concept-id}]})]
    (println ids)
    []))

refset12:05:27

Hi šŸ™‚ Is your crux function really meant to be returning only [] on that last line of the let?

ouvasam13:05:11

No it is only for tests

šŸ‘ 4
ouvasam13:05:20

If i remove the crux call it works

ouvasam13:05:34

if i call the crux function from the ring handler it works too

ouvasam13:05:43

That's only in the resolver

ouvasam13:05:02

but if i remove the crux call in the resolver, it works too

ouvasam13:05:06

Here is the crux conf

(defonce node
  (crux/start-node
   {:crux.node/topology :crux.standalone/topology
    :crux.node/kv-store "crux.kv.lmdb/kv"
    :crux.standalone/event-log-dir "resources/data/eventlog-1"
    :crux.kv/db-dir "resources/data/db-dir-1"
    :crux.standalone/event-log-kv-store "crux.kv.lm/kv"}))

(defonce db (crux/db node))

ouvasam14:05:31

@U899JBRPF Do you think i am correct with crux ?

ouvasam14:05:54

It seems waiting the previous query close or seomthing like this ?

refset14:05:05

I can't see anything obviously wrong with what your doing with Crux. Except I'm not sure what this line is for:

(defonce db (crux/db node))
It feels like that might produce some kind of issue, even if you .close the node across multiple evals. How are you using it?

ouvasam14:05:14

it was a try otherwise i did start with (crux/db node) but the problem is the same Do i have to start and close the node for each ring request ?

refset14:05:27

Ah okay. And no, that definitely shouldn't be necessary šŸ™‚

refset14:05:38

Is it possible to reproduce with a minimal example in a gist or repo that you can share? I think that would be easier to debug both for people here and also the folks over on #pathom

ouvasam11:05:00

Hi Jeremy, many thanks ! I did a little project as you suggested but no error there. So the problem should come from somewhere else. I continue to search ... Thansk !

ouvasam12:05:26

Ok the problem was that i had two resolvers that return the same signature ...

ouvasam12:05:38

Sorry for the derangement and many thanks for your help

refset14:05:23

ahhh - glad I could help a little bit šŸ˜€

refset14:05:43

well done for persevering, at least you won't make that mistake again!

šŸ‘ 4
Jacob O'Bryant16:05:55

Hey, hope it's ok to cross-post this here. This is built with Crux and includes some features similar to what Firestore has. I thought people here might be interested. The main Crux-related features are: ā€¢ Subscribable queries (not datalog, but roughly the same in power as Firestore queries) ā€¢ Authorization rules, so you can submit arbitrary queries and transactions from the frontend ā€¢ Triggers, so you can run a function whenever a certain type of document is created/updated/deleted Also: a big thank you to everyone working on Crux. It's been a joy to work with. https://clojurians.slack.com/archives/C8NUSGWG6/p1588782267181500

šŸ˜ƒ 28
šŸ™ 16
refset16:05:22

This is great stuff - thanks for sharing your work and ideas so generously in return! By the way, we have a new event bus API merged and ready for the next release so you won't have to do your own polling šŸ™‚

Jacob O'Bryant16:05:14

perfect, I can't wait to try it out!

kosengan11:05:03

This is cool šŸ™‚ just wondering why Crux was choosen over Datahike?

Jacob O'Bryant23:05:01

I haven't used Datahike, but it seems to still be at an early/experimental stage, whereas Crux is more mature. With Crux I'm not worried about hitting a wall when I need scale since I know I can switch to Kafka for the storage backend. Having Juxt behind it is also a plus.

Jacob O'Bryant23:05:59

Here's a relevant snippet from https://github.com/replikativ/datahike#when-should-i-pick-what: > Pick datahike if your app has modest requirements towards a typical durable database, e.g. a single machine and a few millions of entities at maximum. > Pick Datomic if you already know that you will need scalability later In this case, s/Datomic/Crux/. IMO Crux gives you the best of both worlds: you can run it on the local filesystem for dev and small deployments, for small to medium size you can use managed SQL, and for large deployments you can use Kafka

kosengan23:05:13

Got it! I see that datahike extends datascript which is well done. So by choosing Crux also means not choosing to use datascript or is there a way to leverage the power of datascript ? The below article* reads that datascript+crux is not that viable(for now) as much as in datascript+datahike. *https://opencrux.com/docs#queries_datascript_differences

Jacob O'Bryant07:05:40

Crux implements its own version of datalog (whereas datascript's datalog is very close to Datomic's datalog). So you can't use datascript's datalog from Crux, but you can still use datalog which is the important thing.