datomic 2020-08-06 | Slack Archive

onetom17:08:45

i've just started to use the client api recently (after a few years of dealing only with the peer api). when i tried to test something simple, i was getting this error:

Execution error (ExceptionInfo) at datomic.client.api.impl/incorrect (impl.clj:43).
Query args must include a database

on a closer look, i can provoke this error by trying to run some of the official examples from the cloud docs (https://docs.datomic.com/cloud/query/query-data-reference.html#calling-static-methods), against a com.datomic/client-pro 0.9.63 backed by a datomic-pro peer-server & transactor 1.0.6165 (and i also have a com.datomic/dev-local 0.9.183 loaded into the same process too):

(dc/q '[:find ?k ?v
        :where [(System/getProperties) [[?k ?v]]]])

or this even simpler query:

(dc/q '[:find [?b]
        :in [?tup]
        :where [[(untuple ?tup) [?a ?b]]]]
      [1 2])

is that expected or a bug?

onetom17:08:14

for the 1st case, where the :in clause is omitted, i might understand the error, but for the second case, i definitely don't expect it

marshall17:08:15

client query must take a db, can’t be run against a collection the way peer query can

onetom17:08:04

so it's a "bug" in the documentation then, isn't it?

marshall17:08:28

i can look at the docs, if they are client-specific docs then yes

onetom17:08:02

the cloud docs - i linked - is all client api specific, no?

marshall17:08:21

yes, that example should take a db

onetom17:08:39

but now that we are talking about it, of course it needs a "db", since that's how the query itself can reach the query engine over the network...

marshall17:08:02

right 🙂

marshall17:08:37

@onetom ^

onetom17:08:07

i keep finding myself reaching for the peer api functions, like entid, ident etc. the https://docs.datomic.com/on-prem/clients-and-peers.html#peer-only section of the docs makes it clear that we should use the pull api instead of these functions, but that's just much more verbose. same issue with the lack of the find-coll and find-scalar from find-spec. is there any official or popular compatibility lib which fills this gap?

onetom17:08:10

or is there any good examples how to sturcture an app in a way that it's concise to test?

onetom17:08:02

for example, given this function:

(defn by-name
  [db merchant-name]
  (-> {:query '{:find  [?merchant]
                :in    [$ ?merchant-name]
                :where [(or [?merchant :merchant/name ?merchant-name]
                            [?merchant :merchant/name-en ?merchant-name])]}
       :args  [db merchant-name]}
      (dc/q)
      (ffirst)))

my test would look like this:

(deftest by-name-test
  (testing "exact match"
    (let [db (db-of [{:db/ident      :some-merchant
                      :merchant/name "<merchant name in any language>"}])]
      (is (match?
            (->> :some-merchant (dc/pull db [:db/id]) :db/id)
            (merchant/by-name db "<merchant name in any language>"))))))

where db-of is just a with-db with some schema, made from a dev-local test db. that (dc/pull db [:db/id]) :db/id is the annoying part and it's even more annoying if im expecting multiple values.

onetom17:08:15

the benefit of operating with idents is that the test failure messages are symbolic and i don't have to muck around with destructuring string temp-ids, potentially across multiple transactions

onetom18:08:21

i can understand that the client api doesn't want to provide find-scalar and find-coll and ident, entid, so the interface size is small, which helps providing alternative implementations, like the dev-local one, but these functions are just too useful for REPL work and automated tests. i can also understand how they might seep into application code, promoting inefficient code, but that's not a strong reason for not providing them officially.

👍 3

onetom19:08:06

for now, i made a custom matcher, which results in tests like this:

(is (match?
      (idents-in db
                 :matching-merchant-1
                 :matching-merchant-2)
      (merchant/named-like db "matching")))

where idents-in looks like this:

(ns ...
  (:require
    [matcher-combinators.core :refer [Matcher]] ...))

(defrecord MatchIdents [db expected-idents]
  Matcher
  (-matcher-for [this] this)
  (-matcher-for [this _] this)
  (-match [_this actual-entity-refs]
    (if-let [issue (#'matcher-combinators.core/validate-input
                     expected-idents
                     actual-entity-refs
                     sequential? 'in-any-order "sequential")]
      issue
      (#'matcher-combinators.core/match-any-order
        expected-idents
        (mapv (comp :db/ident (partial dc/pull db [:db/ident]))
              actual-entity-refs)
        false))))

(defn idents-in [db & entity-idents]
  (->MatchIdents db entity-idents))

kschltz23:08:49

I have an incident where datomic cloud started giving me busy indexing errors

kschltz23:08:17

I supposed I exceeded my node limit, the stop transacting entirely

kschltz23:08:33

but after 40min it still gives me busy indexing errors

marciol00:08:08

#Also sent to the channel

@marshall can you help us with some tip!

marciol00:08:36

@U0CJ19XAM or @U09R86PA4 you have some tip about this kind of issue?

favila15:08:07