datomic 2021-10-14 | Slack Archive

I am occasionally getting some really long :kv-cluster/get-val read times, such as

DEBUG {:event :kv-cluster/get-val, :val-key "60a5e1e2-b81a-49a1-9161-b19df11c3934", :msec 965000.0, :phase :end, :pid 26, :tid 190}

Why would this happen? What could explain this? Zero services, other than datomic backend, have problem reading from postgres. This is on-prem and the storage is postgres.

Daniel Jomphe15:10:07

This query fails to run. It fails to marshal because it contains a regex. Execution error at com.cognitect.transit.impl.AbstractEmitter/marshal (AbstractEmitter.java:194). Not supported: class java.util.regex.Pattern Can we add a transit handler for Datomic to successfully marshal and unmarshal the regex? Or is there a simpler solution?

✅ 1

Daniel Jomphe15:10:22

Daniel Jomphe15:10:40

If I'm right: • This works fine in dev-local and when the app is deployed as an Ion inside the Datomic Cloud cluster. No marshalling is required in such environments. • But it doesn't work when the app runs outside the cluster and connects remotely to a DB. Marshalling is required in such environments.

Lennart Buit16:10:15

Can you pass the regex as string, then use re-pattern to compile?

Lennart Buit16:10:27

Although the regex api is a bit imperative, wonder how that goes…

Lennart Buit16:10:19

(Note that you can’t do nested exprs, you need to bind single exprs one by one)

Daniel Jomphe17:10:09

@UDF11HLKC that's something I tried, but no, we can't. Even a hardcoded search term like this one makes it throw the same error.

'[:find (pull ?cpl cpl-repres)
  :in $ cpl-repres ?tenant-id ?search-str
  :where [?tenant :tenant/name ?name]
  [(.toLowerCase ^String ?name) ?lower-name]
  [(re-find #"(?i)si" ?lower-name)]
  ...]

Daniel Jomphe17:10:29

#Also sent to the channel

Or, for sure, we could install a transaction function just for the purpose of instantiating a regex pattern "server-side" out of args passed as strings, but again, is there a simpler solution or a config knob for transit handlers? Edit: oh no, a tx-fn isn't a solution since this is a query...

✅ 1

Lennart Buit20:10:11

I meant constructing the pattern inside the clauses of the query:

(d/q '{:find  [?match]
       :in    [$ ?pattern-string ?input]
       :where [[(re-pattern ?pattern-string) ?pattern]
               [(re-find ?pattern ?input) ?match]]}
     (d/db conn)
     "\\d+"
     "abc12345def")
=> #{["12345"]}

Lennart Buit20:10:11

although this is all in-process, so YMMV

Daniel Jomphe14:10:49

@UDF11HLKC, this does indeed seem to work! Thanks a lot for taking the time to come back!

Lennart Buit14:10:59

Yeah, so pattern instances can’t be serialized, and we circumvent that by passing a string and compiling the pattern on the peer. You have access to the majority of clojure.core on the peer, after all

Daniel Jomphe14:10:02

Yes. It now makes sense to me. My Datomic-query-fu is quite dusted/rusted.

🎉 1

popeye16:10:51

I am new to datomic and I am doing self running , going through https://docs.datomic.com/cloud/dev-local.html and I installed it and configured local storage

popeye16:10:16

added the data sample downloaded and running a simple application as below

popeye16:10:28

(ns datomic-practice.core
  (:require [datomic.client.api :as d]))

(def client (d/client {:server-type :dev-local
                       :system "datomic-samples"}))

(println (d/list-databases client {}))

popeye16:10:07

Unzip the datomic-samples zip into your - i have completed this step and ran ./install

Fredrik16:10:54

Do you have the .datomic/dev-local.edn file?

popeye16:10:46

yes

popeye16:10:42

it is in /home/g/.datomic folder

Fredrik17:10:30

And it's content is