Fork me on GitHub
#datomic
<
2022-10-19
>
IM08:10:09

hi, was wondering are there any cases/caveats where a relatively fast query would take excessively long time? In other words, are there any deadlock-ish/blocking behavior when doing d/q with on-prem in-process peer (using d/history too)?

Ivar Refsdal08:10:50

Is this on-prem or cloud? I (and others) have been hit by network issues, i.e. silently broken TCP connections. The default behaviour of e.g. the postgresql driver is to wait forever for a packet. For postgresql you may set the socketTimeout property on the connection string. That might help if that is what is hitting you. The problem is well described here: https://github.com/brettwooldridge/HikariCP/wiki/Rapid-Recovery (Datomic on prem uses a different connection pool though, but the socketTimeout setting is the same.)

Ivar Refsdal08:10:20

oh I see you are using on-prem

Ivar Refsdal09:10:12

I also wonder if you are running in the "cloud"? My company services is running in Azure, and we have had plenty of these silently broken TCP connections. All of our code needs to have proper network timeouts, otherwise -well- requests will just block forever it seems. Waiting forever is the default of quite a few libraries (including datomic). We did not have this problem on prem (though that doesn't mean it cannot happen there of course), or at least it hardly ever happened.

IM09:10:12

Yeah, I've seen that ask.datomic thread while looking around, but not sure how much that applies, considering dynamodb backend is used in my case, not even sure if it supports some timeout parameter.

IM08:10:20

also unrelated question, is there any documentation about what the metrics from Datomic Metrics Reporter (`datomic.process-monitor`) mean exactly and how to interpret them? Things like PodGetMsec or AcceptIndexMsec? I could only find about transactor metrics using registerd callback

rolt12:10:47

I remember reading somewhere that undocumented metrics were considered internal

😒 1
emil0r09:10:43

For restoring backups to a SQL storage, how do you format the jdbc-url, given that the connection already has a ? in the string you send in as a db-uri

emil0r09:10:06

Also, is there any limitations in which authentication you need for datomic towards postgres? The installation I am using uses scram-sha-256 as the authn method. Is there need to downgrade to md5?

emil0r10:10:30

OK. Figured it out. You need an md5 based authentication on the postgres instance, and you escape the ? and & characters in your JDBC url with \

Dustin Getz17:10:50

Client API – is it true that the client API is the same across all Datomic product lines? (Onprem, Cloud, Ion, Dev local) the docs for onprem and cloud both link to the same place https://docs.datomic.com/client-api/index.html

yes 1
favila18:10:58

There are minor differences, e.g. you can’t create/delete dbs (or perform administer-system?) with peer-server (= onprem)

👍 2
favila18:10:23

but these are all the same fns from the same namespace

favila18:10:32

and all the usual stuff works the same

Pascale Audet18:10:40

oh, that is cool

JohnJ18:10:47

In cloud, is the transactor and peer one process?

JohnJ18:10:39

@U09K620SG haven't try it, but I don't think devlocal supports custom transaction functions since that requires ions

favila18:10:49

dev-local is a single process---seems like the transaction function would only need to be available to the classloader

JohnJ19:10:55

Ok, is not clear from the docs how you would make them available to devlocal/transactions since the examples rely on the ions api https://docs.datomic.com/cloud/transactions/transaction-processing.html#cancel

JohnJ19:10:41

buy maybe datomic.ion is available in devlocal, haven't check

favila19:10:42

In on-prem, nothing is done to make them available--just make sure the var is in the classpath.

favila19:10:56

I suspect dev-local is the same, because it’s the same process model

favila19:10:28

ions are different, because you have to deploy the txfn to the transacting process somehow.

favila19:10:27

This question can be answered very easily in a repl with dev-local loaded. I don’t have a dev-local project handy though.

JohnJ22:10:29

last time I checked there was no datomic.ion namespace per (ns-all)

favila22:10:53

why do you need an ion namespace to write a transaction function?

favila22:10:13

oh, you mean what environment is available to the txfn

favila22:10:47

I misunderstood what you meant by this:

favila22:10:49

> but I don’t think devlocal supports custom transaction functions since that requires ions

favila22:10:58

It’s easy enough to add a shim for cancel

favila22:10:09

cancel is just (throw (ex-info ..))

favila22:10:49

with some standard keys in the ex-data map

JohnJ22:10:28

thx, will look into it, don't have the project near by right now