datomic 2019-05-07 | Slack Archive

Hi, what would cause Transactor not available errors from Datomic ?

I've seen these during HA transactor failovers and occasionally on unreliable networks. if you're transacting often, even if the transactions are small as you say, you're more likely to see this purely because of statistics

Ivar Refsdal09:05:47

I'm fairly certain I have gotten Transactor not available as a red herring (false positive). I reproduced this locally three times this morning. The real issue was that a cross product query was being run concurrently and starving memory/CPU. This led to the application freezing/slowing and d/transact failing. For my case the cross product query took about 10-15 seconds. 3 x concurrent queries starved the machine's resources, and after about 3 minutes of run time the datomic AQ connection died and d/transact failed with Transactor not available. Hope that helps! And apologies if I'm wrong. Best regards.

👍 4

jdkealy00:05:01

It happens when i'm trying to transact relatively small transactions.

jdkealy00:05:52

it's too infrequent to recreate, but happening enough to cause problems

hadils05:05:49

I have a design question. I am still new to Clojure/Datomic. I am thinking about passing around maps instead of records and, lately, I have been thinking to use the Datomic key names directly, instead of repeatedly renaming them. I have a GraphQL interface and external APIs which need to rename the keys anyway; I don't see why I can't just use the Datomic :db/ident names. Any thoughts on this subject? The only downside I see is that if I refactor the schema, I have to change it in a lot of places. Thanks.

donaldball14:05:04

You might consider adding an attribute to those attributes you want to expose on graphql indicating their public name, e.g. :schema/published.ident

conan14:05:46

i've had good experiences with aligning clojure.spec and datomic schema, and passing this right the way up to the frontend of my app. I even wrote a bit about it here: https://conan.is/blogging/clojure-spec-tips.html#datomic there are some libraries that attempt to bridge the gap between lacinia (for graphql) and datomic schema, although i haven't had a chance to try them yet: https://github.com/workframers/stillsuit https://github.com/workframers/catchpocket https://juxt.pro/blog/posts/through-the-looking-graph.html

conan14:05:09

i think there are other libraries, so be sure to do some digging

conan14:05:20

maybe @U0654RQ1F has made this work

hadils20:05:51

Thanks all for your input. This is very helpful.

Ivar Refsdal08:05:24

I've used a the same names for Datomic and GraphQL fields. And I've instructed cheshire (the JSON serializer) to strip the namespace part of the keys (as well as :db/id) to make the response as GraphQL expects. This way I can work with the data as datomic pull returns it, and when the web response is given it is finally transformed into the GraphQL format. This has worked well for me

val_waeselynck07:05:20

#Also sent to the channel

@hadilsabbagh18 controversial opinion: don't use Clojure's namespacing convention, use one that can be supported by GraphQL and all the places where your data will end up (e.g x_y_z_k instead of the more clojury x.y.z/k). You'll still get the essential benefits of namespacing, and only lose a bit of syntactic sugar. In my experience on real-world Datomic projects: having an ubiquitous namespacing convention is more important than having an idiomatic one. Data traceability is more valuable than concision or esthetics. https://clojureverse.org/t/should-we-really-use-clojures-syntax-for-namespaced-keys/1516

hmaurer14:05:42

Hmm interesting. I’ve also been annoyed at the fact that namespaced keywords are a great feature of clojure that isn’t present in other languages, which inevitably leads to the question of what to do with namespaces as soon as you interact with another lang.

henrik18:05:09

Are there systems that don't support uppercase letters? I have encountered databases that don't support lowercase letters in column names.

val_waeselynck19:05:21

@U06B8J0AJ I believe SQL names are case insensitive, so you get lowercase output

bhurlow13:05:20

@U5ZAJ15P0 though not very satisfying, we use the string representation of clj keywords in our javascript like: ":user/firstname". This still has the benefit of being able to merge safely

timur04:05:31

@U0FHWANJK what happens to your string keywords when they get JSON parsed? That is, once they arrive into the Javascript world over the wire, they are usually parsed from a string into a key.

val_waeselynck05:05:18

@U07HW6PNW any string can be used as a key in js, you just could not use that one with dot notation.

timur06:05:41

const foo = {“foo/bar”: 42} will work because it’s a string, but my question was what happens to these string keys that contain forward slashes when the containing objects get JSON parsed?

val_waeselynck06:05:25

They remain the same? I may not understand the question.

henrik09:05:12

@U07HW6PNW Do you mean if the contents of the string is interpreted as JSON? I'd say all bets are off, as they indeed are with any string.

bhurlow13:05:16

@U07HW6PNW in our app we simply leave them as string keys which means like val noted we cannot use the dot syntax. It looks like entity[":user/username"]. This is certainly a downgrade from actual keyword value types, but we still retain some of the advantages. We did consider at one point camel casing into userUsername but this we felt was not necessarily stronger

timur19:05:01

Right, so anywhere in the system between boundaries and across wire where you might not have control, and some code is doing the automatic JSON.parse on the payload, you’re bound to run into issues, I would think.

conan17:05:34

sorry if the topic has moved on, but this might be of interest https://github.com/workframers/catchpocket

donaldball18:05:40

We looked at that and at least one other take on “generate graphql schema from datomic” system, but given that we want our graphql schema to be our public contract, we decided we were happier using a literal graphql schema expressed as lacinia-compatible edn that we annotate with datomic bindings and some higher order niceties.

val_waeselynck06:05:30

You could also derive both the Datomic and GraphQL schemas from a common source of truth: https://vvvvalvalval.github.io/posts/2018-07-23-datascript-as-a-lingua-franca-for-domain-modeling.html

joshkh20:05:41

does the [?a ...] find specification syntax for returning collections work on Datomic Cloud? i get the following on Cloud when running the on-prem example [1]:

(d/q
  '[:find [?release-name ...]
   :in $ ?artist-name
   :where [?artist :artist/name ?artist-name]
   [?release :release/artists ?artist]
   [?release :release/name ?release-name]]
  (client/db) "John Lennon")
ExceptionInfo Only find-rel elements are allowed in client find-spec, see   clojure.core/ex-info (core.clj:4739)

[1] https://docs.datomic.com/on-prem/query.html#find-specifications (also i think the link in the Exception is outdated)

kenny20:05:52

It is not supported.

joshkh20:05:16

darn. okay, thanks @U083D6HK9

val_waeselynck07:05:20

replied to a thread:I have a design question. I am still new to Clojure/Datomic. I am thinking about passing around maps instead of records and, lately, I have been thinking to use the Datomic key names directly, instead of repeatedly renaming them. I have a GraphQL interface and external APIs which need to rename the keys anyway; I don't see why I can't just use the Datomic :db/ident names. Any thoughts on this subject? The only downside I see is that if I refactor the schema, I have to change it in a lot of places. Thanks.

2019-05-07

Channels