Fork me on GitHub
#datomic
<
2020-10-22
>
onetom07:10:38

To make Ion deployment more comfortable, we started using the datomic.ion.dev/push and friends directly. Those function however seem to shell out to call clojure and expect it to be available on the PATH. Since we are using nix-shells to manage our development environments, we deliberately have no clojure on our PATH by default, only within direnv managed shells. Would it be possible to just call the necessary functions directly from the JVM process which runs the datomic.ion.dev/push function? It seems like it's only doing some classpath computation, which should be possible to do directly with clojure.tools.deps...

cmdrdats09:10:37

We've got a datomic on-prem transactor running, saving the data in mysql - we got this, and it died... how would I go about diagnosing the cause and fixing so it doesn't die?

cmdrdats09:10:07

we're running datomic-transactor-pro-1.0.6165.jar

favila09:10:34

super-high-level, the transactor tried to update one of the top-level mutable rows in the database (“pods”) and it couldn’t so it self-destructed. If this happened immediately on txor startup, I would check the connection parameters and credentials are correct and the mysql user has the right permissions (it needs SELECT UPDATE DELETE on table datomic_kvs). Assuming the transactor started correctly and connected to mysql correctly and this happened randomly later, I don’t know. It could be something transient on mysql itself.

favila09:10:19

I would actually look at mysql logs first

cmdrdats09:10:07

hmm - ok, makes sense - this happened randomly much later

cmdrdats10:10:24

I do know we've had weird random connection issues on our mysql hosts that we've had to workaround with reconnecting.. so that seems the most likely explanation, thanks for the info!

Michael Stokley17:10:56

i'm seeing some old materials around datomic that suggest you can query vanilla clojure data structures, such as vectors of vectors. eg:

(d/q '[:find ?first ?height
       :in $a $b
       :where [$a ?last ?first ?email]
              [$b ?email ?height]]
     [["Doe" "John" ""]
      ["Doe" "Jane" ""]]
     [["" 73]
      ["" 71]])
;; => #<HashSet [["Jane" 73], ["John" 71]]>
this does not run for me. is there a version of this that does work?

favila17:10:12

This is on-prem with the peer api; if you are using the client api, you need to include a “real” database as a parameter (first parameter?) even if you don’t use it because that is how the client api finds a machine to send the query to

favila17:10:51

on-prem runs the query in-process; client (typically-not necessarily) sends the query over the network to another process

Michael Stokley17:10:02

i see, thank you. it would be terrific to be able to use generic data structures as databases, seems like that would have been the clojure way of doing things as opposed to locking you in to a nominal type

favila17:10:44

the client api doesn’t provide a query engine, so, they’re kind of at cross purposes

favila17:10:05

to be clear: this works with the client api just fine, but you have to send it to something that can evaluate it

Michael Stokley17:10:58

can you say more, i'm not sure i follow, yet. this works - this being, using generic data structures as the db?

favila17:10:12

(d/q '[:find ?first ?height
       :in $ $a $b
       :where [$a ?last ?first ?email]
              [$b ?email ?height]]
     (d/db client-connection)
     [["Doe" "John" ""]
      ["Doe" "Jane" ""]]
     [["" 73]
      ["" 71]])

Michael Stokley17:10:26

i am using datomic.client.api, it's not on-prem

favila17:10:47

I’m saying that this should work

favila17:10:51

note I added a client db, but I didn’t use it in the query

Michael Stokley17:10:17

oh, interesting.

favila17:10:18

all of those arguments will be sent to the server (probably not in-process), the query will run, and you will get the result

favila17:10:33

the server that is backing the db object

Michael Stokley17:10:50

yeah, that works!

Michael Stokley17:10:17

i wonder if there's a way to use this for testing? maybe not, since my production query will necessarily be referring to $ (ie not $a or $b)

Michael Stokley17:10:04

it would be great if i could throw together a very simple database out of generic data structures and exercise my production query on that, instead of a real db

favila17:10:39

You could, but real databases normalize entity references to entity ids for you (e.g. it knows :some-attr-keyword is eid 123). Without that you would have to construct your query or data carefully so that the comparisons are exact

favila17:10:04

also many query helper functions only work on a real database because they use indexes directly

favila17:10:08

(e.g. d/datoms)

favila17:10:23

I’m pretty sure get-else would fail, for example

Michael Stokley17:10:23

perhaps it's more practical to use a real db in tests, then.

Joe Lane19:10:32

Hey @michael740 , try the new memdb feature in the latest dev-local!

Michael Stokley02:10:35

thanks @U0CJ19XAM, I'll check it out

motform17:10:37

Is it possible to express a recursive query with the pull api where the data looks like :c/b m..1-> :b/a m..1 -> :a syntax starting from a/gid ? I.e. walking down refs (not components) that point “upward” from the top of hierarchy. I can easily do it from the bottom up, from c/gid, but I guess I just don’t get how to reverse the query. EDIT: Never mind, I just realised that this is what _ is for.

Michael Stokley18:10:18

it looks like the two argument comparison predicates such as < work perfectly well to compare instants when inside of a datomic query but not in normal clojure. it's confusing because the documentation says that most of the datomic query functions are the same as those found in clojure.core. anyone have any insight?

Michael Stokley18:10:00

the documentation does not indicate that < works with strings, inst, etc.

Michael Stokley18:10:37

i glad they do, though!

jaret21:10:34

Hi All! I wanted to announce the release of the Datomic Knowledgebase: http://ask.datomic.com/

🙌 12
👍 9
jaret21:10:19

For anyone wondering... we will be migrating over all of the pendo/receptive requests we've received in the past. So if you don't see something you've requested with me or on our old portal feel free to re-ask or check back again in a week or so.

kenny21:10:34

Curious when this should be used over the forum.

jaret21:10:48

The big gain from the forums, which is still the place to have discussions about Datomic Applications and to see announcements -- is to have the upvote button for features and harder questions.

jaret21:10:16

Our previous tool Pendo/receptive had several limitations. It just wasn't as accessible as we wanted it to be to get that feedback loop on what features are important to the community.

jaret21:10:50

I'll definitely be cross linking/posting from forum posts going forward if we get to a point where a feature is the best answer for whatever is being discussed.