Fork me on GitHub
#datomic
<
2017-10-26
>
eoliphant00:10:36

I also have some versioning questions 😉 I’ve made the newbie mistake of conflating datomic’s ‘native’ sense of time/version with what my application actually needs. I read @val_waeselynck’s blog post about reifying versions and what have you. I think I need to do something simliar for a use case of mine. I’m storing dynamic form definitions and their data. I need to link an instance of form data to the def of Form A - Version 3, so it seems like I need to have actual ‘current’ entities for each version of the form, copying/updating for each rev. My concern is that this seems rather inefficient, anyone done something similar ?

danielcompton02:10:18

@eoliphant not answering your exact question, but the recommendation I've been given from Cognitect employees is "If it matters to your business domain, then model it as an entity, and model it explicitly"

mac0102102:10:22

Is Datomic's query language really datalog? Here's a tiny, canonical example of datalog from wikipedia (https://en.wikipedia.org/wiki/Datalog#Example):

% Store some data
parent(bill, mary).
parent(mary, john).

% Define "ancestor" in terms of "parent".
ancestor(X,Y) :- parent(X,Y).
ancestor(X,Y) :- parent(X,Z),ancestor(Z,Y).

% A query to find all of bill's ancestors
?- ancestor(bill,X).
Does Datomic's query language support anything like this inductive definition of ancestry?

Empperi04:10:12

Datomic supports recursive queries and it is logic based so yes. Your example as it is cannot be defined directly via EDN Datalog but it’s functionality can be duplicated.

eoliphant02:10:50

thanks @danielcompton yeah I’m on board with the modeling aspect, Im just trying to work out the specifics functionally.

Brendan van der Es04:10:19

@mac01021 Datomic's datalog supports recursion in rules. This is a handy recursive rule to get the extent of an entity: https://gist.github.com/stuarthalloway/2002582. Here is thread talks about that exact example (the issue was with datomic version): https://groups.google.com/forum/#!topic/datomic/sD2m810kfrQ

daveeel13:10:16

Hi All, I setup a boot based project with Datomic Pro (0.9.5561.62) on dynamoDB. Connection to the DB is fine from a local transactor and console. However when I try to start my project, I am stuck with the following error:

Exception while starting the system
                              java.lang.Thread.run              Thread.java:  745
java.util.concurrent.ThreadPoolExecutor$Worker.run  ThreadPoolExecutor.java:  617
 java.util.concurrent.ThreadPoolExecutor.runWorker  ThreadPoolExecutor.java: 1142
               java.util.concurrent.FutureTask.run          FutureTask.java:  266
                                               ...
               clojure.core/binding-conveyor-fn/fn                 core.clj: 1938
                   datomic.kv-cluster.KVCluster/fn           kv_cluster.clj:  222
                datomic.kv-cluster.KVCluster/fn/fn           kv_cluster.clj:  224
                                               ...
                           clojure.core/partial/fn                 core.clj: 2534
                                clojure.core/apply                 core.clj:  652
                                               ...
                       datomic.kv-cluster/retry-fn           kv_cluster.clj:   82
                    datomic.kv-cluster/retry-fn/fn           kv_cluster.clj:   82
             datomic.kv-cluster.KVCluster/fn/fn/fn           kv_cluster.clj:  226
                    datomic.kv-dynamo.KVDynamo/get            kv_dynamo.clj:   44
                              datomic.ddb/get-item                  ddb.clj:   94
                             datomic.ddb/get-item*                  ddb.clj:   62
                               datomic.datafy/fn/G               datafy.clj:  136
                                    datomic.ddb/fn                  ddb.clj:   47
         java.lang.NoClassDefFoundError: com/amazonaws/AmazonWebServiceResult
java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: com/amazonaws/AmazonWebServiceResult
The project works fine with in-memory datomic. Googled for quite some time and no clue. Any pointer?

daveeel02:10:16

Hi @marshall

[com.amazonaws/aws-java-sdk-dynamodb "1.11.6"]
already there. I have another lein based project that could connect to the same transactor.

daveeel04:10:42

Turns out if I downgrade datomic from:

[com.datomic/datomic-pro "0.9.5561.62"]
(or .56) to:
[com.datomic/datomic-pro "0.9.5561"]
The issue no longer exists. And I can do what’s expected from the project repl From what I googled, feels like later datomic versions has specific dependencies on aws-java-sdk version ?

marshall13:10:43

@daveeel you need to add the AWS SDK to your dependencies

eoliphant17:10:45

Hi, I’m getting a weird error trying to transact in a db/fn in my REPL. I thought it might be my function, but getting the same error when I try the code from the datomic docs.

(d/transact conn [{:db/ident :add-doc
                   :db/fn #db/fn {:lang "java"
                                  :params [db e doc]
                                  :code "return list(list(\":db/add\", e, \":db/doc\", doc));"}}])
CompilerException java.lang.RuntimeException: Can't embed object in code, maybe print-dup not defined: clojure.lang.Delay@14e6e019,
anyone seen this before?

marshall18:10:30

@eoliphant try this:

@(d/transact conn [{:db/ident :add-doc 
                    :db/fn (d/function 
                             {:lang "java" 
                              :params '[db e doc] 
                              :code "return list(list(\":db/add\", e, \":db/doc\", doc));"})}])

eoliphant18:10:54

Ugh thanks @marshall I read this (https://support.cognitect.com/hc/en-us/articles/215581438-When-to-Use-Data-Literals) forever ago as I was just learning datomic/clojure and kind of forgot about it lol

uwo20:10:40

I have an entity in my system that throws an error I can’t explain when I (d/pull (db) '[*] 17592188295819) Exception Key not found: 59f20f54-00df-4e30-a0dc-267dd0bdd9fc datomic.common/getx (common.clj:191)

uwo20:10:15

I can pull specific attributes from it without error

favila20:10:35

I'm suspecting db corruption, missing block of data

favila20:10:51

does d/datoms work on this entity?

uwo20:10:44

like this? (first (d/datoms (db) :eavt 17592188295819)) => #datom[17592188295819 416 17592196436651 13194149925399 true]

uwo20:10:26

any obvious practice that will lead to corruption?

uwo20:10:51

@val_waeselynck oh weird. I don’t know why it would be, but the issue i describe above only happens when I’m running off of a ‘forked’ (datomock) connection

favila20:10:52

not first but all

favila20:10:57

or, try to visit all the datoms (they are lazy-loaded). Theory was a block was missing. maybe datomock is messing it up

favila20:10:45

the exception superficially looks like attempt to retrieve a fressian block from the underlying kv store failed (keys look like uuids)

favila20:10:11

not sure how datomock would manage to do that

favila20:10:21

so I may be wrong

uwo20:10:29

yuck. nvm. I’ve tested again with a forked connection. that’s not it. sorry for false alarm @val_waeselynck

uwo20:10:45

yeah, it wouldn’t make sense for reads to be affected by datomock

favila20:10:16

my mention of datoms is an attempt to trigger the error via something other than a pull

favila20:10:59

what is your storage?

uwo20:10:07

sqlserver

favila20:10:25

is that key in the datomic_kvs table?

uwo20:10:42

I haven’t checked

uwo20:10:57

this error was happening when I connected my dev peer, and it appears to be intermittent

uwo20:10:08

because it didn’t survive a restart

favila20:10:23

hm, bad sql connection?

uwo20:10:38

perhaps? other reads were working fine

marshall20:10:49

Is it possible you ran a GCstorage?

favila20:10:04

ah good call

uwo20:10:13

actually, come to think of it yes

marshall20:10:59

If you were holding onto an old value of the db and GC storage was run with a more recent time

uwo20:10:46

makes sense. thanks y’all!

uwo20:10:33

@marshall some of the code that was failing, an api endpoint, appears to request a new db on each query. Would that mean that GCstorage wasn’t the culprit?

hmaurer21:10:59

@marshall is the datomic client API sufficiently documented to build a stable PHP client?

favila22:10:54

AFAIK the actual on-the-wire stuff is not documented. Only the client code API interface.

favila22:10:14

You would have to reverse-engineer the existing impl to build your own client

favila22:10:49

the rest server is the only option

hmaurer22:10:39

@U09R86PA4 it is deprecated, isn’t it?

marshall12:10:52

Not yet. When we ship Datomic Cloud we will also document the wire protocol

daveeel04:10:42
replied to a thread:Hi All, I setup a boot based project with Datomic Pro (0.9.5561.62) on dynamoDB. Connection to the DB is fine from a local transactor and console. However when I try to start my project, I am stuck with the following error: Exception while starting the system java.lang.Thread.run Thread.java: 745 java.util.concurrent.ThreadPoolExecutor$Worker.run ThreadPoolExecutor.java: 617 java.util.concurrent.ThreadPoolExecutor.runWorker ThreadPoolExecutor.java: 1142 java.util.concurrent.FutureTask.run FutureTask.java: 266 ... clojure.core/binding-conveyor-fn/fn core.clj: 1938 datomic.kv-cluster.KVCluster/fn kv_cluster.clj: 222 datomic.kv-cluster.KVCluster/fn/fn kv_cluster.clj: 224 ... clojure.core/partial/fn core.clj: 2534 clojure.core/apply core.clj: 652 ... datomic.kv-cluster/retry-fn kv_cluster.clj: 82 datomic.kv-cluster/retry-fn/fn kv_cluster.clj: 82 datomic.kv-cluster.KVCluster/fn/fn/fn kv_cluster.clj: 226 datomic.kv-dynamo.KVDynamo/get kv_dynamo.clj: 44 datomic.ddb/get-item ddb.clj: 94 datomic.ddb/get-item* ddb.clj: 62 datomic.datafy/fn/G datafy.clj: 136 datomic.ddb/fn ddb.clj: 47 java.lang.NoClassDefFoundError: com/amazonaws/AmazonWebServiceResult java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: com/amazonaws/AmazonWebServiceResult The project works fine with in-memory datomic. Googled for quite some time and no clue. Any pointer?

Turns out if I downgrade datomic from:

[com.datomic/datomic-pro "0.9.5561.62"]
(or .56) to:
[com.datomic/datomic-pro "0.9.5561"]
The issue no longer exists. And I can do what’s expected from the project repl From what I googled, feels like later datomic versions has specific dependencies on aws-java-sdk version ?