Fork me on GitHub
#datomic
<
2016-01-13
>
ljosa01:01:05

Does a stack trace like this indicate connectivity issues with the storage (couchbase in this case)?

ExecutionException java.lang.RuntimeException: Exception waiting for value
	java.util.concurrent.FutureTask.report (FutureTask.java:122)
	java.util.concurrent.FutureTask.get (FutureTask.java:192)
	clojure.core/deref-future (core.clj:2180)
	clojure.core/future-call/reify--6320 (core.clj:6420)
	clojure.core/deref (core.clj:2200)
	datomic.catalog/get-catalog (catalog.clj:30)
	datomic.coordination/cluster-conf->resolved-conf (coordination.clj:160)
	datomic.cache/fn/reify--2419 (cache.clj:342)
	clojure.lang.RT.get (RT.java:672)
	datomic.cache/lookup-cache/reify--2416 (cache.clj:287)
	datomic.cache/lookup-cache/reify--2416 (cache.clj:280)
	clojure.lang.RT.get (RT.java:645)
Caused by:
RuntimeException Exception waiting for value
Caused by:
ExecutionException java.lang.RuntimeException: Cancelled
Caused by:
RuntimeException Cancelled

ljosa01:01:44

(worked after pagerduty and automatic restart)

Ben Kamphaus15:01:43

@donaldball: I’m fairly certain the data model and approach are a mismatch for what you’re trying to do re: book quality. Some of the changes I would make to the approach (a) use pull to get attr/value from a know entity id, (b) if you use keywords that have a corresponding integer value, add that value to the enums in the database. Otherwise, use numbers in the database to store values and track their meaning in your application. (c) if you pass in ordinal map, don’t resolve/get etc. in the query itself, use pull to get quality ident from db, then get corresponding value and generate transaction. (d) I wouldn’t do this in its own transaction function, if you need isolation this is a good fit for the built-in transaction function cas http://docs.datomic.com/transactions.html#built-in-transaction-functions - do the lookup work on the peer, you don’t want that overhead in the transactor’s serialization.

Ben Kamphaus15:01:27

@ljosa: it’s possible, hard to diagnose conclusively without more information — do you have more stack trace and/or logs (assuming you want to dig further). Did you encounter this on peer?

ljosa15:01:01

I think "it's possible" is a good enough answer. It only happened once, and yes, it was a peer. No other logs; the only thing to note is that this happened well after the peer had connected and had been running for some time. This peer fails fast when something goes wrong. Marathon restarted it, and all was fine after that.

Ben Kamphaus15:01:02

@ljosa: Ok, sounds good. If you encounter anything similar in the future and opt for a deeper dive, feel free to ping me.

hugod17:01:20

I’m trying to connect to a datomic instance but am getting HornetQException[errorType=SECURITY_EXCEPTION message=HQ119031: Unable to validate user: …elided…]. What does this mean?

Ben Kamphaus17:01:48

@hugod - possible that you’re exceeding the peer count?

hugod17:01:22

@bkamphaus: The peer count is the number of processes connecting to datomic?

hugod17:01:04

As far as I know this is the only process connecting, but I can check that.

Ben Kamphaus17:01:18

@hugod: right, i.e. on free or starter, transactor + two peers. When up, REST server and Console count against limit. You can also look through transactor logs, will see a message where they’re logged with datomic.transactor - {:event :transactor/remote-ips …}

hugod17:01:07

This is with datomic-pro, btw

Ben Kamphaus17:01:56

by default transactor logs will go to log/ subdir of the datomic dir with naming by date.

hugod17:01:31

I don’t see transactor/remote-ips anywhere in the logs

Ben Kamphaus17:01:25

Which version are you on? I would expect to see even a blank message, e.g.:

Ben Kamphaus17:01:33

2016-01-12 00:01:39.312 INFO  default    datomic.transactor - {:event :transactor/remote-ips, :ips #{}, :pid 15750, :tid 28}

hugod17:01:22

Would two transactors running on the same table show this symptom?

donaldball17:01:52

Thanks, @bkamphaus, I realize there are probably better data models that would yield a better txn design, but I’m quite curious now what apparent constraint of txn fns I’m violating hereby

hugod17:01:14

@bkamphaus: This is datomic-pro-0.9.4880

Ben Kamphaus17:01:29

@hugod: nope, two transactors can’t run live against the same table. With paid pro, only one transactor can be active, others will see it writing heartbeat and enter standby. With pro starter/free, transactors will constantly fail with AlarmHeartbeatFailed :cause :conflict.

Ben Kamphaus17:01:10

It may be that the remote-ips logging wasn’t yet included in that version (over a year old at this point), if you can consider upgrading transactor it should contain that logging info, should still be compatible with same peer lib version.

hugod17:01:07

@bkamphaus: Thanks, I’ll assume that having two transactors is the cause of the issue for now, and test with just the one transactor up.

Ben Kamphaus18:01:08

@donaldball: I can’t reproduce your failure using code in your gist - the transaction function works for me with two caveats: (1) I’m using namespaced datomic.api/q for query, and (2) I’m installing the db fn through standard process, not using your deftxfn, e.g.

{:db/id (d/tempid :db.part/user) 
             :db/ident :db.fn/increase-book-quality
             :db/fn #db/fn {:lang "clojure" 
                            :params [db eid attr quality] 
                            :code ...

donaldball18:01:21

Cool, thanks, I’ll dig in a little deeper then

domkm19:01:25

@bkamphaus: Related to my dev/mem issue yesterday, it seems like dev is swallowing the root cause of exceptions while mem is properly wrapping them. See: https://gist.github.com/DomKM/bedbb9ef2f281c1254fe