Fork me on GitHub
#datomic
<
2018-04-09
>
zlrth01:04:25

I'm starting datomic via this command:

bin/run -m datomic.peer-server -h localhost -p 8998 -a myaccesskey,mysecret -d hello,datomic:
per a tutorial. i think i have to specify what logback.xml file i'm using. what's the flag for that? I'm looking for something like this:
bin/run -L ./bin/logback.xml -m datomic.peer-server -h localhost -p 8998 -a myaccesskey,mysecret -d hello,datomic:
Context in case I'm going about this the wrong way: In cider-repl. I want logging to cider-repl to be less noisy; right now it's at least 30kb of logging per datomic repl command. I have commented-out sections of the logback.xml file in the bin directory of datomic-pro-0.9.5561, but they're not showing in my cider-repl. And as a test, I changed this line
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level %-10contextName %logger{36} - %msg%n</pattern>
to this
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level %-10contextName %logger{36} - testfoobar - %msg%n</pattern>
and didn't see testfoobar in my cider-repl logging, so I think the run datomic program isn't picking up my logback.xml file.

zlrth01:04:12

If there's comprehensive for Datomic pro starter, I'm happy to read it. I couldn't find it. Thanks everyone!

Alex Miller (Clojure team)02:04:32

logback.xml is probably loaded as a resource so you should just put it on your classpath

zlrth16:04:06

Thanks for your response! I thought I had a much more tricky problem than I did. I had put logback.xml in my classpath, but it wasn't picking up my modified logback. Turns out, there was already a logback.xml in my project, which was clobbering my modified one. Silly problem!

caleb.macdonaldblack12:04:47

Any way to pull the tx-instant from an entity’s db/id using the pull or pull-many function?

gcast12:04:21

im guessing putting the ident :db/txInstant in the pull pattern doesn't work?

robert-stuttaford13:04:37

@caleb.macdonaldblack no, neither entity nor pull support access to the ‘t’ in ‘eavt’. you need to use d/q, d/datoms, or d/tx-range. entity and pull start with ‘e’ and give you ways to discover ‘v’ through ‘a’. ‘t’ is never an ‘v’ in this scheme.

Chris Bidler14:04:12

Do I understand correctly that a consequence of this would be that if your schema doesn’t expose synthetic timestamps for things (e.g., if you rely upon :db/txInstant as a proxy for “when did thing happen” instead of explicitly storing a :thing/created-at datom), then you cannot use the Datomic Client API to know about when things happened in your db?

Alex Miller (Clojure team)14:04:25

no, you just need to query for it, not pull

Alex Miller (Clojure team)14:04:25

but I think it’s also helpful be clear in your thinking about the difference between “when did thing happen” and “when did I record that thing happened”

Chris Bidler14:04:10

ah, I see, thanks for the clarification

Alex Miller (Clojure team)14:04:16

it can be a useful thing to conflate the two, but don’t forget that you’re doing so :)

Chris Bidler15:04:14

I am very clear in the difference between the two, but I also have inherited a medium-sized schema where the decision to conflate the two things was made long ago. I think it’s a reasonable choice for the data at hand since the events in question don’t have a need for high resolution to wall-clock time, but I like to keep a bead on what all we can and can’t do with Client

Chris Bidler15:04:25

(for example we have a lot of logic in DB fns, so I’m keeping my on-prem-on-AWS footprint up-do-date and waiting to see what Cloud will offer to handle them - this restriction would have been another thing I had to know about while doing the Indiana-Jones-sandbag-and-idol dance trying to decide when to migrate to Cloud :D)

Alex Miller (Clojure team)15:04:21

well if you haven’t, that’s probably something to talk to @marshall about

Chris Bidler15:04:32

It’s on my list. I’m very interested in Datomic Cloud but I have other dragons to slay before I get to the point of looking at upgrading our stack.

wotbrew15:04:08

Hi, I'm not running datomic - I am interested in some consistency edge cases around GC and excision. What happens if your db 'value' holding one root pointer see's GC'd or excised nodes? Do you get an exception? Do you just not see the datoms? Lets say i (def dbval (db conn)) and then run a GC job or excise some data, several days later, I seek all datoms in dbval - what happens?

wotbrew15:04:36

I hope somebody just knows because it would be a pain to try it out 🙂

marshall16:04:11

@danstone you get an exception

marshall16:04:29

that is why the gc-storage API has an ‘older than’ date argument

marshall16:04:37

“The reason that garbage is not deleted immediately on creation of a new tree is that not all consumers will immediately adopt the latest tree. Garbage collection should not be run with a current or recent time, as this has the potential to disrupt index values used by long-running processes. Except during initial import, garbage collection (gcStorage) older-than value should be at least a month old.”

PB16:04:17

Hey all. Inside of my tx-data of a transaction I have a list of datoms. I.e. #datom[17592220704601 134 17592186149960 13194174193496 true]. I realise I can retrieve the eav as well as the tx. My understanding is that the true denotes as to whether it was actually transacted. My question is, how do I get at that value?

marshall16:04:39

@petr (get datom :added)

marshall16:04:49

or (:added datom)

PB16:04:03

Is there documentation around this? I couldn't find it

PB16:04:10

Great, thank you

bmaddy18:04:03

Does anyone know of an easy way to do something like d/touch but include all reverse references? Context: I find myself typing (inspect-tree (d/touch (d/entity db 1234))) and would like to be able to traverse reverse references instead of having to look them up.

marshall19:04:23

@bmaddy The set of reverse references from any entity is an open set (since any entity can have any attribute). I’d probably use the Datoms API and the VAET index to find what you’re looking for

bmaddy19:04:48

Ok, I'll probably look into that. Thanks @marshall.

Chris Bidler19:04:33

I have designed a cross-region failover capability like so: - backups are taken every 10 minutes automatically to an S3 bucket in us-east-1 - those backups are copied to a different bucket in us-east-2 by cross-region-replication Now I am trying to exercise it: - create new, empty DynamoDB table in us-east-2 - bin/datomic restore-db <the us-east-2 bucket> <the new, empty table> (<optionally, a t-value from bin/datomic list-backups)

Chris Bidler19:04:41

I see the restore fail like this:

Chris Bidler20:04:22

bin/datomic restore-db  datomic: the-t-value
Copied 0 segments, skipped 0 segments.
Copied 0 segments, skipped 0 segments.
java.util.concurrent.ExecutionException: java.lang.Exception: Key not found: 583bfcc4-3b77-4dec-b0ba-2c0bda16dbed
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at clojure.core$deref_future.invokeStatic(core.clj:2292)
	at clojure.core$future_call$reify__8097.deref(core.clj:6894)
	at clojure.core$deref.invokeStatic(core.clj:2312)
	at clojure.core$deref.invoke(core.clj:2298)
	at datomic.backup_cli$status_message_loop.invokeStatic(backup_cli.clj:24)
	at datomic.backup_cli$status_message_loop.invoke(backup_cli.clj:18)
	at datomic.backup_cli$restore.invokeStatic(backup_cli.clj:53)
	at datomic.backup_cli$restore.invoke(backup_cli.clj:43)

Chris Bidler20:04:07

restores from the backups in us-east-1 appear to work okay (I haven’t let one run to completion yet but they do at least begin to copy segments)

Chris Bidler20:04:56

Does this likely mean that all my backups in Ohio are corrupted somehow? Is there something about how backup-db puts data into S3 that would not be amenable to CRR?

Chris Bidler20:04:23

to be clear, bin/datomic list-backups shows the same set of t values available in both the us-east-1 and us-east-2 buckets, but restoring from the us-east-2 bucket for any of the t values I’ve tried leads to the above error (including the key that is not found being the same key each time)

marshall20:04:55

sounds like the cross-region replication hasn’t completed

marshall20:04:03

or had som kind of failure