Fork me on GitHub
#datomic
<
2015-06-18
>
jthomson10:06:18

Has anyone encountered datom conflicts after restores? I.e. two entities with the same value for a :db/unique attribute.

jthomson10:06:17

e.g.

(d/q '[:find ?e ?unique-ident
       :in $ ?a ?v
       :where
       [?e ?a ?v]
       [?a :db/unique ?unique]
       [?unique :db/ident ?unique-ident]]
     (d/db (db/connection))
     :cat.block/key
     "designers22")

->

[[17592187845110 :db.unique/identity]
,[17592189560149 :db.unique/identity]]

marshall12:06:55

@jthomson: can you also return ?a in the :find spec and report that output?

marshall12:06:33

@jthomson: sorry, never mind, i missed that you were passing it in.

marshall12:06:48

@jthomson: what version of Datomic?

jthomson12:06:09

i'm pretty sure it's a real conflict, as when I try to retract the entity I get a two datoms conflict error.

jthomson12:06:32

datomic-pro "0.9.5130"

kbaribeau15:06:17

restores are a bit funny aren't they? I noticed yesterday that datomic will still do a differential restore even if I've delete-database'd the target db

jthomson15:06:01

hm, interesting

jthomson15:06:46

FYI in my case it this was due to trying to restore on divergent dbs (the target had some transactions that were not in the source)

jthomson15:06:02

which isn't supported at this point.

kbaribeau15:06:31

yeah, that definitely seems like it could get weird

kbaribeau15:06:07

if I really want to guarantee i've got two identical databases after a restore (like if I want my staging env to match prod exactly), I think I'd have to clean out everything related to my target db in the storage layer. does anyone have a decent way to do that in dynamo besides deleting the table? I've got several database in my dynamo table, but the raw data just looks like gibberish to me.

Lambda/Sierra15:06:32

There is no "differential restore"

Lambda/Sierra15:06:11

Restore always starts from an empty database.

kbaribeau15:06:32

I am confused then, the output I see is that most segments are skipped

kbaribeau15:06:59

"Copied 146 segments, skipped 2672 segments." etc.

marshall15:06:00

restore always checks the whole DB. it only copies when necessary

marshall15:06:15

differential backup doesn’t need to check the whole DB

kbaribeau15:06:52

"starts from an empty database" and "only copies when necessary" sound like conflicting statements to me?

marshall15:06:25

you can restore to an existing uri of the same DB (at a different t)

marshall15:06:52

but, as jthomson mentioned above, if you have a divergent DB, restoring to that existing DB is not supported

kbaribeau15:06:25

right, so if I have DBs that have diverged, and I want to resolve that, what do I do?

kbaribeau15:06:46

basically I want to force prod -> staging while ignoring anything from staging that has diverged

marshall15:06:02

restore to a new/blank DB in staging

jthomson15:06:21

So I guess this means use a different name? Thinking about it in my case (which was actually the same use case @kbaribeau) I had deleted the old DB but reused the name, thinking this would create a new db.

kbaribeau15:06:37

yup, same issue ^

marshall15:06:16

You need to call bin/datomic gc-deleted-dbs

Ben Kamphaus15:06:21

yep, different name won't work because db has unique id that will report conflict if you try to restore same db to different name in the same datomic storage (i.e. into the same bucket/table, w/e)

jthomson15:06:07

ah yes of course

bhagany16:06:46

curious about this, "* Fixed bug in Pull API when using selectors built from strings."

bhagany16:06:50

I reported a bug in which a scalar find spec (aka, :find (pull ?e [*]) .) returned an edn string over the rest api that didn't match what you get at the repl

bhagany16:06:54

is that what this refers to?

bhagany16:06:32

iirc, over the rest api, you would get a map per attr/value pair

bhagany18:06:00

okay, I updated datomic, and it appears that it's not yet fixed

bhagany18:06:16

also, I was wrong - you get a 2-vector for each attr/value pair

bhagany19:06:02

as though you called vec on the correct result