Fork me on GitHub
#datomic
<
2015-07-28
>
ljosa14:07:57

Does anyone here use Datomic with multiple Couchbase clusters and cross-datacenter replication (XDCR)?

Lambda/Sierra18:07:40

@ljosa: Datomic is, in general, not designed to support cross-datacenter operation with one Transactor pair.

Lambda/Sierra18:07:40

Cross-datacenter replication strategies usually allow data to diverge between the two datacenters, with some kind of arbitrary rule for conflict resolution. This is not a strong enough guarantee to preserve Datomic's consistency model.

Lambda/Sierra18:07:17

For example, for Couchbase, http://docs.couchbase.com/admin/admin/XDCR/xdcr-architecture.html 'XDCR … provides eventual consistency across clusters. If a conflict occurs, the document with the most updates will be considered the “winner.” '

ljosa19:07:08

Hmm … but doesn’t Datomic store immutable segments, always with new keys?

tcrayford19:07:45

not for the roots

tcrayford19:07:57

the roots require CAS or consistent put

tcrayford19:07:08

(they just contain uuids to the immutable segments afaik)

Lambda/Sierra19:07:30

Yes, as @tcrayford says, there is one important piece that is not immutable: the pointer to the "root" of each database value.

tcrayford19:07:57

ack, wrong term s/root/"pointer to the root"/g

tcrayford19:07:14

there are afaik like 4-6 or something of those as well, not just a single thing

Lambda/Sierra19:07:28

Also, the immutable segments are nodes in a tree structure… if the tree has a new root but not all the leaves have been replicated across the datacenters, you would see inconsistent results. Datomic doesn't allow this, so it would appear as unavailability.

tcrayford19:07:45

(uh: db heartbeat, log tail, log, indexes, gc)

Lambda/Sierra19:07:37

Basically, you can't get Datomic's strong consistency guarantees and cross-datacenter (or cross-region) replication at the same time. simple_smile

tcrayford19:07:47

physics, a thing

ljosa19:07:58

I believe conflicts cannot happen in this case because the replication is one-way from the cluster that Datomic writes to. But I see that point that Datomic will be confused if the mutable documents are updated in the wrong order or if the leaves of the tree are delayed. Do you know how Datomic would react in such cases? Would it throw an exception? (That might be OK: from playing with Datomic and XDCR, it seems that replication delays are usually masked because recent datoms are cached in the memory index, which is transferred directly from the transactor to the peers.)

ghadi19:07:00

are entities comparable?

ghadi19:07:53

like if I access a ref (through navigation) on two different database values.

ghadi19:07:04

I should just test it out... but chat room

Lambda/Sierra19:07:04

@ljosa: In general Datomic will always prefer an error to returning inconsistent results. But you should be aware that cross-datacenter replication is not a supported use case so anything it does is, by definition, undefined behavior.

ljosa19:07:27

understood. thank you for good answers.

ljosa19:07:47

I suppose we’ll have to get by with a single Couchbase cluster in a single AZ and hope that caching in the peers together with the memory index is enough to smooth over AWS glitches.

ljosa19:07:47

I suppose Datomic must be using strongly consistent reads when it’s running on DynamoDB?

tcrayford19:07:16

for the pointers to roots, yeah, CAS too iirc

tcrayford19:07:23

(I think dynamo supports it)

arohner20:07:07

yes, dynamo supports strongly consistent reads

arohner20:07:25

some systems (not sure if dynamo is one), only have problems w/ consistent reads when (ab)using mutability

arohner20:07:52

if you never update-in-place, it will give you correct results for “give me the segment w/ this UUID”, even when using eventual consistency