Fork me on GitHub
#datomic
<
2016-08-17
>
danielstockton08:08:05

Are there any interesting strategies for dealing with data spread over geographical regions? The single transactor seems like a weakness here, although it's easy to replicate the storage for reads.

danielstockton08:08:56

Just accept slightly slower writes? Separate DBs for different regions and accept slower reads when you have to query across DBs from other regions?

danielstockton08:08:18

Those are the obvious trade-offs I can think of

danielstockton08:08:14

Wait, in fact you don't have to accept slower reads if storage is replicated...That seems like the best choice, having different DBs per region and querying across multiple DBs?

danielstockton08:08:45

Use case: I have an application served from multiple regions all making writes and a cron job in the background which needs to compile an XML using data from all regions to send off to a third party..

val_waeselynck09:08:27

@danielstockton: I would definitely try to see if I can live with slow writes and 1 transactor before jumping to geographical sharding 🙂

danielstockton09:08:23

@val_waeselynck: yeah, my application isn't very write heavy, that would probably be just fine

robert-stuttaford10:08:39

@danielstockton: do you need transactionality across regions? or can you consider them to be sharded from each other?

robert-stuttaford10:08:47

i would definitely try 1-transactor first, and see what the latency does to reads and writes from other regions

val_waeselynck10:08:15

I know that's what my SQL-users friends do: they distribute their read replicas and have one master write replica

val_waeselynck10:08:57

And this is so much easier to do with Datomic that we have no excuse for not trying it 🙂

val_waeselynck10:08:44

Surprisingly, low-latency writes is not such a common requirement

danielstockton10:08:42

i can consider them to be sharded, i think, but i do need to be able to see a full view of the data from the background job

danielstockton10:08:04

the requirements are forever changing, which makes it difficult

danielstockton10:08:20

one thing that worries me is making a bad choice that is hard to reverse

vinnyataide12:08:03

how to create a nil reference when the entity has a cyclic reference?

vinnyataide12:08:22

of cardinality one

danielstockton12:08:00

you can just omit the attribute

vinnyataide12:08:12

hm, that's odd, it's saying that I'm inserting a string type, even when I ommit

vinnyataide13:08:18

seems like an odd change in the schema that made my db react

vinnyataide13:08:25

just deleted it and now its ok

cezar21:08:08

I'd like to use Datomic on Postgres but I have a deep concern about it using one table with a Primary Key (which means a BTree index in postgres). Is the uniqueness constraint actually necessary for it to work or could I create the datomic_kvs table without declaring the id column as the Primary Key and instead creating a hash index for it?