Fork me on GitHub
#datomic
<
2015-08-05
>
martinklepsch09:08:06

when backing up to S3 — is there a way to specify a region?

martinklepsch09:08:00

Apparently not needed ☺️

robert-stuttaford09:08:11

yeah, S3 is not region specific as far as i’m aware

domkm13:08:34

Are there any good resources for modeling user permissions with Datomic?

tcrayford13:08:56

domkm: the real hard part there is modeling user permissions at all. Shit is incredibly hard to do well imo

domkm13:08:11

tcrayford: Agreed. I was thinking that Datomic might enable some interesting patterns where permissions could be modeled very granularly on the entity and attribute level.

damionjunk13:08:55

Does anyone have any pointers to discussions of pro/cons of various storage services? Or is it more the case that you’ve got to first weigh the pros/cons of things like riak vs. cassandra vs. postgresql on their own merits?

damionjunk13:08:34

Aside from configuration/administration/cluster issues, I’m sort of wondering if there are any base performance / capability comparisons of the underlying storage services.

tcrayford14:08:35

damionjunk: from my perspective: datomic's gonna be faster on riak/cassie/dynamo than one of the sql stores

tcrayford14:08:45

(and more resilient)

damionjunk14:08:49

tcrayford: I was thinking so as well. especially given Riak’s similarity to Dynamo. I was just wondering if the extra administration effort was worth it. I guess I’ll actually have an opportunity to try SQL and Riak, this is a greenfield project, and step one is a data-load-throughput simulation anyway.

tcrayford14:08:17

@damionjunk: as for comparing riak/cassie/dynamo: they have pretty different sets of tradeoffs. Dynamo is obviously completely commercial, the other two are open source. Cassie seems to be better maintained these days to me (disclosure: I use to know a bunch of folk who worked at the company that makes riak). I use riak, but then I use it for other things as well simple_smile

damionjunk14:08:10

this may be a bit noob’ish, but if you’re not actually running a 3+ node cluster, would Riak still make sense as a storage service?

Lambda/Sierra14:08:05

@damionjunk: My take is: If you're using AWS, use Dynamo. Can't be beat. If you're on-premise, pick whatever distributed storage you have the most experience / comfort with.

damionjunk14:08:20

@stuartsierra: so, if you’ve not no experience with distributed storage on premise, would Postgres? I’m leaning towards Riak, because it’s something additional to ‘learn’, and I’ve got the time to do it though.

damionjunk14:08:51

There’s nothing “special” about the data, it’s entirely read heavy, medical sensor data, eventually around 200billion “rows"

Lambda/Sierra14:08:51

@damionjunk: Well then that's another issue you have to contend with first: sharding those 200 billion rows. Even Datomic can't fit that much in a single transactional database.

damionjunk14:08:19

yeah, that’s a projection, but initially, the data set is in the 100 million range

damionjunk14:08:08

it’s entirely domain partitionable, afaik. (i’ve actually yet to see the full “plan” unfortunately)

damionjunk14:08:44

at any rate, @stuartsierra @tcrayford , thanks for the replies. I’ll definitely be reporting back. simple_smile

robert-stuttaford14:08:53

@domkm: i have interesting stories to tell around user permissions

Lambda/Sierra14:08:56

Any one of the distributed storages has a maintenance burden, and you can lose your data if you don't configure things like replication correctly. Make sure you understand how your storage works, and test for failure scenarios!

robert-stuttaford14:08:03

it’s a source of considerable complexity for us

domkm14:08:13

robert-stuttaford: Please share simple_smile

robert-stuttaford14:08:48

can’t right now. wonderful south african power cuts are landing in 20 minutes. let’s chat on skype some time?

domkm14:08:03

robert-stuttaford: I'd love that! Thanks. I'll email you.

wasser16:08:00

domkm: at last year's conj Lucas Cavalcanti & Edward Wible gave a presentation called "Exploring four hidden superpowers of Datomic" which touched on access control as one of the four. https://m.youtube.com/watch?v=7lm3K8zVOdY

domkm16:08:32

wasser: Thanks! I'll check it out.

akiel17:08:59

@damionjunk: I would not use Riak with less than 5 nodes.

robert-stuttaford20:08:10

just a sanity check, @bkamphaus, but we definitely want to down all our peers when restoring databases, right? 😁

robert-stuttaford20:08:30

it’s not like when upgrading the transactor version with the double-failover trick

Ben Kamphaus20:08:31

@robert-stuttaford: correct — if you’re using a storage other than dev/free, should take txor down as well, see http://docs.datomic.com/backup.html#other-storages

robert-stuttaford20:08:33

this’d be RDS -> DDB

robert-stuttaford20:08:54

so the txor would go down and back up thanks to that we’re using CFN and whatnot