This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2015-08-05
Channels
- # admin-announcements (2)
- # beginners (20)
- # boot (13)
- # cider (36)
- # cljs-dev (11)
- # clojure (31)
- # clojure-berlin (3)
- # clojure-czech (2)
- # clojure-dev (2)
- # clojure-japan (9)
- # clojure-russia (16)
- # clojurebridge (3)
- # clojurescript (182)
- # cursive (8)
- # datascript (2)
- # datomic (35)
- # editors (33)
- # hoplon (18)
- # ldnclj (11)
- # off-topic (6)
- # re-frame (3)
- # reagent (39)
when backing up to S3 — is there a way to specify a region?
Apparently not needed ☺️
yeah, S3 is not region specific as far as i’m aware
domkm: the real hard part there is modeling user permissions at all. Shit is incredibly hard to do well imo
tcrayford: Agreed. I was thinking that Datomic might enable some interesting patterns where permissions could be modeled very granularly on the entity and attribute level.
Does anyone have any pointers to discussions of pro/cons of various storage services? Or is it more the case that you’ve got to first weigh the pros/cons of things like riak vs. cassandra vs. postgresql on their own merits?
Aside from configuration/administration/cluster issues, I’m sort of wondering if there are any base performance / capability comparisons of the underlying storage services.
damionjunk: from my perspective: datomic's gonna be faster on riak/cassie/dynamo than one of the sql stores
tcrayford: I was thinking so as well. especially given Riak’s similarity to Dynamo. I was just wondering if the extra administration effort was worth it. I guess I’ll actually have an opportunity to try SQL and Riak, this is a greenfield project, and step one is a data-load-throughput simulation anyway.
@damionjunk: as for comparing riak/cassie/dynamo: they have pretty different sets of tradeoffs. Dynamo is obviously completely commercial, the other two are open source. Cassie seems to be better maintained these days to me (disclosure: I use to know a bunch of folk who worked at the company that makes riak). I use riak, but then I use it for other things as well
this may be a bit noob’ish, but if you’re not actually running a 3+ node cluster, would Riak still make sense as a storage service?
@damionjunk: My take is: If you're using AWS, use Dynamo. Can't be beat. If you're on-premise, pick whatever distributed storage you have the most experience / comfort with.
@stuartsierra: so, if you’ve not no experience with distributed storage on premise, would Postgres? I’m leaning towards Riak, because it’s something additional to ‘learn’, and I’ve got the time to do it though.
There’s nothing “special” about the data, it’s entirely read heavy, medical sensor data, eventually around 200billion “rows"
@damionjunk: Well then that's another issue you have to contend with first: sharding those 200 billion rows. Even Datomic can't fit that much in a single transactional database.
yeah, that’s a projection, but initially, the data set is in the 100 million range
it’s entirely domain partitionable, afaik. (i’ve actually yet to see the full “plan” unfortunately)
at any rate, @stuartsierra @tcrayford , thanks for the replies. I’ll definitely be reporting back.
@domkm: i have interesting stories to tell around user permissions
Any one of the distributed storages has a maintenance burden, and you can lose your data if you don't configure things like replication correctly. Make sure you understand how your storage works, and test for failure scenarios!
it’s a source of considerable complexity for us
can’t right now. wonderful south african power cuts are landing in 20 minutes. let’s chat on skype some time?
domkm: at last year's conj Lucas Cavalcanti & Edward Wible gave a presentation called "Exploring four hidden superpowers of Datomic" which touched on access control as one of the four. https://m.youtube.com/watch?v=7lm3K8zVOdY
@damionjunk: I would not use Riak with less than 5 nodes.
just a sanity check, @bkamphaus, but we definitely want to down all our peers when restoring databases, right? 😁
it’s not like when upgrading the transactor version with the double-failover trick
@robert-stuttaford: correct — if you’re using a storage other than dev/free, should take txor down as well, see http://docs.datomic.com/backup.html#other-storages
this’d be RDS -> DDB
so the txor would go down and back up thanks to that we’re using CFN and whatnot