Fork me on GitHub
#datomic
<
2016-08-07
>
podviaznikov03:08:32

quick question: is there terraform templates for setting up datomic on aws?

robert-stuttaford06:08:57

@podviaznikov: @mrmcc3 has Datomic + Terraform running.

mrmcc310:08:50

I’ve got a terraform module which from my understanding sets up a more or less identical system on AWS as the datomic scripts. It ended up being pretty succinct and readable.

mrmcc310:08:02

Only tedious bit was the transactor bootstrap/userdata script which i converted to a terraform template. Because I essentially copied the userdata script produced by datomic scripts I wasn’t sure if I could share it publicly.

codonnell13:08:51

is it possible to connect to a datomic database while the transactor is unreachable?

yonatanel14:08:15

@codonnell: Not when using dev protocol

codonnell14:08:56

@yonatanel: my database is on RDS; sql protocol

yonatanel14:08:36

@codonnell: I honestly don't know. I just tried now with dev protocol. Even if you tried and it worked with RDS, I don't know what the nuances are, so better wait for an authoritative answer

codonnell14:08:37

@yonatanel: appreciate the attempt

yonatanel14:08:38

@codonnell: Not sure how helpful this is, but here it says "Transactor does not participate in queries. Peers with no transactor connection can still do reads from storage": http://tonsky.me/blog/unofficial-guide-to-datomic-internals/

codonnell14:08:58

@yonatanel: yeah, I think I heard that somewhere in the datomic tutorial videos. That page looks like a nice resource, though. I'll definitely look through it. If I try to connect to the database from a box that can't reach the transactor, I get an exception (reasonable behavior). I just want to know if there's a way to get my peer to ignore the transactor and set up a kind of "read-only" connection.

robert-stuttaford14:08:29

@codonnell, @yonatanel: peers connect to storage first, which has the connection details for the primary and backup transactors. this is how failover is possible 🙂

robert-stuttaford14:08:51

have you tried connecting to storage with your transactor down, and issuing only queries?

codonnell14:08:44

@robert-stuttaford: Yeah, I tried connecting to storage, and I got CompilerException clojure.lang.ExceptionInfo: Error communicating with HOST x.x.x.x on PORT 4334

robert-stuttaford14:08:49

i don't think it'll work, but it's worth trying. when first connecting to a database, peers have to grab the latest in-memory live indexes, which the transactor has, but storage does not. this may prevent reads against 'now' dbs. you may be able to read from the past

codonnell14:08:15

I didn't know that live indexes lived in the transactor; that's good to know.

robert-stuttaford14:08:29

the live indexes have to live there (and in all peers) because they haven't made it to storage yet. that's what indexing's job is: integrating all live index into storage

robert-stuttaford14:08:37

also memcached if you have one connected

codonnell14:08:26

@yonatanel: this blog post is really fantastic; thanks for the link

yonatanel14:08:58

@codonnell: yep. kinda makes you wanna write an open source clone ;)

marshall14:08:56

@codonnell: You can’t start up a connection to a Datomic DB without access to the transactor. It is possible that under certain circumstances, loss of connectivity to the transactor will not prevent peers from running queries, but this is not guaranteed behavior, and it will depend on specifics of storage/config/deployment/etc

robert-stuttaford14:08:51

can confirm that queries continue on fine during transactor outages

robert-stuttaford14:08:13

just wasn't sure about starting new connections

codonnell14:08:45

@marshall: Alright, thanks for the definitive answer. Given what @robert-stuttaford mentioned about live indexes living on the transactor and each live peer, it makes a lot of sense.

pesterhazy17:08:52

@mrmcc3: any chance you could put that up as a gist or on github? that would be tremendously helpful

pesterhazy17:08:31

I'm not too comfortable with the default CF-based set up where you can't ssh into the machines and tail logs in realtime

podviaznikov19:08:04

does anyone have problem with getting datomic license via email? Registered yesterday, but didn’t get email yet.