Is there any way to store “ordered” values on a card/many attribute?

Or do you have to add an entity with and :entity/order attribute?

@zentrope: Nope, card many is unordered.

@bkamphaus: what method can i use to identify which transactor in a HA pair is the primary?

@zentrope: https://clojurians.slack.com/archives/datomic/p1450042662000836

Since DynamoDB is based on Amazon cloud service, we do not want to save the data there. Although DynamoDB has a local version, but obviously that's not recommended for production. So except DynamoDB, which storage server is recommended or perform better considering the load balance of peers?

you can use clustered postgresql, or the fault tolerant versions of any of the storages available, basically

@robert-stuttaford: thanks, robert, read this article: http://martintrojer.github.io/clojure/2015/06/03/datomic-dos-and-donts/, it's said DynamoDB is the best choice, since, we decide to not to use it, what do you think of Riak?

I am not aware of any testing with Datomic on clustered PostgreSQL.

i’ve only got experience with postgres (a single node) and ddb. never used the rest

ah, thanks for correcting me stuart.

It might work (clustered Postgres), I just don't know if anyone has tried it.

Support for SQL as a storage engine is mostly targeted at organizations that already have substantial investment in SQL infrastructure.

that’s as i understood it as well - which i took to mean 'at scale'

from wiki: https://en.wikipedia.org/wiki/Riak, Riak implements the principles of AWS's DynamoDB paper

Performance of Datomic will be roughly similar on any of the distributed storage engines. Cassandra is probably the most popular distributed storage among Datomic users, after DynamoDB.

Whichever one you choose, read the Datomic docs carefully — improperly configured storage can lead to data loss.

@stuartsierra: Thanks for recommendation and advices.

We briefly used clustered Postgres before switching to DynamoDB. We had no issues with Datomic, only with managing clustered Postgres.

We also tried Riak. Again, no issues with Datomic. Managing 5 Riak nodes was difficult for us.

@statonjr: Thanks for sharing experiences.

the advice shared at Datomic Conf this year -- which seems both eminently wise and supported by comments here -- is to use the storage you already know. There is probably some performance differential in the limit, but operational issues with a new storage backend are far more likely to bite you, and more badly (= data loss).

@curtosis: OK, I will keep it in mind, thanks for advices.

does anyone know of a hosted version of Datomic? I love it but would prefer not to take on the ops overhead. Looked around and couldn’t find anything out there

does the license allow it?

no

Is this something Cognitect would think about providing at some point?

@davebryand: we are aware of the request for a hosted Datomic. I’ll note that there’s additional interest being expressed in it (you’re welcome to add your voice to the group post as well). https://groups.google.com/d/msg/datomic/Vcx7LtaK65U/-rVfFLrILPQJ

In the mean time, happy to gather feedback on whether you think the ops overhead looks intimidating due to intrinsic reason (i.e. just running all the pieces), or if there are specific things unclear in docs, etc. that you’ve had trouble with.

thanks ben—just posted

After all is said and done, I probably will end up taking on the ops overhead because I truly love Datomic.

I had a bunch of trouble where I didn’t understand the side effects of things like ensure-transactor, for instance. When I initially ran it, I didn’t have an s3 log bucket uncommented. I got everything up and running and then decided I wanted one. Here is are my notes from that experience.

Basically my transactor just kept getting terminated and restarting and I had no logs to look at. It would have also been helpful to SSH into the transactor but appears to not be on that AMI

right, the AMI can’t be ssh’d into. Logs can be helpful, a lot of problems can be addressed with metrics though of course if problem is getting started in the first place, logs won’t rotate and metrics won’t be put.

Anyway, it was a rocky few days to get things running but I think it’s stable now

right, no logs, no help

Not sure if you reviewed this section of the docs, but the manual setup walks through everything ensure does for you: http://docs.datomic.com/storage.html#automated-setup

It handles role, transactor, and permissions for table and s3 access but doesn’t create the bucket.

In terms of where the boundaries are (i.e. if it’s not listed in manual setup there, ensure-transactor doesn’t do it for you).

Net net is that I think Datomic is the PERFECT database for most startups to use and I want less friction for my friends to try it

ahhh, that’s super helpful—thanks

I did see that but didn’t read it closely

But yeah, rolling your own stuff on AWS is definitely tricky, and navigating the docs for the different possibilties AWS + Dynamo, AWS + something else, on site + something else, mix and match AWS and Heroku, etc. — all of the different blends out there — is tricky.

Of course now that I’m on the other side of the experience it feels much less tricky, but yes, it was. Thanks for your help!

@bkamphaus: does the Datomic license prohibit third parties providing hosted datomic?

yes

running hosted databases is hard work

(it's not actually as hard as you might think though. We have much better automation than the average devops person at a startup)

is datomic easy to administer on heroku? i.e 3 commands and you’re up and running like heroku postgres ?

noooooooooooooooooooooooooooooo

well I guess I have my answer

that’s good to know. Is there any plan for it in the future? Or what’s the easiest way to administer it with “no hands"

scroll up, but there's no currently available option for that

the license prevents third parties from doing that for cognitect, and they haven't released or publicly talked about doing hosted datomic at all

With all this talk about storage, there's one option I rarely see mentioned: the file system. Given that you take backups, is there a good reason to avoid file storage? We're not on AWS, so should I set up Cassandra just to have a "real" data store?