Is this the best way of doing sorting/pagination in Datomic?: https://groups.google.com/forum/?nomobile=true#!msg/datomic/NgVviV9Sw8g/nXm3Lvkk2dwJ

If I see a huge spike in DynamoDB WriteCapacity units used (20,000 instead of the usual 200), could that be the transactor re-indexing/re-partitioning/doing some other kind of maintenance work?

@teng, I think it depends on your requirements

if you return modest number of results (<10,000 ?) I'd do this:

- d/q [:find ?e] (only query the entity id) - sort - subvec using the offset/limit provided - then do further transformations (another query, or d/pull or map d/entity(

if you want to use d/q, to my knowledge that's the best you can do as it always returns results unsorted

(looking at my list, you probably also need to include some sort of sort key in the query)

Ok thanks @pesterhazy - looks very similar to what Francis Avila wrote.

@pesterhazy Spikes in DDB write usage are definitely present during indexing jobs. You can look at memory index metrics and/or the logs to get an idea if there is an indexing job running

@marshall if usage exceeds provisioned capacity, does datomic work gracefully in that case?

you will get backoff and exponential retry for a period of time. too much throttling will result in transactor termination

makes sense

we haven't seen transactor failures due to indexing spikes yet, so the retry mechanism seems to be working

@marshall thanks!

sure!

Hi, I'm trying to back up a datomic db and am running into an obscure error that ends with java.lang.NoClassDefFoundError: javax/xml/bind/DatatypeConverter... not sure where to go from here. Anyone have any suggestions? https://gist.github.com/jdkealy/948068e40a47861c5d84b097bcbf39d5

@jdkealy can you post the backup command you’re running (elide any sensitive info like s3 paths or db names if necessary)

bin/datomic -Xmx4g -Xms4g backup-db datomic:<ddb://us-east-1/my-datomic-new/test-table> <s3://datomic-backups/backup1>

are you running this from a clean unzip of the Datomic distribution?

i went through the steps to set up my transactor

this is on a ubuntu server that's currently running the transactor

My suspicion is a classpath issue. Can you try downloading and unzipping a fresh distribution somewhere else on the box and running from there?

Is it possible to get the last transaction id for an entity when using the pull syntax?

ok will do thanks marshall

@marshall just tried on a fresh unzip, same thing... how is the read / write being authenticated in AWS

it should be using IAM

if you have roles configured for that instance

if not you’d need to have creds exported into the env

http://docs.datomic.com/aws-access-control.html

got it thanks

It looks like the roles were configured using the configure script