Clojurians Log v2
datahike
hello, we’re thinking of implementing a Datahike DynamoDB backend. Just wanted to check how come it doesn’t exist yet? Do you think it’s a good idea and reasonably easy to do?
@stathissideris There is now https://github.com/replikativ/datahike-dynamodb
It probably needs some experimentation and tuning with write throughput and capacity planning still. It stalled on a large transaction for me yesterday and did not propagate the error properly it seems. This still needs fixing. But if you have feedback and are willing to check it out that would be appreciated.
Thank you for this! It’s been pretty hectic for me lately so I can’t promise anything…
Hi, there was one konserve-dynamo backend: https://github.com/csm/konserve-ddb It's quite old so it doesn't reflect the current api of konserve but it should be easier now since the api got leaner. I did implement a backend for cassandra a while ago and it wasn't bad.
the datahike-backend is just a small wrapper then
so you’re saying it’s a small step from that to datahike with a dynamodb backend?
yep
ok thanks!
👍 If you need help, this is the right place. You can look at konserve-jdbc and konserve-s3 for an up-to-date implementation
redis as well https://github.com/replikativ/konserve-redis/tree/main
thank you very much!
@stathissideris I had started working on dynamoDB a few years back then gave up when I came across the fact that the reads are eventually consistent by default and cost more to be strongly consistent. It's something worth keeping in mind in your implementation. https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.ReadConsistency.html
@alekcz360 Right, you will have to use proper quorums, depending on how DynamoDB handles things. @stathissideris Lmk if I can help, I think this would be very useful and it should not be too hard to do. The S3 backend was fairly simple and it already supports multiple stores per bucket, which you can maybe ignore to get started.