Clojurians Log v2

Maybe @viesti can share the way how to correctly initialise Clojure code from Java to get snap start working?

What is the exact error? In the example I did, there should not be anything particular https://github.com/viesti/clj-lambda-datahike/blob/main/src/clj_lambda_datahike/handler.clj, but maybe there is something different in this case. I haven't followed closely the thread, but on top of my head, DynamoDB has a batch API, https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_BatchWriteItem.html, for writing with less roundtrips, TransactWriteItems is more about supporting different types of operations in one go (so if you are only writing, suggest using BatchWriteItem). Testing on EC2 (for being in close to the network that dynamodb service is) is easier than in lambda, I agree 🙂

No, wait.. I am about snap start. Or I got confused and this topic is not to you?

Okay. I figured out about Snap Start. It was related to my publishing mechanism in .tf scripts.

But I encounter another big bug. What happens on attempt to connect (DynamoDB backend)? Functions database-exists? and connect are doing something for unbelievable ~10 seconds! Can it be realted to :allow-unsafe-config options? I am using it because IP of Lambda could be different on every run and it was throwing exception...

probably good to check out snapstart docs https://docs.aws.amazon.com/lambda/latest/dg/snapstart-best-practices.html, maybe disabling java DNS cache might help

I don't think this is related to allow-unsafe-config. Basically you only do a read from the store. If you don't have an EC2 instance at hand to check whether it works well from the REPL then I can try later myself.

I would test both the konserve get and assoc latency first and then the high-level API latency.

You also will have to warm up the JIT first on the JVM.

(if you measure things with latencies in milliseconds that is)

@whilo So. Good news are that backend works. Bad news about performance... around one second for hundred of Datoms... need to check algorythms, because metrcs in AWS shows 3-4ms of average latency on PutItem operation (15-25ms maximum). Maybe better to use TransactWriteItemsRequest ? Is it possible in konserve logic? Such request can write and delete items in one request, but need to check quantity (not more then 100 items)... or maybe something else is wrong. I will take a look more, but later

That is weird. I would test konserve directly and see how long individual operations cost in there first. Datahike needs two roundtrips at least per transaction.

(or rather per commit, it can do multiple transactions per commit)

And snap start doesn’t work. That sucks, I am so tired of it already(

This is why suggested to first test the datahike backend in an EC2 instance with the REPL, then we can do some benchmarking to see whether it even would work for the native image. The native image shrinking is still valuable, I am trying to figure out how to get the binary to be small.

I will try to figure out what is wrong with snap start at first. If this feature is officially supported and people are using it there is no real reason to cold start for ten seconds in my case 🤦‍♂️

Right