You're going to have to be more specific - not all http calls were made equal. Is your api for submitting data, reading data or performing some calculation? This can inform your solution

If it's just submitting data throw it in a queue and let someone else handle it. If it's for reading, push the processes data to the edge as much as possible. If it's for calculations and transactions you get do "enjoy" async programming

You’re right, was a little sparse on information. 99% is a HTTP API accessing XTDB. Writing data is putting documents, reading data is mostly getting docs by their ID. When submitting data, we validate the payload accept the call, return an ID before the async save operation is complete. If the write is not successful afterwards, it’s not that big of a deal. Currently everything happens in the same instance, but we already set up some kind of request dispatch which could be extended to different machines asynchronously. There are very few operations which do calculations and data processing, and these calculations are usually returning fast. For reads we return the documents and try to make use of the in memory key/value store as much as possible Edit: Thanks for the reply!

This is pretty specific to xt's implementation details, are chunks cached in memory like with datomic?

I’m not all to familiar with Datomic but that’s my understanding. There is an index store (in memory in our case, if it grows LMBD/RocksDB on a volume) which syncs with the transactions of the DB itself, so most read calls will not need network i/o from our application. From my understanding we are quite close to your suggested answer “If it’s just submitting data…” with those concepts

Don’t forget to look at other things than just the web stack. For instance among other things, we’ve hit IO EBS limits on our AWS stack once and thinking it was our stack. Also scaling load testing itself is quite hard, but necessary to draw conclusions on “improvements”

Thanks for the reminder! Infrastructure is a big topic on our list, incl. observability, scaling, cloudfront etc. Lots to read 😉

https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/ are also interesting concepts when scaling out with multiple instances (Look into Hystrix and related sources)