Fork me on GitHub
#datahike
<
2022-11-24
>
Tom H.23:11:26

Hey folks, I’m looking into datahike as an alternative to Datomic. If I have multiple instances of my back-end (usually just 2) and they both are making transactions to a postgres-backed datahike db via datahike-jdbc is that likely to cause issues?

timo11:11:43

Unfortunately so, currently @U1C36HC6N is thinking about solving distributed Datahike connection management afaik. In case you are transacting into the same database the index of the Datahike instance that does not transact is not updated. That's the problem right now I think. But others might be more educated to answer that question. However, we had some trials with Apache Pulsar as event broker that replicates the transactions into mulitple Datahike instances each having there own storage. https://github.com/replikativ/datahike/pull/281

👍 1
Tom H.01:11:41

Ahh gotcha, so datahike is similar to Datomic in that the indexes live in memory. I’m still learning these technologies so apologies if this doesn’t make any sense but with a backing store like postgres would it be crazy to store the indexes in there as well?

whilo06:11:48

The indexes are always durably stored, but can be cached in runtime memory. The important asepct about Datomic is that it is a copy-on-write memory model, i.e. it data is never overwritten, but each version of the database is created efficiently anew, similarly to how git handles code.

👍 1
Tom H.08:11:15

@U1C36HC6N Is this not also the case for datahike and its hitchhiker tree? I suppose my question is, am I able to disable/avoid the runtime caching of the indexes in datahike to avoid the issue timo mentions above? > In case you are transacting into the same database the index of the Datahike instance that does not transact is not updated My (I think incorrect) understanding was that the datahike database could potentially be fully stored in postgres and then multiple clients could make queries and transactions via their own connections. But it sounds like the connection is used to populate local data in memory which is what’s used for running queries against, making transactions etc?

whilo06:12:40

Hey @U06077728, sorry for the delayed reply I was at NeurIPS and a bit busy. Yes, I am working on this, hopefully the connection management is done by year end.

❤️ 1