Fork me on GitHub
Jakub Holý (HolyJak)11:08:26

Not sure where to report, there is a typo at : "database, or or colocating multiple databases" and "For sh, is is not uncommon"


Thanks for catching this. I’ll fix it today.

👍 6
Jakub Holý (HolyJak)11:08:25

A question about multiple DBs and transactors. > When you serve multiple databases with Datomic, you have a choice of provisioning separate transactor pairs per database, or or colocating multiple databases on a single transactor pair. But I cannot find where to configure that. The only or that can be is related to the storage cluster, nothing about individual DBs. I know I can start multiple peers / peer group and point each to a particular subset of DBs but I do not see how to run a number of transactors, each serving only a subset of DBs, and how to get the peers to connect to the correct transactor?


Transactor properties do not include a database

Jakub Holý (HolyJak)12:08:13

Yes, it seems so. So how should I understand this sentence: "you have a choice of provisioning separate transactor pairs per database"? 🙏


Make more transactor with their own storage


A connection string has storage + datomic-db info


You can have one transactor pair per storage but multiple dbs

Jakub Holý (HolyJak)12:08:59

I see, thank you. So in the case of Cassandra, I would need one Cassandra cluster per a set of DBs. That makes sense.

Jakub Holý (HolyJak)12:08:48

The Datomic on-prem license is "per system". Do you know what a "system" is? Is it essentially per one HA transactor pair?


No you could use the same Cassandra cluster but a different key space

👍 3

Re:system, I think so.

👍 3

Hope you don’t mind that I try again. I’m seeing No space left on device - /opt/codedeploy-agent/deployment-root/.../bundle.tar types of errors during deployments DownloadBundle step. Seems like code deploy agent is running out of space for revisions archive. Is there a way to change the root volume size that’s attached to the compute instance (running 884-9095, on t3.small w/ the default 8GB storage)?

Jakub Holý (HolyJak)13:08:15

Any idea why I might be getting "Could not transfer artifact com.datomic:datomic-pro:pom:1.0.6316 from/to ( status code: 401, reason phrase: Unauthorized (401)" even though I have added <server<id></id>... to ~/.m2/settings.xml as suggested by ? 🙏 Could it be because my account is too new?

Joe Lane13:08:24

Are you using Lein?

Joe Lane13:08:51

Does it work now that you have ?

Jakub Holý (HolyJak)13:08:52

Found it, I misplaced the <server> section 😅

👍 3
Linus Ericsson13:08:06

Is there any open source code for the functionality of (selective) respooling datomic databases?


What is “respooling”?

Linus Ericsson13:08:32

selectively lift data from an old db to a new one (as mentioned in some talks from Nubank).


in a history-preserving way? or just copying?

Linus Ericsson13:08:15

history preserving but possibly rewriting/never transact certain things (GDPR use cases among other things).


The history-preserving method is usually called “decanting”. I’m not aware of any open source code for this and having written a few I’m not sure what it would contain--it tends to be very bespoke and depend on the data model

Linus Ericsson13:08:19

decanting is the word. Thanks for confirming that this tends to be use-case specific. This means we don't waste our time thinking about if with the ambition to build something from scratch.


the basic technique is read the tx log in order, transact that into the target db, and retain somewhere a mapping of eids in the old db to the new id as you go.

Jakub Holý (HolyJak)13:08:36

I guess could be a good starting point? It also supports filtering/rerwriting data so that prod data can be used in a stage env in a privacy-friendly way

👍 3
Linus Ericsson13:08:47

Thanks for the pointers and hints. We will look into this for our particular use-case. The devil is in the details, as always.

Jakub Holý (HolyJak)20:08:30

Hi! If I understand it right, you can enforce integrity constraints on data but must remember to include the relevant :db/ensure on any transaction that creates/modifies the entity in question. How does that work for you in practice? Is it a non-problem or does it happen that a developer forgets to include it and makes a change that leads to invalid data (such as creating an employee without a (required) department)? Thank you!