Clojurians Log v2
datomic
The time has come when I need to "copy" my Datomic Cloud database to create a high-fidelity acceptance environment. This task might be less nuanced than "decanting" since I'm happy with an exact copy of all datoms. Does anybody have any experience performing such an operation? In my case, the source and destination system, database and CF templates are named identically but exist in different AWS accounts. It is feasible to copy the DDB table(s) from the source to the destination (while the destination system is shut down)? Is that sufficient?
@cch1 you can "decant" without filtering. "Just" (™️) copy the datoms over to a new DB in the new system. This is of course a less satisfactory answer if you are dealing with a very very large DB and time pressures, but should be suitable for smaller DBs. I recommend against copying the underlying resources (DDB table etc).
OK. I will build a copy tool to do that.
Is there a reasonI am missing that your https://github.com/recbus/caribou tool doesnt already work for this task?
Caribou takes source clojure data (typically sourced from edn files) and transacts it in a way that is guaranteed to be both repeatable (across databases and time) and verifiable. It is intended for schema (which has a source representation somewhere) -not all transacted data -most of which derives from the day-to-day of user actions.
If it is at all helpful, I had to do some flavor of this in the past. There are lots of edge cases. I believe the code here is up-to-date(ish). Docs are the code 🙃 https://github.com/ComputeSoftware/datomic-backup
IIRC, if you don’t care about history and have Big DB, it’s much faster to https://github.com/ComputeSoftware/datomic-backup/blob/7c9e920423a15be2e2ed8b39da1a5ffd70f8fa0e/src/computesoftware/datomic_backup.clj#L139.
That's likely to be the case for me, at least for a couple more months. This looks like it might do the trick...
Fun to re-read this code. Weird edge case w/ d/datoms failing in the middle & need to retry. https://github.com/ComputeSoftware/datomic-backup/blob/7c9e920423a15be2e2ed8b39da1a5ffd70f8fa0e/src/computesoftware/datomic_backup/current_state_restore.clj#L186