Fork me on GitHub
#datomic
<
2022-12-27
>
Setzer2216:12:51

I'm using datomic (on-prem) and I need to move data from one datomic instance to another. More specifically, I need to send data from A and B in such a way that when the operation ends, B contains a replica of A's data. For that, I was trying to use the backup-db and restore-db scripts. My logic seems correct: I can backup and restore from the same database (from A to A), but I'm getting an error when attempting to restore a backup made from another database (from A to B). Any idea of how to fix this?

java.lang.IllegalArgumentException: :restore/collision The name 'datomic' is already in use by a different database
        at datomic.error$arg.invokeStatic(error.clj:79)
        at datomic.error$arg.invoke(error.clj:74)
        at datomic.error$arg.invokeStatic(error.clj:77)
        at datomic.error$arg.invoke(error.clj:74)
        at datomic.backup$create_restore_target.invokeStatic(backup.clj:460)
        at datomic.backup$create_restore_target.invoke(backup.clj:449)
        at datomic.backup$restore_db.invokeStatic(backup.clj:490)
        at datomic.backup$restore_db.invoke(backup.clj:481)
        at datomic.backup$restore.invokeStatic(backup.clj:571)
        at datomic.backup$restore.invoke(backup.clj:568)
        at datomic.backup_cli$restore.invokeStatic(backup_cli.clj:53)
        at datomic.backup_cli$restore.invoke(backup_cli.clj:44)
        at clojure.lang.AFn.applyToHelper(AFn.java:154)
        at clojure.lang.AFn.applyTo(AFn.java:144)
        at clojure.lang.Var.applyTo(Var.java:705)
        at clojure.core$apply.invokeStatic(core.clj:667)
        at clojure.core$apply.invoke(core.clj:662)
        at datomic.require$require_and_run.invokeStatic(require.clj:22)
        at datomic.require$require_and_run.doInvoke(require.clj:17)
        at clojure.lang.RestFn.invoke(RestFn.java:423)
        at datomic$_main$fn__163.invoke(datomic.clj:150)
        at datomic$_main.invokeStatic(datomic.clj:149)
        at datomic$_main.doInvoke(datomic.clj:142)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.lang.Var.applyTo(Var.java:705)
        at clojure.core$apply.invokeStatic(core.clj:667)
        at clojure.main$main_opt.invokeStatic(main.clj:514)
        at clojure.main$main_opt.invoke(main.clj:510)
        at clojure.main$main.invokeStatic(main.clj:664)
        at clojure.main$main.doInvoke(main.clj:616)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.lang.Var.applyTo(Var.java:705)
        at clojure.main.main(main.java:40)

Setzer2216:12:54

Note that this error does not occur when restoring a backup made from the same database (it works fine in that case), only when trying to move data between databases.

Joe Lane16:12:38

@jsanchezf Do you already have a different database in the destination system named “datomic”? the :restore/collision error message claims that is the case

Setzer2217:12:07

@joe.lane yes. It seems what I'm trying to do is removing that existing database and overwrite it with this new one keeping the same name datomic. But I'm not sure how to do that

Setzer2217:12:00

I'm having trouble understanding what datomic even means when it tells me there's an existing database called "datomic". I don't remember creating anything like a datomic "db" or giving it a name. I am using postgres as my storage, and I remember creating a schema for datomic inside postgres, but I don't think this what datomic is referring to

ghadi17:12:00

you may have restored twice unwittingly

Setzer2218:12:05

I think I'm not explaining myself clearly 😅 Let me explain a bit better. A is a production system, B is a staging system. What I want to do is to move the data from A (a fully-functioning datomic database holding our user data) into B (another fully-functioning datomic database holding whatever dummy data we use for testing). Both systems apparently have a database named datomic, and I can backup and restore from A to A, or from B to B as many times as I want, but I cannot restore a backup made for A into B. The purpose of doing this is to be able to test a data migration in a controlled environment with our real user data, but in a controlled environment where doing this won't cause any unnecessary downtime.

Setzer2218:12:56

Other databases have similar features. Postgres has pg_dump and pg_restore. MongoDB has mongodump and mongorestore. Both allow you to transfer data across databases. Is there a way to achieve the same thing with datomic?

favila20:12:48

A datomic db made by create-database has both its name and a uuid. The uuid is to distinguish dbs with the same name created by distinct create-database calls. Your error message is because you are attempting to restore over another db with the same name created via a different call

favila20:12:15

You need to first remove the staging db before restoring

favila20:12:10

The fastest way is to kill staging txor, truncate the underlying Postgres table (assuming there are no other dbs you need in it), restore from your prod backup into staging, then bring up the txor again

Setzer2208:12:30

Hi @U09R86PA4, thanks a lot! By "truncating the underlying postgres table" you're referring to the datomic_kvs table, right?