Clojurians Log v2

our test suite doesn't even delete the temporary database, because it would complicate the testing process. im considering the use of https://github.com/NoahTheDuke/lazytest to ensure a concise and reliable test state clean up, because we do see a gradual slow down on repeated (kaocha) test runs however, manually deleting all test dbs doesn't seem to help with this magical slow down. either way, i would be happy to jump through some hoops to delete databases at least UNTIL there is a way to restore them easily :) BUT when i'm using datomic-local, i wouldn't want to jump through hoops!

our test suite doesn't even delete the temporary database, because it would complicate the testing process.We use d/with-db d/with for all our database tests so there's no need for cleanup. For me it is one of the "superpowers" of Datomic. Although some scaffolding in the form of fixtures and testing utilities is needed for domain specific reasons, developer experience writing tests is amazing. Nothing like I've experienced in the past. We do not mock or stub anything. Actual data, actual queries, actual transactions, actual db values and super quick. Pretty amazing in my opinion 🙂

u still need at least 1 in-mem, datomic-local database to call d/with on, which u would need to re-create, when your schema changes (or retain the connection across test runs to make sure it has the latest schema, if u have just added attrs, not modified existing ones) datomic-local did support io-stats either and there were some other cases, where the speculative wasn't sufficient, but can't remember where exactly, but probably when • dealing with :db/txInstants • changing cardinality on an attr • when a query handles more than one db-vals as inputs • when the tx-log is looked up with tx-range, which expects a connection or similar, more advanced use-cases. would be good, if the Datomic team would have some official testing recommendations, which detail the pros and cons... something like https://github.com/vvvvalvalval/datomock

u still need at least 1 in-mem, datomic-local database to call d/with on, which u would need to re-create, when your schema changesWe use a fixture, that for every test, redirects all requests to a clean in-memory db with required schema. It is pretty straightforward and works well enough for our needs which is nothing advanced. Mostly testing API handlers. > • when a query handles more than one db-vals as inputs Interesting 🤔 We have an upcoming feature where queries are planned to heavily rely on this feature. I guess I need to look how it should be tested then.

Out of curiosity, if I call d/delete-database in the Peer API on a dynamo-backed instance, does the deletion work get performed by the Peer side or is it going to instruct the transactor to do that?

looks like it goes through datomic.peer/send-admin-request so I'm assuming it's telling the transactor what to do. This'd be another good argument for my colleague's request to allow building connections/dbs without needing to let an application that needs to do read-only Peer stuff talk to the transactor at all. https://clojurians.slack.com/archives/C03RZMDSH/p1723147402520899

“Deletion work” from d/delete-database is just marking the database deleted. The marking is done by the transactor. Actually deleting all its contents is done by running gc-deleted-databases, a cli tool. This does not involve the transactor

can you add that to the docstring please https://docs.datomic.com/clojure/index.html#datomic.api/delete-database

https://docs.datomic.com/client-api/datomic.client.api.html#var-delete-database

@dustingetz it does not work this way in Cloud. In Cloud this reclaiming of storage space is handled for you and gc process is run automatically. For Pro, this is documented see: https://docs.datomic.com/operation/capacity.html#garbage-collection-deleted > When you delete a database with https://docs.datomic.com/clojure/index.html#datomic.api/delete-database, the database immediately becomes unavailable for use. As a separate step, you may later choose to reclaim all storage associated with deleted databases on a system. There are two ways to do this, corresponding roughly to production and development/testing needs.

I will discuss with the team adding a link or similar statement to the pro api doc string

I was responding specifically to @chancerussell’s https://clojurians.slack.com/archives/C03RZMDSH/p1755724672224909?thread_ts=1755391083.668369&cid=C03RZMDSH and didn't want anyone to get the impression that either the peer or the transactor is performing potentially gigabytes of DDB DeleteItem ops the moment you call peer d/delete-database, which is what someone might think from the phrase "deletion work"

I wish there was something "office hours" for Datomic during the Clojure/Conj2025 where it would be possible to get some unofficial insight into Datomic development - plans, priorities, Q&A etc. Last year I participated at the Amazing Day of Datomic workshop just so I could bug Joe Lane to death about all the questions that had accumulated over the years (including cloud backups, GDPR etc.). Did not feel 100% right but it was super helpful. It would also be a great way to connect in person with other Datomic users. I would guess that many of us are facing similar challenges.

It was great to hear that Cloud backups is something that is not completely disregarded and what to do if d/delete-database is accidentally called on a database etc. That kind of stuff 🙂

https://clojurians.slack.com/archives/C03RZMDSH/p1751154326943559

tl;dr no.

(alter-var-root #'d/delete-database
                (fn [old]
                  (fn [client args]
                    (if (instance? DurableClient client)
                      (old client args)
                      (throw (ex-info "Deleting a cloud db is NOT allowed! (See user.clj for override instructions.)"
                                      {:client client
                                       :args args}))))))

that’s my workaround fwiw

Thanks, but I fear what you have in user.clj will tell it how to anyway. 😅

Having d/delete-database available on application classpaths is a mistake, and if it were removed, would even a single Datomic user notice?

I would support its removal - a new dep called “datomic-delete-database” could be used where needed.

or like, use aws to rm -rf the system is more likely what most people are doing when they want this, i do not understand why the api needs to exist at all

"There is no DROP DATABASE statement in the SQL standard."

It’s useful for test runners, etc or if you’re regularly provisioning databases programmatically, but I’d rather opt-in to delete-database being on the classpath

yeah, I'd also be okay with being able to lock a system at the ion deployment level. at the very least that would create a double-check.

Yea something like the AWS concept of "Deletion Protection"

however I still think it's a mistake to allow full rw access to a DB like that, if it can't call d/delete-database it could easily write its own little clj -Sdeps incantation to pull in the function and delete it, or use your aws cli or other tool to delete it. The only safe solution is to mediate access via a trusted intermediary

> or like, use aws to rm -rf the system is more likely what most people are doing when they want this, i do not understand why the api needs to exist at all Cloud supports multiple databases in a single system.

I've captured the concern about delete database in a previous story, I will add this thread to that and check in with dev on the weekly feature/story review.

the multi-db-val issue might "only" be a present when connecting to a datomic cloud ion server remotely (with (d/client {:server-type :ion :endpoint "" ,,,}), but running it from outside of the datomic cloud vpc)