Any acid jdbc db will do. I’ve used SQLite

good to know, thanks

Prior to Free we had one customer I knew of using MariaDB. The only downside of FREE for me is not being able to heartbeat with these projects at license renewal time to see how things are going. Please do let me know if you encounter any hiccups setting up MariaDB if you go that route.

One way to move storage is to do a datomic backup and then restore into a new transactor. It requires you to stop the world during the transfer, but it gives you full freedom in choice of storage.

@U07FCNURX recently moved a Datomic pro transactor to SQLite, which seems like a good place to start - cheap and easy to get up and going.

Yes, using SQLite as a storage solution for Datomic pro was an elegant solution for me. SQLite is optimised for a single writer and many readers, which is a good fit for Datomic's model - and there is no need for a separate running process. It does require you to run on hardware with direct access to disk, however. If your startup needs to scale up later, moving to a different storage solution is no issue.

@U07FCNURX So moving to another storage solution effectively means restoring a datomic backup into a new transactor as @U9MKYDN4Q suggested? Is that the method generally used?

Backup and restore is the preferred method for migrating storages

https://docs.datomic.com/pro/operation/backup.html

Just saw no one had added the link. Backup and restore and moving underlying storages is a great super power IMO (providing future leverage and choices). Datomic uses all supported underlying storages the same as a K/V store so you can use what works best for you.

Reading through SQLite’s documentation, I get the impression that this piece of software is actually quite robust, within its limitations, and has the potential to support a substantial amount of traffic. As @U07FCNURX noted, it is a single writer, but to that end I found this statement in its documentation: “Actually, SQLite will easily do 50,000 or more https://www.sqlite.org/lang_insert.html statements per second on an average desktop computer. But it will only do a few dozen transactions per second. Transaction speed is limited by the rotational speed of your disk drive. A transaction normally requires two complete rotations of the disk platter, which on a 7200RPM disk drive limits you to about 60 transactions per second. Transaction speed is limited by disk drive speed because (by default) SQLite actually waits until the data really is safely stored on the disk surface before the transaction is complete. That way, if you suddenly lose power or if your OS crashes, your data is still safe.” This seems to imply that it will/may handle more than 60 write transactions per second on an SSD. Here’s another statement from the documentation: “SQLite works great as the database engine for most low to medium traffic websites (which is to say, most websites). The amount of web traffic that SQLite can handle depends on how heavily the website uses its database. Generally speaking, any site that gets fewer than 100K hits/day should work fine with SQLite. The 100K hits/day figure is a conservative estimate, not a hard upper bound. SQLite has been demonstrated to work with 10 times that amount of traffic.” Given that Datomic Pro has been released without licensing charge to promote its use, I think it would be nice to see documentation how to use SQLite as it storage service for first time users and small to medium sized applications. The SQLite docs say that, contrary what many might assume by its name, it is by far the most used database engine in the world, and is most likely in the top 5 most used pieces of software in the world.

I haven’t had issues using with for this in the past (check to see if tx is a no-op and abort if so), not sure I follow while you’d anticipate that being difficult or slow?

Did you use it in a transaction function or on the peer?

on the peer I am not too concerned … on the transactor I’d worry that basically I’m asking the transactor to do double the work (a dry run of every transaction followed by transacting).

Ah, I see. I probably wouldn’t approach it that way in a transaction function? Though I guess it depends. From a high level, it would kind of flow from the domain semantics? I.e., two simple cases: Either everything that’s getting into the system does so as part of some guarded, curation process. In which case, speculative work slowing things down to keep noise out of the system is justified. Or it’s a noisy stream of events being collected in which e.g. tx queue subscribing peers are already filtering down to only a subset of interest.

I wouldn’t want to satisfy too many constraints broadly across the system at the same time. It’s also not clear to me how you want this to proceed from the peer side, domain semantics wise? I.e., what does it mean if two peers submit a tx at the same time, and there’s order dependent behavior, in that if one peer gets in first, it will be a no-op, but if it gets in later, it would overwrite the transaction it wasn’t aware of.

if you just mean eg a race to commit the same fact, I’d probably prefer to use [cas](https://docs.datomic.com/pro/transactions/transaction-functions.html#dbfn-cas) or something with the same semantics: reject if the value has changed since the peer last saw it before submitting the tx. Rather than relying on assuming that a tx I expected to result in datoms didn’t being an implicit cause for rejection that way.

Yes, we’re using cas here already on the last-updated timestamp--I guess that does make it safe to do the d/with on the peer and rely on the result. I had not really thought about that.

We do dry run all transactions using d/with on the peers and it does ~half our throughput and doubles the latency on API requests to the peers but it’s horizontally scalable, whereas the transactor is not, and it reduces transactor load if you have a lot of invalid transactions or dupe facts.