Clojurians Log v2
clojure-losangeles
Hey! Debating between Datomic vs XTDB for an experimental project. I think I'll insert about 100k records a day mostly from web crawlers, and then the query side is probably negligible for both databases. I don't think it will get above 100 users any time soon 🙂 The problem I have that I don't really know how the data will be queried and I'll need temporal queries to see changes in pages between "crawls". And of course, I'd love it to be in Clojure, since I've been doing python for the last 4 days and I don't want to see Classes and mutable structures in my life ever again 😄 What would you recommend?
XTDB has been working a lot on ingest performance, that may be relevant
also Datomic is now free to use (announced this morning at the Conj)
I went with XTDB mainly because of Biff framework. Didn't feel like gathering everything together myself for a prototype. Works quite well. I love "valid-time" queries. But will do another reevaluation with some load testing if it ever goes to production.
Cool! Let us know how it goes. Seems like a lot of data and a good experiment to see how Christ handles it.
Will you know what your schema is before you write to the database? One key distinction between Datomic and XTDB is that Datomic is schema on write, which means you need to know your schema before inserting anything into the database, whereas XTDB is schema on read, which means you can just stuff data in the database and query for structure later. I personally have used Datomic for several projects, and only done some light investigation into XTDB, but for exploratory work I might be tempted to try things out in XTDB just because ostensibly there's less decision making to do up front (and also to get some experience using it in anger). However Datomic is a lot more mature, well documented, and battle tested, and if the schema on write thing is not an issue, should be able to handle your use case without too much trouble.