Clojurians Log v2
datalevin
I am looking for key range delete operation, I can fetch a range, but then I have to delete them one by one.
you mean the KV store? Please file a GitHub issue.
I set up a Datalevin db aggregate data from several different services to do data verification and bulk etl jobs. My intention is to just mirror the data in the remote services. I had claude help me set up the scaffolding and data loading. (since i am a EAV db noob). This wasnt a one-shot. I had to keep going over it in a clumsy way. Even though I declared my intentions, it initially never set up any code to evict state stuff. Weeks later, an multiple rounds of “hardening”, I am still getting bit by stale data. Claude put eviction logic into the loaders, but it still seems to miss things and is not installing this behavior by default. I’m coming from a system where i would just download json, load it into memory, and work of that; So it was always guaranteed fresh. I thought that’s how things would be working... What am i missing? Are they any established patterns for this (without the repetition of bespoke evicition logic I currently seem to need?)
Unlike other DBs, in Datalevin, retracting entities is extremely fast, close to raw datoms loading speed.
Don't know what you are doing. You can retract entities easily in datalevin.
if you are doing ETL, you can just use an in memory db, then nothing will be preserved.
Ok. Not suggesting any problems with Datalevin. I guess I just stumbled about awkwardly getting the the retraction logic (using system timesamps, etc) set up. Thought maybe there was a basic pattern or fn I might be missing.. But i guess retracting appropriately is all that is needed.. I am persisting because to load everthing from api’s can take over 40min. And i want that to survive clojure/repl restarts, and to be able to reload (synchronize) parts selectively. It’s working pretty nicely. The snapshot function is great for before/after comparison. I’m just clunky with the schema and loading so far.