Clojurians Log v2
datahike
Hey everyone, I’m running a loop where each iteration creates a new database, runs some queries, and then deletes it afterward. However, I’ve noticed that the query time keeps increasing with each iteration — in some cases, it can take up to ~70 seconds just to recreate the DB and run the query. The total number of objects transacted is around 1,300. Any idea what might be causing this? My hunch is that it’s related to the repeated DB creation, even though I’m calling release and delete each time.
It’s a bit hard to provide a concrete example right now. I’ve tried disabling keep-history?, and that seems to have helped for the moment. I’ll keep an eye on it to see if the issue persists — it might be related to an internal issue
In-memory DB?
Yes
what's the memory footprint during the runs?
Hey @aviv846, thanks for reporting this. Can you provide an example to reproduce?
My business logic references an attribute (`:org`) but I modeled it poorly in my schema. I need to change :db/valueType. I want to "update" this attribute in my schema model to avoid renaming every instance in the codebase. Is this possible?
I actually like to keep the model and its attributes as general as possible. (Solo projects.) One would think that schema definitions also have a history, such that as-of could also apply to schema definitions at a particular instant. (Or maybe that's a horrible idea?)
If there's git history for one's codebase, maybe there should be a history for one's schema definitions. (Schemas are just data, after all, and change as the project and codebase does.)
I believe that what you're describing would be extremely difficult to reconcile with the indices (and probably also the underlying storage model) used by Datalog DBs. Datahike's docs https://cljdoc.org/d/io.replikativ/datahike/0.6.1607/doc/schema-migration#how-to-migrate-a-database-schema: > When we are speaking of changes to your schema, these should always add new definitions and never change existing definitions. In case you want to change existing data to a new format you will have to create a new schema and transact your existing data transformed again.
Unless you're ok with paying the cost of deleting and recreating the database entirely, the general advice here is "add a new attribute with the correct name and type." I'm not 100% sure about Datahike specifically, but the datalog DBs are not really built for editing schema - I think in large part because of the history-preserving nature of recording all the transactions in the DB. Might be worth using this as an opportunity to add a namespace to that keyword to disambiguate it from other concepts!