Fork me on GitHub
#asami
<
2021-11-19
>
zeitstein16:11:13

Trying out durable storage and I have some questions: 1. Is there a quick way to get a fresh durable database from the REPL? Something like delete-database for in-memory. 2. The workflow I have in mind is: a) on start, load from durable to memory b) periodically transact new facts to durable. Any suggestions on how to handle entity IDs? Clearly, I can transact keywords as eids to durable, but I understand this is not optimal. 3. Thinking about how I would achieve the desired workflow when in-memory is done from CLJS, I had an idea to use the GraalVM binary to achieve steps a) and b) above. (I should be able to execute CLI commands from Node.js, if I'm not mistaken.) Would it technically be possible to add transact and export-data to the CLI API? 4. Transacting a single entity to a fresh durable db, the db takes up over 40 MB of disk space. Does that seem right or am I doing something wrong?

quoll16:11:36

1. Just connect to the appropriate URL. It if doesn’t exist then it will be created: (def c (connect "asami:")) 2. Entities are allocated their own nodes. In memory, these are keywords. In durable storage, these are InternalNode objects. This was done because as soon as the JVM sees a keyword then it gets interned, so if your DB is large enough, then that could exhaust memory. If your entire local storage is to be loaded into memory, then this isn’t a problem. I would either create everything in memory (hence, keywords) and then export to disk, or else, specify :db/id for each entity to be a keyword. 3. Access to transact and export-data is easy and not a problem. The question is the appropriate syntax. I’ve been thinking of doing a Datomic-syntax extension that looks a bit like SPARQL Update. (e.g. :insert / :data and :insert / :find 4. Durable databases reserve a lot of space when they’re created (this works well with memory mapping files). Those files should be truncated to only the filled space when they’re closed. When they’re reopened they will expand again. 40MB sounds larger than I remember, but it may at that level.

zeitstein19:11:36

Fantastic, thank you!