Fork me on GitHub
#datalevin
<
2022-03-22
>
fs4222:03:08

Just started to eval datalevin with nextjournal - very impressed with datalevin!!! - any others who managed to integrate it into the nextjournal-jupyter environment? - trying to figure out where to store the datalevin db-files... nextjournal has this versioned persistent filesystem mounted on /results... which doesn't allow for directories - could keep the db-files in the ephemeral filesystem (e.g. "/db") and then back those files up in /results - any experience/suggestions?

fs4223:03:17

note that I got it to work well by simply adding the datalevin to the deps.edn file in nextjournal - the only thing left to really deploy it is to figure out where the database files should reside and how they are persisted...

Huahai00:03:03

You may try adding :nosubdir to :flags when run open-kv. Please let me know if it works.

fs4201:03:47

thanks for the suggestion! I see two data files being created for datavin-db: lock.mdb and data.mdb. Are those the only two?

fs4201:03:28

the reason I ask is that nextjournal has a somewhat (convoluted?) way to refer to files in their version filesystem: you have to literally ask nextjournal GUI for the name and then manually add a reference to those files in your code (https://nextjournal.com/help/uploads-and-results?version=published).

fs4202:03:02

Also... when i close the db-connection, does the lockfile get discarded/deleted (this could possibly be an issue when a new file gets created and an old lockfile exists on the versioned filesystem? Also to backup the db, should I only worry about the data.mdb file, forget the lock.mdb, or are there potentially others? Sigh... sorry for all those Qs - should have been easy...

fs4203:03:03

So far I had been using "get-conn", which feels a little different from your suggested "open-kv" - do both return a same kind of connection to work with? (I can see that only open-kv allows you to pass in a :nosubdir in the option's :flag vector...)

Huahai03:03:06

If you are using the Datalog db, then there’s no option to avoid the data directory. open-kv is for working with the key-value store.

Huahai03:03:53

Yes, the two files are the only files and they have fixed names. They will not be deleted.

Huahai03:03:21

when you do backup, you only need to specify the directary, not the file names inside.

Huahai03:03:11

Backup should be done with Datalevin copy, not file system utilities.

fs4205:03:12

would datalevin work with symbolic links to the data&lock.mbd files? one option would be to copy the two files after creation into the versioned filesystem directory /results, get their file references, and create symbolic links to those files in the ephemeral filesystem like "/db/", and use that /db in the get-conn call... (also waiting for some more advise from the nextjournal folks...)