Fork me on GitHub
#xtdb
<
2020-03-26
>
dgr14:03:47

What’s the best way to contribute to Crux documentation? As I’m learning the system in more detail, I’m finding places where the documentation could be made more clear or provide additional information to users. Should I just submit pull requests to the Crux repo on Github?

jarohen14:03:07

hi Dave, thanks for getting in touch 🙂 would be great to get your thoughts and suggestions on the docs. if they're relatively small/local changes, yep, PRs would be great; for larger changes - either here or a Github issue would be lovely thanks again 🙂

dgr17:03:06

OK, will do. Thanks.

dgr17:03:19

Is it possible to get the Crux CLA in another format than just LibraOffice? Something portable like PDF, RTF, or even ASCII? Could it be added to the documentation itself so that people could review it?

refset18:03:08

Yes, certainly. The odt format was purely expedient but there's no reason why it shouldn't be something else. I will check with my colleagues about the options. RTF seems like a very sensible choice to me

dgr20:03:17

Does it need to be editable, or just downloaded, printed, scanned, and sent in? If so, then PDF is probably the best. If there’s something that needs editing, then maybe RTF.

refset21:03:09

No need to print and scan, generating a PDF (from RTF or whatever) with an embedded signature is fine. Editing PDF ex post facto is still a bit too magical for me 🙂

dgr21:03:49

OK, I get it. Makes sense.

dgr21:03:45

Sorry, another question. 🙂 What is the status of an evicted document in terms of being re-added to the DB. In other words, if I have a document with :crux.db/id of :foo, then I evict :foo, can I re-add another document with :foo as key? I’m thinking about two situations, where I might want to create information for the purpose of testing in a live database, but the fixture wants to reset everything with each test run. Eviction would seem the best, as long as information can be add back with the same key the next time the tests are run. I assume that deletion would not be as good as the history would build up and remain forever, even if the current DB showed it as deleted. Thoughts? The second situation would be for things like user accounts that might come and go. If I have an account with key :bob and Bob deletes his account and then another Bob comes, would eviction cause any problems later.

👍 4
refset21:03:37

These are good questions! As things stand today you can't un-evict something, so for testing you would need to delete the db directories. We do this throughout our tests https://github.com/juxt/crux/search?q=with-tmp-dir&amp;unscoped_q=with-tmp-dir You could pull in our with-tmp-dir into your own fixtures, although it won't be officially supported. It's probably best to copy it for now. If you are using :bob as a document ID to enforce a uniqueness constraint then it's best to not evict it, and only evict the other entities it refers to. However, if the ID itself contains sensitive data such as an email address (that needs evicting to comply with the right to erasure) then I suppose there's quite an interesting contention. I'll need to think about that some more. But otherwise yes, you won't be able to insert a future :bob - I think the subsequent put will fail with an error log (...just confirming that now...)

dgr21:03:40

OK, thanks. That’s another good addition to the manual, then. I’ll check out with-tmp-dir. Thanks for the pointer.

👍 4
refset10:03:19

Actually, the more sensible option (for the majority of tests you may want to run) is to just use the default standalone topology without Rocks as everything stays in memory by default and disappears once you close the node - so no need to think about directories on disk.

dgr13:03:56

OK, how does one not use Rocks? The instructions in the docs seem to imply that Rocks is used in the standalone config. Specifically, :crux.standalone/event-log-kv-store defaults to 'crux.kv.rocksdb/kv and the example sets :crux.kv/db-dir to a value in the start-node options. So, is standalone config really two possible configs: in memory only or on-disk locally with Rocks? That might be another topic to document.

refset14:03:17

Ah, it looks like "Table 5. Standalone configuration" is stale/misleading. I'll add it to my list! Standalone supports memory or Rocks/LMDB, however to use Rocks/LMDB you need to add the respective modules into the topology vector. If you don't add either Rocks/LMDB then it will default to in-memory, no extra config required. :crux.kv/db-dir has a role even for memory-only standalone nodes as it provides options for exporting/restoring the indexes to disk: https://github.com/juxt/crux/blob/master/crux-core/src/crux/kv/memdb.clj (again, these aren't first-class APIs)

dgr16:03:16

OK, gotcha. Yea, maybe clarify that. Given what you said and reading down to the RocksDB section of the Standalone Node section, I can see what you’re talking about. Maybe a note for the first section that states explicitly that this listed example results in a memory-only configuration.

👍 4