Hey @U8ZQ1J1RR - no recent discussion I can think of, but broadly speaking this should already be possible wherever there are underlying facilities. Which storage options are you particularly interested in this for?

We are still at psql level, not using kafka yet. I'm not aware of simple ways to do that (like on JDBC level), I suppose because it easily breaks indexing

would encrypting at a disk/partition level maybe enough for your requirements?

(just a thought, nothing more)

> We are still at psql level You/we could probably extend xtdb-jdbc with pgcrypto's pgp_sym_encrypt/`pgp_sym_decrypt` functions by wiring in the dialect-specific SQL a little more deeply. Only the v column would need encrypting, and only for documents (not tx-log entries). I haven't considered that option before, interesting!

Sounds surprisingly simple

wouldn’t you also want to crypt any checkpoints

(Not) Knowing that kind of stuff is just what makes me feel that I couldn't really implement this idea. Normally you can fix bugs when you notice them, but with security things should be a bit more comprehensively planned and designed

guarding data against your own ops people is difficult, what stops them from getting the encryption key from the same place your app does?

It wouldn't be our ops people. Concretely, we would have our databases from Aiven. A smallish company that would handle the nitty gritty of DBA work. While the ops people behind our virtual servers can of course access the memory, and our own company can do whatever if they are able to deploy code, we could at least limit the access of Aiven people to our databases.

It's not much, but it would be somewhat feasible to limit the exposure a bit. If XTDB had that option, I would put an encryption key to our own application services and encrypt what's in the database servers

in that case it sounds like it would be better to do the en/decrypt in the xtdb node side… your db provider could log your SQL statements and have the encryption password if you use pgp_sym_* afaict

^ yeah I think that's right actually, this pgp_sym_* idea is only good for addressing data-at-rest and the keys are still going to be accessible to server admins (IIUC). Maybe that's still compelling enough though. I must say thanks for all the feedback here so far, it's useful! And short of submitting a PR...feel free to open an issue for (any of) this in the meantime, so we can try to find time for it eventually. Alternatively, please note that as a commercial venture we can always be $persuaded to make the time 🙂

I wrote https://github.com/xtdb/xtdb/issues/1812, hopefully the idea is clear