Fork me on GitHub
#xtdb
<
2021-11-25
>
Jacob O'Bryant20:11:16

Any docs/advice on transitioning from jdbc doc store to s3 doc store (still using jdbc for tx log)? how (non)trivial of an operation is that? I'm at a stage where it'd be aesthetically nice to save disk space on the postgres cluster but business-wise it doesn't really matter, so not worth dealing with yet if it'll be complicated.

refset23:11:04

Migration is certainly doable but given all the possible combinations we haven't yet attempted to write a slick & comprehensive tool, or docs . But I'd be happy to help anytime 🙂 For rough inspiration you can look at these examples of migrating via the underlying stores using edn files: KV doc-store (and tx-log) export https://github.com/xtdb/xtdb/commit/efc5f93f372ecbf18445fbcd60c43359f0f07f86 and JDBC doc-store (and tx-log) import https://github.com/xtdb/xtdb/commit/7999b8e17366a4cd3d6bd4dd66631052bdeb02c0 An online migration would require a little more thought although if you're only migrating the doc-store it's definitely less hairy

Jacob O'Bryant04:11:31

Thanks! I also had the thought just now that migrating would probably be pretty easy/doable from "userspace" if I just migrate the tx log to a new postgres db as well. e.g. I could run two XT nodes at the same time, go through the old tx log and import each tx into the new node (with a tx listener to keep it up to date), wait for indexing to catch up, then redeploy the app to a version that uses only the new node.

🙏 1
refset08:11:22

Yep that should work also, although you may need to take inspiration from and be mindful of https://github.com/xtdb/xtdb/issues/1343

👌 1
👍 1