This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-11-25
Channels
- # announcements (1)
- # asami (3)
- # aws (2)
- # babashka (8)
- # beginners (60)
- # biff (3)
- # calva (45)
- # clj-http (3)
- # clj-kondo (19)
- # clojure (50)
- # clojure-europe (23)
- # clojure-nl (8)
- # clojure-spec (4)
- # clojure-uk (3)
- # clojurescript (32)
- # conjure (3)
- # datomic (7)
- # events (2)
- # fulcro (24)
- # graalvm (3)
- # inf-clojure (3)
- # jackdaw (1)
- # lsp (3)
- # malli (8)
- # missionary (14)
- # off-topic (1)
- # polylith (3)
- # portal (11)
- # rewrite-clj (63)
- # shadow-cljs (21)
- # spacemacs (7)
- # tools-build (6)
- # xtdb (4)
Any docs/advice on transitioning from jdbc doc store to s3 doc store (still using jdbc for tx log)? how (non)trivial of an operation is that? I'm at a stage where it'd be aesthetically nice to save disk space on the postgres cluster but business-wise it doesn't really matter, so not worth dealing with yet if it'll be complicated.
Migration is certainly doable but given all the possible combinations we haven't yet attempted to write a slick & comprehensive tool, or docs . But I'd be happy to help anytime 🙂 For rough inspiration you can look at these examples of migrating via the underlying stores using edn files: KV doc-store (and tx-log) export https://github.com/xtdb/xtdb/commit/efc5f93f372ecbf18445fbcd60c43359f0f07f86 and JDBC doc-store (and tx-log) import https://github.com/xtdb/xtdb/commit/7999b8e17366a4cd3d6bd4dd66631052bdeb02c0 An online migration would require a little more thought although if you're only migrating the doc-store it's definitely less hairy
Thanks! I also had the thought just now that migrating would probably be pretty easy/doable from "userspace" if I just migrate the tx log to a new postgres db as well. e.g. I could run two XT nodes at the same time, go through the old tx log and import each tx into the new node (with a tx listener to keep it up to date), wait for indexing to catch up, then redeploy the app to a version that uses only the new node.
Yep that should work also, although you may need to take inspiration from and be mindful of https://github.com/xtdb/xtdb/issues/1343