is there a handy example on how to setup the jdbc store?

@hoppy not "handy" but I can accelerate our release plan a little 🙂 which backend DB are you hoping to use?

postgres 11.5

@hoppy some pointers: https://github.com/juxt/crux/blob/master/crux-test/test/crux/jdbc_test.clj

I'm basically hunting the recipe that aligns with the start-standalone-node / start-cluster-node thingy. I can puzzle through that and see how to do it eventually.

I'm hoping to house this in an existing db, if that's possible

@hoppy this seems to work:

(ns crux-jdbc-example
  (:require [crux.api :as crux])
  (:import (crux.api Crux ICruxAPI)))

(def opts {:dbtype "postgresql"
           :port 5432
           :dbname "postgres"
           :user "postgres"
           :kv-backend "crux.kv.memdb.MemKv"
           :db-dir "kv-store"})

(def node (Crux/startJDBCNode opts))

I can't see an obvious reason why reusing an existing db within a running instance would be a bad idea, as long as you have plenty of headroom for the storage of backups -- but I'm not a database operations expert 🙂

I'm playing with ^^^

I'm wanting to do that (move in) because I have a hybrid situation and I wan't a singular consistent backup story

Oh, and I'll stick with rocks for the kv-store

I've surmised that if I completely wipe the rocks stuff, it will rebuild itself, is that correct?

Cool, yeah the sweet spot for JDBC mode is that it fits into an existing ops landscape very trivially

yep, any kind of cluster node (i.e. JDBC or Kafka) can rebuild itself

I wouldn't necessarily plan on doing that too much given the time it takes to rebuild will only increase, but I'm guessing you don't have huge quantities of data

so the other fun question. what if the kv store is older than the database backup. Does it "catch up" or is that a nuclear event

aka, my strategy is to not care about the kv store aside from say, back it up once a month.

postgres would be either replicated or wal archived

yep it catches up, regardless of how old it is, and that's the basis of how clustering works with each node continuously trying to stay up to date with transaction log independently

Is there a way to block until a submit-tx completes?

(let [id (m/random-uuid)]
  (crux/submit-tx node [[:crux.tx/put {:crux.db/id id}]])
  ;;??
  (crux/entity (crux/db node) id)) ;;how to make this return the submitted entity

oh wait, I think sync is what I want

Here we go, working:

(let [id (m/random-uuid)]
  (let [result (crux/submit-tx node [[:crux.tx/put {:crux.db/id id}]])]
    (crux/sync node (:crux.tx/tx-time result) (Duration/parse "pt1s")))
  (crux/entity (crux/db node) id))

@taylor.jeremydavid the above sorta works (the first time).

after that, it tries to do the create_table again on the 2nd run