This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-01-10
Channels
- # aleph (1)
- # announcements (21)
- # babashka (3)
- # beginners (98)
- # calva (2)
- # circleci (3)
- # clara (58)
- # clj-kondo (123)
- # cljs-dev (1)
- # cljsrn (7)
- # clojure (162)
- # clojure-europe (2)
- # clojure-finland (7)
- # clojure-italy (5)
- # clojure-nl (6)
- # clojure-sanfrancisco (1)
- # clojure-spec (1)
- # clojure-survey (17)
- # clojure-uk (70)
- # clojuredesign-podcast (2)
- # clojurescript (46)
- # cloverage (5)
- # cursive (2)
- # data-science (22)
- # datascript (1)
- # datomic (60)
- # emacs (3)
- # figwheel-main (1)
- # fulcro (26)
- # graalvm (5)
- # jackdaw (3)
- # leiningen (8)
- # luminus (1)
- # off-topic (8)
- # other-lisps (2)
- # pedestal (27)
- # re-frame (17)
- # reagent (20)
- # reitit (3)
- # shadow-cljs (37)
- # spacemacs (23)
- # sql (69)
- # tools-deps (2)
- # utah-clojurians (9)
- # xtdb (3)
Hi there! I want to use Crux to replicate some data on a running system. The problem is, this system have about 10 million rows on a relational DB, so I'll probably implement some sharding. The thing is, how to configure crux to work with this volume of data? I read that you need Kafka and RocksDB, but where do RocksDB is installed (or, is it installed anywhere)?
Hi @mauricio.szabo! 10 million doesn't sound too bad - it should take well under an hour on a reasonably sized machine (assuming your rows aren't giant). If you are just trying out Crux as single-node replica for the running system you can run it in the standalone mode, which embeds Rocks and means you don't have to worry about setting up Kafka or a JDBC log store behind the scenes. Rocks should be able to handle that volume without sweating using the default config (though there's certainly a lot of config to try out if desired). You just need to require crux-core
and crux-rocksdb
to make it happen: https://www.opencrux.com/docs#config-rocksdb
Is there an "explain how to set up everything as if I was a 5 year old" ready? 😄 Like, do I need to setup a machine to index on Rocks, and then connect my clojure app on it, or does Rocks stays on my clojure app?