Fork me on GitHub
#xtdb
<
2020-01-10
>
mauricio.szabo20:01:41

Hi there! I want to use Crux to replicate some data on a running system. The problem is, this system have about 10 million rows on a relational DB, so I'll probably implement some sharding. The thing is, how to configure crux to work with this volume of data? I read that you need Kafka and RocksDB, but where do RocksDB is installed (or, is it installed anywhere)?

refset21:01:34

Hi @mauricio.szabo! 10 million doesn't sound too bad - it should take well under an hour on a reasonably sized machine (assuming your rows aren't giant). If you are just trying out Crux as single-node replica for the running system you can run it in the standalone mode, which embeds Rocks and means you don't have to worry about setting up Kafka or a JDBC log store behind the scenes. Rocks should be able to handle that volume without sweating using the default config (though there's certainly a lot of config to try out if desired). You just need to require crux-core and crux-rocksdb to make it happen: https://www.opencrux.com/docs#config-rocksdb

mauricio.szabo21:01:09

Is there an "explain how to set up everything as if I was a 5 year old" ready? 😄 Like, do I need to setup a machine to index on Rocks, and then connect my clojure app on it, or does Rocks stays on my clojure app?