xtdb 2021-01-22 | Slack Archive

Fork me on GitHub

ordnungswidrig08:01:52

What’s the go-to solution for a non-cloud-hostet small scale HA crux deployment? 3-nodes running kafka? (I’d hate to have to maintain that)

Maybe running Postgres in HA yourself will be easier than Kafka?

➕ 3

ordnungswidrig09:01:35

IIRC this basically a failover architecture which I’m not a fan of. It’s also tricky to get STOMITH right etc.

jarohen09:01:18

If it helps, we've used the Confluent Kafka Docker images in the past with relatively little trouble

ordnungswidrig09:01:43

Does this need zookeeper be configured?

👍 4

ordnungswidrig09:01:03

With a basic setup I assume, you can run zookeeper and kafka nodes side by side on 3 nodes.

👍 4

ordnungswidrig09:01:24

This looks tempting https://github.com/spotify/docker-kafka

kosengan10:01:32

https://youtu.be/cfZQYVQcspE?t=227 crux

🙂 3

9

jonpither14:01:56

Hi all, we've published a new dev-journal you may be interested in: https://opencrux.com/blog/dev-diary-jan-21.html.

📓 15

✏️ 3

Aleksander Rendtslev04:01:28

Amazing! This is great. Keep up the good work

🙏 3

🙂 3

kosengan18:01:49

https://www.youtube.com/watch?v=zAlvM33UpGA Can Ketu bear a fruit for Crux?

👀 3

Yeah it's definitely possible that Ketu (or similar) would simplify a few things. We currently rely on Java interop and track our own offsets, but it isn't so bad. Maybe Ketu would be a good option if our usage of Kafka grows more advanced in future. Asides from a few Java classes for serdes classes and ID partitioning, all the Kafka-related parts of Crux are handled here in a single ns: https://github.com/juxt/crux/blob/master/crux-kafka/src/crux/kafka.clj

👍 3