This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-02-11
Channels
- # announcements (6)
- # babashka (61)
- # beginners (85)
- # calva (21)
- # cider (6)
- # clara (9)
- # clj-http (1)
- # clj-kondo (35)
- # cljfx (6)
- # clojure (91)
- # clojure-australia (11)
- # clojure-europe (23)
- # clojure-italy (7)
- # clojure-losangeles (2)
- # clojure-nl (27)
- # clojure-uk (107)
- # clojurescript (4)
- # community-development (1)
- # cursive (69)
- # emacs (12)
- # fulcro (29)
- # graalvm (25)
- # honeysql (10)
- # hugsql (3)
- # integrant (13)
- # jobs (4)
- # kaocha (3)
- # keechma (1)
- # lambdaisland (3)
- # leiningen (2)
- # meander (17)
- # mount (3)
- # observability (1)
- # off-topic (86)
- # pathom (3)
- # polylith (2)
- # practicalli (14)
- # reitit (14)
- # shadow-cljs (61)
- # startup-in-a-month (1)
- # tools-deps (9)
- # vim (54)
- # xtdb (16)
https://vectorized.io/ Seems like a potential alternative to Kafka for Crux.. any thoughts?
It seems pretty neat, but we've not experimented with it ourselves. There's an open question about whether or not idempotence=true is critical for Crux to work correctly (I'm not sure whether Redpanda has implemented that yet) which would require some investigation before I think we could say it was a suitable replacement. The performance boost certainly couldn't hurt :)
They are almost done removing the zookeeper from Kafka. And with docker it's easy to setup anyway. But just for Crux Kafka might be overkill.
it's still 3+ separate jvms to run the coordinators though. rp is single-binary with a very convenient cli tool. it's gratis as well
Not sure if here is the right place to follow up on this discussion, but I am pretty curious about Redpanda. Jock is also interesting but seems more beta status.. I wonder how stable Redpanda is, what parts of Kafka it can actually replace plus how compatible it will be then. I wonder if you can still run Kafka Streams and Kafka Connect on top of Redpanda.. but I suppose it should work?
Guess here is fine, I did see some issues with consumer groups, but they seem supported, unlike jocko. Once you want to scale you really need/want to use use consumer groups. Redpanda even seems to have a schema registry. Further off-topic but one of my ideas was to build something like Redpanda in Rust, but if it already exists is not that usefull..
writing a scylla/cassandra doc store; so it seems that the id gets passed into the doc record as strings? am I going through id->buffer and then converting the agrona thing into bytes, if I want to store the id,doc pairs both as BLOBs?
argh, now alia wants me to convert a java.nio.HeapByteBuffer
to something nippy can thaw. I just want to pass around 0's and 1's!
ah, I think ztellman is my savior https://github.com/clj-commons/byte-streams
actually, crux.memory has an impl for ByteBuffer already so I just used that. It works!
inchoate scylla doc store, barely tested, for anyone interested: https://gist.github.com/nivekuil/75981cfb573caeb2818e01bdddfc46ed