I like not needing to stand up kafka......

But I also like not paying for software as a tiny entity.

It seems pretty neat, but we've not experimented with it ourselves. There's an open question about whether or not idempotence=true is critical for Crux to work correctly (I'm not sure whether Redpanda has implemented that yet) which would require some investigation before I think we could say it was a suitable replacement. The performance boost certainly couldn't hurt :)

They are almost done removing the zookeeper from Kafka. And with docker it's easy to setup anyway. But just for Crux Kafka might be overkill.

it's still 3+ separate jvms to run the coordinators though. rp is single-binary with a very convenient cli tool. it's gratis as well

Not sure if here is the right place to follow up on this discussion, but I am pretty curious about Redpanda. Jock is also interesting but seems more beta status.. I wonder how stable Redpanda is, what parts of Kafka it can actually replace plus how compatible it will be then. I wonder if you can still run Kafka Streams and Kafka Connect on top of Redpanda.. but I suppose it should work?

Guess here is fine, I did see some issues with consumer groups, but they seem supported, unlike jocko. Once you want to scale you really need/want to use use consumer groups. Redpanda even seems to have a schema registry. Further off-topic but one of my ideas was to build something like Redpanda in Rust, but if it already exists is not that usefull..

they get passed in as crux.codec.Id objects - that should work, yep 🙂

argh, now alia wants me to convert a java.nio.HeapByteBuffer to something nippy can thaw. I just want to pass around 0's and 1's!

ah, I think ztellman is my savior https://github.com/clj-commons/byte-streams

actually, crux.memory has an impl for ByteBuffer already so I just used that. It works!