Fork me on GitHub

Hi, I'm looking into Kafka - Connect, the docs says "Download the connector from Confluent hub" but I can only find old alpha version there. no as the documentation suggests. Is there any other source where I can get that connector?


Hi, thanks for checking out the connector - sorry we've not uploaded a more recent version! You should be able to build the latest version without much trouble if you have the Crux repo cloned See: Please let us know if you have any trouble 🙂


Yes I have it.. let me try


@U899JBRPF I ended up writing my own sink connector in Java.. but I encountered the following problem when starting the connector:

[2020-09-02 09:00:08,906] ERROR WorkerSinkTask{id=CruxSink-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask)
Syntax error compiling new at (taoensso/nippy/compression.clj:43:15).
Caused by: java.lang.SecurityException: sealing violation: package org.tukaani.xz is sealed
	at by: java.lang.SecurityException: sealing violation: package org.tukaani.xz is sealed
mvn dependency tree is stating that org.tukaani.xz is included only once via crux-cor--> nippy --> xz Any idea how to get around this issue?


Hmm! Is this for 1.10.1 ?


<crux.version>20.08-1.10.1-beta</crux.version>        <clojure.version>1.10.1</clojure.version>


thanks, I'll have a look. We had a similar looking issue earlier in the year, so I need to read my notes

👍 3

thank you


I excluded tukaani.xz package from crux-core dependency and I got around this. Not sure if this will cause any other problems but at the moment it seems that Sink is working


cool, thanks for the update. I will keep you posted also 🙂

Ivan Fedorov20:09:46

Also in Kafka-Connect / Confluent area – I can’t understand yet how to estimate how much I will need pay for their awesome storage and how much storage I will need.


Hey! GCP is usually cheaper, from what I've seen. Unless you're doing something super serious (requiring multi-AZ HA) the Basic service is probably good enough:


Why they haven't created a calculator themselves...I have no idea 🙂


When you're using Confluent for Crux then you can assume that all data store will need to be retained indefinitely

Ivan Fedorov05:09:06

Thanks for pointing out pricing section!

🙂 3
Ivan Fedorov20:09:45

I’ve skimmed the surface on Confluent website and didn’t find a “calculator” for that. Maybe if you could include some estimates on opencrux website that would improve the funnel.

👍 3