This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-09-01
Channels
- # announcements (6)
- # atom-editor (4)
- # babashka (59)
- # beginners (51)
- # bristol-clojurians (6)
- # calva (5)
- # cider (22)
- # clara (6)
- # cljfx (28)
- # clojure (182)
- # clojure-australia (1)
- # clojure-berlin (1)
- # clojure-dev (21)
- # clojure-europe (12)
- # clojure-italy (8)
- # clojure-nl (7)
- # clojure-norway (13)
- # clojure-uk (20)
- # clojurescript (2)
- # code-reviews (24)
- # conjure (9)
- # cursive (9)
- # datomic (35)
- # defnpodcast (45)
- # fulcro (18)
- # graalvm (3)
- # graphql (1)
- # hugsql (1)
- # jobs (3)
- # jobs-discuss (2)
- # klipse (1)
- # leiningen (1)
- # malli (5)
- # membrane (4)
- # off-topic (7)
- # portal (2)
- # reitit (30)
- # remote-jobs (1)
- # rewrite-clj (5)
- # ring (2)
- # shadow-cljs (25)
- # test-check (2)
- # tools-deps (5)
- # vrac (26)
- # xtdb (17)
Hi, I'm looking into Kafka - Connect, the docs says "Download the connector from Confluent hub" but I can only find old alpha version there. no juxt-kafka-connect-crux-20.08-1.10.1-beta.zip
as the documentation suggests. Is there any other source where I can get that connector?
Hi, thanks for checking out the connector - sorry we've not uploaded a more recent version! You should be able to build the latest version without much trouble if you have the Crux repo cloned See: https://github.com/juxt/crux/tree/master/crux-kafka-connect#packaging-the-connector Please let us know if you have any trouble 🙂
@U899JBRPF I ended up writing my own sink connector in Java.. but I encountered the following problem when starting the connector:
[2020-09-02 09:00:08,906] ERROR WorkerSinkTask{id=CruxSink-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask)
Syntax error compiling new at (taoensso/nippy/compression.clj:43:15).
...
...
Caused by: java.lang.SecurityException: sealing violation: package org.tukaani.xz is sealed
at java.net.URLClassLoader.getAndVerifyPackage(URLClassLoader.java:400)Caused by: java.lang.SecurityException: sealing violation: package org.tukaani.xz is sealed
at java.net.URLClassLoader.getAndVerifyPackage(URLClassLoader.java:400)
....
mvn dependency tree is stating that
org.tukaani.xz
is included only once via
crux-cor
--> nippy
--> xz
Any idea how to get around this issue?<crux.version>20.08-1.10.1-beta</crux.version> <clojure.version>1.10.1</clojure.version>
thanks, I'll have a look. We had a similar looking issue earlier in the year, so I need to read my notes
I excluded tukaani.xz package from crux-core dependency and I got around this. Not sure if this will cause any other problems but at the moment it seems that Sink is working
Also in Kafka-Connect / Confluent area – I can’t understand yet how to estimate how much I will need pay for their awesome storage and how much storage I will need.
Hey! GCP is usually cheaper, from what I've seen. Unless you're doing something super serious (requiring multi-AZ HA) the Basic service is probably good enough:
When you're using Confluent for Crux then you can assume that all data store will need to be retained indefinitely
I’ve skimmed the surface on Confluent website and didn’t find a “calculator” for that. Maybe if you could include some estimates on opencrux website that would improve the funnel.