Hey @U022N1DU7GQ - do you see any errors in the console or REPL about Kafka connectivity? Is there any evidence that the producer is able to send anything to the broker at all?

Hi @U899JBRPF, Yes this is the first half of the error message.

`FAILED_TO_LOAD_NATIVE_LIBRARY] no native library is found for os.name=Mac and os.arch=aarch64 ERROR: Unhandled REPL handler exception processing message {:op stacktrace, :nrepl.middleware.print/stream? 1, :nrepl.middleware.print/print cider.nrepl.pprint/pprint, :nrepl.middleware.print/quota 1048576, :nrepl.middleware.print/buffer-size 4096, :nrepl.middleware.print/options {:right-margin 70}, :session f5d5aa98-e692-405d-b7b0-a33f75e4f5ea, :id 50} java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: Could not initialize class org.xerial.snappy.Snappy at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191) at clojure.core$deref_future.invokeStatic(core.clj:2300) at clojure.core$future_call$reify__8439.deref(core.clj:6974) at clojure.core$deref.invokeStatic(core.clj:2320) at clojure.core$pmap$step__8452$fn__8456.invoke(core.clj:7025) at clojure.lang.LazySeq.sval(LazySeq.java:42) at clojure.lang.LazySeq.seq(LazySeq.java:51) at clojure.lang.RT.seq(RT.java:531) at clojure.core$seq__5387.invokeStatic(core.clj:137) at clojure.core$map$fn__5855.invoke(core.clj:2757) at clojure.lang.LazySeq.sval(LazySeq.java:42) at clojure.lang.LazySeq.seq(LazySeq.java:51) at clojure.lang.Cons.next(Cons.java:39) at clojure.lang.RT.next(RT.java:709) at clojure.core$next__5371.invokeStatic(core.clj:64) at clojure.core.protocols$fn__8144.invokeStatic(protocols.clj:169) at clojure.core.protocols$fn__8144.invoke(protocols.clj:124) at clojure.core.protocols$fn__8099$G__8094__8108.invoke(protocols.clj:19) at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31) at clojure.core.protocols$fn__8131.invokeStatic(protocols.clj:75) at clojure.core.protocols$fn__8131.invoke(protocols.clj:75) at clojure.core.protocols$fn__8073$G__8068__8086.invoke(protocols.clj:13) at clojure.core$reduce.invokeStatic(core.clj:6828) at clojure.core$into.invokeStatic(core.clj:6895) at clojure.core$mapv.invokeStatic(core.clj:6903) at clojure.core$mapv.invoke(core.clj:6903) at cider.nrepl.middleware.stacktrace$flag_duplicates.invokeStatic(stacktrace.clj:205) at cider.nrepl.middleware.stacktrace$flag_duplicates.invoke(stacktrace.clj:194) at cider.nrepl.middleware.stacktrace$analyze_stacktrace.invokeStatic(stacktrace.clj:219) at cider.nrepl.middleware.stacktrace$analyze_stacktrace.invoke(stacktrace.clj:213) at cider.nrepl.middleware.stacktrace$analyze_cause.invokeStatic(stacktrace.clj:323) at cider.nrepl.middleware.stacktrace$analyze_cause.invoke(stacktrace.clj:314) at cider.nrepl.middleware.stacktrace$analyze_causes$fn__4651.invoke(stacktrace.clj:348) at clojure.core$map$fn__5847$fn__5848.invoke(core.clj:2742) at clojure.core$take_while$fn__5898$fn__5899.invoke(core.clj:2901)`

Thanks, that gives a big clue. Looks like the Kafka client isn't loading on your system (it pulls in Snappy for compressing the messages that XT generates). I haven't seen this before, that I can recall

Is this XT 1.21.0? Can you share the classpath? I'm particularly curious about the kafka version shown

Sure!

:dependencies [[org.clojure/clojure "1.10.0"] [ring/ring-jetty-adapter "1.7.1"] [environ "1.1.0"] [buddy/buddy-sign "3.4.333"] [clj-http/clj-http "3.12.3"] [clojure.java-time/clojure.java-time "0.3.3"] [compojure/compojure "1.6.2"] [environ "1.1.0"] [org.clojure/tools.logging "1.2.4"] [metosin/malli "0.8.9"] [ring "1.9.5"] [ring/ring-json "0.5.1"] [ring-cors "0.1.13"] [ring/ring-defaults "0.3.2"] [com.xtdb/xtdb-core "1.21.0"] [com.xtdb/xtdb-kafka "RELEASE"] [org.clojure/tools.logging "1.1.0"] [ch.qos.logback/logback-classic "1.2.3"] [ch.qos.logback/logback-core "1.2.3"] [org.slf4j/slf4j-api "1.7.30"]

I went with RELEASE as I couldn't find xtdb-kafka on clojars.

huh, we actually host on Maven these days (since https://www.juxt.pro/blog/maven-central), and xtdb-kafak is definitely there https://repo1.maven.org/maven2/com/xtdb/xtdb-kafka/1.21.0/ - not sure why it wouldn't show for you :thinking_face:

Let me try with that version.

cool 🤞 in the meantime, the the output of running lein classpath might be useful too

Okay. With the version 1.21.0, I am getting the same error.

And this is the classpath output

I think I should go with RockDB within EC2 instead. There aren't many who use confluent so I am a bit unsure.

I'm confused... I don't even see XTDB in that classpath 😄

or Kafka for that matter, but I do see Snappy

> with RockDB within EC2 instead yeah...this is probably the better bet while you're in a prototyping (/ learning the API) phase, at least. Kafka isn't for the faint of heart!

Hmm, that's weird. But isn't classpath a reflection of what project.clj declares?

that classpath command should should flatten the whole tree of Clojure + Java deps and spit out the "final" set of Java-level deps, IIRC

I think we have a Snappy/Kafka and M1 issue here, fix incoming /cc @U0GE2S1NH

So there is an issue related to m1 Mac?

Yeah, this is the upstream fix https://github.com/xerial/snappy-java/pull/303 - we will need to bump our deps here

👍

OpenTelemetry would be great to add eventually, and some of the team here has used it before successfully too. In the meantime though do you think the JMX integration would be lacking anything obvious? https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/jmx-monitoring-integration/

It looks like OpenTelemetry offers some fairly interesting options for extension, e.g. https://github.com/open-telemetry/opentelemetry-java-contrib/blob/main/jmx-metrics/README.md :thinking_face:

We haven't looked at the JMX New Relic stuff. We use their OT lib directly to report all sorts of metrics direct to New Relic for dashboards/alerts/etc.

@U04V70XH6 I wish I felt comfortable tossing out the usual "maybe you want to try creating an XTDB module!" suggestion... but I somehow doubt you've recently discovered a bottomless pit of spare time amongst your other projects? 😉

When (if!) I actually start using XTDB, I'm much more likely to contribute stuff. Right now I'm just in very early exploratory reading. At some point I want to talk to the XTDB team about our applications and our data and see whether a migration to XTDB is even feasible (Datomic wasn't, due to various limitations).

I know it's one of the first results on google, but I really liked https://www.youtube.com/watch?v=oo-7mN9WXTw

Thanks, I came across that talk a few weeks ago. I really liked it too, probably the talk that sold me most on datalog I edited the question to indicate the resources I've already used