Fork me on GitHub
#jackdaw
<
2019-07-22
>
xi13:07:05

Anyone here please? Having trouble understanding how Jackdaw maps onto the Java API. In streams.clj I can see a bunch of calls to protocols, and I can see the protocols, but... I don't get how they join up. For example, the protocol method is kstream but on the Java API that's called stream. Something must be doing the mapping? I've had a poke through the source and I just don't see it 😐

xi13:07:06

Also not sure how to materialize a join of two KTables to a store

xi13:07:43

Great to have this library for free but well, the doc can be rather sporadic. A lot of methods take functions and there's no detail of the signature required.

cddr14:07:30

Hey @dbernal, Yes you can use KafkaStreams with the avro serde but you need to pass it as an argument to the .stream or .table function rather than rely on the default configured in the application config. Here's where this is done in the example app... https://github.com/FundingCircle/jackdaw/blob/master/examples/word-count/src/word_count.clj#L76 This is because if you let kafka look it up by name, it won't find dynamically created Clojure classes of the type created by the avro-serde.

👍 4
cddr14:07:28

Hey @slack1274, https://github.com/FundingCircle/jackdaw/blob/master/src/jackdaw/streams/interop.clj is where the rubber meets the road. I can't remember why stream was changed to kstream. Wherever you see a topic-config map that is referenced in the docs, the keys for this map are spec'd in "src/jackdaw/specs.clj". This should be better documented. When I get a chance, I'm going to add a high-level guide to "configuring" to go alongside the existing guides.