I'm wondering (new to the library), if I wanted simply to consume messages from a kafka topic and invoke a function, passing in each message, without any further action (no sending on to other topics), which jackdaw API would suit? Sort of like a peek, but a do 🙂 Foreach?

Hey @dharrigan, Thanks for trying it out. The jackdaw.client.log/log function gives you a lazy seq of "datafied" consumer records. So e.g. something like this...

(with-open [consumer (-> (jackdaw.client/consumer consumer-config)
                         (jackdaw.client/subscribe topic-config))]
  (doseq [elem (jackdaw.client.log/log consumer 1000)]
     (do-it elem)))

You're welcome, thanks for providing a nice library! So, I can call (client/log 1000 my-function) and my-function would receive a message (if there are ones to consume of course)

thank you!

thanks for the code snippet! Very handy! 🙂

The devil is in the details though. Once you start breaking your topic up into partitions, and having opinions about what should happen when things fail, get re-started, etc, then kafka streams has support for implementing the desired behavior.

Thanks again 🙂 Very useful - just starting small and working up to it. Thanks for your time.

@cddr Thank you for the help previously, I got my client to work successfully now! Have a nice afternoon!