Clojurians Log v2

@jarvinenemil has joined the channel

Are there any guides on how to create a consumer? I currently get Missing required configuration "key.deserializer" which has no default value.

Sure

Here's a snippet:

(defn -main
  [& _]
  (echo-environment)
  (with-open [consumer (-> (jackdaw-client/consumer consumer-config)
                           (jackdaw-client/subscribe [my-topic]))]
    (doseq [message (jackdaw-log/log consumer (-> env :topic :poll-interval-ms))]
      (process/foreach message))))

(def consumer-config
  (merge (-> env :broker)
         {"key.deserializer" (.getName StringDeserializer)
          "value.deserializer" (.getName StringDeserializer)}))

(:require [my-foo-process :as process]
            [jackdaw.client.log :as jackdaw-log]
            [jackdaw.client :as jackdaw-client]
            [jackdaw.serdes.json :as json-serde]
            [config.core :refer [env]]
            [clojure.pprint :refer [pprint]])
  (:import [org.apache.kafka.common.serialization Serdes StringDeserializer])
  (:gen-class))

I use config.core to store in a config.edn values, such as :topic, :poll-interval-ms and :broker

hope that helps to get you on the path

Ah yeah, thanks for the example. ✌️

it's s bit, bitty, since it's copy&pasta from a program in use, so can't show company stuff

but it's a generic enough setup

Still not 100% about when to use value-serdes and key-serdes in the client.consumer api though 😄

I know the java api for kafka, just something I dont want to use 🙂

nvm I figured it out now

Do you process your messages "async" with core.async @dharrigan?

nope

no need to complicate things for now, works fine 😉