nbb

ray 2023-07-29T12:58:28.328249Z

playing around with https://github.com/Blizzard/node-rdkafka

ray 2023-07-29T12:59:20.364619Z

it's no wonder some people (ahem) are not fans of async code ... here's how to connect

ray 2023-07-29T13:00:13.692779Z

(defn ->producer []
  (p/let [producer (Producer. kafka-config)]
    (let [resolve (p/deferred)
          reject (p/deferred)]

      (set! (.-on producer "ready") (fn []
                                      (js/console.log "Producer is ready")
                                      (p/resolve! resolve producer)))

      (set! (.-on producer "delivery-report") (fn [err report]
                                                (if err
                                                  (println (js->clj err))
                                                  (println (js->clj report)))))

      (set! (.-on producer "event.error") (fn [err]
                                            (js/console.log "event.error" err)
                                            (p/reject! reject err)))

      (.connect producer)
      (.setPollInterval producer 100)
      (p/any [resolve reject]))))

ray 2023-07-29T13:01:27.542219Z

and of course it's reporting ready via event.error thinking-face ... so might not be quite right 😆

borkdude 2023-07-29T13:52:47.958319Z

Do they have a promise based API?

ray 2023-07-29T15:24:06.986749Z

yes

ray 2023-07-29T19:11:01.910679Z

after more fiddling ... this is how it works

ray 2023-07-29T19:11:08.827139Z

(defn create-producer []
  (let [producer (Producer. kafka-config)
        connection (p/deferred)]
    (.on producer "ready" (fn [] (p/resolve! connection producer)))
    (.on producer "delivery-report" (fn [err report]
                                      (if err
                                        (js/console.warn "Delivery error: " err)
                                        (let [{:keys [key value]} (js->clj report {:keywordize-keys true})]
                                          (println "Delivery data - key:" (str key) "value:" (str value))))))
    (.on producer "event.error" (fn [err] (p/reject! connection err)))
    (.connect producer)
    (.setPollInterval producer 100)

    connection))

👍 1
ray 2023-07-29T21:04:32.877899Z

I did a search on github for uses of this lib and could not find any CLJS examples so I'll make a small repo for future reference once I've filled out the consumer and a few test cases

ray 2023-07-29T21:05:15.736489Z

it's the recommended lib by Confluent so it could be good to have a working example

borkdude 2023-07-29T21:26:31.981469Z

cool. in the nbb project examples directory there's also lots of libraries. perhaps you can put it there as well

👌🏼 1