Fork me on GitHub
#onyx
<
2018-12-10
>
lucasbradstreet04:12:52

I gave it another run. Chrome starts up with this:

lucasbradstreet04:12:46

If I load the page manually, I see: https://imgur.com/a/PvvDiQk

youvere21:12:22

https://github.com/onyx-platform/onyx-kafka#attributes :kafka/deserializer-fn what is the recommended way to add some state (not serializable) in onyx ? The way I’ve found so far would be to set a global state and get the global state inside the deserializer function. I’ve also tried to create my own deserializer in my own task, but I have the same issue. You must use a namespace keyword. When you use a namespace keyword you can’t inject a state. I can use lifecycle, create a new task for deserialization and inject the state using the lifecycle I could also create a new def at runtime with the state, but all those solutions seems bad

(defn build-catalog [kafka-zookeeper kafka-topic bootstrap-servers]
  [{:onyx/name               :kafka-in
    :onyx/type               :input
    :onyx/medium             :kafka
    :onyx/plugin             :onyx.plugin.kafka/read-messages
    :onyx/max-peers          1
    :onyx/batch-size         50
    :kafka/zookeeper         kafka-zookeeper
    :kafka/topic             kafka-topic
    :kafka/deserializer-fn   ::deserializer-avro
    :kafka/offset-reset      :earliest}
(defn set-deserializer [schema-registry]
  (defn deserializer-avro [data]
    (avro/decode schema-registry data)))