Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2019-09-08

Channels

#jackdaw
<
2019-09-08
>
pdmct01:09:29

Hi, I am playing around with Jackdaw and Willa (and new to them and Kafka) and trying to build a TimeWindowed workflow and having some troubles. How do I define an entity and topology for aggregating by a time window? Here is my attempt munging the Willa example: 

(def workflow
  [[:input-topic :aggregate-stream]
   [:aggregate-stream :output-topic]])

(def entities
 {:input-topic {::w/entity-type :topic
                :topic-name "timed-readings-topic"
                :replication-factor 1
                :partition-count 1
                :key-serde (serdes.edn/serde)
                :value-serde (serdes.edn/serde)}
  :aggregate-stream {::w/entity-type :ktable
                     :willa.core/group-by-fn (fn [[k v]] (:device-number v))
                     :willa.core/window (TimeWindows/of (* 2 60 1000))} 
  :output-topic {::w/entity-type :topic
                 :topic-name "timed-readings-output-topic"
                 :replication-factor 1
                 :partition-count 1
                 :key-serde (serdes.edn/serde)
                 :value-serde (serdes.edn/serde)}})
When I try to build this topology I get a No method in multimethod 'coerce-to-ktable' for dispatch value: class jackdaw.streams.interop.CljTimeWindowedKStream exception. Am I doing this right? or on the right path?

davewm14:09:00

hi @U066S7PQC, that looks like a bug in Willa to me - I’ll take a look this evening if I have chance 🙂

davewm14:09:20

could you create an issue in the Willa github repo please? Just conscious that your message will disappear in a few days, and it’d be good to have a more permanent record of this problem

pdmct02:09:14

Changing entity-type to KStream seems to have fixed this.