This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-10-01
Channels
- # announcements (4)
- # aws (1)
- # beginners (60)
- # calva (10)
- # cider (21)
- # clj-kondo (38)
- # cljdoc (1)
- # clojure (59)
- # clojure-austin (1)
- # clojure-conj (1)
- # clojure-europe (19)
- # clojure-italy (9)
- # clojure-nl (29)
- # clojure-spec (6)
- # clojure-uk (85)
- # clojurescript (54)
- # community-development (11)
- # core-async (2)
- # cursive (21)
- # data-science (4)
- # datomic (39)
- # events (2)
- # fulcro (48)
- # funcool (1)
- # graalvm (5)
- # jackdaw (9)
- # kaocha (17)
- # luminus (2)
- # off-topic (10)
- # parinfer (22)
- # quil (1)
- # re-frame (4)
- # reagent (29)
- # shadow-cljs (7)
- # sql (9)
- # tools-deps (31)
- # yada (1)
Hello again... I'm trying to mix the streams + processor APIs. Specifically I want to add a Processor to the end of my topology which sends messages to multiple topics. The issue I'm hitting is that Streams (/Jackdaw) gives my processor a random name, so I have no easy way to call Topology.addSink(String processor-name, ...)
.
I'd prefer to avoid introspecting the topology to dig out that name, I can't see a way to do that robustly
Are you using the Processor
because you need some other feature not provided by the DSL? I think you can achieve what you've described with something like this...
(fn [builder]
(let [foo (k/stream builder foo-topic)]
(-> foo (k/map identity) (k/to foo-1))
(-> foo (k/map identity) (k/to foo-2))
(-> foo (k/map identity) (k/to foo-3))
builder))
This is related to a wider concern that if 1 message (a command) is broken up into many events and those events are committed separately, that a command may get partially executed in case of downtime. Otherwise I could branch
.
(defn dispatch-processor
"Why not `flat-map f` beforehand instead of passing it in here? That would
split one message into many and therefore commit many times instead of once."
[f ctx k v]
(doseq [[topic msg-key msg] (f [k v])]
(.forward ctx msg-key msg (To/child topic)))
(.commit ctx))