This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-02-02
Channels
- # admin-announcements (33)
- # announcements (2)
- # beginners (75)
- # boot (340)
- # braid-chat (6)
- # cider (30)
- # cljsrn (44)
- # clojars (19)
- # clojure (169)
- # clojure-austin (12)
- # clojure-czech (1)
- # clojure-japan (6)
- # clojure-miami (1)
- # clojure-poland (7)
- # clojure-russia (83)
- # clojurebridge (4)
- # clojurescript (166)
- # community-development (55)
- # component (2)
- # core-async (39)
- # core-matrix (3)
- # cursive (32)
- # data-science (3)
- # datavis (3)
- # datomic (58)
- # dirac (28)
- # emacs (4)
- # events (7)
- # hoplon (254)
- # immutant (29)
- # jobs (2)
- # jobs-discuss (4)
- # ldnclj (35)
- # lein-figwheel (3)
- # mount (202)
- # off-topic (9)
- # om (123)
- # onyx (22)
- # parinfer (112)
- # proton (11)
- # re-frame (6)
- # reagent (43)
- # ring (3)
- # spacemacs (2)
@sirsean: Any luck?
Ah, alas no. But I only tried for a little bit after you left, and now I’m about to get some dinner. I’ll be back at it in the morning, with a different “get stuff into Kafka” approach than kafkacat. (It would probably be useful for me to figure out the kafkacat problem too, but honestly that’s lower priority for me right now than the app I’m trying to build.)
Yup gotcha! The way I see it is if you can get it working with onyx-seq
/`onyx-core-async` then thats 90% of the struggle. The input/output plugins are just configuration at that point
thanks, @mention me if you want to work through this more!
@lvh I'm keen to turn the meetup part of the template into a template option, but we didn't have the time yet. We wanted to get something out and leaned towards including it
For now we think it's better to include more and strip it down
lucasbradstreet: Yep, absolutely. That way at least I can backport stuff that’s interesting
now if only I knew what a lisp was.
Hahaha
@greywolve: I’ve decided that more the merrier for onyx-metrics - we’d love a PR
@lucasbradstreet: will try get to that this Friday 😄
So I’m wondering if anyone can help me debug this problem? Rather than using kafkacat, I made a different Docker image that produces messages to Kafka. I’ve verified this works by making it send to Kafka, and then setting up a consumer that runs on my host and connects to the Kafka topic running inside Docker, and I can see the messages. (Likewise, I can send messages from outside and see them come out. So Kafka seems to be working.)
peer_1 | 16-Feb-02 21:40:50 3c89bb885d2b INFO [onyx.peer.task-lifecycle] - [566c0298-7791-408a-894b-ecbc591c4146] Stopping Task LifeCycle for :read-lines
peer_1 | 16-Feb-02 21:40:50 3c89bb885d2b WARN [onyx.peer.task-lifecycle] - Caught exception inside task lifecycle. Rebooting the task.
peer_1 | java.lang.Thread.run Thread.java: 745
peer_1 | java.util.concurrent.ThreadPoolExecutor$Worker.run ThreadPoolExecutor.java: 617
peer_1 | java.util.concurrent.ThreadPoolExecutor.runWorker ThreadPoolExecutor.java: 1142
peer_1 | java.util.concurrent.FutureTask.run FutureTask.java: 266
peer_1 | ...
peer_1 | clojure.core/binding-conveyor-fn/fn core.clj: 1938
peer_1 | onyx.plugin.kafka/start-kafka-consumer/fn kafka.clj: 164
peer_1 | onyx.plugin.kafka/reader-loop kafka.clj: 80
peer_1 | onyx.plugin.kafka/reader-loop/invokeStatic kafka.clj: 97
peer_1 | onyx.peer.operation/kw->fn operation.clj: 27
peer_1 | onyx.peer.operation/kw->fn/invokeStatic operation.clj: 34
peer_1 | clojure.core/ex-info core.clj: 4617
peer_1 | clojure.core/ex-info/invokeStatic core.clj: 4617
peer_1 | clojure.lang.ExceptionInfo: Could not resolve symbol on the classpath, did you require the file that contains the symbol ?
peer_1 | data: {:kw nil}
(I’d been wondering how I can get read-lines
to log more, but this gives me the impression it’s failing to start read-lines
because it can’t set up a Kafka consumer correctly. I am so open to being corrected on that.)
Looks like you haven't supplied the deserializer-fn to the Kafka task
Ah! Since my message is just plaintext, I removed the JSON deserializer … and didn’t replace it. Trying...
Ah. We should have a better error message there too