Fork me on GitHub
#onyx
<
2018-01-05
>
Quan Nguyen03:01:37

ah, if so then I think the behavior i saw was expected. Why is after-batch still called after the task is done processing all of the batches?

fdserr05:01:55

Hi Onyxeers, happy 2018 to all! Is there a plug-in, lib, gist, or article to help me get started with idiomatic in-outing of onyx messages to/from transit over websocket? TIA!

michaeldrogalis15:01:31

@quan.ngoc.nguyen After-batch is after a single batch, not all the batches.

michaeldrogalis15:01:05

@fdserr Hi! I don't think there's a recent plugin for web socket IO. There is one for HTTP though.

kenny21:01:26

The Kafka plugin producer silently fails when a message does not adhere to the expected format. Subsequent values are then simply ignored. It seems an exception or warning should be thrown if a segment does not adhere to the expected format.

michaeldrogalis21:01:22

@kenny As it turns out, Im investigating that bug right now 🙂

michaeldrogalis21:01:58

We're going to throw here and crash the job, deferring to lifecycle restarts. Tracking down why it's been silenced though.

kenny22:01:51

Another possibly related issue that I just ran into is if an exception is thrown when attempted to serialize the Kafka message, that exception is swallowed. For example when using Nippy to serialize, I attempted to pass in a Datomic Datom as part of the Kafka :message. That silently fails when an exception should be thrown. Normally if you attempt to serialize something Nippy doesn't know how to serialize an ExceptionInfo is thrown.

michaeldrogalis22:01:06

Sounds like it's right around the same area.

michaeldrogalis22:01:27

I'll keep you posted when I figured it out, though do let me know if you find the root cause first

michaeldrogalis22:01:41

@kenny The venerable @gardnervickers will be picking up the bug from here, I need to task switch. He's got a reproducer and isolated it to onyx-kafka.