Fork me on GitHub
#onyx
<
2017-11-28
>
kenny03:11:48

I think the docs on onyx-kafka should mention which keys are required. I just ran into a NPE due to not having set :onyx/n-peers or :onyx/max-peers caused by https://github.com/onyx-platform/onyx-kafka/blob/86f422389d32d9feb526355954ec4ffa08638df5/src/onyx/plugin/kafka.clj#L80. It would've been helpful to have known I need to specify one of those.

lucasbradstreet03:11:17

@kenny thanks. Could you please open an issue? That should have been detected much earlier.

lucasbradstreet03:11:22

Thanks. I think we assumed a max is enforced by the onyx schema, but that’s not true.

lucasbradstreet03:11:03

Either way, it should never NPE

kenny03:11:25

Agreed 🙂

kenny20:11:26

When using the Kafka output task, do you guys usually attach an :onyx/fn to transform your segment into the format :onyx.plugin.kafka/write-messages expects?

lucasbradstreet20:11:34

It depends on whether data comes from multiple tasks. If so you may need to do it on the previous tasks, as the format may differ.

kenny20:11:35

Not sure what you mean. I assume you mean the format of my segment?

lucasbradstreet20:11:29

Main point is you can do it on either the writer task or the previous task. Which is best depends on your job.

lucasbradstreet20:11:42

I’d prefer to put it closer to the output task in general though.

kenny20:11:55

Right, makes sense. Thanks!