@michaeldrogalis @lucasbradstreet I’m working on a onyx project (using onyx-kafka-8 plugin) that has 3 Kafka partitions but it only reads from the last one. When I look in zookeeper the consumers in Kafka only report one consumer being read from.

Has anyone else had the same issue? Or is there something I should be looking at in the config.

@jasonbell Are you assigning a partition in the catalog entry?

It works once through then hangs on the next run. Deployed on mesos.

@michaeldrogalis using min/max peers set to the number of partitions.

so they should auto-assign

Mmkay.. We do auto-discovery to make sure there are enough peers to cover all the partitions, so it’d throw if there weren’t enough.

I’m not sure how easy this would be to test — but can you set up a consumer outside of Onyx to test that connectivity to all partitions is working?

I’ll see what I can do

What’s the preferred method, assign one task per partition

or use min/max to auto assign

Nah, what you’re doing is fine.

ok