Fork me on GitHub

I see that Onyx has a Kafka plugin but not a RabbitMQ plugin(yes, there is an unofficial plugin). But, my question is why Kafka is preferred?


Any pointers will be helpful


@sreenath.n: It's not preferred. None of the people who work on Onyx use RabbitMQ much, so no one ended up building one. I'm sure us or someone else will get to it eventually.


I have one more question. With whatever I know, I see that N_PEERS should be at least equal to the number of tasks right? But, Onyx provides a way to mention Min/Max peers for each job. Wouldn’t it be easier if we just calculate N_PEERS based on the catalog? I am asking this because I had run into a problem where N_PEERS was less than the number of tasks I had defined. I am sure I might be missing something here...