Fork me on GitHub
#onyx
<
2024-05-02
Drew Verlee17:05:15

Has anyone used a streaming system like onyx, that made it easy to run the same jobs in memory in the same way as they might be run across multiple machines?

Travis17:05:36

Guess it depends on what your looking for but I use kafka streams pretty heavily

Drew Verlee17:05:34

Thanks Travis, i'm looking for the dataflow abstraction (windows, triggers, etc...) but in a single threaded process.

Drew Verlee17:05:22

that can also be deployed in a distributed system

Drew Verlee17:05:21

about 7 years ago when i looked into this, only Onyx seemed like they were trying to do this. Everything else seemed like they wanted to claim they could, but when you actually got into the docs, i felt what you learned is that serious modifications (not just additions) had to be made to transition.

Travis17:05:49

you can do a lot of windowing in kafka streams

Travis17:05:24

they support most of the same ones onyx did

Travis17:05:09

It also easy to run locally in a test with out infrastructure and to scale