@michaeldrogalis: Hello, I have a question: would Onyx make sense on a single node because you may want to use its "information model for the description and construction of distributed workflows" to ensure that you are following a pattern that describes a data flow without having to devise it from scratch

Hi @stathissideris. Yes, we have users who are currently using it in this way, and it works very well for this use, and ensures that you can leverage all the cores that your machine has.

When operating on a single machine, short circuiting will always be used to ensure that there is no serialisation between tasks, and thus the performance overhead is minimal.

wow, short circuiting sounds great

thanks, I've started reading the documentation and onyx looks extremely well thought out

Yep, it’s generally useful feature that works when the virtual peers are on the same machine - which on a single node is all the time. With our recent scheduler improvements we’re improving performance a lot by increasing the amount of co-location between communicating peers on multi-node setups.

Good to hear that

are you referring to :onyx/group-by-fn or something else?

Just the messaging layer / peer scheduler. Sorry, it’s mostly irrelevant information if you’re running on a single node

oh so peers that communicate a lot a co-located in the same physical node by the scheduler?

Yes, that’s the idea. We’re at early stages with it but we’re already seeing a lot of gains

I'm overseeing a project that ended up looking like a messy coupled dataflow... I'm considering using onyx to clean it up a bit and to future-proof the project for larger volumes of data... I'll keep reading but it sounds very promising

Sounds like a good fit. Let me know if you have any questions later on