hey folks, Does onyx uses Dire ?

It used to, but we ditched it after a while because we didn’t want any performance overhead, since we care more about our users than our convenience.

Nope, took it out a few years ago

so do you use any similar tool ? or similar approach ?

We don’t, no. Most of the code base is performance sensitive, so we’re handling errors pretty meticulously.

@michaeldrogalis by meticulously you mean let it fail ?

of course if there is no catch or something

@lxsameer No I mean we’re careful around how we handle every exception, we don’t use a blanket strategy like using Dire everywhere.

@michaeldrogalis Aha, cool. I'm interested to know more on this subject, could you please refer me to a starting point in the source code ?

The code has gotten moderately hard to read over time, between building up a state machine with lifecycles and doing a lot of performance sensitive stuff, but I can point you to where we handle it in a second

@lucasbradstreet thanks man. 🙂

Main loop is here https://github.com/onyx-platform/onyx/blob/0.10.x/src/onyx/peer/task_lifecycle.clj#L456

you can see the handle-exception-fn which calls our pre-comp’d lifecycle/handle-exception code to decide what to do

in here https://github.com/onyx-platform/onyx/blob/0.10.x/src/onyx/peer/task_lifecycle.clj#L131

task-lifecycle used to look more elegant. But it also used to be able to process thousands of segments a second, not millions 😉

@lucasbradstreet thanks man, I'm sure that it's quiet good