Thanks for the reply @michaeldrogalis , I'm not sure we can handle it in the application, without a ton of extra complexity, basically order matters to some degree in our system, it's hard to make it not care about order. It's just transacting that needs to happen in order. I don't mind stopping progress, because if datomic is down, we can't progress anyway regardless, so there's actually no point in requeuing the segment. Is there a way not to retry segments for certain tasks?

You can in order data happen with windows, however it will have performance characteristics that may be unfavourable until we land ABS in onyx core

How do you configure no-retry policy, or maybe freeze the job when datomic is down? Is this possible in a distributed context?

@greywolve This is not related to Onyx, but it would be helpful to know if your datomic transactions depend on the current db value or just append independent data, and whether you can skip failed transactions without retrying at all, assuming some dropped data is ok or that newer segments can fill in the missing data eventually. You have some nice options in Datomic. For example: to get the db value as-of a point in time where the processed Onyx segment was first created. If you don't have to transact every segment, you can check if a newer segment was already transacted, but indeed it is more complex. I think you can't avoid it.

Yeah we were doing as-of at the time of the tx coming in, perhaps we could just use the current db value, but that's not ideal.

ABS?

@greywolve You could let the transaction failure exception go uncaught and throw it, thus letting it kill the job.

Use a monit process to only start the job back up when Datomic is healthy again.