I'm tracking down an odd bug in our onyx server, which may or may not be related to the platform itself. Via request ids we have determined that (rarely) a request will get handled many times. Does this sound like a known onyx issue? just fishing for clues (also I am writing a stateful logger that will write error messages if it sees the same request id twice etc. to track it down of course)

@noisesmith: Onyx will retry a message if it doesn't complete within the default time window of 60s.

OH!

can I adjust that window? this is a big task

Yeah. In the input catalog entry, turn up :onyx/pending-timeout to a value in ms

Its tracked at the root of the workflow, which is why its configured on the input task that it comes out of

OK - this bug was driving me batty (so many places this behavior could have been coming from)

thanks

Haha sure thing

@noisesmith: BTW, we added new life cycle calls. You can hook into a retry callback: https://github.com/onyx-platform/onyx/blob/master/doc/user-guide/lifecycles.md#after-retry-message

Maybe log a warning and ensure that is the case.

oh, nice

yeah, that's an excellent idea

the problem was it was rare that the issue was triggered

What exactly is the task doing that it exceeds 60s?

thousands of networked api calls followed by a bunch of graph calculations

the task needs to be more granular

but thats on the "after it works" list - or at least it was

Haha nice. Are you finding things to be relatively stable so far? Curious about your experience.

yeah, it's definitely been the case that it's just worked after I got things set up, this has been the first hiccup

and the setup issues could have gone smoother with more time reading the docs and less time randomly typing

(as usual)

Its alright. I still do the same using infrastructure I didnt write 😛 Bash away until I hit an error, then go find the docs

Glad to hear it though. Give me a shout when you guys move to prod. Ill crack open a celebratory beer 😛