Yeah, while we are using it, it does not run as well inside Kubernetes as we would like even after a lot of tweaking. I don’t know of any better alternatives to switch to for the dynamically creatable batch job use-case.

fwiw we're running it inside kubernetes and it runs reasonably well, but our approach is to create & throw away entire onyx clusters all the time, so they're relatively short-lived

it did take a lot of tweaking to get right