Fork me on GitHub

Anyone know if Onyx is supposed to balance peers across multiple jobs if you set the task scheduler to :onyx.task-scheduler/balanced? Let's say I submit a job and it gets all the peers assigned. Then I submit another job an hour later… Onyx will rebalance the peers to make sure my new job gets some of the available vpeers, right?


I believe that is the case


@camechis me too, but I can't see it confirmed in the docs.


Oh, wait. Found it in the docs! > If a job is submitted while this scheduler is running, the entire cluster will be rebalanced. For example, if job A has all 100 peers executing its task, and job B is submitted, 50 peers will move from job A to job B.


yep, was just going to send that to you


Question answered. Thanks @camechis!


Hey guys, I have an interesting question for you. I am going to try and explain this. We are attempting to use spandex ( an elasticsearch lib ) in a bulk task. What is interesting about this library is that the ES bulk task is built around using core.async channels. We are wondering how this might work inside of a bulk task or if it will?


Currently we are going to setup the channels in a lifecycle which get passed in to the bulk task using a param


You can use lifecycles to inject any state you may need, including core.async channels.


ok, I think its just feeling wierd to put stuff on a channel and then deal with the results on an ouput channel in a single task


but thats the way the bulk ops work on this library


i think we will have to do a blocking take on the output and check for errors


Depending on what you need for reliability guarantees, it might be a better choice to use a plugin.


yeah, i don’t think we have enough time right now to build one out


for our needs