I'm thinking of closing the from channel from within xf. Does that make sense?

Do you want it to abort ongoing uploads, too?

Well, ideally, yes. But I'm happy with just skipping new uploads

you can close the from channel in the xform but it would be cleaner to do it at the process which consumes from to

oh, nice, yeah it may be a bit cleaner :thinking_face:

thanks for the reassurance on the approach, I was not sure if I was doing something horrific

You could just use a thread pool, by the way

yeah, I think I'll switch to those, core.async seems overkill for this

I like core async, it's useful, but unless I need CSP I don't reach for it. You could implement a worker pool with cancelation and other mechanisms, but that would just be reinventing the wheel. Thread pools exist already

well, those thread pools aren't as friendly as I thought. throttling to a max of n uploader threads is not a straightforward task. If I submit more than n files, submit throws. This is where things get ugly

What about using a blocking queue in the executor service constructor?

:thinking_face: will try later, I used SynchronousQueue as that's ultimately what core.async uses