updated with deps and missing fns

I dunno, I suspect want you might actually want is some kind of parallel fold over the tree

the promise-chan thing is weird

you are dropping everything except the first result to come out of to

the whole done function and it usage is more callbacky then I like to see, the whole point of core.async's go macro is to try and eliminate those

core.async is really about building stateful machines that communicate, if you just want parallel data processing, reducers or maybe using forkjoin directly might be better

the logic for done doesn't seem to be correct

it looks like you are feeding a bunch of stuff into the pipeline to be run in parallel, but then getting the "done" time when the first thing is complete, which likely isn't what you want

oh, just saw these responses! thanks!

the pipe at the end just pipes to the promise chan when it closes the once

it’d probably be less noise to monitor when to closes

previously I was returning a single accumulation on the channel, but the process we need is just going to write to another system in xf!, hence the dropping-buffer just to drain the output. I had seen a pattern like that (dropping the output of to on the floor in datomic’s tx-pipeline example

I totally agree on core.async empowering us to remove callbacks. I was trying to contain core.async code in a manner that tbaldridge suggest in this talk, and one of his suggestions is hiding the implementation behind a callback. Not everyone on my team is comfortable with core.async (hehe. not that I’m comfortable clearly!)

whoops. the mentioned talk https://youtu.be/096pIlA3GDo?t=909

I would love an opportunity to use fork/join;reducers, unfortunately in this case I have a stream of info that can’t fit into memory (1st issue), and the transducer passed to pipeline-blocking has to do IO. I was under the impression that reducers were for computationally expensive work only, am I off here?