Fork me on GitHub

looking for critique of the approach taken here: Any feedback welcome


updated with deps and missing fns


I dunno, I suspect want you might actually want is some kind of parallel fold over the tree


the promise-chan thing is weird


you are dropping everything except the first result to come out of to


the whole done function and it usage is more callbacky then I like to see, the whole point of core.async's go macro is to try and eliminate those


core.async is really about building stateful machines that communicate, if you just want parallel data processing, reducers or maybe using forkjoin directly might be better


the logic for done doesn't seem to be correct


it looks like you are feeding a bunch of stuff into the pipeline to be run in parallel, but then getting the "done" time when the first thing is complete, which likely isn't what you want


oh, just saw these responses! thanks!


the pipe at the end just pipes to the promise chan when it closes the once


it’d probably be less noise to monitor when to closes


previously I was returning a single accumulation on the channel, but the process we need is just going to write to another system in xf!, hence the dropping-buffer just to drain the output. I had seen a pattern like that (dropping the output of to on the floor in datomic’s tx-pipeline example


I totally agree on core.async empowering us to remove callbacks. I was trying to contain core.async code in a manner that tbaldridge suggest in this talk, and one of his suggestions is hiding the implementation behind a callback. Not everyone on my team is comfortable with core.async (hehe. not that I’m comfortable clearly!)


I would love an opportunity to use fork/join;reducers, unfortunately in this case I have a stream of info that can’t fit into memory (1st issue), and the transducer passed to pipeline-blocking has to do IO. I was under the impression that reducers were for computationally expensive work only, am I off here?


it won't work if there's false on the channel being drained


you can use when-some rather than when to explicitly check for nil


also, minor style nitpick, you never need _ bindings at the end of a let binding block - you can just move things done for side effects down into the body


thanks. I have always wondered if the trailing _‘s were ugly!


done can be replaced with take! which already exists and does the same thing


excellent! fixed the when-some. thanks


and now i’ve replaced done with take! I had always wondered when to use that function picard-facepalm


generally take! without the callback can replace a go block that consumes from a chan but doesn't care about the value, and with the callback can replace a go block that only exists to do something to one value on a channel


(fixed for accuracy)


I’ve removed the promise-chan indirection, and just monitor the to chan to see when done