Fork me on GitHub
#core-async
<
2018-07-17
>
uwo20:07:22

looking for critique of the approach taken here: https://gist.github.com/uwo/9a6de35d17c73337ffde3dc2c5f53c54 Any feedback welcome

uwo20:07:45

updated with deps and missing fns

hiredman20:07:44

I dunno, I suspect want you might actually want is some kind of parallel fold over the tree

hiredman20:07:04

the promise-chan thing is weird

hiredman20:07:22

you are dropping everything except the first result to come out of to

hiredman20:07:30

the whole done function and it usage is more callbacky then I like to see, the whole point of core.async's go macro is to try and eliminate those

hiredman20:07:30

core.async is really about building stateful machines that communicate, if you just want parallel data processing, reducers or maybe using forkjoin directly might be better

hiredman21:07:32

the logic for done doesn't seem to be correct

hiredman21:07:20

it looks like you are feeding a bunch of stuff into the pipeline to be run in parallel, but then getting the "done" time when the first thing is complete, which likely isn't what you want

uwo21:07:05

oh, just saw these responses! thanks!

uwo21:07:30

the pipe at the end just pipes to the promise chan when it closes the once

uwo21:07:53

it’d probably be less noise to monitor when to closes

uwo21:07:15

previously I was returning a single accumulation on the channel, but the process we need is just going to write to another system in xf!, hence the dropping-buffer just to drain the output. I had seen a pattern like that (dropping the output of to on the floor in datomic’s tx-pipeline example

uwo21:07:02

I totally agree on core.async empowering us to remove callbacks. I was trying to contain core.async code in a manner that tbaldridge suggest in this talk, and one of his suggestions is hiding the implementation behind a callback. Not everyone on my team is comfortable with core.async (hehe. not that I’m comfortable clearly!)

uwo21:07:08

I would love an opportunity to use fork/join;reducers, unfortunately in this case I have a stream of info that can’t fit into memory (1st issue), and the transducer passed to pipeline-blocking has to do IO. I was under the impression that reducers were for computationally expensive work only, am I off here?

noisesmith20:07:36

it won't work if there's false on the channel being drained

noisesmith20:07:56

you can use when-some rather than when to explicitly check for nil

noisesmith20:07:31

also, minor style nitpick, you never need _ bindings at the end of a let binding block - you can just move things done for side effects down into the body

uwo21:07:14

thanks. I have always wondered if the trailing _‘s were ugly!

noisesmith21:07:00

done can be replaced with take! which already exists and does the same thing

uwo21:07:58

excellent! fixed the when-some. thanks

uwo21:07:02

and now i’ve replaced done with take! I had always wondered when to use that function picard-facepalm

noisesmith21:07:33

generally take! without the callback can replace a go block that consumes from a chan but doesn't care about the value, and with the callback can replace a go block that only exists to do something to one value on a channel

noisesmith21:07:37

(fixed for accuracy)

uwo21:07:59

I’ve removed the promise-chan indirection, and just monitor the to chan to see when done