core-async

winsome 2023-09-12T15:22:55.594649Z

I'm reducing over a collection, and I need to take from a channel inside of the reducing function.

(reduce (fn [acc item]
          (update acc :x conj (<! (get-some-async-stuff acc item)))))
        {}
        items)
This doesn't work because I need to be in a go-block and that means the accumulator needs to be a channel and everything gets messy very quickly. Is there a recipe for this sort of a thing, or is it just messy? I've started trying to implement via pipeline-async or maybe using transducers, but if this is a familiar problem I'd welcome some advice.

Jan K 2023-09-12T15:35:35.399769Z

You can always turn it into a loop (or go-loop )

2023-09-12T15:50:34.128469Z

Turn it into a map instead of a reduce, use pipeline for the map, use async/into to turn the result of the pipeline back into a collection

winsome 2023-09-12T17:37:08.658589Z

The problem was a little more complex, it's a series of nested reduces for traversing arbitrarily-shaped trees. I ended up rewriting it as go-loops, which is less elegant but does the job.

2023-09-12T17:39:12.629449Z

reduce is particularly problematic because it is sort of its own callback based system (which is what CollReduce is) and that conflicts with the callback system that core.async is

winsome 2023-09-12T17:40:06.670979Z

Yeah, I really wanted to try it as a transducer on a chan, but I wasn't able to make it work and loop was just sitting there the whole time.

2023-09-12T17:45:31.188829Z

I have played around a little bit with a version of iteration that iterates a process (a thing that runs over time on some channels) instead of a function, and I wonder what that would look like for reduce

2023-09-12T17:47:32.273999Z

(the reduce-async above is fine, but doesn't scratch that itch because af is a function that returns a channel, not a process)

winsome 2023-09-12T17:49:18.610339Z

I've been playing around with Promesa's csp namespace with virtual threads and I think for green field work in the future I'll be skipping core.async - it's so much easier to just block and have the runtime do the right thing. Virtual threads are really great!

2023-09-12T17:50:44.354709Z

core.async's blocking ops should work just fine with virtual threads

winsome 2023-09-12T17:51:35.187139Z

I think they do, but I like Promesa's api more.

2023-09-12T17:58:52.315629Z

I guess, promesa's csp too me looks like a rather mechanical copying of parts of core.async's api, without fixing any of the internal issues in core.async(particularly the resource handling around alts), using virtual threads, which are a brand spanking new feature.

winsome 2023-09-12T18:04:04.519689Z

I think that's a pretty on-the-nose description. I do like that go just spawns a vthread instead of doing the whole state-machine macro dance, and all the limitations that brings, but obviously core.async couldn't do that because vthreads are new. And I also like the core promise library so using csp is just an extra bonus.

ghadi 2023-09-12T19:40:23.814369Z

clojure.core.async/reduce exists. different use case

2023-09-12T21:16:14.881179Z

https://downey.family/p/2023-09-12/async-reduce.clj.html maybe something like this is the higher order async-reduce, not sure anyone would actually want it ever

2023-09-12T21:17:23.896599Z

like a copy with a feedback loop