Fork me on GitHub

For a channel c, I’ve some code like:

(let [latch (atom (count urls))]
  (doseq [url urls]
      (let [resp (http/get url ...)]
        (when (some-condition resp)
          (>!! c (do-something resp))))
      (when (zero? (swap! latch dec))
        (close! c)))))
Is there a more idiomatic way to do this? Particularly the bit about closing the channel. The latch method is the best way I can think.


@xiongtx take a look at async/pipeline-blocking, it maintains ordering (might not be what you want.


But that's the general idea, create a pipeline, flow data through it


Hmm…is pipeline-blocking or pipeline-async more appropriate in this case? I don’t care about the order of the responses on the output chan. Also: - Does pipeline-*’s parallelism have the same caveat r.e. blocking ops as go blocks? - I’d assume pipeline-* spins up n threads, where n is the degree of parallelism. Then it’d be bad to block any of those n threads, yes? - Does the channel parameter of af in pipeline-async correspond to the to chan of pipeline-async itself? I assume so, but the docs aren’t clear on that.


From the src of pipeline* it seems that :blocking uses thread, while :async and :compute use go blocks, so I guess pipeline-blocking is the right thing to use.

Alex Miller (Clojure team)13:01:34

Yes, pipeline-blocking is designed for you to potentially block


Is there a a more idiomatic way to "get all values off a channel until the channel is closed" than:

(loop []
  (when-let [v (<!! c)]
    (do-something v)
I suppose I could use pipeline-* again, or map, but in this case I don't care to put the result of (do-something v) onto another channel. So this'd be more like doseq.

Alex Miller (Clojure team)13:01:26

You can reduce off the channel


Ah, yes, saw reduce. :thumbsup: