Fork me on GitHub
#core-async
<
2018-10-23
>
dangercoder12:10:33

Why is it not recommended to do any type of IO in GO-blocks in Clojure? I am doing some IO now: 1. I have 10 users which I want to make an api request (per user) and buffer the result. 2. I create a go-block and a channel per user and put the user on the channel, the go-block will call a 3rd part api. 3. When the api call is done in each go-block I put the result on a "invited" chan which has a go-loop that buffers all messages untill all users are invited, and then continues..

Alex Miller (Clojure team)12:10:42

Re io, go blocks are dispatched across a fixed pool of (by default 8) threads. If you block, you tie up a thread and prevent other go blocks from proceeding. In the worst case, you can stop all progress.

Alex Miller (Clojure team)12:10:15

It’s better to do this kind of work in a dedicated thread

dangercoder12:10:11

I see, thank you for the explanation @alexmiller

mkvlr13:10:10

but it’s a bit hard where to draw the line, right? Would be interested to hear how you do this in practice. I assume logging is fine. What about reading from a datomic db using the peer library?

mkvlr14:10:16

are <! >! alts alt! the only things that can park and resume from a different thread? Context: we’re introducing a consistent logging context (using pedestal’s log/with-context). Or is that generally a bad idea to mix that and core.async?

noisesmith16:10:30

outside of a pathological case, logging shouldn't be a significant block to the thread

noisesmith16:10:24

there's a simple pattern to putting something in a thread and parking on it: (<! (async/thread (f)) will avoid blocking a core.async go thread and uses a separate expandable pool

mkvlr07:10:59

thanks! Not sure why I never got that before 🙈

mkvlr10:10:38

and async/thread uses a global thread pool, right? So I don’t really have to worry about creating too many threads this way as they’ll just be reused then the work is done.

noisesmith17:10:01

they'll be reused and eventually collected if unused for some period of time, yes