Hi, I'm using core async for coordinating IO and have doubts about my architecture. My app offers collaborating editing using operational transforms on documents (think Google docs). One go-loop reads all messages being sent by clients over websocket. It keeps a map of document-id -> chan and forwards them. Each document-chan is consumed by a go-loop that receives the messages, updates the state of the document (transforming the message if necessary), persists the messages in the db (blocking!), and forwards them to the other clients editing the same document. Clients only have one message in flight at a time and latency is not a huge priority – I mainly want to avoid everything grinding to a halt. Would this setup work for a larger number of documents being edited? Alternatively, I could keep a fixed number N of go-loops, that each handle M/N documents. Or is there an entirely different solution?

IO inside go blocks will break at medium scale, the fixed number of threads are low enough that you could lock all core.async up hard with less than ten documents

this is the reason clojure.core.async/thread exists, it uses a thread in an expandable pool (as opposed to the fixed pool go blocks use) and returns a channel that you can properly park on (rather than blocking on IO)

because of core.async's back pressure it is very easy to get in to a situation where a slow consumer is blocking things for everyone else

so, instead of go-loop I use async/thread and loop and then things should work until I hit the thread limit, correct?

you can call and park on thread inside go-loop

and you are very unlikely to hit any thread limit if you do that (unless you mean your OS limit for usable threads)

ah, so I create a go-loop for each doc, but then do the message handling (with the blocking IO) inside a thread call?

yeah - that's how I would do it at least

(go-loop [... ...] (let [msg (<! doc-chan)] (when (some? msg) (let [foo (<! (thread ...))] ... (recur ...))))) - something like this

gotcha, that makes perfect sense to me... thanks, noisesmith!

I treat (<! (thread ...)) as my placeholder for anything that might block, I rarely want to call some blocking operation inside a go block without doing a park on it immediately