Fork me on GitHub
#core-async
<
2021-04-15
>
Steven Katz18:04:55

I have a basic channel closing question. If I am the creater of a channel (and the only writer) should I close the channel after putting on its one and only value? The use case is connecting the call back from http-kit into core async/channels. My original thought is:

(defn call->channel [url]
  (let [c (chan)]
    (http/get url (fn [{:keys [status headers body error] :as resp}]
                                        (if error
                                          (put! c error)
                                          (put! c resp))))
    c)) 

Steven Katz18:04:22

But should it be:

(defn call->channel [url]
  (let [c (chan)]
    (http/get url (fn [{:keys [status headers body error] :as resp}]
                                        (if error
                                          (put! c error)
                                          (put! c resp))
                    (close! c)))
    c))

ghadi18:04:40

the second one is incorrect because the channel is unbuffered

ghadi18:04:09

unbuffered (aka "rendezvous") channels, require the putter and taker to rendezvous

ghadi18:04:48

when you put! but no one there to rendezvous with take, then close, the put will never complete into the channel

ghadi18:04:11

so either 1) don't close, or 2) close && buffer the chan (chan 1)

phronmophobic18:04:50

if you only intend on putting one value, then you may want to use https://clojuredocs.org/clojure.core.async/promise-chan

Steven Katz18:04:35

still have the issue of closing the channel though?

phronmophobic18:04:55

Is there a requirement that the chan must be closed? For most use cases, simply putting a single value on the promise chan is all that is required.

Steven Katz18:04:55

ok, but in case 1 who would be responsible for closing, the reader?

ghadi18:04:09

no, reader closing is often a bad pattern

ghadi18:04:18

you could choose not to close

ghadi18:04:32

I think your best bet is buffer=1 + close

Steven Katz18:04:36

so no one closes it, maybe I’m miss-understanding the need to close channels at all

ghadi18:04:58

thread does something similar to your usecase internally

Steven Katz18:04:37

Thanks, I’ll take a look. In general is it more typical to have channels with a buffer >= 1 when the channel is going to be consumed in a go block? (and I am correct in thinking that put! does not park/block the caller?)

ghadi18:04:56

in general you have to think carefully about the buffers

ghadi18:04:13

the same algorithm might be correct without buffers, then deadlock with buffering

ghadi18:04:48

in this case, you know there will be one thing put into the channel, and that it might need to be put into the channel before the consumer takes it

ghadi18:04:31

buffering is not required, and there are various strategies (dropping/sliding buffers)

ghadi18:04:53

rendezvous channels (no buffer) are very useful and are essentially the core primitive in the CSP paradigm

ghadi18:04:20

a lot of the pictures from the old JCSP project are relevant to core.async

Steven Katz18:04:03

This is from the doc on close:

Logically closing happens after all puts have been delivered. Therefore, any
blocked or parked puts will remain blocked/parked until a taker releases them.
This implies to me that puts on an unbuffered channel would be allowed to be taken even if it has been closed?

Steven Katz18:04:00

(going to write some test code for my own understanding anyway)

Steven Katz19:04:12

testing seems to indicate that this worked:

(defn blah []
  (let [c (chan)]
    (go (>! c "hello")
        (close! c))
    c))

atom.core> (def c (blah))
#'atom.core/c
atom.core> (<!! c)
"hello"
atom.core> (<!! c)
nil
So putting a value on an unbuffered channel and then closing it still allows the value to be delivered and then the channel is closed. I think the code snippet I first posted might have been misleading as it make the close look like it is part of the outer function and not the callback.

ghadi19:04:58

you are misinterpreting

ghadi19:04:20

it's the first <!! that is allowing the >! to proceed

ghadi19:04:29

the channel is closed after the >!

ghadi19:04:37

the docstring on close! is related to put that are "delivered" -- delivered means you're putting into a channel with an open space in a buffer or a put with a rendezvous take

Steven Katz19:04:38

Here’s whats in my head: Without any buffer: 1. Try to put something on the channel in a go block, get parked 2. Someone tries to take, it works because their is a pending put 3. execution in the go block resumes because of the take, channel gets closed With buffer == 1 1. Try to put something on the channel in a go block, succeeds because buffer is empty 2. execution continues, channel is closed 3. Someone tries to take, it works because there is a value in the buffer Is this interpretation correct?

raspasov19:04:09

@steven.katz why do you need to close the channel? If you can avoid closing, that usually makes things simpler.

Steven Katz19:04:51

Thats part of my original question, “Should I care about closing channels the same way I care about closing files etc?” (maybe its a language thing)

raspasov19:04:02

I don’t think so

raspasov19:04:26

A promise chan that receives a value is effectively “closed”, since it will keep returning the same value and no other value can replace it

raspasov19:04:09

Don’t close channels unless you can’t solve your problem without closing. If there are no more references to that channel, it will be garbage collected (barring any gotchas and bugs). Closing shouldn’t make a difference.

3
cassiel09:04:35

Sorry for jumping in at the end of a conversation but: in the Java/Clojure world with real multithreading, if I let a non-closed channel, with a thread waiting to read from it, drop out of scope, is everything GCed? Or are threads treated as roots for GC?

raspasov18:04:23

I assume it will be GC-ed, but somebody more experienced with the details of GC should chime in. Assuming that the GC has established that nothing can reach that object (the channel), it should be GC-ed (as far as my understanding of GC goes). Recently watched this talk, I recommend it. Very good high level overview of different memory management models (manual, reference counting, garbage collection) and then more detailed explanation of the different garbage collectors available on the JVM https://www.youtube.com/watch?v=k4vkd0ahWjQ

raspasov18:04:38

If I am understanding correctly, I believe your question touches on the topic of “Can GC effectively deal with retain cycles?” (discussed in the video above). The answer is yes, it can.

cassiel18:04:29

Way back when I was a wee nipper, we learned that the GC roots included any threads, since they could still be active even if nothing they were doing was still reachable from the main thread’s environment and stack. But my knowledge here is old.

raspasov18:04:23

I assume it will be GC-ed, but somebody more experienced with the details of GC should chime in. Assuming that the GC has established that nothing can reach that object (the channel), it should be GC-ed (as far as my understanding of GC goes). Recently watched this talk, I recommend it. Very good high level overview of different memory management models (manual, reference counting, garbage collection) and then more detailed explanation of the different garbage collectors available on the JVM https://www.youtube.com/watch?v=k4vkd0ahWjQ

raspasov18:04:38

If I am understanding correctly, I believe your question touches on the topic of “Can GC effectively deal with retain cycles?” (discussed in the video above). The answer is yes, it can.