This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-02-01
Channels
- # announcements (11)
- # babashka (71)
- # beginners (34)
- # calva (25)
- # chlorine-clover (38)
- # cider (13)
- # clj-kondo (1)
- # cljsrn (2)
- # clojure (40)
- # clojure-australia (4)
- # clojure-europe (16)
- # clojure-france (3)
- # clojure-nl (4)
- # clojure-uk (16)
- # clojurescript (27)
- # conjure (2)
- # core-async (41)
- # core-logic (3)
- # cursive (1)
- # data-science (1)
- # datomic (16)
- # depstar (19)
- # emacs (7)
- # fulcro (33)
- # graalvm (4)
- # honeysql (20)
- # hugsql (4)
- # jobs (1)
- # juxt (4)
- # off-topic (48)
- # pathom (41)
- # reagent (9)
- # reitit (19)
- # remote-jobs (1)
- # shadow-cljs (20)
- # startup-in-a-month (2)
- # tools-deps (29)
- # vim (3)
- # xtdb (30)
purely for learning purpose why does
(require '[clojure.core.async :refer [go >!! close! <!! >! <! thread]])
(def c (chan))
(do
(go
(do
(doseq [i (range 1000000)]
(>! c i))
(close! c)))
(go
(time
(loop [i (<! c)]
(when i
(recur (<! c)))))))
perform better than
(require '[clojure.core.async :refer [go >!! close! <!! >! <! thread]])
(def c (chan))
(do
(thread
(do
(doseq [i (range 1000000)]
(>!! c i))
(close! c)))
(thread
(time
(loop [i (<!! c)]
(when i
(recur (<!! c)))))))
i understand when there are lots of go-processes, the switching time pays off if goroutines are used as compared to thread
but with chan size = 1 and two goroutines, ultimately the threads from go threadpool are going to block on the channel until a put happens. so essentially at low level it is the same as thread blocking right?for the go
example, it could theoretically just bounce back and forth between pushing and pulling on the same thread.
ya that’s what i thought too but the thread ids were surprisingly different
Interesting. not sure how to investigate further without using a profiler like https://github.com/clojure-goes-fast/clj-async-profiler
at a low level a process / thread context switch is much more expensive than a goroutine state machine context switch
for example the saving of state needs to save all state when doing an OS context switch, while go reuses most of its state and just switches out state machines to pick up different blocks. even if multiple thread ids are being used, it's a fixed number of them with cooperative context switching, and it never spends time in OS allocated work slices waiting on a signal like thread
can.
Rather a newbie-flavoured question here re: combining core.async
with Stuart Sierra’s Component machinery.
Which would be better style?
(i) Have one of the component modules own all the channels in the system, and do (chan)
and (close!)
on them all, with other components referring to them when firing up their (go)
blocks
(ii) Assuming a pipeline of consume-produce components, have each one create and close the channel it sends to only (which it can be assumed to “own”)?
Bonus question: given that components are firing up their own go threads/coroutines, should they also be responsible for shutting them down? Or should the go blocks be required to exit on closed input channels, so that close!
everywhere brings them down? I’m veering towards the latter: close the channels to shut down the go blocks.
i is only more reliable if you don't understand that component does things synchronously, so if you have asynchronous tasks (via go blocks, threads, threadpools, whatever) you need to bridge that divide
your stop function shouldn't return until all the async tasks you have started have exited (it is not enough to signal them to stop)
(Aside: I’m in CLJS so it’s all coroutined.) The only way to ensure a shutdown then is to make every go block hang on an alt!
and have explicit shutdown channels. That feels like a lot of machinery to achieve something which feels like it should be simpler.
it is often easier to use an explicit shutdown channel, but you can make all your go blocks check for reading nil from the input channel
so in clj the component's stop will close the input, which will signal the go block to exit, and then do a blocking take from the channel returned when that go block is started, to ensure that after the component is stopped the go loop has exited
(if you just close the channel, your component may return from stop while the go loop is processing a back log of messages in the channel)
There’s no way to block on a channel from within the “main thread” in component, so I guess all I can do is arrange for go blocks to stop on their next read-from-closed, and be content with that.
the "correct" thing would be to write your own version of component where the lifecycle protocol is itself asynchronous
OK, so it feels like I have to live with some decoupling here - I can shut down a component, and make it shut channels, but have to code the go blocks so that they obligingly shut themselves down when the thread jumps to them.
Am happy to go with a singleton channel “owner” which mints fresh ones on start
and closes them all on stop
. I don’t think that leads to any deadlocks or orphaned go contexts.
"orphaned go contexts" - IIRC go blocks are stored on channels, if they aren't running or waiting on a channel they go out of scope and get gc'd
I believe so - they’re just parked contexts. But I think I read something recently that hinted that they came from a pool. As I think about it, that doesn’t make sense to me - I don’t see any reason why they’d need to be limited.
threads are pooled, the blocks are "unlimited"
Sure - though in CLJS I don’t have threads and am just jumping around inside a coroutine state machine, so I guess there are no limits of any kind.
well "only one thread ever exists" is a bit of a limit 😄