This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-06-05
Channels
- # announcements (10)
- # beginners (59)
- # calva (172)
- # cider (13)
- # clj-kondo (1)
- # cljdoc (10)
- # cljs-dev (4)
- # cljsrn (65)
- # clojure (144)
- # clojure-europe (2)
- # clojure-italy (26)
- # clojure-losangeles (1)
- # clojure-nl (14)
- # clojure-spec (26)
- # clojure-uk (91)
- # clojurescript (75)
- # core-async (53)
- # cursive (11)
- # datomic (16)
- # fulcro (42)
- # graalvm (29)
- # graphql (9)
- # kaocha (3)
- # leiningen (22)
- # off-topic (26)
- # qa (1)
- # re-frame (3)
- # reagent (7)
- # reitit (10)
- # rewrite-clj (56)
- # robots (1)
- # shadow-cljs (107)
- # spacemacs (10)
- # specter (5)
- # sql (15)
- # tools-deps (39)
- # vim (11)
a queue somewhere has too many callbacks in it I guess
I wonder
it's probably a heap issue, or maybe there's a bound on the queue
someone else here probably has a real answer, for now I'm experimenting
(dotimes [n 5000000]
(async/go (async/>! c n)))
results in a bunch of these:
java.lang.AssertionError: Assert failed: No more than 1024 pending puts are allowed on a single channel. Consider using a windowed buffer.
(< (.size puts) impl/MAX-QUEUE-SIZE)
user=> (def a (atom 0))
#'user/a
(cmd)user=> (dotimes [_ 500000] (>/go (>/timeout 10000) (swap! a inc)))
nil
(ins)user=> @a
500000
adding another 0 to the end of that number made it take 10x as long, but still no error
oh I wasn't taking from the timeout!
that just changes wait time though
oh cool, with enough of those timeouts I was able to get the state machine to start crawling and my CPU is heating up :D
anyway, I bet @ghadi would be much more informative on this subject
experimentally with enough parked go blocks the throughput gets really low and bursty
the numbers in the atom move slower but everything still looks correct
but that's not real proof of anything, it just means my toy test worked without making core.async break
I think the assertion error you got was because everything was trying to use c
or too many things using some queue that my example doesn't use, clearly
I am seeing something in the cljs implementation of core.async that looks like a serious bug. It appears to me that using a promise-chan will break non-deterministic choice in cljs. Like, if I have an alts! choosing between a timeout and a fulfilled promise-chan, it appears to never choose the timeout chan at all.
(go
(let [p (promise-chan)
t (timeout 1000)]
(>! p {})
(go-loop []
(let [[val port] (alts! [p t])]
(if (= port p)
(do (println "loop") (recur))
(println "huh it timed out"))))))
From looking at the implementation, I don't understand why this would be, since the promise chan is just implemented in terms of a special buffer. We ran into this when porting some code from clj into cljs and started seeing some confusing behaviors (basically infinite loops). I'd be happy to be mistaken here but I'm not sure this is user error.
I don't understand why, since it has an alts! in the middle, and it's rewritten in terms of being a state machine. This isn't just like a normal tight loop, is it ?
I mean, to be clear, if you replace the promise chan with a separate go-loop that infinitely puts onto a chan, you don't get this same behavior.
That seems like it breaks the entire idea of non-deterministic choice, from a CSP perspective, doesn't it ?
It seems like it makes it a challenge to write safe code unless you know the provenance and type of every channel you might be alts!-ing on ...
but js is a tricky runtime, and I hate it, and as far as I can tell everyone who writes clojurescript(using core.async or otherwise) is basically teeting on a razors edge of fast behavior for the optimal case and falling off some weird cliff for other cases
but at the same time, if you replaced your random number generator with 5 for non-deterministic choice, I am not sure csp says that is "wrong"
Yeah. It feels pretty dangerous, to be honest. It sucks having code that runs stably for months in the clj version seemingly to 'randomly' hang on the cljs version for mysterious reasons that are hard to understand.
I guess it depends on whether we think non-determinism is allowed to mean 'determinism'.
In any case, it seems to me that if I just stop using promise-chans (or anything else where the buffer might indefinitely supply results), and just use a manually-written go-loop equivalent, the problem won't occur. I hope!
Hey @hiredman, do you happen to know the code well enough to give me a reference to the place where it will choose not to yield? I'd love to read it (although my understanding of the core.async internals is poor at best).
alt! is non-deterministicly choose the first available, and in your case unless the timeout has expired, a fulfilled promise will always be chosen
as for not yielding, the state machine callback is passed to ioc-alts!, which calls do-alts! and if do-alts! doesn't return nil, ioc-alts! returns :recur
which continues the state machine loop
My understanding is to some degree informed by https://wingolog.org/archives/2017/06/29/a-new-concurrent-ml which isn't about core.async per se, but has some analogous parts. it describes channel operations as either happening in the optimistic case or the pessimistic case. The core.async code that corresponds to the optimistic case results in no yield to the js thread.
Yeah, it may be that my understanding is a bit broken as well, because I hate the idea that I have to know what is on the other end of chans (or what types of chans they are) to have an idea of how my code is likely to behave.
Especially given that chans have no easy mechanism to know their identities or types or anything like that (ie, they do not have names and probably do not implement IMeta). It seems fundamentally more fragile than it needs to be.
I guess? it still isn't clear to me if you are passing an expired timeout in or not, if you are passing in an unexpired timeout, and a promise-chan that has been delivered to, the fact that it picks the promise-chan every time is 100% everything working as expected and as designed, and if you expect otherwise then you don't understand the kind of choice alts! makes