Fork me on GitHub
#core-async
<
2022-07-06
>
Alex Miller (Clojure team)15:07:49

if anyone wants to crack into this https://clojure.atlassian.net/browse/ASYNC-247, would love a hand /cc @bronsa @hiredman

😱 1
Ben Sless16:07:50

I ran across this in a production service getting OOM!

hiredman16:07:09

vs would be overwritten as the loop loops, so it must be coll

dpsutton16:07:52

this would be a phenomenal screen recording of how you debug it or perhaps an article if whoever fixes it is interested

bronsa16:07:38

yeah I can't repro with that fix applied

[~/src/clojure]> clj -J-Xmx32m -Sdeps '{:deps {org.clojure/core.async {:mvn/version "1.5.648"} org.clojure/clojure {:mvn/version "1.12.0-master-SNAPSHOT"}}}'
Clojure 1.12.0-master-SNAPSHOT
user=> (require '[clojure.core.async :as a])
nil
user=> (let [c (a/chan)] (a/onto-chan! c (range 1000000)) (while (a/<!! c)))
nil
user=>

bronsa16:07:14

(I can repro without it)

bronsa16:07:37

trying again in case it was just a lucky run..

bronsa16:07:45

yeah, I confirm the fix. I've just glanced at the core.async code so I can't offer a precise explanation, but the code shape looks "right" to hit the issue CLJ-2145 describes

Alex Miller (Clojure team)16:07:29

I'm not saying you're wrong, but that doesn't match a couple things I saw in debugging

bronsa16:07:38

hmmm, though I didn't get an OOM in 1.12.0-alpha1 on this second run either, so hang on, maybe I keep getting lucky 🙂

bronsa16:07:28

so ignore this for now

Alex Miller (Clojure team)16:07:41

the expansion of onto-chan's go-loop is something like this

Alex Miller (Clojure team)16:07:16

Indexes 6/7/8 are the locals - you can see in lines 19-39 those closed over :once thunks are invoked, then saved and vs is saved as index 9 (and 10) later

Alex Miller (Clojure team)16:07:33

in a debugger I see I think the original coll as the one that is retained (index 7)

hiredman16:07:16

yeah, I don't see it being nil'ed anywhere

Alex Miller (Clojure team)16:07:01

so that's why I don't see a match to CLJ-2145 (no conditional, and different place where I see the head retained)

👍 1
bronsa16:07:43

yeah the macroexpansion is clear about that

Alex Miller (Clojure team)16:07:55

you both have more experience than me in here, I'm not sure what the analogous approach to locals clearing in jvm clojure would be - how easily can we know enough to not save those locals in the state array?

Alex Miller (Clojure team)16:07:42

(separately, that expansion looks like it's redoing the resolution of all of the locals multiple times - ch, then ch and coll, then ch and coll and close?, a lot of this smells unnecessary)

hiredman16:07:18

I don't know how locals clearing works in the go macro

hiredman16:07:02

the enviroment re-creation stuff is meant to make it possible to emit big chunks of normal code that doesn't require rewriting as a state machine, there just aren't any big chunks here

bronsa16:07:46

there is a pass in tools.emitter.jvm to annotate locals at the point where they can be cleared. maybe I can try reusing that and emitting an aset to nil

hiredman16:07:53

tricky because it aliases things in locals instead of using direct refs to the state array

bronsa19:07:19

I am refreshing my memory on the ioc macro 🙂 so many subtleties I forgot about