Fork me on GitHub
#core-async
<
2022-10-14
>
Ben Sless08:10:34

I have a service with a huge memory leak. After analyzing the heap dump I see it all ties back to timeouts in alts. Is it fixed in the latest version of core async? Any workarounds or mitigations? I remember seeing a few Jiras about it

Alex Miller (Clojure team)12:10:15

There is an outstanding issue about head holding of closed over values, but that’s not directly tied to timeouts

Alex Miller (Clojure team)12:10:44

Can you explain more about what you’re seeing?

Alex Miller (Clojure team)12:10:18

And are you using the latest version or something older? 1.5.640 fixed an issue with clearing alt handlers

hiredman12:10:09

In systems with callbacks in java, timers can often be a source of leaks, because the inversion of control plus timers being a global resource means a timer is a strong reference to whatever data until it expires, and core.async doesn't remove timers when they are not chosen by an alt, it just marks them as cancelled

Ben Sless12:10:12

That may be it. We'll rerun with latest version and check again

Ben Sless12:10:41

And async-234 is the one I remembered

Ben Sless12:10:52

I can't explain it succinctly and I don't have the heap dump at hand, if you'd like I could probably share it with you. When you analyze it with Eclipse MAT it points to the synchronized queue behind timers at the dominating object which holds all the references in the heap

hiredman13:10:17

I am perfectly happy to blame the issue laid out in async-234 for any leaks in a program that uses core.async without any more evidence (I reported it and wrote the patches)

hiredman13:10:53

At work we had a lot of trouble with memory leaks in our project that heavily uses core.async and I ended up reducing our direct usage of timeouts. This was all a year or two before I figured out async-234(reducing timer usage was a wild guess "what is global and could be holding a ref?!"), so hard to definitively say.

Ben Sless13:10:18

And yes, all our leaks seem to point to timers in alts

vlaaad14:10:01

Why is there no drain in core.async? I can create my own impl for sure, but it feels so essential it should be in the library as a basic primitive…

Ben Sless16:10:34

Doesn't reduce accomplish the same thing?

vlaaad16:10:49

reduce’s fn is 2 args, I only need one

vlaaad16:10:16

90% of the time it will be something like tap> or println

Ben Sless16:10:57

Reduce and ignore the accumulator? Basically copy run!'s definition

vlaaad16:10:26

Yes, my question is why this doesn't exist in core.async?

Ben Sless17:10:44

Oh. Uhh, reasons? 🙃