Fork me on GitHub
#clojure
<
2023-06-22
>
jeroenvandijk08:06:46

Anyone know how to trigger a java.lang.OutOfMemoryError consistently? Even better a Compressed class space error? 🧵

jeroenvandijk08:06:55

This seems to slowly fill up memory, but it is taking some time

(dotimes [i 4] (doto (Thread. (fn [] (dotimes [i 1000000000] (.intern (str (java.util.UUID/randomUUID)))))) .start))
Also I’m actually trying to tirgger the Compressed class space error which I get time to time in some project, but not consistently

igrishaev08:06:55

(throw (new java.lang.OutOfMemoryError "Ahh"))
no?

jeroenvandijk08:06:30

hmmm not a bad idea I guess 😃

igrishaev08:06:31

or just (throw (new OutOfMemoryError "Ahh")) as it's shorter

2
👍 4
2
jeroenvandijk08:06:19

I will keep this in mind next time it happens and see if this could fulfill the requirement for catching it, I’m guessing that this would do it

oyakushev08:06:40

(byte-array 1e10)
Adjust the number depenending on your heap size.

👍 2
2
jeroenvandijk08:06:35

Thanks for the suggestions! This error sometimes comes up when using #C013B7MQHJQ so I was thinking of a workaround

emilaasa09:06:02

Why are you trying to catch it?

igrishaev09:06:00

+1, catching OOM is a bad idea because it means the system is unwell but you just silence it.

igrishaev09:06:41

Throwing OOM is used to simulate such case only

jeroenvandijk09:06:40

@U6T7M9DBR Good question! Polylith uses a classloader to load the projects. After X amount of time (hours/days) this error pops up. The only way to recover from this error in this situation is restarting the Polylith shell. As this breaks the flow and also has some heavy startup time, I was thinking of something more user friendly

jeroenvandijk09:06:02

(btw, this error doesn’t seem to occur for all users of Polylith, but only specifc bigger projects it seems)

jeroenvandijk09:06:12

My idea would be to wrap the try/catch around this classloader part specifically

jeroenvandijk09:06:08

And maybe another important part of my idea would be is that I would destroy the process when this happens, and start a new one 🙂

jeroenvandijk09:06:53

More specifically, the process would be managed by another babashka process (through pods). This process needs to be told through the bencode format that something bad happened. I think now I’ve explained all the parts of my idea 😅

emilaasa09:06:14

Just crash the jvm and have something else restart it

2
emilaasa09:06:36

I don’t think you have any guarantees after a OOM error

2
emilaasa09:06:05

It’s an internal VM error, so recovering probably works most of the time but is not reliable.

emilaasa09:06:20

Which JVM version are you using?

jeroenvandijk09:06:51

Right now

java version "11.0.7" 2020-04-14 LTS

emilaasa09:06:48

You can analyse the root cause of the OOM from a heap dump if you’d like. The metaspace memory is inspectable after that as well. It sounds like something is not being garbage collected correctly, or you need more memory

jeroenvandijk09:06:12

Yeah you are probably right. Unfortunately, it is not happening too often to find it easily, but often enough that it is annoying

emilaasa09:06:58

We usually run -XX:+HeapDumpOnOutOfMemoryError

jeroenvandijk09:06:52

I will add that to the polylith profile and wait for the next time. Thanks!

oyakushev09:06:45

@U0FT7SRLP If it happens again, you can take a heap snapshot manually by doing this in the terminal:

jmap -dump:live,format=b,file=heapdump.bin 12312
The last argument is the PID your Clojure process. You can do this after the OOM happens, it is okay. After you've obtained the heapdump, you can use MAT (https://www.eclipse.org/mat/) to discover what is leaking memory. It's not a trivial thing to do and takes some time to learn how to use MAT and interpret its results, but overall it's a useful skill to have.