I haven't seen this phenomenon, but I can say that having looked at string deduplication I came to the conclusion I shouldn't use it. Did you do it to save on memory?

Yeah it seems like a bit of a slam dunk to save memory. We have observed it reducing a heap that was roughly 2250mb down to about 1750

Why did you conclude you shouldn't use it?

you're saving 500mb of memory for a massive performance hit

If it saved 80-90% of your heap I'd say go ahead, for ~20% I wouldn't bother, not worth hitting a synchronized cache for every new string

Did you measure the perf, or find an article that did? All I've seen is 'there is a cost' but as our app's GC time is so small anyway I didn't think it could be much

It's not about GC, it's about a global throttle on strings

I thought it just adjusted some char array pointers at GC time if it found it to be a duplicate, that all string creation on young gen is unaffected (deduplication only happens on old gen)

It does. I can try digging up those articles, but it's worth to compare the throughput and not just heap space

Interesting thank you, I'll do a bit more reading. At this point having stared at this problem for two days without making much progress I'm tempted to just turn it off to get our app stable again

You should turn on GC logs or use jfr in any case if you want to get to the root of the problem

Yeah been doing that, NMT and GC logs and heap analysers and all the things, made my head spin a bit. The answer is probably there but I'm quite inexperienced at reading it all

Thanks for your advice :)

Would it be possible to temporarily replace httpkit with clj-http or something to confirm your hypothesis? I would prioritise stability over memory in this case.