Hum, maybe could be related with new java analysis we now have java-member-definitions

@UKFSJSM38 I'm trying 2023.02.27-13.12.12

I think some release from this year added the java-member-definitions from ASM

try disabling the java feature first

{:java false}

we should probably improve the kondo java analysis call to be in batch like we do for clojure analysis

it's interesting how the OOM happens during the save to transit cache and not kondo call, maybe there is somthing we can do there to improve memory usage

hrm ok so :java false made a small difference. 2022.07.24-18.25.43 works fine. 2023.02.27-13.12.12freezes? but does not produce an error, it was stuck at this 50% for ~5 minutes then died without a message (the old version says "Finding diagnostics... No diagnostics found!")

I'm guessing it's still OOM but being swallowed

Yeah, I think so we had so many changes since july 2022 😅

I'll do some bisescting

we had keywords, edn analysis and more stuff, but I suppose there are other people using more recent clojure-lsp via metabase?

yeah, but not the debug-cli

yeah, native-image works and JVM not is weird

Yeah double checked, and the homebrew release does work. There is a huge wait time between 50% analysis and spitting out the result tho unlike the old release that gradually climbed to 100%. Maybe in the switch to bb debug-cli we just lost a jvm opts for heap space somewhere? 2022.10.05-16.39.51 fails w screenshot (only tag that didn't do that. 2022.09.01-15.27.31 works

Hum, maybe, we had considerable redactors since July release that changed to use bb for everything I think

Checking the changelog, I think that release of Oct added the index by URI, which is made in kondo.clj normalize function, maybe we could check if there is some improvement there as well

@U0BUV7XSA try on the branch from https://github.com/metabase/metabase/pull/30202/files There was an issue relating to clj-kondo’s parsing that caught us out. This should hopefully fix it. I’ve been on the january release as i haven’t spent the time to fix it but cam just fixed it

@U11BV7MTK i did, it's a different issue w the build

I'm seeing perhaps the same issue. I'm running a bb debug-cli build from nvim and get an error that Stack trace of root exception is empty on startup which I suspect is the same OOM but the message isn't getting through. (I've also got the Metabase-specific fix Dan alluded to above.)

Only in metabase @UCY0GT0QM?

maybe metabase is the first project to reach "the limit" of analysis without need to increase heap, since 2022 july, we had lots of improvements and feature on clojure-lsp which require more analysis, like keywords, java stuff etc, so we could check if there is a improvement for this in clojure-lsp to avoid that much memory

yeah, I'm only seeing this in Metabase currently. that is a big codebase with a big depset, so maybe it really is memory exhaustion. on the other hand, it's possible there's an analysis bug that's causing a runaway analysis.

let me try a debug-cli build at master on another codebase and see if it works there.

yeah, it's fine.

so I guess it's related to analysis size indeed, we could try to understand what is making clojure-lsp use that much analysis, not blowing in kondo is already a good start since is what we had issues in the past, so probably something we could do on clojure-lsp side like avoid having a whole vector of items and do things lazy and things like that

I'm trying a tweaked build to set -Xmx4g and see what happens on Metabase.

The stacktrace snoe posted already gives a hint where we should start improving, https://github.com/clojure-lsp/clojure-lsp/blob/4448954d9fecff4b777abbcdb4c606305a60ff27/lib/src/clojure_lsp/kondo.clj#L148

there is even a TODO from @U07M2C8TT , maybe he has some clue :)

for the record it is working at 4g after a lengthy startup 🙂

Not sure I can help… it looks like snoe’s OOM occurred in the assoc on https://github.com/clojure-lsp/clojure-lsp/blob/4448954d9fecff4b777abbcdb4c606305a60ff27/lib/src/clojure_lsp/kondo.clj#L153. That assoc doesn’t look like it would allocate much itself, so presumably we were right on the border of being out of heap already. As usual with OOMs, the real problem is probably elsewhere