your mono-repo and config looks correct

but never saw 6gb, actually there is a limit of 2GB per clojure-lsp process :thinking_face:

https://github.com/clojure-lsp/clojure-lsp/blob/57d39a0c31b733f5c71d7f33265cad39e23bed6b/cli/deps.edn#L37

Yeah that got me confuses too. This is what it looks like at the moment.

I assumed those jvm-opts only applies to the compilation of the clojure-lsp graalvm native image and not baked into the native image at runtime. I don’t have enough experiences with graalvm to be sure tho. :thinking_face:

That's interesting, first time I see that with more than 2GB

Yeah, I never managed to test that, but seems like you are correct

Would be nice to run a JMC to profile the memory, we can probably find mem per improvements somewhere

Would be nice to plug the repl on the running clojure-lsp and check how much is the size of clojure-lsp.db/db atom

It keeps all the analysis of the project

Kk thanks that’s where I’ll start! Thank you! Also FYI I just found this: https://www.graalvm.org/latest/reference-manual/native-image/optimizations-and-performance/MemoryManagement/#java-heap-size - seems like the option to bake the memory limit into the native image is -R:MaxHeapSize

Oh, that's quite interesting, but I wonder if the process would just stop working when reaches the limit

Ah, the repl plug will only work with a JVM clojure-lsp not a Graal one

Yeah I wouldn’t change that setting without testing it thoroughly. 😛 Just want to share what I found so far.

With a JVM one we can use https://github.com/clojure-goes-fast/clj-memory-meter as well

To generate a JVM one just clone the repo and run bb debug-cli

But I suppose you should get OutOfMem exceptions since the limit is 2GB

Yeah I’m getting OOM exception so far with the debug-cli executable. It’s okay I’ll bump the Xmx setting to get it going. Because my project is pretty complex, I’d like to inspect the db atom to see if I happen to be loading redundant analysis for my project.

Yeah, also, check the .lsp/.cache/db.transit.json which is a big part of the db that goes in the atom

If that is huge, (more than 100mb) probably that's the problem

Oh mine is 276MB at this point.

Would you help me understand what goes into this cache file?

basically https://github.com/clojure-lsp/clojure-lsp/blob/57d39a0c31b733f5c71d7f33265cad39e23bed6b/lib/src/clojure_lsp/startup.clj#L161-L167, but probably :analysis is the huge map there, it contains all clj-kondo analysis of the project and external deps

load that into mem was never a issue, but your case seems a little bit corner case since it's a huge project, maybe for that case we may want to have a flag to make clojure-lsp have less things on db :thinking_face:

i'd play with these values https://github.com/clojure-lsp/clojure-lsp/blob/57d39a0c31b733f5c71d7f33265cad39e23bed6b/lib/src/clojure_lsp/kondo.clj#L254-L276 in a local clojure-lsp JVM, to see if there is something that decreases the size of the db by a lot

Kk thank you 🙇 I’ll give it a try! 🙏

@U7PQH43PS Were you successful? My db.transit.json is 900mb and ram consumption is huge.

@UBRV1HXPD could you try .lsp/config.edn

{:java nil} 

@UKFSJSM38 The effect is there, but not big. It’s hard to say. When I open Emacs, load a project, and start lsp, and wait, it once consumed 5.90GB, and another time at 5.53GB. When I set it to {:java nil}, RAM consumption was 4.9GB and 5.3GB. while 2023.08.06-00.28.06 consumes less memory than 2023.07.01-22.35.41 (with that version my project consumed ~7GB) i have got a feeling that a new version leads to new bugs (flymake, xref got timeout) but it is really hard to reproduce it and do some testing as I have experienced this after 3 hours working on project and I didn’t have time to do some debugging. But that is another topic (i still need to reproduce to be sure) :-)

yeah so maybe it's related with your clojure code being the biggest part indeed

the 2gb limit works in what cases?

I think the 2gb limit is not working for graalm native images otherwise you would get Out of memory errors

I want to focus on some perf improvements like that until the end of the year, but not a promisse we will have huge improvements in there, at least options to disable features maybe

Ah, you already discussed it here. Sorry to ask again. Good luck!

@UKFSJSM38, huge thanks for the https://github.com/clojure-lsp/clojure-lsp/issues/1700. The memory usages for my use case is down from 6GB to 3GB. This is a significant improvement for me. Thank you thank you 🎉 🙏

That's really good to hear @U7PQH43PS, it's important to know how significant those improvements are :)

I agree. These improvements are really significant. With the last version I got the same results like @U7PQH43PS on Macbook M1PRO.