Fork me on GitHub

Hi, I often (daily?) find Emacs & clojure-lsp consuming an ungodly amount of memory. This has been happening over weeks and persisted over clojure-lsp upgrades. I’ll put more info into the thread. Anybody know some other things I could try to debug or fix this?


memory usage


clojure-lsp --version
clojure-lsp 2022.01.03-19.46.10
clj-kondo 2021.12.20-SNAPSHOT


I’m running macOS Monterey 12.0.1 (21A559)


when I attempt to run lsp-doctor in Emacs in a state like this it hangs and I can only force quit it


this is the output of lsp-doctor after a Emacs restart

Checking for Native JSON support: OK
Check emacs supports `read-process-output-max': OK
Check `read-process-output-max' default has been changed from 4k: OK
Byte compiled against Native JSON (recompile lsp-mode if failing when Native JSON available): OK
`gc-cons-threshold' increased?: OK
Using `plist' for deserialized objects? (refer to ): OPTIONAL
Using emacs 28+ with native compilation?: OPTIONAL


That's odd, I know there was a memory leak on lsp-mode but it was fixed some months ago. Does restarting the server with lsp-workspace-restart help? If so, probably is some memory issue with closure-lsp otherwise lsp-mode


would that be per project?


does it restart when I restart Emacs?


this above is after I restarted emacs


and I have three processes running after killing the one with a ton of memory


guess I should check next time what files it has open using lsof -p


If you close emacs it should close running clojure-lsps


It seems like clojure-lsps process is freeze or something, by any chance does your project use Potemkin?


We had a kondo issue where there was a infinite loop on Potemkin macros lint


not directly but this could be in any dep I guess, correct? I'll check in a bit


yep, looks like I have a transitive dep on *pote*mkin/*pote*mkin 0.4.5


yeah, I'm releasing a new clojure-lsp version which contains the kondo fix

🙏 3
Drew Verlee15:01:14

can someone help me install the latest clojure lsp server? This is what i just tried: `sudo bash < <(curl -s` This is what i got

Downloading : Not Found/ to /tmp/tmp.6bNiSeAJB4
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of or, and cannot find, period.


Do you use macos?


any reason why not installing it via brew?


there are plenty ways to install it as well:


@U0DJ4T5U1 there is a bug on that script


@U0DJ4T5U1 I fixed one of the bugs, but the reason to not finding the version is because I'm deploying a new version right now 😅


it should work in a few minutes/hours if CI helps 😅

Drew Verlee15:01:42

thats awesome, ill try again later


fyi, via brew upgrade, @UKFSJSM38 - i guess you need a new version in brew too?

==> Upgrading 1 outdated package:
clojure-lsp/brew/clojure-lsp-native 2022.01.03-19.46.10 -> 2022.01.20-14.12.43
==> Downloading 
curl: (22) The requested URL returned error: 404

Error: clojure-lsp-native: Failed to download resource "clojure-lsp-native"
Download failed: 


just tried brew upgrade clojure-lsp-native and got the same 404 as well


yes! I'm releasing the fix that should fix brew as well, I let you know when it's done


You know what would be nice, if you have an alias, such as [ :as baz], if you place the cursor on baz and goto definition, it knows that baz is really and goes to that (presently, it only works if you have the cursor anywhere on


Would that be nice?


yes, and possible


already faced that issue as well


makes sense, feel free to create a issue


clojure-lsp Released clojure-lsp with a lot of exciting changes! • General ◦ Bump clj-kondo to `2022.01.15`. (Fixes critical freezing issues with potemkin macro analysis.) ◦ Add `:classpath` to `serverInfo` command for downstream usages. ◦ Fix issues in `extract-function` that arise when operating over `.cljc` files. ◦ Add setting `:copy-kondo-configs?` to allow disable auto clj-kondo config copy, enabled by default. ◦ Lint unused public keywords, like unused re-frame events/subs. ◦ Fix rename of namespaced keywords -> aliased keywords. ◦ Huge refactor on clojure-lsp codebase, spliting code into submodules, this made possible remove a lot of deps from API jar and help separate clojure-lsp features in the future. ◦ Send testTree notification for all project test files after clojure-lsp start. • Editor ◦ Improve 'create function' refactor code action handling multiple cases. ◦ Fix 'resolve macro as ...' code action not working. ◦ Fix `showDocumentRequest` issues when triggered via some refactor code action. ◦ Add new code actions + commands `Move coll entry down` and `Move coll entry up` to move entries within collections. • API/CLI ◦ Make `format`, `clean-ns` and `rename` features not need to scan whole classpath, analyzing only project code improving performance a lot. First, we had a big refactor on clojure-lsp code, to achieve a more modular codebase, there still a lot to do but with this release we separated the lib/core depenedncies from the cli/native ones, with this we should have a API jar with a lot of less dependencies 🎉 Most clojure-lsp API/CLI features now just lint the project code and not the whole classpath, this should help bringing faster lint especially on CIs! We have now the unused-publi-var linter for re-frame handlers and subscribers! re-frame We have new move coll entry code actions and commands thanks to the huge work of @jacob.maine, check the gif! 🚀

🎉 7
clojure-lsp 4
🙌 1
💜 1
Noah Bogart19:01:26

bug report for the new version: when i load a file in vim (using coc.nvim), lsp seems to never indicate that it’s done loading. it’s been spinning on “Loading clojure-lsp” for 5 min

Noah Bogart19:01:55

verbose log output:

2022-01-20T19:32:00.534Z  INFO [clojure-lsp.source-paths:192] - Automatically resolved source-paths from project.clj: #{"test/clj" "src/css" "src/cljs" "src/cljs/nr" "src/cljc" "src/clj"}
2022-01-20T19:32:00.543Z  INFO [clojure-lsp.db:?] - Reading analysis cache from Datalevin db took 0.01 secs
2022-01-20T19:32:00.544Z  INFO [clojure-lsp.classpath:16] - Finding classpath via `lein classpath`
2022-01-20T19:32:05.980Z  DEBUG [clojure-lsp.classpath:28] - Classpath found, paths: [... many many paths ...]
2022-01-20T19:32:05.982Z  INFO [clojure-lsp.classpath:16] - Finding classpath via `npx shadow-cljs classpath`
2022-01-20T19:32:11.429Z  DEBUG [clojure-lsp.classpath:28] - Classpath found, paths: [... many many paths ...]
2022-01-20T19:32:11.446Z  INFO [clojure-lsp.crawler:106] - Analyzing classpath for project root /Users/noah/Personal/netrunner
2022-01-20T19:32:11.449Z  INFO [clojure-lsp.kondo:184] - Analyzing 420 paths with clj-kondo with batch size of 9 ...
2022-01-20T19:32:11.449Z  INFO [clojure-lsp.kondo:190] - Analyzing 1/9 batch paths with clj-kondo...
2022-01-20T19:32:26.670Z  INFO [clojure-lsp.kondo:190] - Analyzing 2/9 batch paths with clj-kondo...
2022-01-20T19:32:40.846Z  INFO [clojure-lsp.kondo:190] - Analyzing 3/9 batch paths with clj-kondo...
2022-01-20T19:32:55.507Z  INFO [clojure-lsp.kondo:190] - Analyzing 4/9 batch paths with clj-kondo...
2022-01-20T19:33:16.691Z  INFO [clojure-lsp.kondo:190] - Analyzing 5/9 batch paths with clj-kondo...
2022-01-20T19:33:53.674Z  INFO [clojure-lsp.kondo:190] - Analyzing 6/9 batch paths with clj-kondo...
2022-01-20T19:34:04.812Z  INFO [clojure-lsp.kondo:190] - Analyzing 7/9 batch paths with clj-kondo...
2022-01-20T19:34:22.232Z  INFO [clojure-lsp.kondo:190] - Analyzing 8/9 batch paths with clj-kondo...
2022-01-20T19:34:42.251Z  INFO [clojure-lsp.kondo:190] - Analyzing 9/9 batch paths with clj-kondo...
2022-01-20T19:34:54.404Z  INFO [clojure-lsp.crawler:?] - External classpath paths analyzed, took 162.96 secs. Caching for next startups...
2022-01-20T19:35:00.290Z  INFO [clojure-lsp.crawler:?] - Manual GC after classpath scan took 1.53 secs
2022-01-20T19:35:00.291Z  INFO [clojure-lsp.crawler:226] - Analyzing source paths for project root /Users/noah/Personal/netrunner
2022-01-20T19:35:00.292Z  INFO [clojure-lsp.feature.clojuredocs:40] - Refreshing clojuredocs cache...
2022-01-20T19:35:01.562Z  INFO [clojure-lsp.feature.clojuredocs:?] - Refreshing clojuredocs cache took 1.27 secs.
2022-01-20T19:35:19.509Z  INFO [clojure-lsp.db:?] - Upserting analysis cache to Datalevin db took 19.22 secs
It’s now 19:45 and still “loading”

Noah Bogart19:01:25

restarting my editor gets to that point much faster:

2022-01-20T19:46:32.157Z  INFO [clojure-lsp.source-paths:192] - Automatically resolved source-paths from project.clj: #{"test/clj" "src/css" "src/cljs" "src/cljs/nr" "src/cljc" "src/clj"}
2022-01-20T19:46:37.916Z  INFO [clojure-lsp.db:?] - Reading analysis cache from Datalevin db took 5.76 secs
2022-01-20T19:46:37.927Z  INFO [clojure-lsp.crawler:200] - Using cached db for project root /Users/noah/Personal/netrunner
2022-01-20T19:46:37.928Z  INFO [clojure-lsp.crawler:226] - Analyzing source paths for project root /Users/noah/Personal/netrunner
2022-01-20T19:46:37.928Z  INFO [clojure-lsp.feature.clojuredocs:40] - Refreshing clojuredocs cache...
2022-01-20T19:46:40.131Z  INFO [clojure-lsp.feature.clojuredocs:?] - Refreshing clojuredocs cache took 2.20 secs.


Yes, I'm observing the same now


It spins forever on vim + coc


For me, it's hanging on this:


2022-01-20T19:53:14.980Z  INFO [clojure-lsp.kondo:184] - Analyzing 218 paths with clj-kondo with batch size of 5 ...
2022-01-20T19:53:14.980Z  INFO [clojure-lsp.kondo:190] - Analyzing 1/5 batch paths with clj-kondo...


I wiped the .lsp directory also on first attempt


Seeing similar with emacs + eglot

[server-notification] Thu Jan 20 13:59:29 2022:
(:jsonrpc "2.0" :method "$/progress" :params
          (:token "clojure-lsp" :value
                  (:kind "report" :message "Analyzing project files" :percentage 95)))


I'll log a bug report

Noah Bogart20:01:37

what’s real funny is I thought “Oh, that’s okay, I don’t need lsp, I spent 3 years doing clojure without any sort of help” but it turns out, I deeply rely on lsp now to help me out


I'll take a look, can we centralize in a thread? 😅


Here okay?

👍 1

I’m not able to reproduce on a very small project, FWIW


I'm completing a github issue, I'm sure others can chime in when I've posted it


Ditto. I’m using emacs. It finishes analyzing a small project, but stalls analyzing a large project (`clojure-lsp` itself actually)

👀 1

Does anyone have a project that this happens?


I can't repro with the static binary on the clojure-lsp repo


I've posted an initial issue, others can add to it


Alright, managed to repro, it seems related with graalvm, it freezes when analyzing with clj-kondo :thinking_face:

Noah Bogart20:01:20

sorry @UKFSJSM38 for not making a thread to begin with!


I had this issue before, and I thought it was a corrupted jar in my m2, after removing it it solved my issue


but since more people and I managed to repro again, seems to be something else


I'll revert the release for now


@U04V15CAJ any chance this is related with the new clj-kondo version? It seems to be freezing on the call to clj-kondo, not easy to repro though


In an example project I have up on github , the issue occurs too (if I open up the main.clj)

👍 1

So, if you're looking for another project on which this fails, that is another example.


I removed latest release until find the proper fix, I can take a closer look later today, thanks for the input everyone!

❤️ 3
Noah Bogart20:01:36

appreciate the quick response


Will you be pushing out a new version? As a maintainer of the clojure-lsp version on arch, it's tricky to go back for a release...


(even if the new version is identical to the previously good version - it needs a higher version number)


I see, not easy to revert all changes though


I just removed the latest github entry so it's possible to get previous version from install script


I'll probably revert brew as well


ehm... can you reproduce with clj-kondo only?


I'll try it!


I can't repro with JVM clojure-lsp, so I'd need to test with clj-kondo native


I'm building a clojure-lsp static binary with previous clj-kondo version to make sure as well


For clojure-lsp project, it seems to stuck when analyzing /home/greg/.m2/repository/com/taoensso/timbre/5.1.2/timbre-5.1.2.jar , the same jar I faced the same issue before :thinking_face:


I can't repro with kondo native though


My startrek one doesn't use timbre


it seems to be a inner timbre deps


because it fails with taoensso/encore as well


you can't repro with kondo itself?


yes, I tried to repro with kondo static binary, but it linted correctly


I ran clj-kondo v2022.01.15 on my startrek one, with no issues


can you repro with clojure-lsp JVM version?


are you using clj-kondo master in lsp? try with that one?


@U11EL3P9U I'building a new native clojure-lsp with more logging on what we are sending to kondo to lint, it should be avaialbe here soon to download


It doesn't happen with JVM clojure-lsp the issue


I'm using latest stable clj-kondo which doesn't include only latest clj-kondo commit, do you think that commit could be related with this issue?


I'm happy to hear you didn't include master since the latest commit contains a false positive with bb.edn ;)

😅 1

timbre project does lint correctly to me with the native clojure-lsp


it's weird, seems to be related with that jar on my m2


that's why I thought it was a corrupted jar first time I saw that issue, let's see on what jar stuck for @U11EL3P9U’s project when the CI finish


removed the timbre jar and the issue keeps happening, so it's not a corrupted file or something like that


this native bin contains more logs regarding kondo file being analyzed:


@U04V15CAJ it seems that is freezing indeed on the kondo call, I know the best would be to repro with clj-kondo itself, I'll keep investigating


The issue doesn't happen to me with your project though @U11EL3P9U, so it seems to not be consistent, harder to debug :(


I wonder if others have any leads too?


other projects to test against 🙂


yeah, that could help, could you try my zip above and check what file is stuck on the logs?


I can't unfortuantely, I need a linux native image


let me dm you


this one is a linux static one


okay, trying


okay, spinning it up

loading 1

@U11EL3P9U project hangs on other jar in his machine, so it seems to have no pattern the issue


seems to be a hairy one 😔


I'll keep trying narrow down


I’m trying to prune out a private, single src namespace, single test namespace project that seems to exhibit the issue

👍 1

(yes, it was hanging for me on kaocha library)


For me it freezes on jars: after I remove one of them from my m2 manually it stucks on the next



I wonder what they have in common

😂 1

tee hee, from your examples, I hope not all from taoensso 😄


haha yeah, but for you it's kaocha, so 🤷


Let me see what it is with my bigger project...


hanging with kaocha


any news? if there's anything I can do to help, I would


Thanks, I'm still debugging trying to compile native with previous commits to narrow down


for now I can't find any issues with kondo, but it's indeed freezing on the kondo call


and you can't repro with the JVM ?


which graalvm version are you using?


that exhibits the issue for me


(eglot and emacs)


I'll try your project @UPEKQ7589 @U04V15CAJ 21.3.0


thank eric


i hope it suffices


@UKFSJSM38 try 22.0.0 which came out yesterday? ;)

🤯 1

btw, also try linting with --parallel if you are reproducing with clj-kondo only


use the exact same arguments as lsp does

☝️ 1

hum, good point, forgot about that parameter


@UPEKQ7589 your project indeed freeze with me, but linting the project source not the jar!


that's a perfect sample it seems, I'll take a closer look


does it freeze for you in the project code source analysis as well or during classpath scan?


not sure eglot shows that though


[server-notification] Thu Jan 20 15:58:26 2022:
(:jsonrpc "2.0" :method "$/progress" :params
          (:token "clojure-lsp" :value
                  (:kind "report" :message "Analyzing project files" :percentage 95)))

👍 1

@U04V15CAJ Using @UPEKQ7589’s project, I found it freezes clojure-lsp during the analysis of this simple code:

(ns lsp-kondo-repro.core-test
   [com.gfredericks.test.chuck.clojure-test :refer [checking]]))

(checking "illegal charges" 100)
(trying still to repro with clj-kondo only)


oh, if I start clojure-lsp with

;; (checking "illegal charges" 100)
it starts well, if I uncomment and lint with clj-kondo (via clojure-lsp) it then freezes 👀


nice simplification


only that code?


exactly 😱


and with clj-kondo itself?


Now I doubt all these projects are using test.chuck


no freezes with clj-kondo static binary 😔


I pushed up the simplification to the repo


have you tried with clojure-lsp from previous release?


perhaps you can git bisect


yes, clojure-lsp from previous release works perfectly


ok, then bisect

loading 1

uh, I have a clue of what could be


the re-frame unusued-public-var code, that is used on the custom-lint-fn


that's why I can't repro with CLI clj-kondo, not sure there is a option to pass custom-lint-fn there


no, but you can on the JVM


will try to revert that code and compile a native one


on the JVM it works


oh, is not that, the bisect I tested and it worked already contains the re-frame changes, damm


take it easy, people can just use the previous release

👍 1
☝️ 1

and we’ve got a very small repro case now

🙏 1

I wish I could use a REPL in a graalvm native image 😂


I pushed up another narrowing commit. Dropped some dependencies

💯 1

As I was afraid the issue started to happen after, which unfortunatelly was the refactor of splitting to submodules and had a lot of changes (just moved code)


the issue is that I already commited some other changes, so no easy to just revert that


Not sure this is helpful, but… I waited for a long time and it finally finished prepping clojure-lsp. Checking whether it stalls when I restart the lsp workspace and when I restart emacs…

🤯 1

Oh wait, I take that back. It’s still not working for clojure-lsp. But I had some ~/.emacs.d/workspace/.cache/rewrite_clj.* buffers open from using lsp-find-definition. And LSP is working in those buffers. Weird. Anyway, probably best to ignore what I said before.

👍 1

Alright, issue fixed, I tried to explain the details on fixed it. thank you very much everyone for the help especially @UPEKQ7589 and @U11EL3P9U for the project samples! I'm releasing the fix right now, it should be available in some minutes/hours

🎉 1

TBH I didn't understand the issue exactly, but I hope @U04V15CAJ will help understand this later:joy:

Noah Bogart01:01:03

Thank you so much for the quick fix. Really appreciate all the hard work you do to keep this thing working


😄 Thank you @UEENNMX0T for the quick reply and the feedback!


Same Eric. Thank you. As someone mentioned earlier, LSP became part of my workflow so quickly

gratitude 1
🚀 1

CI is having issues to build the native image for windows as always, will leave the release to tomorrow morning


@UKFSJSM38 Glad you solved it. Are you able to reproduce the issue on the JVM with your project that didn't explicitly include clj-kondo? I've seen a similar issue recently, it may be due to a different version of SCI being pulled in somewhere or so


so it would be good to look at the deps tree of the problematic project, I think you should be able to reproduce it on the JVM


Yeah, I couldn't repro with JVM, but what you say makes sense, clojure-lsp has a javac task that uses americano to compile the java sources to a classes folder that is included on the classpath, when compiling the uberjar, depstar put the compiled code into classes as well and I noticed that sci was not there when that deps was not on the cli submodule


perhaps move to


depstar is deprecated


yeah, forgot about that, will do!


@U04V15CAJ you were right, after changing to I can repro the same issue with JVM, but I'll leave to debug tomorrow


move-coll-entry isn’t working quite as I had hoped. More in thread…


First, it doesn’t work when you’re on whitespace or a comment, even though the tests prove that it can. That is, in the tests, if you start on a whitespace/comment node, you can successfully move up or down. But in the editor, if the cursor is on a whitespace/comment node, invoking move up / down doesn’t do anything. I have a guess of what’s happening, based on something I saw when writing tests. I think that the code path we’re using must translate a cursor position into a zloc by looking only at “real” nodes, i.e., not whitespace or comments. So, if you’re in a map and on whitespace or a comment, it chooses the map itself as the zloc. … Actually, yes, I can confirm this must be the case. If I put the map inside another sequence, and invoke up/down, it moves the whole map. It must be choosing the whole map as the zloc.


The second issue is related to the first. When moving up/down, the cursor follows the moved node. This is by design, so that you can repeatedly invoke the action to keep moving the node. However, sometimes the cursor ends up on a comment associated with the node, and then the above situation arises… and so movement stops


I’m going to dig around to figure out where the code chooses the zloc that it passes to the actions. But, I’m skeptical of changing that. I bet there’s other code that assumes that zloc will be a real (non whitespace/comment) node. But I’ll experiment anyway and report back what I find.


And a final thing, not so much a bug as a new feature… It’d be nice if it moved pairwise in case , cond and other similar statements.


> … Actually, yes, I can confirm this must be the case. If I put the map inside another sequence, and invoke up/down, it moves the whole map. It must be choosing the whole map as the zloc. Yes, that's the issue. I'm not sure but I think this behavior was related with completion issues, if you change that, take a look if could be any completion issues related to that (the tests should help though)


As usual, you have great foresight @UKFSJSM38. 🙂 I have a patch that makes this behavior change: the parser will move onto whitespace zlocs. As you predicted, I had to be careful to keep the completion tests passing. Once I had that figured out, there were surprisingly few breakages. So, I think my patch is worth discussing. There are a few different possible implementations, so I’ll put together a PR and we can go over the options there. The patch is built on top of this other (which is just some test cleanup) so it’ll be easier to look at after that’s merged. Will you review it when you have time? (I know there’s a lot of other stuff going on, so no rush.)


Sure, I'm still fighting with windows CI to release a new clojure-lsp version with the fix from yesterday, but I'll take a look at your PR after that! Yeah, besides the completion issues I don't see why we can't follow this approach, looks a valid option indeed to me


@UKFSJSM38 Thanks for merging I’ll follow up on your questions there. I also plan to merge or rebase, to resolve those conflicts. And I have a few other branches built on 716 that I’ll submit as PRs soon. One of them is the fix to stabilize the cursor positioning when moving collection entries. I’ll put the changelog entry in that one. 716 is a big change, so I’ll try to keep an eye on things over the next few days. Let me know if any surprises come up.


Yeah, 716 was quite big, maybe we should that kind of changes but I know it was necessary for that issue, thank you for all the help!

👍 1

I removed latest release until find the proper fix, I can take a closer look later today, thanks for the input everyone!

❤️ 3