return to 2022.07.24-18.25.43 solves the problem

I've had this too and reverting to pre-lsp4clj solved the problem for me. https://github.com/clojure-lsp/clojure-lsp/issues/1240

Perhaps you could add to the issue if you see a different problem that you can easily reproduce

I think @U07M2C8TT is already on it, if I'm not mistaking

thanks, looks like my problem

are you using emacs as well?

neovim

Might be worth mentioning in the issue then since we also suspected it would be something with emacs lsp-mode

Thanks for the report @U0BBFDED7. As @U04V15CAJ said I think I have a handle on what the problem is and have made some progress on fixing it. I hope to have a build together by next week (I'm traveling atm). If you'd be able to test that, whenever it's ready, it'd be much appreciated.

no problem

I'm watching the repo, so as soon as the release comes out I'll test it out

Is lsp development easier in vscode somehow or does it not matter that much?

Do you mean working on clojure-lsp? Or using it while working on other projects? I'd say working on clojure-lsp itself is equally easy in any editor. I use Emacs, but that's personal preference. Using it is similar. Although the editors each offer slightly different ways of interacting with clojure-lsp, it's mostly the same, and primarily comes down to personal preference.

Or was your question about the development of the client (editor) side? I don't have any experience with that

I mean working "on" it, in this case the vscode, emacs, etc.. clojure lsp clients.

I suppose I'm asking the maintainers specifically which editor Sparks more joy.

I'm a little confused about what you mean by “emacs clojure lsp client”. That's kind of a contradiction in terms. There's an “emacs lsp client” as well as one or more for each of the other editors. But they're language agnostic—not Clojure specific. Then there's a “clojure lsp server”, which is language-specific, but editor agnostic. The maintainers of the clients usually develop their side using the same editor, because they like that editor. And the maintainers of the server use whatever editor they prefer. That's part of the beauty—everyone gets to use what they like

Yep, you seemed to have gleamed my unclear meaning. Sure, i guess the easier question is, what editor do they prefer. It's easier to maintain and grow something you need and benefit from. I believe Eric uses doom, i'm not sure what borkdude uses in his day to day clojure development...

I think they both use some flavor of Emacs. snoe uses nvim.

thanks jacob. I'm slowly (years) becoming more invested in emacs to, so thats good to hear. Though it comes with some tradeoffs obviously 🙂

For lsp development specifically, I'll actually say more diverse editor usage would be good.

I use emacs for day to day development, but when I develop lsp servers, I usually use VSCode too since it's pretty easy to test there

I’ve submitted a https://github.com/clojure-lsp/clojure-lsp/issues/1240 for the lsp4clj side of this and made a draft https://github.com/clojure-lsp/clojure-lsp/pull/1259 for the clojure-lsp side. To test a build, you’ll need the lsp4clj branch checked out in the same directory as clojure-lsp. If that sounds too complicated, please wait until the clojure-lsp branch is out of draft. /cc @UKFSJSM38

Sorry wrong link. Should be https://github.com/clojure-lsp/lsp4clj/pull/25

https://clojurians.slack.com/archives/CPABC1H61/p1664028206972669

@U0BBFDED7 if you have a chance, will you report on whether https://clojurians.slack.com/archives/C032YH7P0R2/p1664133908073819 build (or a more recent one) improves performance for you?

Hi @U0BBFDED7, did you run into any performance issues with nightly above ☝️ ?

FWIW I still have the same issue and will keep using the 2022.06 release for now

@U07M2C8TT @UKFSJSM38 looks good.

I checked one of the longer files and can't find any performance issues

Cool! Thanks for testing, so it does fix your perf. Issues, one more reason to think it's something specific with the borkdude's case @U07M2C8TT

previously in neovim lsp/diagnosis appeared literally letter by letter, so it would be quite hard to miss now if it was the same

no problem, thanks for the instant solution, with clojure-lsp dev expirience is much higher

Glad to hear your problem is solved

@U0BBFDED7 glad that worked for you. @U04V15CAJ, I apologize we haven’t got performance back on par for you. Were you able to follow up on any of my questions about Emacs garbage collection? I’m still very curious about this because it’s the only way I’ve been able to truly reproduce what you’re seeing. I know the problem is with clojure-lsp, not Emacs, but it’s possible that the Emacs settings affect garbage creation differently in the two versions of clojure-lsp. In particular I’m curious what describe-variable says about lsp-log-io, lsp-clojure-server-command, lsp-idle-delay, gc-cons-percentage, gc-cons-threshold, and read-process-output-max. Anecdotally, do the freezes correlate with times that Emacs is doing GC? I checked this by watching top but there may be a better way. The Emacs profiler https://emacs-lsp.github.io/lsp-mode/page/performance/#reporting-performance-problems memory usage. And in theory setting garbage-collection-messages should show you when GC is done, though I can’t actually find those messages. If you truncate your clojure-lsp log file, does that help? I want to follow up on these threads, but I also don’t want to get stuck on GC as the problem. I asked @UKFSJSM38 if he could duplicate the lag when Emacs was handling giant log files, but he couldn’t.

@U07M2C8TT Appreciate all your efforts. In the last conversation we had you asked for my emacs GC settings and I posted that

I'll try your other suggestions here when I'm back home

Ok cool, thanks 👍

Didn't get to it yesterday, will look into it today

No problem. Sorry to keep making you switch builds. Thanks for the help debugging

Now running the nightly from yesterday, let's see

lsp-log-io is a variable defined in 'lsp-mode.el'.

Its value is nil

lsp-clojure-server-command is also nil

lsp-idle-delay is 0.5

gc-cons-percentage 0.1

gc-cons-threshold `Its value is 104857600 Original value was 800000`

read-output-process-max Its value is 1048576

Darn… that all looks like normal config. Do you happen to have any *lsp-log* files? You should have one, but if there are more that’d be surprising

Err… buffers, not files

yes, it's about 700 lines

there is one

OK, that’s normal

There are also these:

everything looks correct

You might still try truncating your server log. I doubt it’ll affect anything, but good to eliminate that variable. The next thing to look at is whether the freezes correlate with Emacs GC

I immediately notice the lags after switching, it's very clear to me

$ rm /tmp/clojure-lsp.out

doesn't seem to help

Not too surprising

Can you tell whether Emacs is doing GC whenever a freeze happens @U04V15CAJ ?

how can I tell?

Sorry afk. Will leave instructions here as soon as I can

👋

If you want, we could also do a video chat to debug things, if that helps

I ran with the nightly all weekend but I've switched back to the June version now as the lag is still noticeable to me to the point where it becomes annoying

Is there a chance we can move back to lsp4j perhaps? I don't know how large that change was. Perhaps there is a way we can hide this behind a protocol so lsp4clj can see more testing before switching definitively?

I realize this may be a case similar to people being able to hear high pitch sounds where others don't notice it, but for me it's enough that the thought of forking for personal use has even crossed my mind briefly (which I find undesirable)

cc @UKFSJSM38 @U0BUV7XSA

I see your pain @U04V15CAJ, sorry for the headache, not sure if @U07M2C8TT has anything more to debug related to that, it's really weird this only happen with you and I don't think that is related with the way we code, we usually can repro all performance issues and test even in other editors. From my point of view, after Jacob's change, clojure-lsp is fast as before, even faster with some other improvements we did, so I still believe it's something related with your emacs or something as we didn't have any more complains even posting on #lsp channel. It's not that easy to rollback to lsp4j I think and not sure it's worth to maintain 2 versions of that unfortunately. My suggestion is to release a new version with that as we already have lots of changes for next release which would be good to be released and check for some time if there are any other people with issues like yours.

What I suspect is that other people have these issues too, but they just don't notice it (yet)

The June version doesn't have this issue for me with the same emacs config, so I have a hard time believing that it's my emacs config

but maybe co-incidentally some issue will be found in the future and perhaps I will be able to upgrade at some point in the future... :(

Yeah, having it only on nightly limits a lot the people that will test that, that's why if we have it as release we may find other people with that issue and maybe in other editors

Makes sense. Btw, I'll probably release a new clj-kondo too this week

Cool, so I'll wait for that

probably Wednesday

FWIW, I see the big lag using doom emacs, when doing the xxxxxxxx test in analyzer.clj from clj-kondo sources. I can confirm it is pretty bad. I also see a vast improvement in the nightly build. The nightly build lag seems fine for me but I work at leespeed not borkspeed. For folks who work at borkspeed the lag could feel like eons. I’m not currently working on any larger files right now (the lag is correlated to larger files?), when I get back into working on a larger file I might get a better feel for the effect of the lag on my psyche.

so for you the nightly fix the issue right? same for me, and I consider that I usually type fast as well IMO 😅 (that's why releasing this fix may bring more users or no)

> fix the issue right? I think what we are seeing is that the lag got reduced to an amount that for some people is tolerable and for some people is not. And it's still not close to what it was imo.

That's what looks like for you right? we are not sure yet, for all other users that complained they consider fixed

I think @UE21H2HHD told me earlier that he does notice the lag, but it doesn't bother him enough to feel it.

Your gif representing your issue is quite noticeable, so I don't think it's a "Some people may notice or not" issue, the lag was quite high on the gif.

Yes, I'm referring to the new nightly

but even at this moment, I don't get why not more people reported the problem with the currently released lsp

The first one who reported this was me, 5 minutes after upgrading and then you all could reproduce the problem, while you had been working with this for probably weeks

I noticed the problem too, just didn't have time to take a look/not sure was really clojure-lsp or any other thing on my setup The idea is to have it released as latest release is really being affected

I’ll retry the xxxxxx test with current nightly on doom emacs.

Apologies for not being able to follow up on this. I’ve been traveling for the last few days. I’ll have a few hours to look at it this morning, but I have a cold so my energy is pretty low.

@UE21H2HHD are you having freezes too? Are you running the nightly or the last official release?

@U07M2C8TT Keep in mind we don't need to hurry IMO, we should do the release this week soon since we have most users using the latest release with the bug, and this new release should fix or at least improve a lot, so take your time

@U04V15CAJ will you (setq garbage-collection-messages t)?

I know… I’m not feeling a TON of urgency, but I do want to figure this out. I’m starting a new job next week, so I won’t have as much time for this. I’d like to make some more progress on it this week

@U07M2C8TT ❤️

I'll try it now: (setq garbage-collection-messages t)

Emacs says that it will log whenever it does GC if you set garbage-collection-messages. This doesn’t actually work for me—I get nothing in Messages—but if it works for you, it’s a good thing to try first

I don't see anything either (assumed that this lag is caused by GC)

Another quick thing to check… does your clojure-lsp server log include Trace messages—the logs of the actual JSON going back and forth—or just the regular clojure-lsp logging?

You mean the file to which lsp logs on disk?

no JSON there

Yes, that’s the file I meant. An aside… In the old code there were 3 log sources. 1. lsp-mode trace (JSON) logs (available in a buffer in Emacs) 2. clojure-lsp normal logs (a file on disk) 3. clojure-lsp trace (JSON) logs (a different file on disk, but which required a special build to set up) In the new code, the two flavors of clojure-lsp logs have been combined into one file. The tracing can be turned on or off, either by making a special build, or by starting the clojure-lsp executable with --trace

About 1. I'll make lsp-mode stop tailing that file autoamtically, removing gcs

> I’ll make lsp-mode stop tailing that file autoamtically Uh… will you explain in more detail?

lsp-mode tails which file automatically?

Yes, when I created lsp-clojure-server-log I thought it'd be a good idea to automatically emacs tail that file, I just didn't know emacs was terrible on performace doing that :) https://github.com/emacs-lsp/lsp-mode/blob/master/clients/lsp-clojure.el#L246-L269

Does that tail start whenever you open a Clojure file? Or only when you run lsp-clojure-server-log? That is are you always tailing that log, or only sometimes?

But that should only affect performance if: • user has the log file huge • user has the log buffer opened

Only when running that command manually

OK. @U04V15CAJ do you know if you’ve run that command in your current Emacs session?

dunno, but I'll restart just to make sure I haven't

restarted but no difference

@UKFSJSM38 my Emacs “remembers” open buffers, so restarting sometimes re-opens stuff I had tried to close. Do you know if there’s a way to disable that?

Not really, I didn't know about such feature

OK, not a big deal

I'll be happy to try out some minimal other emacs config with lsp enabled

@U04V15CAJ that’d be great. I have a few other ways to check whether Emacs GC might be to blame… want to try those?

There is one lsp-mode's lsp-start-plain which we recommend to test bugs

@U07M2C8TT I wanna try everything

https://emacs-lsp.github.io/lsp-mode/page/troubleshooting/#check-if-any-package-or-wrong-config-could-be-causing-the-issue

haha… sounds good. I just don’t want to send you on too many errands simultaneously. Which do you want to try first? More GC research or a simple Emacs config?

simple emacs sounds good, preferably with CIDER and clojure-mode etc already enabled so I can test it in clojure

perhaps it would be good to have such a minimal config for clojure-lsp users in general

which you can just swap with your .emacs.d

I do not use vim keybindings so anything with that, I can't work with

We have this one: https://emacs-lsp.github.io/lsp-mode/tutorials/clojure-guide/#basic-configuration

Maybe we should link it on clojure-lsp troubleshooting, although it's emacs specific.

ok, I'll try this one

it's installing now...

I just tried lsp-start-plain.el, but I get Debugger entered--Lisp error: (cl-assertion-failed ((seq-every-p (apply-partially #'stringp) command) "Invalid command list")) as lsp is starting. I’ll try the Clojure specific config

I had no lag with the minimal setup, but when running lsp-clojure-server-info it turned out it was running the lsp4j one ;)

Now trying to switch to the nightly again...

ok back on :server-version "2022.09.29-20.05.41-nightly", now... let's try

Hmmmm, I can't reproduce now with the minimal setup, interesting!

Oh, so glad to see progress!!

Btw, this is useful to add to the minimal setup for debugging with a specific clojure-lsp: lsp-clojure-custom-server-command '("/Users/borkdude/bin/clojure-lsp-2022-09-30")

For sure, I'll update the docs soon with this kind of debugging

@U04V15CAJ I suggest you start enabling some lsp features you have on your config, like lens etc and check if that changes anything

Very exciting. Still a lot to bisect, but at least we have a starting point

FYI: Removed the tail from lsp-clojure-server-log , it should not cause performance issues anymore on latest lsp-mode.

I can't get my separate clj-kondo install working alongside this minimal setup

Ok, I’m back… retried xxxxxxx test on clj-kondo analyzer.clj on latest nightly clojure-lsp and do not notice any lag.

@U04V15CAJ wait a sec, all this time you were using clj-kondo diagnostics with lsp?

this could be important to test, if clj-kondo could be causing any conflicts with lsp-mode, I know you use like that for years though

No, I disabled those, I'm always running my own version of clj-kondo

oooh… that explains why i wasn’t getting squiggles with your lsp-mode config @U04V15CAJ

I now was able to enable it. For some reason I had to toggle flycheck-mode

but still no lag 😅

and also why i haven’t noticed the new unused expr warning

Ah well, maybe I'll just use this minimal setup then :P

(I'll keep digging)

How do I start another emacs with my own config again? I can hardly work here ;)

I think you can pass a flag to emacs command to point to a file :thinking_face:

maybe

emacs -q --load <path-to-your-init.el>

Awesome: emacs -q --load ~/.emacs.d.bak/init.el

Agh… wrong file

It’s subtle, but there’s a tiny pause as I’m deleting the ~10th character on the 3rd line of xxxs. Emacs’ memory usage also drops right around that time

yep, the tiny pauses.... but I only have them when I stop and then edit again

@U04V15CAJ would you keep an eye on Activity Monitor to see if you notice a drop in Emacs memory usage around the time that the pauses happen? Remember that Activity Monitor only samples every 2 seconds or so, so there may be a delay before a new reading. Can anyone recommend a better tool than Activity Monitor?

https://emacs-lsp.github.io/lsp-mode/page/performance/#reporting-performance-problems should help

(dinner, be back tonight)

I am bisecting my config now and I excluded some packages, it still works now... slowly adding stuff back

Well, it seems throwing away my elpa directory and recompiling all packages fixed the issue

How on earth could that fix the issue and what has this got to do with the clojure-lsp version I wonder... but it would maybe be good to add to the "troubleshooting" section of clojure-lsp...

Welcome back to the awesome world of up-to-date clojure-lsp @U04V15CAJ!

Thank you all for hanging in there with me!

It's probably not the first time that an emacs issue got fixed by deleting elpa, but I always forget to do this

I didn’t do much but I do know we must invest in keeping the borkmachine productive.

If I ever meet you, I owe you guys a beer or two

Likewise! But coffee beans for me please!

"beer" as placeholder for any drink you would like

Awesome! Never faced a issue where the backend change could affect emacs elpa packages, pretty wild! Doom uses straight el to manage packages so probably that's related why I never faced that

It was a worthwhile adventure. I’m really glad that we found and fixed the CompletableFuture problem. I’m quite confused why the clojure-lsp version makes a difference, and still wonder if there’s some lingering problem. I’ll think about it

I’m also using straight.el

I should probably rewrite my config to one of those things and ditch 50% of what I'm using right now

I use doom and mostly with a config stolen from @UKFSJSM38

one of the problems I haven't solved to satisfaction is version-control my elisp packages, this is why I even save the elpa dir in git

> version-control my elisp packages I’m not an expert on straight.el, but I think that’s the problem that it’s supposed to solve

So a few things that helped debugging/narrowing down the issue: • Minimal .emacs.d for clojure(script) users: this showed the issue didn't occur with the other config • (which lead to): Removing elpa (after bisecting) Both things are probably good to have in the trouble-shooting section

will add to docs, thank you!

@UKFSJSM38 @U07M2C8TT @UE21H2HHD Would any of you be interested in a free babashka t-shirt, as a thank you? (Feel free to say no if this is not your cup of tea). https://www.etsy.com/borkdude/listing/1241766068/babashka-clj-kondo-nbb-shirt

That would be amazing! Totally unnecessary though

Dude! Cool! Didn’t know that these existed. I’d be happy to buy one.

And nbb and kondo on the back! Neat!

@U07M2C8TT I mean it, let me know your address and I'll make sure one gets delivered

and also the size etc.

Oh, I'd love it!

I’m already pretty darn handsome, I’m not sure the world could handle me in a shirt like this.

DM me the color, size and address if you want one, take it or leave it :)

Someone just mentioned eglot to me: https://github.com/joaotavora/eglot I'd never heard of that. Is it any good, what are the differences?

It has less feaatures than lsp-mode, I personally prefer lsp-mode, but have some friends that use it with clojure and say that work well. There is a https://github.com/joaotavora/eglot/issues/661 though that you can't go to jars or goto def of java

The development is usually slower than lsp-mode as it's shipped on elpa or something like that IIRC

Eglot doesn't support code lens and call hierarchy too AFAIK

I see, so no "x references"?

I'll stay with whatever the maintainers of clojure-lsp are using for now, seems the safest ;)