I haven't seen this yet with the latest stable release. Have you?

I have, I went on nightly hoping it would be fixed

Ah ok then

I disabled a bunch of modes just to be sure it's not coming from stg else.

(like whitespace-mode etc)

it just happened in the middle of a string "yolo|asdf"

I did have a problem that was gone after I wiped my elpa dir and re-compiled

I did that as well

are you using lsp-mode?

I ll try to catch it on the profiler later today if I have some time

yes

it happens once every now and then (like every few hours)

I was wondering if I was not hitting https://github.com/clojure-lsp/clojure-lsp/pull/1322 looking at the wip things in the repo, but not sure

it only happens while I am typing

and in the middle of a string no completion would happen

I did have a similar problem a while back, did you follow that?

nope

https://clojurians.slack.com/archives/CPABC1H61/p1664028339428649

and this one: https://clojurians.slack.com/archives/CPABC1H61/p1663620137210819

but those problems are now solved for me

❯ clojure-lsp --version clojure-lsp 2022.10.12-19.04.48-nightly clj-kondo 2022.10.05

For me the lag happened when I typed, then briefly stopped and then continued. Just the same letter, like: xxxxxxxx<pause>xxxxxx

But no freezes, just not being able to input the next letter

it freezes for seconds at a time for me

that's a different problem then, darn

@U07M2C8TT was hinting at garbage collection. I do see the freezes in an html-mode when editing JS inside HTML btw but there lsp-mode isn't activated (I think).

yeah, with a 5-10s freeze I am not sure it's just gc

but I could be wrong

I have a feeling I might have seen the same problem. Sometimes when doing completions in emacs it hangs for a moment. Usually I just C-g which clears it immediately and then everything works fine, so I just shrug and keep going. I'll try to diagnose it better when/if it occurs again.

In my case c-g has no effect

I think I have an idea:

not sure but I ll share anyway

we have a custom http proxy (actually written in clojure) to retrieve maven artifact internally, and there was a bug where it would fail to return .jars in some cases, I kind of suspect that could have been linked. Like the request somehow fails and that makes the clojure-lsp bit that fetches jars to do analysis hang or something.

but maybe I am pusing it, and it's not related, but I didn't hit the bug since we corrected that earlier today, we ll see

Does clojure-lsp fetch jars?

maybe, not sure, I guess it might get them from the local fs

It doesn't fetch jars. @U050SC7SV could you follow the https://clojure-lsp.io/troubleshooting/ and get the logs between server and client and confirm clojure-lsp is taking 5s to respond with completion items?

Also, test with older releases and double check the https://emacs-lsp.github.io/lsp-mode/page/performance/

@U050SC7SV Did you find out what was happening? I disabled this at one point: https://github.com/borkdude/prelude/blob/2a951dfa28d8b7c535430cc86ead7ccff18c43a3/personal/init.el#L382 which could also have been related to some similar problem

A caveat about disabling file watchers (which is what @U04V15CAJ’s doing in his init.el)… If you do, clojure-lsp will become out of sync with the file system. This will happen whenever files are changed without the participation of the editor, most often when you switch branches. The outcome is incorrect lint, broken or incorrect navigation, and potentially syntax mangling, especially when doing renames or other edits across files. Before clojure-lsp implemented file watching, we all dealt with this and it was annoying. Sometimes you can fix things by finding the files that have changed and touching them so that their contents are re-synced with clojure-lsp, but this is tedious and error prone. The other fix is to restart clojure-lsp. Anyway, be warned—you’ll be trading one set of problems for another. If you decide that file watching is the source of your problems but don’t want to disable it in all cases the https://emacs-lsp.github.io/lsp-mode/page/file-watchers/ offer some alternatives.

I haven't experienced any sort of trouble, but I could just be ignorant and my emacs behaved a lot better after that ;)

I mean, when I suspect something is incorrectly coming from another namespace, I go check there, and go back and the problem disappears - for me it's worth the trade-off and I'm used to that behavior: it's also how clj-kondo + flycheck works - a simple model that I can understand

I think the reason I started doing this was that lsp-mode had no way to ignore certain folders and this caused way too many watches

and when talking into their discord, they had no plans of implementing some kind of ignore thing

but I could be misremembering it

Have you tried

lsp-file-watch-ignored-directories

lsp-file-watch-ignored-files

Those are probably relatively new things then

but no I haven't and I don't feel the need to enable things since I don't experience a problem

I’m not that familiar with the lsp-mode internals, but I get the feeling file watching works okay on small projects, but causes performance problems on large projects

that was indeed the problem

and also issues like this: https://github.com/emacs-lsp/lsp-mode/issues/2741 (I think there were various such issues coming up now and again)

or actually it was this issue: https://github.com/emacs-lsp/lsp-mode/issues/713 which is still open

The file-watch ignore exists since I know what is a LSP. We had huge performance issues with watch on server side, and we now analyze in batches, IMO it's pretty stable even on big projects

It should be as bad as just start the project again when you change branches as we will lint it in batch as well

I might enable it when I experience I'm missing out on something 😆

yup, i opened a regression issue but it didnt make it into the latest release because the freezing issues were more important to get fixed fast :(

Oh yeah, I just noticed that yesterday too.

👍

https://github.com/clojure-lsp/clojure-lsp/issues/1287 if anyone wants to try to find the offending commit it would be helpful. I'm unable to spend anytime on lsp for the foreseeable future.

Yeah, I'm ok making a new release as soon we fix that ☝️ it's hard to debug neovim for me, so if anyone finds the commit, LMK

(Side remark: Don't forget to bump clj-kondo, just released a new one)

I just did!

Doing a bisect now

❯ git bisect bad
d6ca21a6d2235d7438fd8901db93930b80ae257b is the first bad commit
commit d6ca21a6d2235d7438fd8901db93930b80ae257b
Author: Jacob Maine <[email protected]>
Date:   Tue Sep 20 11:33:21 2022 -0500

    Indexes analysis and findings by URI (#1231)
    
    * Indexes analysis and findings by URI
    
    By doing URI/filename conversion at the boundary with kondo, most of the
    app can use URI without needing to convert to filename.
    
    * Fix missed refactoring identified by integration tests
    
    * Introduce helper
    
    * Expand created/deleted directories before analyzing or deleting
    
    * Document places where we could display URI
    
    * Fix lint
    
    * Remove comment accidentally restored during merge

The regression starts with this commit

Cool, thanks @U11EL3P9U, unfortunately I can't tell exactly why that would affect only zipfiles uris, but it's related indeed with uris. Maybe you could check server logs responses and check what comes different between both commits (nightly and before this one)?

FYI @U07M2C8TT if you have any clue as well

Sure, although pretty late here now, I'll see if I can grab some time over the weekend.

@U11EL3P9U thanks for doing the bisect. I’m not surprised that’s the commit that’s causing problems, although when @U0BUV7XSA originally reported this he said he’d ruled out d6ca21a by testing a4a44f55, the commit immediately after it. But d6ca21a touches things that all have to line up for goto-definition to work, so I trust your analysis. I’ll start working on getting my nvim running again so I can debug this. In the meantime, will you give me some more detail about how you did the bisect? What repo were you testing in? What namespaces were you navigating from and to? Will nvim tell you the name of the JAR file?

Also, are you using coc.nvim? I’d like to be able to run :echo CocRequest('clojure-lsp', 'clojure/cursorInfo/raw'), but it’s giving me an error

Hi, I am on master of clojure-lsp so started from the pulling the latest. I have a very simple test project that I used (I'll push to git), I started from head, marking it as bad, then looking at the github issue, I marked a4a44f55 as good, then I just went from there. I did bb debug-cli each time I marked a bad commit, reopened my neovim editor, waited until the analysis of my project was done, then gd'ed into the validate function (at line 7). Then if the library (malli) went highlighted, I knew it had worked, confirmed by being able to gd around the jar dependency.

I do also use CoC. I normally (if debugging stuff like this) just do CocCommand workspace.showOutput to see the request/responses back and forth

Here is a dump of the text on the moment I gd on the validate function:

I've zipped up the text file.

Hope that helps!.

@U11EL3P9U thank you! I see that client sends URI escaped on didOpen after findDefinition: zipfile:/home/david/.m2/repository/metosin/malli/0.8.9/malli-0.8.9.jar%3A%3Amalli/core.cljc , could you please paste the analysis of the previous version when it was working? (you don't need to zip)

I think that was the one where it was working...was it? I can't recall.

haha so we need the other to compare, have no idea which one was what you sent 😅

I'll do both

Give me a few minutes 🙂

They are big files, so I'll zip them up again if you don't mind.

Right, both logs looks correct, which is interesting...

i suspect its internally how we index or lookup analysis thats changed. it's weird tho that the integration test wouldnt have found it.

yeah, I suspect we receive it escaped but save unescaped, but some place is still finding it as escaped

I think kondo analysis come unescaped, @U11EL3P9U maybe running clojure-lsp dump > my-analysis.edn may help here, it should produce a really big file, so feel free to zip it

Do I run that in the sample project I have?

yes

@U0BUV7XSA the integration test probably is not sending didOPen with escaped URIs, we should probably add one doing that

You're not kidding it's large

Shall I magic wormhole it to you?

(even after zipping it's 1.4M in size)

Is it a problem paste the 1.4M here?

Not at all, no problemo 🙂

that's using clojure-lsp master head

yep, we save kondo analysis uris unescaped which looks correct, but when we receive requests from client we don't "unescape" the URIs, probably that's the issue.

not sure why start happening recently though, we did so many changes, but I'll try to add a integration test using escaped URis and see if I can repro the issue via code, thanks for the help @U11EL3P9U

np, happy to try out a branch if/when ready 🙂

Oh, I think I know why was working before, because we used to always uri->filename and inside that function we always unescape uris, but now we don't use filanems, only uris, but we forgot to unescape c/c @U07M2C8TT

@UKFSJSM38 that makes sense. thanks for doing the research. sorry i haven’t been able to investigate. the goal of that pr was to make the analysis match the client’s uris. so i think the fix would be to ensure that the analysis is indexed by escaped uri (and that the elements :uri is also escaped)

as long as that would also work for non-zipfile schemes

Agreed, not sure if kondo will return escaped URIs as well, but the integration test will confirm that

Out of curiosity, any exceptions on https://clojure-lsp.io/troubleshooting/#server-log @U11EL3P9U?

lettmecheck

Nvm, I was checking a old log, I think you won't find any exceptions as expected

No exceptions

Would you like the log anyway?

no, tks, I have a integration test reproing the issue finally

👍

Was thinking about this on my drive home just now, which is like a mildly stressful version of hammock time. I think storing escaped URIs in the analysis is only half of the fix. Let me write up my thoughts so you all can see if they make sense

Yep @U07M2C8TT I'm trying to fix it and just found we are not handling escapes correctly on clojure-lsp, we only found that because nvim is the only client sending encoded IIRC

I made a uggly quick-fix, will commit to a branch with the integration-test

Will need to be AFK for some time, but this branch should fix the issue but not sure we should merge it as I kind of hard coded that zipfile will come always encoded. @U07M2C8TT feel free to take a closer look, we should find a better way to infer URIs will come encoded

https://github.com/clojure-lsp/clojure-lsp/pull/1327

There’s one more piece… gimme a sec to explain

Thanks @UKFSJSM38. I’ll review

As I was writing everything down, the epiphany I had in the car sort of melted away. But the analysis might still be interesting: https://github.com/clojure-lsp/clojure-lsp/pull/1327#issuecomment-1279855700. @UEENNMX0T I think we’d have a nicer fix than what @UKFSJSM38 is proposing if we made a simultaneous change to coc-clojure. See details in that Github issue comment. What do you think?

I’m at a wedding this weekend so to help me remember, open a barebones issue on coc-clojure linking to the lsp issue?

But I’m amenable to basically any change

Will do—thanks!

https://github.com/NoahTheDuke/coc-clojure/issues/8

@U07M2C8TT I didn't understand why changing NoahTheDuke’s coc-clojure would work, is that the only canonical way to use clojure-lsp? I thought vim users could use nvim native lsp client without coc at all, if so, changing only coc-clojure wouldn't be enough, right?

@UKFSJSM38… I’m really not sure. But, FWIW, I no longer think my proposal for coc-clojure is the way forward anyway

I’ve found a different approach, which seems to be working

yeah, agreed, changing server would be the safer

I was able to reproduce the problem in nvim, and have a build that fixes it. Now I’m going through all the other editors making sure they work. The fix I developed for nvim doesn’t work for Emacs, and the fix for that didn’t work for Calva. I have a third iteration which I think works for all 3, though I’m still testing edge cases

Great, thanks @U07M2C8TT, that sounds really dangerous indeed, we should improve that in the future as you mentioned

@U0BUV7XSA and/or @U11EL3P9U I’ve made some changes to https://github.com/clojure-lsp/clojure-lsp/pull/1327. Would you test that branch again?

Yup, that seems to work (at first I thought it didn't, but then I removed the cache files - then it did)

Nice, we should bump clojure db version to avoid the news of removing cache @U07M2C8TT

@UKFSJSM38 there are several clean up activities... 1. Bump db version 2. use with-redefs 3. Switch integration test from hover to definition 4. If possible, confirm that the branch works on Windows. See discussion in the Issue about Calva's encoding of drive letters

I won't have a chance to work on the branch until this evening at the earliest, so if you want to take ownership today, feel free

Sure, if the branch fixes it we can merge IMO, and open PRs for the missing cleanups

@U07M2C8TT as another data point, the branch works for me too! Thanks for the fix.

@U07M2C8TT I made a commit adding the extra find-definition from the extrnal file and it fails, so I wonder if we are missing anything or if the test is wrong. clojure-lsp return URIs not encoded, will the client send the next request encoded? (the integration test was not doing that

@UKFSJSM38 will you double check that last commit that was pushed to Github? The final assertion looks wrong—the expected output of the find-definition command is the same as the expected output of the hover command. To answer your question, generally it seems that the clients transform the URIs we pass to them before they pass them back to us. We can't really make many assumptions because the spec is silent on the issue. But in practice, yes, often the client will send URIs that have been encoded, even if we gave them a decoded URI. In real life, I've noticed clients make the following changes: remove the // authority section; encode the double colon :: separating the jar file from the jar entry; encode the single : following jar:file. I'm not sure how to translate all that into an integration test, but maybe it'll help get the existing tests passing.

Note that different clients make different combinations of the changes mentioned above. And there may be other changes that happen on Windows that I haven't seen.

Oh my bad, I forgot to change the expected, but my concern was before, since I was not sure if we should re-encode the uri returned by server, will take a look soon

Yeah, I guess the mock client should re-encode URIs returned by the server. Not every client does that, but that’s the case we need to be the most careful about

No problem, if the integration test passes re-encoding the uri I'm fine

I fixed the tests and merged the PR, thanks everyone for the help!