Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2021-08-03

Channels

#lsp
<
2021-08-03
>
pez11:08:47

Is there a way where I can make Calva reach the clj-kondo problem analysis? My use case is that clj-kondo is better than Calva at finding unbalanced brackets, so am considering if I should try use lsp for this.

borkdude11:08:41

@pez You could also invoke clj-kondo on the currently edited file (since this problem is very local)

borkdude11:08:57

but I don't know if you are using a JVM necessarily

borkdude11:08:18

@pez FWIW, edamame also warns about this

borkdude11:08:44

and it works also in ClojureScript

borkdude11:08:23

but yeah, if you could piggy back on LSP that would probably make sense, since that's already being used

pez11:08:25

I might turn to edamame for it in case the LSP piggybacking seems tricky.

borkdude11:08:59

@pez This should allow you to parse most code: 

user=> (e/parse-string "#?(:clj 1 :cljs 2)" {:read-cond :allow :all true :features #{:clj :cljs :bb} :auto-resolve (fn [ns] 'foo)})
1
user=> (e/parse-string "#foo {:a 1}" {:read-cond :allow :all true :features #{:clj :cljs :bb} :auto-resolve (fn [ns] 'foo) :readers (fn [tag] #(tagged-literal tag %))})
#foo {:a 1}
user=> (e/parse-string "::a/x" {:read-cond :allow :all true :features #{:clj :cljs :bb} :auto-resolve (fn [ns] 'foo) :readers (fn [tag] #(tagged-literal tag %))})
:foo/x
user=> (e/parse-string "::x" {:read-cond :allow :all true :features #{:clj :cljs :bb} :auto-resolve (fn [ns] 'foo) :readers (fn [tag] #(tagged-literal tag %))})
:foo/x

borkdude11:08:03

And this is the error for the mismatched paren: 

user=> (e/parse-string-all "1 (")
Execution error (ExceptionInfo) at edamame.impl.parser/throw-reader (parser.cljc:102).
EOF while reading, expected ) to match ( at [1,3]

pez12:08:00

Thanks! Almost looks like I could replace Calva’s own parser with this. 😃

pez12:08:20

I really would like to keep the number of AST:s down to a minimum, which is why I like the idea of piggybacking on LSP.

ericdallo13:08:31

clojure-lsp Released https://github.com/clojure-lsp/clojure-lsp/releases/tag/2021.08.03-13.33.03 with awesome news 🎉 • General ◦ Parse correctly unescaped URIs sent from clients like vim avoiding errors on some features. (kudos @dharrigan) ◦ Bump clj-kondo fixing analysis position issue with `declare`, making rename and other features work. ◦ Don't use PowerShell profiles on Windows when analyzing classpath. Fixes https://github.com/BetterThanTomorrow/calva/issues/1050 ◦ Support babashka classpath and source-paths discovery via bb.edn file. (needs babashka >= 0.5.1) • Editor ◦ Add `:hover :hide-file-location?` settings option to disable displaying the source path on hover. ◦ Use new clj-kondo `:custom-lint-fn` for the `:unused-public-var`, this should improve performance and give the ability to suppress unused vars via code with `#{:clj-kondo/ignore [:clojure-lsp/unused-public-var]}` or `#:clj-kondo/ignore` I'd like to thank you @borkdude for the help on #babashka and #clj-kondo side making it possible to clojure-lsp now understands babashka projects/scripts 🚀 From now on, clojure-lsp will make available completion, rename, find-references and all features you already know work great with bb as well babashka Another huge improvement/refactor on this release is how clojure-lsp lint public unused vars, we now use a new feature from clj-kondo called `custom-lint-fn` where it's possible to "create" its own linter but using clj-kondo features, giving the possibility to ignore this linter via code 🎉 Also this improves performance and open other possibilities to other clojure-lsp features 👀

🙏 12
🎉 9
dpsutton16:08:31

Has there been a request to handle clojure.test in a special way? I suppose this is really a clj-kondo request but we often use (:require [clojure.test :refer :all] and then lsp will underline deftest testing is etc as if they are unknowable symbols. Perhaps clojure.test deserves a bit more bespoke attention?

dpsutton16:08:52

in this screenshot deftest and testing are highlighted as unresolved symbols

ericdallo16:08:09

this is a misconfiguration with your project

ericdallo16:08:18

probably your project was not analyzed correctly with clojure-lsp

ericdallo16:08:43

so clj-kondo doesn't know what macro/functions/vars come from the :refer :all

dpsutton16:08:44

i'll look into that after i finish this up. that's good to hear that this should work

👍 3
borkdude20:08:48

Normally this should even work without analysis, because clojure.test is built-in

borkdude20:08:03

If you can repro this using clj-kondo only, then feel free to create an issue at clj-kondo

👍 3
wcohen20:08:18

have any doom emacs users successfully gotten lsp to work over tramp?

ericdallo21:08:24

I think this can be improved on lsp-mode to make it easier for users

wcohen21:08:01

wow — i was so busy digging in clojure-lsp and doom config github issues that i didn’t even think to look in lsp-mode’s. thank you so much!

tvaughan21:08:42

(use-package lsp-mode
  :hook
  ((clojure-mode clojurec-mode clojurescript-mode) . lsp)
  :config
  (setq gc-cons-threshold (* 100 1024 1024))
  (setq lsp-enable-completion-at-point nil)
  (setq lsp-lens-enable t)
  (setq lsp-log-io t)
  (setq read-process-output-max (* 1024 1024))
  (lsp-register-client
   (make-lsp-client
    :major-modes '(clojure-mode clojurec-mode clojurescript-mode)
    :new-connection (lsp-tramp-connection "clojure-lsp")
    :remote? t
    :server-id 'clojure-lsp-remote)))
I'm not a doom user, but this works for me over tramp. When opening a clojure file locally I get an error message about not being able to start clojure-lsp which I don't have installed locally. I simply use C-g to skip over this error message and things seem to carry on ok

ericdallo21:08:35

@U0P7ZBZCK this way of registering a new client usually brings a lot of bugs, since you are not registering the client like lsp-mode originally register the local one, that's why I think that issue will help that

tvaughan21:08:48

Good to know. I do encounter problems starting lsp-mode. I simply combined this https://emacs-lsp.github.io/lsp-mode/page/remote/#sample-configuration and https://emacs-lsp.github.io/lsp-mode/tutorials/clojure-guide/#basic-configuration to produce this setup. I'll watch the issue above to know when and how to correctly update this. Thanks @UKFSJSM38

👍 3
zane22:08:43

Might want to cross-share this with #doom-emacs!

walterl21:08:28

I've noticed that clojure-lsp quite likes melting my CPU. Seems like it's completions, but it happens even when editing docstrings. Is there anything I can do to mitigate this? Log in 🧵

walterl21:08:13

Notice the pause at the blank line.

ericdallo21:08:47

uow, 47s of completion is too much, is your buffer huge?

walterl21:08:01

This particular one is only 49 lines 🤷

ericdallo21:08:12

that's weird, are you using emacs/lsp-mode?

walterl21:08:28

You think the problem lies there?

ericdallo21:08:28

even so, this is taking time on server side

walterl21:08:57

Oh, and htop tells me it's clojure-lsp eating on all my cores 😅 🔥

ericdallo21:08:08

I found in the logs you have a outdated clojure-lsp, maybe update it?

walterl21:08:13

Happy to help debug any way I can 🙂

walterl21:08:26

K, let's see...

walterl22:08:08

clojure-lsp --version
clojure-lsp 2021.07.28-14.24.06
clj-kondo 2021.07.28

walterl22:08:22

A little better, but still pretty hot.

ericdallo22:08:30

after updating and testing again, could you share the code or a minimal sample project which you are testing?

walterl22:08:56

Will do 👍

walterl22:08:06

❯ clojure-lsp --version
clojure-lsp 2021.08.03-13.33.03
clj-kondo 2021.08.03

walterl22:08:38

Test file was literally just this.

walterl22:08:53

I first added the function, and then the docstring's first line. Up to this point is where the completions seem to still respond quickly (<200ms), until line ~283.

walterl22:08:05

After that I added the second line to the docstring.

walterl22:08:32

And that's where you can see the completions all took >30s

walterl23:08:54

The project is 19kloc without tests, 29kloc with tests. Maybe project size is relevant?

seancorfield23:08:53

FWIW, I occasionally see lsp lock up like this on projects -- primarily the project at work (which is large). Mostly, I just wait for it to "catch up" but it is kind of annoying 😕

seancorfield23:08:36

I haven't managed to nail it down to anything reproducible tho'...

walterl23:08:46

Yeah, I'm not complaining. It's not a show stopper. 🙂

ericdallo23:08:43

maybe the project size is relevant indeed, still I can't easily guess where the bug could it be without a repro where I could test it 😕

ericdallo23:08:01

using that file in a simple project doesn't repro the issue to me

walterl23:08:39

Yeah, for me too

walterl23:08:08

I'll let you know if I find a repro'able, shareable example 🙂

👍 3
ericdallo23:08:29

@UJY23QLS1 checking the logs we can see that the issue is not completion, but the didChange (it says 0ms but it's processed async, and most features need to wait its completeness)

walterl23:08:09

It seems like it fires one of those on each keypress. Is that expected?

ericdallo23:08:15

so probably is something delaying the changes made in a file or multiple changes (but that should be handled correctly by clojure-lsp with a debounce logic)

ericdallo23:08:29

yes, it's expected on each file change

👍 3
walterl23:08:25

Perhaps also relevant: CPU usage drops off as the completion items are logged, and it's definitely something CPU-bound.

ericdallo23:08:31

I don' t think it's a completion issue, this is a consequence of the need to wait for the completion request

walterl23:08:38

K. If you have any ideas for tests to run, I'll be happy to do so 🙂

walterl23:08:09

My only idea is to start a test project, and add a bunch of deps, until I get the same behavior 🤷

ericdallo23:08:04

yeah, anything where I could repro the issue on my side would help a lot

👌 3
Tomas Brejla08:08:34

Hi, I also experience 100% cpu usage of clojure-lsp process. I see output such as this. This is on tiny hello-world-like project in calva. Times such as this scare me 🙂 

1 505 697 milliseconds = 25.09495 minutes
2021-08-05T08:06:45.000Z brdloush-nb DEBUG [clojure-lsp.server:?] - :didSave 0ms
2021-08-05T08:06:45.000Z brdloush-nb DEBUG [clojure-lsp.server:?] - :didChangeWatchedFiles 0ms
2021-08-05T08:06:47.275Z brdloush-nb INFO [clojure-lsp.feature.diagnostics:92] - Linting public vars took 7ms
2021-08-05T08:06:47.279Z brdloush-nb DEBUG [clojure-lsp.server:?] - :codeAction 11288ms
2021-08-05T08:06:47.283Z brdloush-nb DEBUG [clojure-lsp.server:?] - :documentSymbol 1575ms
2021-08-05T08:06:47.714Z brdloush-nb DEBUG [clojure-lsp.server:?] - :codeAction 61ms
2021-08-05T08:06:48.047Z brdloush-nb DEBUG [clojure-lsp.server:?] - :documentHighlight 4ms
2021-08-05T08:06:49.049Z brdloush-nb DEBUG [clojure-lsp.server:?] - :didChange 1ms
2021-08-05T08:07:01.049Z brdloush-nb INFO [clojure-lsp.feature.diagnostics:92] - Linting public vars took 0ms
2021-08-05T08:07:14.392Z brdloush-nb DEBUG [clojure-lsp.server:?] - :codeAction 25915ms
2021-08-05T08:07:14.393Z brdloush-nb DEBUG [clojure-lsp.server:?] - :documentSymbol 22242ms
2021-08-05T08:15:27.925Z brdloush-nb DEBUG [clojure-lsp.server:?] - :codeAction 673214ms
2021-08-05T08:15:27.941Z brdloush-nb DEBUG [clojure-lsp.server:?] - :codeAction 674274ms
2021-08-05T08:15:36.504Z brdloush-nb DEBUG [clojure-lsp.server:?] - :codeAction 675318ms
2021-08-05T08:27:17.113Z brdloush-nb DEBUG [clojure-lsp.server:?] - :codeAction 1362238ms
2021-08-05T08:27:17.196Z brdloush-nb DEBUG [clojure-lsp.server:?] - :codeAction 1359967ms
2021-08-05T08:31:15.295Z brdloush-nb DEBUG [clojure-lsp.server:?] - :codeAction 2ms
2021-08-05T08:31:50.364Z brdloush-nb DEBUG [clojure-lsp.server:?] - :codeAction 1505697ms
Is there anything I should check?

Tomas Brejla08:08:59

I have a feeling that clojure-lsp is somehow trying to parse/index even the repl output in output.calva-repl. Because at this specific case, I was doing some testing of our updated calva, which has improvements of the repl-window performance. So I intentionally emitted over 100k lines into this window (and effectively output.calva-repl ) and now clojure-lsp is eating cpu as crazy.

Tomas Brejla08:08:07

so this specific case seems to be quite easy to reproduce

Tomas Brejla09:08:00

It also seems that sometimes multiple slow operations somehow run concurrently next to each other. They have almost identical elapsed time and finish next to each other. Perhaps multiple threads doing more or less identical action? Or is that a single action splitting its work between 3 workers? 

...
2021-08-05T09:02:39.945Z brdloush-nb DEBUG [clojure-lsp.server:?] - :codeAction 716292ms
2021-08-05T09:02:40.957Z brdloush-nb DEBUG [clojure-lsp.server:?] - :codeAction 717855ms
2021-08-05T09:02:41.047Z brdloush-nb DEBUG [clojure-lsp.server:?] - :codeAction 716685ms

ericdallo12:08:46

Thank you for the investigation @U01LFP3LA6P it helps a lot, @U9A1RLFNV could you check if calva is sending the repl as a file to clojure-lsp? This is something client should not do

Ben Sless12:08:19

Is there some profiling methodology for lsp?

ericdallo12:08:37

Besides the server log that @U01LFP3LA6P sent, the client <-> server json request logs are useful to know what client is sending and receiving

ericdallo12:08:09

This one should be provided by calva, via some log output window

Ben Sless12:08:05

I was thinking more along the lines of attaching clj-async-profiler or something similar

Tomas Brejla12:08:36

would it be possible (and good idea) to add some more logging details to that :codeAction 716685ms output? Such as some information regarding what kind of code action that was?

ericdallo12:08:54

the issue is not on code action, the issue is on didChange where we analyzethe code, and that is taking some time and holding other features like semanticTokens, codeActions, completion

ericdallo12:08:17

we probably can improve how much time the change is taking, ATM since it's async, it always shows didChange 0ms

ericdallo12:08:40

that's why I think it's related with the repl

ericdallo13:08:50

@U9A1RLFNV I just realized we don't have easily that server <-> client json requests logs in a calva output window, do you think we can add a window for that?

ericdallo13:08:12

I confirmed that Calva is sending the repl content as a file to clojure-lsp analyze, this should easily decrease performance while REPL grows, we should fix that @U9A1RLFNV @pez 🙂

Tomas Brejla13:08:01

that sounds like good news, hopefully it will not be too hard to fix

🤞 3
seancorfield13:08:43

@UKFSJSM38 is it possible that Clover is also doing that?

seancorfield13:08:13

Or is this only down to Calva?

ericdallo13:08:58

AFAIK Calva is the only Clojure LSP client for VSCode that uses clojure-lsp, so it's responsibility of the client to send the files to be analyzed/managed by the server, unless you use both Calva + Clover, I can't see this issue happening

3
seancorfield13:08:25

I use Calva + Clover.

ericdallo13:08:16

Yeah, so there is a chance Calva is sending clover repl content as well, it should send .clj/.cljc/.cljs/.edn files only

3
pez14:08:41

I doubt Calva is sending clover repl content. I could be wrong, of course, but I don’t think that is a file in the way that Calva’s REPL is.

ericdallo14:08:58

I don't know anything about Clover, it was just a guess as Calva is sending files that are not clojure 😅

ericdallo14:08:31

I think we can easily know that with that server <-> client lsp json window log

pez14:08:58

I don’t think Calva is sending any file content to clojure-lsp directly. It’s VS Code doing it (I think, let’s see what @U9A1RLFNV says). If I am right then VS Code sends all Clojure files to clojure-lsp. And the Calva REPL is a Clojure file.

ericdallo14:08:44

it makes sense @pez I know Calva just use vscode lib to integrate with LSP, so probably we can configure some kind of filter there

pez14:08:48

Also, I certainly appreciate clojure-lsp help in the REPL.

ericdallo14:08:39

thinking-face What do you mean?

pez14:08:08

I get completions and all kinds of services from clojure-lsp in the Calva REPL. So as long as I don’t lose that I am fine with a filter. 😃

pez15:08:38

I think clj-kondo is not active there, which is mostly for the good, but still… some linter help would be nice as well. (Speaking for the users of the REPL window now, I am actually not one of those myself so often.)

ericdallo15:08:44

Oh, I see now, the repl is handled/sent as a clojure buffer, that's why completion/code actions and other thing work on calva's repl, I didn't know about that 😅

ericdallo15:08:33

yeah, but probably that will have issues as the buffer keeps growing... clojure-lsp needs to parse the whole text and analyze with clj-kondo for most features, a REPL with 10k lines will certainly affect performance

ericdallo15:08:57

My suggestion is to disable clojure-lsp on the REPL to not affect performance of the whole project for now or something like that

pez15:08:52

Could we rather just disable the analysis of the file?

pez15:08:28

Or will that also disable clojure-lsp features in the file?

ericdallo15:08:56

clojure-lsp features depends on clj-kondo analysis

ericdallo15:08:15

is not related with a lint, but with clojure-lsp understand what is each part of the code

pez15:08:23

It is sort of a promise of the REPL window that it is just a Clojure file, and should behave like that.

ericdallo15:08:17

yes, I agree, but clojure-lsp doesn't work well with huge buffers like 10k lines, as it needs to call clj-kondo and rewrite-clj each time something is changed on that buffer

ericdallo15:08:55

so my suggestion is lose clojure-lsp on the REPL window to not affect whole project performance

ericdallo15:08:18

Also, what kind of clojure-lsp features you use on REPL besides completion?

pez15:08:53

I have never seen this performance issue myself. Even though I often build up quite huge REPL output.

ericdallo15:08:02

yeah, maybe we need a more detailed repro

pez15:08:59

It does update often, that much is certain. 😃

pez15:08:55

Also, it is not always structured. Since stdout and results are mixed there.

ericdallo15:08:59

We can start with an issue on Calva side to debug that @UJY23QLS1 @U01LFP3LA6P, also a minimal repro would be perfect to help debug

pez15:08:57

As for implementing a filter. I think it would be nice with an API where Calva can tell clojure-lsp to ignore certain files and/or extensions. Then this can be exposed as Calva setting, so that users can add whatever to the list.

walterl15:08:00

I'll try and carve off some time for this today

bringe15:08:06

Just caught up here, moving to the issue for further discussion.

bringe15:08:17

> I just realized we don't have easily that server <-> client json requests logs in a calva output window, do you think we can add a window for that? @UKFSJSM38 There is a setting to enable that adds a Clojure Language Client output window for these logs. Are you talking about something else?

bringe15:08:36

"clojure.trace.server": "verbose"

ericdallo15:08:17

Oh, that's what I was looking for, perfect, I probably missed that on the Calva docs

👍 3
Tomas Brejla15:08:20

repro is as easy as 1. open a lein new whatever project in calva 2. cider jack in 3. evaluate and wait until this finishes 

(->> (range 100000)
     (run! #(println "line " %)))
4. watch clojure-lsp eat 100-300 %CPU for many minutes

👍 3
pez15:08:04

Please add repro to the issue, @U01LFP3LA6P

🆗 3
bringe15:08:17

@UKFSJSM38 It was under the Troubleshooting header in the clojure-lsp doc, but I just added a sub-header for it so it's more visible in the side menu.

thanks 3
metal 3
Tomas Brejla15:08:19

btw this might also explain why clojure-lsp was behaving similarly when I just opened some small project in the past. I may have had a huge output.calva-replfrom previous session and poor lsp was trying to process it when code/calva started.

yes 3
pez15:08:07

I’ve seen my computer CPU meter go nuts like that for many minutes in relation to lots of REPL output. I have always assumed it is Calva’s parser that finds it tough.