Clojurians Log v2

@ericdallo a problem with filtering these token out is that now Calva depends on an internal detail of clojure-lsp, iiuc. That comment tokens are indexed with 10. Would you consider a PR that included the token vector with server-info? Then Calva can use that for its filtering decisions.

yeah, that's dangerous, I think it makes sense, feel free to create a issue or a PR for that

The issue is hitting other extensions too. Or at least vscode-go: https://github.com/golang/vscode-go/issues/3724

Should garner more attention 🙂

Interesting, looks like a vscode semantic token coloring issue, I can't repro on Emacs or Intellij (using clojure-lsp)

Would be nice to double check if disabling semantic tokens the problem is solved but I'd not suggest that as a final fix

I’ll try that now.

it's only in the latest vscode-insiders with calva on

With semantic tokens off the problem goes away.

yeah, it's definitely a vscode issue

So should be reproducible somehow in some other language. We just need to figure out how…

Is there some token you add for ignored forms, @ericdallo?

> With semantic tokens off the problem goes away. can you tell me how so I can avoid this for the time being?

Put this is .lsp/config.edn

{:semantic-tokens? false}

yes, latest clojure-lsp added new tokens for comment forms

@pez works for me too fyi. cheers.

I see this. But I can’t repro in TypeScript that easily.

So, it’s with latest clojure-lsp and VS Code 1.99-insiders. Using the previous version of clojure-lsp 2025.02.07-16.11.24 also dodges this bullet.

yeah, but I think as soon vscode releases this we will have a big problem in Calva

would be best to report this issue in vscode

also disabling semantic tokens is pretty bad

Is there a fix that can happen on the clojure side for a specific version of vscode if they don't fix in time?

no, disabling semantic tokens is the only way as clojure-lsp is correct

I mean auto disable it if using a bad version of vscode

hum, not easy to do I think

ok just wanted to throw that idea out there.

which clojurians have the skills to patch vscode and push on a PR? might want to ping them...

good question

Reporting to vscode is the reason I want to repro with TypeScript.

Since it is on Insiders we may not need to patch anything if the vscode team is alerted in time in a way they understand.

makes sense

https://github.com/microsoft/vscode/issues/243923

I think a big part of the issue is that VS Code does not support multiline tokens. As we can see also with the current stable version of VS Code.

Ah I doubt, they created the semantic tokens using vscode as primary case, the LSP spec always use vscode for those things, and multiline tokens are kind of obvious to support

Here’s a version of Calva that filters comment tokens from the semantic tokens provided by clojure-lsp. • https://output.circle-artifacts.com/output/job/90e3fe44-95b8-4394-bd7a-d024d90941e5/artifacts/0/tmp/artifacts/calva-2.0.492-2757-filter-comment-tokens-e5b7e2ea.vsix It then avoids the bug in VS Code and keeps working in files with ignored lists. It also avoids the disabling of syntax highlighting that these tokens cause. Can you give it a spin, @ingy? With semantic tokens enabled again. Related PR and issue: • https://github.com/BetterThanTomorrow/calva/pull/2758 • https://github.com/BetterThanTomorrow/calva/issues/2757

@pez I can try it. What's the best way to install that?

https://stackoverflow.com/questions/42017617/how-to-install-vs-code-extension-manually Something like that?

@pez I did intstall it and then removed the .lsp/config.edn that we used as a workaround. It does seem to work now.

I have to mention that it froze again. It froze after I deleted the #_ form that first brought it to our attention! Then I undid the delete and it worked again. And I can't get it to freeze again, but weird right?

Thanks for testing! 🙏 Super weird with that revisit of the freeze. We’re up against undefined behavior, obviously. I’ll run with this build too and we’ll see what gives.

The pressure is building: • Dart: https://github.com/microsoft/vscode/issues/244266 • And also Lua: https://github.com/microsoft/vscode/issues/243923#issuecomment-2743249274

https://github.com/microsoft/vscode/issues/243923#issuecomment-2745290050 just in

So it’s definitely about multiline tokens.

I tried heredocs in perl and bash and they were ok

also tried

/*
...
*/

Ah I guess its triggered by certain extensions...

nvm

It depends how the token is highlighted. For many languages there is TextMate grammar doing it. It can also be done with a semantic tokens provider. That’s when it hits this bug. It can also be done by just coloring the text in the buffer. Like Calva does for ignore tokens.

I tried with tagged literals in TypeScript and it didn’t produce semantic tokens for it, so guessing it’s TM grammar in action.

There’s a command in VSCode for inspecting tokens that will show you the TM and semantic tokens for things.

That looks like a bug in vscode as well, @andreribeirocamargo can you take a look please?

Which looks like a bug?

The one coloring only the first line

I was afraid you would say that. 😃

The best is to debug semantic tokens request and confirm clojure-lsp is returning proper tokens range, but it's a vector of integers so it might be hard to understand

All I have figured out so far is that index 3 is the token type and that type 10 is comment tokens.

I will filter out these tokens in the lsp client. It will fix the issue with removal of syntax highlighting and also stops Insiders from croaking.

Every 5 integers means a token, first is the line from last token, second the column, third the length of the token to color from the start, fourth is the token type and 5 the token modifiers list as bitwise, but that's irrelevant here

Ok, but would be nice to fix the root cause later

I’ll try to figure out a way to report the Insiders issue. But I understand a bit too little about the problem so may not do a very good job.

https://github.com/BetterThanTomorrow/calva/issues/2757