Clojurians Log v2

I checked out Pauls fork, built the java version with bb debug-cli, removed references to :slow-but-accurate => seems to work fine after abusing it for a few minutes, at least in lsp-mode.

But I'll test this a bit more just to be safe

Yeah, seems to work fine. At first I was a bit confused as to why I couldn't repro this anymore with the latest nightly, but of course that's because the original PR was reverted.

In any case, lots of gratitude for all of you for fixing this 🙂

@lasse.olavi.maatta @shem I've just installed Emacs and have been dredging up old muscle memory (I used to live in Emacs back in the 90s, but it's been a long time since then!). I have a theory about what might be going on: Do you see the problem when the buffer contains syntactically correct Clojure? Or only when it's syntactically incorrect (e.g. mismatched parentheses)?

the issue occurs when the buffer contains valid clojure. when and if the issue occurs, the result is often syntactically incorrect as one or more parens might get overwritten

Damn. So my theory doesn't work. Could you do me a favour and send me a video of the issue occurring with valid Clojure in the buffer?

it feels like the selected completion candidate is not inserted where the completion occurs (iow. it should "push" any code after the completion point forwards to make room for the candidate). instead it overwrites the characters after the point

sure, I can try to grab a video of it

FWIW everything works perfectly for me, unless I have mismatched parentheses in the buffer. It would be good to know what's different between my (very vanilla, because I only installed Emacs a couple of hours ago 🙂) setup and yours.

This is my init.el in its entirety:

(require 'package)
(add-to-list 'package-archives '("melpa" . "") t)
(package-initialize)
(custom-set-variables
 ;; custom-set-variables was added by Custom.
 ;; If you edit it by hand, you could mess it up, so be careful.
 ;; Your init file should contain only one such instance.
 ;; If there is more than one, they won't work right.
 '(package-selected-packages
   '(smartparens parinfer-rust-mode clojure-mode-extra-font-locking cider lsp-treemacs company flycheck lsp-ui lsp-mode)))
(custom-set-faces
 ;; custom-set-faces was added by Custom.
 ;; If you edit it by hand, you could mess it up, so be careful.
 ;; Your init file should contain only one such instance.
 ;; If there is more than one, they won't work right.
 )

(require 'smartparens-config)
(add-hook 'clojure-mode-hook 'smartparens-mode)
(add-hook 'clojurescript-mode-hook 'smartparens-mode)
(add-hook 'clojurec-mode-hook 'smartparens-mode)

(add-hook 'clojure-mode-hook 'lsp)
(add-hook 'clojurescript-mode-hook 'lsp)
(add-hook 'clojurec-mode-hook 'lsp)

(setq lsp-clojure-custom-server-command '("bash" "-c" "~/Downloads/clojure-lsp"))

(setq gc-cons-threshold (* 100 1024 1024)
      read-process-output-max (* 1024 1024)
      treemacs-space-between-root-nodes nil
      company-minimum-prefix-length 1
      ; lsp-enable-indentation nil ; uncomment to use cider indentation instead of lsp
      ; lsp-enable-completion-at-point nil ; uncomment to use cider completion instead of lsp
      )

Prior to the change, clojure-lsp used to just return completions and leave it up to the editor to work out what to do with them. After the change, it tells the editor that it "knows what it's doing" and gives an explicit character range which should be replaced. My theory was that clojure-lsp's model of what's going on was getting out of sync with the editor's buffer when the Clojure is syntactically incorrect. But that can't explain what's going on if you're seeing it with syntactically correct Clojure.

Example of the issue. The first two completion attemps work fine, but the third breaks the forms

Ah! @ericdallo could this be another case of having to use process-after-changes like we did for linked editing ranges?

I think so, we can test that

I can't look at this right now, so any tests or PRs would help

I will put something together. I think we also have a problem with syntactically incorrect files too (which I serendipitously discovered while trying to reproduce the issue @lasse.olavi.maatta noticed).

(not yet sure what to do about that one).

there are some logic about invalid files in both clojure-lsp and clj-kondo, but it's basic, like missing ) and so on

I'll volunteer as a test subject; feel free to ping me if you get a potential fix ready and I can test it out locally when I can.

This seems to be happening https://clojurians.slack.com/archives/CBE668G4R/p1738096901625779, I believe we will have to revert that if we don't find a quick fix @paulbutcher

I reverted https://github.com/clojure-lsp/clojure-lsp/commit/46d6fd16abb9a00fafce6fcf47ef173704288388 until we find the proper fix, nightly build should have the fix soon

Nightly fixes it. Thanks! 🙏 ❤️

(let [x (foo| quuz) y 5]\n : completing foo to foobar will result in something like (let [x (foofoobar 5]\n instead of (let [x (foobar quuz) y 5]\n

BTW let us know if you feel anything in performance with that setting, we never noticed for sure how much slow it's actually

Feedback on performance would be great, but this isn’t the entire solution. @ericdallo I am approaching a proposal for how to address this properly and will let you know when I have everything straight.

(I think there’s a chance we can have our cake and eat it too).

my first impression was that completion is a bit slower now. But that might just be placebo due to naming; if the flag was called :alternative-super-fast-mode I would probably say it felt faster 🙂

Cool, thanks for testing, @paulbutcher are we good to merge it so?

I think so, @ericdallo 👍

merged, thanks for the help

That makes sense @ericdallo. I'll keep looking at this and try to come to a full understanding of what's going on.

@lasse.olavi.maatta it turns out that the process-after-changes code that @ericdallo and I were discussing is already implemented under a setting. Could you try setting :completion:analysis-type to :slow-but-accurate (documented https://clojure-lsp.io/settings/#all-settings) to see if it affects the behaviour you were seeing? (note, I'm not suggesting that this is the fix, but if it does modify the behaviour that will be informative). Thanks!

Sure, I can test that tomorrow and report back my findings

Thanks - much appreciated!

I can confirm that :slow-but-accurate seems to fix the issue with 2025.01.22-23.28.23. I'll leave it on for now, just to see if I notice anything strange with completions later today.

@ericdallo @lasse.olavi.maatta I've just submitted another PR which, I believe, re-implements the textEdit functionality (fixing the problems in Zed which prompted this saga) while both avoiding the problems Lasse found and not requiring :slow-but-accurate to be enabled.

https://github.com/clojure-lsp/clojure-lsp/pull/1960

I will take a look, but would be nice if @lasse.olavi.maatta could test that, there is a ci action that builds a binary

sure, I can take a look, if not today then at least during the weekend

@shem would be nice if you could test that fixes for you as well

I can't repro that and we didn't have anything in latest clojure-lsp version that touched completion, you can double check with previous clojure-lsp version to make sure. Looks like more a lsp-mode issue or so

yeah, sounds plausible

Err, that's not entirely true @ericdallo: I submitted this PR which added textExit to completion items: https://github.com/clojure-lsp/clojure-lsp/pull/1934

It shouldn't have broken anything because (I believe) it's consistent with the LSP spec. But it's possible that I got something wrong (or that Emacs has)?

I see that as well, already before the newest update, this is with emacs 31.0.50, freshly compiled.

I'll try to debug this over the weekend if I can. The issue doesn't happen 100% of the time, not sure if something is caching the completion results which might explain it.

I'm running a relatively fresh install of emacs-30 from git, I think I compiled it last weekend. But as I said, I noticed the problem today, after upgrading clojure-lsp.

I haven't been an Emacs user since the 90s (😱) so I'm unlikely to be much direct help, but let me know if there's anything I can do, or if there's anything in the PR linked above that is suspicious or unclear.

true @paulbutcher forgot about that PR, @lasse.olavi.maatta try disabling lsp-mode completion cache via lsp-completion-no-cache t

There is a chance that lsp-mode is not following something in the spec, but really unlikely

not sure if disabling the cache had a huge impact; it still seems to occur maybe 80% of the time. But I also tested with eglot (... yes, I have a flag in my config to choose the lsp client...) and can observe the same issue, although it appears more like 100% of the time.

Interesting, I'll need to double check what clojure-lsp is doing to make sure it's following the spec or maybe it's a misinterpretation of the spec on editor or server side

can't repro the problem with clojure-lsp 2024.11.08-17.49.29