Fork me on GitHub
#lsp
<
2022-07-28
>
Darrick Wiebe03:07:04

Is it possible to inform lsp that a macro produces a def or defn? For instance, in my genera library, there is a macro called defgenera which defines different type of multi-method. An example usage looks like this:

(defgenera matcher-type 1
  "Return the type of the variable if it is a matcher."
  [var] :value)
For lsp users, I just discovered that this produces "errors" all over because lsp does not recognize that matcher-type is being defined as a function. I assume there is some sort of metadata that I can include in the library that will put lsp back on the right track? I'd like it not to seem broken if someone is using the library in their project.

lassemaatta05:07:13

I think clojure-lsp uses clj-kondo for analyzing the code, therefore someone must teach kondo to understand the macro. for some macros you can instruct kondo to treat it like some existing common macro or symbol (https://github.com/clj-kondo/clj-kondo/blob/master/doc/config.md#lint-a-custom-macro-like-a-built-in-macro). for more complex cases you can define a hook (https://github.com/clj-kondo/clj-kondo/blob/master/doc/hooks.md#transformation). You can define these in your library so your users don't need to worry about it (https://github.com/clj-kondo/clj-kondo/blob/master/doc/config.md#exporting).

☝️ 1
Darrick Wiebe13:07:26

Thanks, I'll take a look.

Darrick Wiebe18:07:24

I've made the following change to my library. Will this be sufficient for clj-kondo to lint it correctly?

diff --git a/resources/clj-kondo.exports/horse.no/genera/clj_kondo/genera_hooks.clj b/resources/clj-kondo.exports/horse.no/genera/clj_kondo/genera_hooks.clj
new file mode 100644
index 0000000..a56f678
--- /dev/null
+++ b/resources/clj-kondo.exports/horse.no/genera/clj_kondo/genera_hooks.clj
@@ -0,0 +1,29 @@
+(ns clj-kondo.genera-hooks)
+
+(defmacro defgenera
+  "Do not use. This is for clj-kondo linting / lsp support only"
+  [name arity & fn-tail]
+  `(defn ~name ~@fn-tail))
+
+
+(defmacro defgenera*
+  "Do not use. This is for clj-kondo linting / lsp support only"
+  [name arity & etc]
+  `(defn ~name [~@(repeat arity gensym)]))
+
+
+(defmacro defgenera=
+  "Do not use. This is for clj-kondo linting / lsp support only"
+  [name arity & etc]
+  `(defn ~name [~@(repeat arity gensym)]))
+
+
+(defmacro defgen
+  "Do not use. This is for clj-kondo linting / lsp support only"
+  [name arity & etc]
+  `(defn ~name [~@(repeat arity gensym)]))
diff --git a/resources/clj-kondo.exports/horse.no/genera/config.edn b/resources/clj-kondo.exports/horse.no/genera/config.edn
new file mode 100644
index 0000000..8f07488
--- /dev/null
+++ b/resources/clj-kondo.exports/horse.no/genera/config.edn
@@ -0,0 +1,5 @@
+{:hooks
+ {:analyze-call {genera.macros/defgenera clj-kondo.genera-hooks/defgenera
+                 genera.macros/defgenera* clj-kondo.genera-hooks/defgenera*
+                 genera.macros/defgenera= clj-kondo.genera-hooks/defgenera=
+                 genera.macros/defgen clj-kondo.genera-hooks/defgen}}}

borkdude18:07:02

yes, but don't use `

clj_kondo/genera_hooks
as the namespace for your hooks, just use your own organization name

Darrick Wiebe18:07:34

I've got the project included in my deps.edn like this:

 {:local/root "../genera"}
I've restarted Calva, but it is still not happy.

borkdude18:07:35

also you can use the .clj_kondo extension to not confuse other tooling like Cursive :)

Darrick Wiebe18:07:57

Use that extension where?

borkdude18:07:02

in the hook code

Darrick Wiebe18:07:08

Instead of .clj?

borkdude18:07:02

you can check if clj-kondo imported your hook code by looking in your project's .clj-kondo/... directory

Darrick Wiebe18:07:51

ok, so I'll change the namespace to http://horse.no.genera-hooks, and rename the files. Will that fix the issue?

➜  pattern git:(main) ✗ tree .clj-kondo/horse.no/genera
.clj-kondo/horse.no/genera
├── clj_kondo
│   └── genera_hooks.clj
└── config.edn

borkdude19:07:04

Oh I see. you wrote analyze-call but the hook you're using now is called macroexpand

borkdude19:07:31

it won't fix the issues, but it's a good convention to not use other people's organizations in your code ;)

Darrick Wiebe19:07:55

Makes sense. The documentation seemed to require it.

Darrick Wiebe19:07:23

It's hard to say which parts of the example are configuration by convention vs just example.

borkdude19:07:52

For your hooks, I would choose:

clj-kondo.exports/horse.no/genera
as the base dir

borkdude19:07:30

and then inside that base-dir:

config.edn

borkdude19:07:15

and:

horse_no/genera_hooks.clj_kondo

borkdude19:07:47

(made a few edits)

Darrick Wiebe19:07:49

ok. I think I also made a mistake by including a . in a folder.

Darrick Wiebe19:07:03

ok going to retry this in a moment here.

Darrick Wiebe19:07:14

And I should rename analyze-call to macroexpand?

Darrick Wiebe19:07:33

ok, new diff, but apparently still not working. I'm going to restart Calva...

diff --git a/resources/clj-kondo.exports/horse/no/genera/config.edn b/resources/clj-kondo.exports/horse/no/genera/config.edn
new file mode 100644
index 0000000..42c865b
--- /dev/null
+++ b/resources/clj-kondo.exports/horse/no/genera/config.edn
@@ -0,0 +1,5 @@
+{:hooks
+ {:macroexpand {genera.macros/defgenera horse.no.genera-hooks/defgenera
+                genera.macros/defgenera* horse.no.genera-hooks/defgenera*
+                genera.macros/defgenera= horse.no.genera-hooks/defgenera=
+                genera.macros/defgen horse.no.genera-hooks/defgen}}}
diff --git a/resources/clj-kondo.exports/horse/no/genera/horse/no/genera_hooks.clj-kondo b/resources/clj-kondo.exports/horse/no/genera/horse/no/genera_hooks.clj-kondo
new file mode 100644
index 0000000..f04b537
--- /dev/null
+++ b/resources/clj-kondo.exports/horse/no/genera/horse/no/genera_hooks.clj-kondo
@@ -0,0 +1,24 @@
+(ns horse.no.genera-hooks)
+
+(defmacro defgenera
+  "Do not use. This is for clj-kondo linting / lsp support only"
+  [name arity & fn-tail]
+  `(defn ~name ~@fn-tail))
+
+
+(defmacro defgenera*
+  "Do not use. This is for clj-kondo linting / lsp support only"
+  [name arity & etc]
+  `(defn ~name [~@(repeat arity gensym)] ~@etc))
+
+
+(defmacro defgenera=
+  "Do not use. This is for clj-kondo linting / lsp support only"
+  [name arity & etc]
+  `(defn ~name [~@(repeat arity gensym)] ~@etc))
+
+
+(defmacro defgen
+  "Do not use. This is for clj-kondo linting / lsp support only"
+  [name arity & etc]
+  `(defn ~name [~@(repeat arity gensym)] ~@etc))

borkdude19:07:16

can you show what is imported in your .clj-kondo directory?

Darrick Wiebe19:07:04

➜  pattern git:(main) ✗ tree .clj-kondo
.clj-kondo
└── horse
    └── no
        └── genera
            ├── config.edn
            └── horse
                └── no
                    └── genera_hooks.clj

5 directories, 2 files

Darrick Wiebe19:07:32

➜  pattern git:(main) ✗ cat .clj-kondo/**/*
cat: .clj-kondo/horse: Is a directory
cat: .clj-kondo/horse/no: Is a directory
cat: .clj-kondo/horse/no/genera: Is a directory
{:hooks
 {:macroexpand {genera.macros/defgenera horse.no.genera-hooks/defgenera
                genera.macros/defgenera* horse.no.genera-hooks/defgenera*
                genera.macros/defgenera= horse.no.genera-hooks/defgenera=
                genera.macros/defgen horse.no.genera-hooks/defgen}}}
cat: .clj-kondo/horse/no/genera/horse: Is a directory
cat: .clj-kondo/horse/no/genera/horse/no: Is a directory
(ns horse.no.genera-hooks)

(defmacro defgenera
  "Do not use. This is for clj-kondo linting / lsp support only"
  [name arity & fn-tail]
  `(defn ~name ~@fn-tail))


(defmacro defgenera*
  "Do not use. This is for clj-kondo linting / lsp support only"
  [name arity & etc]
  `(defn ~name [~@(repeat arity gensym)] ~@etc))


(defmacro defgenera=
  "Do not use. This is for clj-kondo linting / lsp support only"
  [name arity & etc]
  `(defn ~name [~@(repeat arity gensym)] ~@etc))


(defmacro defgen
  "Do not use. This is for clj-kondo linting / lsp support only"
  [name arity & etc]
  `(defn ~name [~@(repeat arity gensym)] ~@etc))

borkdude19:07:41

the first directory structure should have only two levels

borkdude19:07:22

the hook code can have as many levels as you want so that's fine

borkdude19:07:34

just the base dir has to have 2 levels exactly

Darrick Wiebe19:07:04

Now it is no longer finding the code.

➜  pattern git:(main) ✗ tree .clj-kondo
.clj-kondo
└── 
    └── genera
        └── config.edn

2 directories, 1 file
Despite it being present in the library
➜  clj-kondo.exports git:(master) ✗ tree
.
└── 
    └── genera
        ├── config.edn
        └── horse
            └── no
                └── genera_hooks.clj-kondo

4 directories, 2 files

borkdude19:07:33

.clj_kondo with an underscore

Darrick Wiebe19:07:25

Ok, the code has been found this time, but still getting a lint error.

borkdude19:07:51

can you try on the command line and lint a file in your project?

borkdude19:07:56

and see what clj-kondo prints

Darrick Wiebe19:07:23

I'll have to install it...

Darrick Wiebe19:07:28

ok got it via brew

Darrick Wiebe19:07:00

What command would you like me to run?

Darrick Wiebe19:07:19

$ clj-kondo
clj-kondo v2022.06.22
...

borkdude19:07:47

clj-kondo --lint src/foo/bar.clj

borkdude19:07:55

same file you just showed in vscode

Darrick Wiebe19:07:57

Tried a few things. I tried that first :

clj-kondo --lint src/pattern/match/core.clj

Darrick Wiebe19:07:19

...
src/pattern/match/core.clj:27:12: error: Unresolved symbol: var-name
src/pattern/match/core.clj:30:12: error: Unresolved symbol: matcher-type

borkdude19:07:50

ok, but no further errors, like "can't find macro" or so?

Darrick Wiebe19:07:52

so I deleted the .clj-kondo folder. and added the --copy-configs option. Nothing showed up

Darrick Wiebe19:07:57

no nothing like thoat

borkdude19:07:04

can you print something from a macro

Darrick Wiebe19:07:06

➜  pattern git:(main) ✗ clj-kondo --lint src/pattern/match/core.clj
src/pattern/match/core.clj:3:39: warning: #'genera/defgenera* is referred but never used
src/pattern/match/core.clj:3:95: warning: #'genera/defmethod! is referred but never used
src/pattern/match/core.clj:4:29: warning: #'genera/trampoline is referred but never used
src/pattern/match/core.clj:4:40: warning: #'genera/trampolining is referred but never used
src/pattern/match/core.clj:4:53: warning: #'genera/bouncing is referred but never used
src/pattern/match/core.clj:5:14: warning: namespace uncomplicate.fluokitten.core is required but never used
src/pattern/match/core.clj:8:73: warning: #'pure-conditioning/manage is referred but never used
src/pattern/match/core.clj:8:80: warning: #'pure-conditioning/restart-with is referred but never used
src/pattern/match/core.clj:8:93: warning: #'pure-conditioning/handler-cond is referred but never used
src/pattern/match/core.clj:10:14: warning: namespace clojure.walk is required but never used
src/pattern/match/core.clj:27:12: error: Unresolved symbol: var-name
src/pattern/match/core.clj:30:12: error: Unresolved symbol: matcher-type
src/pattern/match/core.clj:50:13: warning: unused binding comp-env
src/pattern/match/core.clj:153:43: error: Unresolved symbol: x
src/pattern/match/core.clj:161:13: error: Unresolved symbol: matcher-mode
src/pattern/match/core.clj:172:13: error: Unresolved symbol: matcher-prefix
src/pattern/match/core.clj:299:29: warning: unused binding dictionary
src/pattern/match/core.clj:423:47: warning: unused binding comp-env
linting took 62ms, errors: 5, warnings: 13

borkdude19:07:20

you can just edit in the imported code

Darrick Wiebe19:07:34

ok will do. 1 sec

Darrick Wiebe19:07:01

no additional output

Darrick Wiebe19:07:26

{:hooks
 {:macroexpand {genera.macros/defgenera horse.no.genera-hooks/defgenera
                genera.macros/defgenera* horse.no.genera-hooks/defgenera*
                genera.macros/defgenera= horse.no.genera-hooks/defgenera=
                genera.macros/defgen horse.no.genera-hooks/defgen}}}
Is this correct? Maybe I'll try making it just 1 level of namespace

Darrick Wiebe19:07:34

Another possibility: I use potemkin/import-vars to expose an API from the top-level genera namespace. Is it possible that kondo doesn't see through that and I need to point it to the genera/ namespace instead of genera.macros/?

Darrick Wiebe19:07:55

That was the last issue.

Darrick Wiebe19:07:15

{:hooks
 {:macroexpand {genera/defgenera horse.no.genera-hooks/defgenera
                genera/defgenera* horse.no.genera-hooks/defgenera*
                genera/defgenera= horse.no.genera-hooks/defgenera=
                genera/defgen horse.no.genera-hooks/defgen
                genera.macros/defgenera horse.no.genera-hooks/defgenera
                genera.macros/defgenera* horse.no.genera-hooks/defgenera*
                genera.macros/defgenera= horse.no.genera-hooks/defgenera=
                genera.macros/defgen horse.no.genera-hooks/defgen}}}
Now it works.

Darrick Wiebe19:07:41

Ok... now I need to expose the more complex macros... 😰

Darrick Wiebe19:07:55

Thanks for your help!

borkdude19:07:50

Congrats on the first success :)

borkdude19:07:21

You could also just use :lint-as {your.macro clj-kondo.lint-as/catch-all}

borkdude19:07:43

this will make the existence of the var clear to clj-kondo but will ignore all lint warnings inside the calls of those macros

borkdude19:07:06

if it gets too complicated

Darrick Wiebe19:07:23

Ok... I'll see what can be done... I assume I need to keep the macros fast and short startup time because kondo is not running as a service, right?

borkdude19:07:49

in clojure-lsp it is running as a service (Calva) but not everyone uses clojure-lsp

borkdude19:07:54

so in general no

Darrick Wiebe19:07:55

Is there a way to tell?

borkdude19:07:31

even if you run as a service, the files get linted on every keystroke, so having shorter macro expansions is going to beneficial there too

borkdude19:07:46

there is currently no way to tell

Richie21:07:50

In vscode, how can I jump from clojure to java and then jump around in java? I can jump from clojure to java but then the java file just complains about “import … cannot be resolved”. I just installed vscode on macos and installed calva and Language Support for Java(TM) by Red Hat. Thanks!

1
Richie21:07:19

I looked through https://github.com/clojure-lsp/clojure-lsp/blob/master/docs/settings.md#all-settings but nothing stands out. Ditto for the redhat extension.

ericdallo21:07:56

Using clojure-lsp is not possible and probably with java LSP too... Clojure-lsp just open the java file, from there it doesn't know about the java code

ericdallo21:07:25

And other LSP that understand java would probably not provide too many features since it only have that file available but no java project context close

ericdallo21:07:47

that would be a hard to support I think...

Richie21:07:48

Ok, I’ll take your word for it. Thanks! I was hoping that the java extension would engage when vscode opened a *.java file and somehow get the context from the pom. Vscode shows the path for the java file as .lsp/.cache/java/decompiled/… though so I figured something would have to change since there’s no pom there. I tried setting :dependency-scheme to :jar without understand what it means but that didn’t work, haha. I think that’s a totally unrelated configuration. Thanks again!

Richie21:07:49

The only feature that I want is to be able to jump to definitions. I don’t need anything beyond that. The project is all clojure and not mixed.

ericdallo21:07:53

Yep, clojure-lsp decompiled the java file to that path, but that's not enough for a java project, so I guess a java LSP would only have basic features like understanding syntax and navigation to same file only I guess... Anyway, if there was anything to improve on clojure-lsp to help with that, I'd love to help with that c/c @U04V15CAJ

👍 1
Richie21:07:44

Can we leverage what’s in the .m2 folder somehow? There are source files and poms in there… I guess it’s out of the scope of clojure-lsp and maybe out of the scope of lsp at that point.

ericdallo21:07:45

not sure if searching on whole .M2, but since clojure-lsp has the classpath and the jar it opened/decompiled the java class, maybe we could copy the jar structure with the pom to some folder as well, I wonder if that would be enough for LSP java or those java extensions

ericdallo21:07:03

Maube you can try copying it manually and see how it works

ericdallo21:07:20

Other files would be needed to be copied too like the whole jar though

snoe21:07:40

right, it would be worthwhile to see what a java lsp needs to navigate a project, possibly we can set it up for it to see the jar as such a project

👍 1
Richie22:07:31

I don’t know how to decompile class files.

ericdallo22:07:58

You mean just unzipping it as it is, is enough to LSP java features work?

ericdallo22:07:49

That's good, I'll make some tests, if so we can unzip the whole zip in the cache folder I think

ericdallo22:07:24

We would need to decompile all classes in the jar though, but we could do that async

ericdallo22:07:33

When user finds the definition

snoe22:07:48

You wouldnt have a dependency on the -sources jar though usually.

ericdallo23:07:29

Could you elaborate @U0BUV7XSA?

Richie23:07:27

My hope is that decompiling the non-sources jar will give you something similar enough since it will have the same pom.xml. I tested with -sources.jar since I don't know how to decompile.

👆 1
Richie23:07:40

If that's a problem, as a user, I'd be happy to have the feature even if it downloaded the -sources.jar.

snoe23:07:02

I think the big difference is that the package structure will need to be recreated since i believe .class files are stored in a single dir.

Richie23:07:04

$ tree

.
├── META-INF
│   ├── MANIFEST.MF
│   └── maven
│       └── com.amazonaws
│           └── aws-java-sdk-cloudwatch
│               ├── pom.properties
│               └── pom.xml
└── com
    └── amazonaws
        ├── auth
        │   └── policy
        │       └── actions
        │           └── CloudWatchActions.class
        └── services
            └── cloudwatch
                ├── AbstractAmazonCloudWatch.class
                ├── AbstractAmazonCloudWatchAsync.class
                ├── AmazonCloudWatch.class
                ├── AmazonCloudWatchAsync.class

snoe23:07:24

oh, good

Richie23:07:03

That's after unzip . Idk if unzip is doing something magical since it's a jar.

Richie23:07:13

Oh, wait. Jars are zips.

Richie23:07:26

Yea, no magic there. It's a zip with folders and stuff.

ericdallo23:07:29

Yeah, not sure if classes follow same arch if the jar was AOTed

ericdallo23:07:28

I think we could just decompile the classes in the jar, it should not be that slow or hard, I'll research later

ericdallo23:07:55

@UPD88PGNT feel free to create a issue on clojure-lsp since we think we can improve that on clojure-lsp side :)

👍 1
ericdallo15:07:48

I did some experiments usinghttps://www.benf.org/other/cfr/ (the current decompiler we use at clojure-lsp), doing something like java -jar cfr-0.152.jar ~/.m2/repository/com/google/guava/guava/31.1-jre/guava-31.1-jre.jar --outputdir . , cfr decompiles all classes properly, and when opening the project, lsp-java understand all classes from the jar, making find definition and references work, but doesn't understand external deps, like the ones from the jar's pom.xml, it seems cfr doesn't extract the pom.xml inside the META-INF folder, so not sure there is a option for that or we would need to manually search and extract that so lsp-java and editors understand that as a java project and load all deps properly. Anyway, it's already a improvement if we manage to decompile the whole jar async

Richie19:07:36

I tried it myself. Yea, cfr doesn't put the pom anywhere. I tried copying over the META-INF folder but that didn't seem to work. i.e. not resolve references outside the project. I moved the pom to the root of the project and then it seems to work. What do you mean by "search" when you say "manually search and extract"? I don't know what you have available in the context of clojure-lsp. Do you just mean that you'll have to copy the pom out of the jar or is there some uncertainty?

Richie19:07:42

Oh, I'm not testing this in the .lsp folder. I should do that.

ericdallo20:07:43

Yeah, clojure-lsp would need to manually do that

ericdallo20:09:25

So, good news, I have a working branch https://github.com/clojure-lsp/clojure-lsp/tree/java-project-decompile, @UPD88PGNT I would love to hear what you think from a user perspective, I still need to fix tests + add some kind of feature flag for that, what clojure-lsp will do in the end is: • User find definition of a java class • clojure-lsp check if the feature flag is enabled, if not decompile as it is today • if enabled, will search for a pom.xml inside the jar, if not found will decompile as it is today • if found, will decompile the whole jar + copy the xml file to the global cache dir • then return the file for the editor which will open the file, ex: ~/.cache/clojure-lsp/java/decompiled/org.benf/cfr/cfr-0.152/src/org/benf/cfr/reader/api/CfrDriver.java • clojure-lsp will not decompile the jar if the pom.xml file is found, which means it already decopmiled before, avoiding the need to re-decompile Which a jar decompiled in a valid java project structure, containing pom.xml + source code in a source-path, this is enough for most editors be able to recognize as a java project and use Java LSP and other features like find references, definition etc, I tested with Emacs + lsp-java and worked perfectly One thing to notice here is, decompile the whole jar can take some time depending on the jar size (CFR has lots of classes and take ~10s), I'd like to timprove that in the future, but for now seems pretty good enough, also, I implemented a custom message which should work for most/all clients while editor is waiting for decompilation happen as the screenshot, that should help a little bit with a feedback

amazed 1
Richie11:09:44

Thanks! I'll check it out over the next couple weeks.

pesterhazy22:07:21

I recently switched to eglot (from lsp-mode). Seems to be working pretty well. Some things I noticed: • I need to use xref-find-definitions and xref-find-references (eglot-find-declaration or eglot-find-implementation doesn't work) • I didn't need to do any configuration at all Apparently eglot is on its way to becoming part of emacs proper

👍 3
mpenet07:07:54

Is it? I recall reading about the author wanting to do that back then

pesterhazy07:07:08

I saw it mentioned in a recent message to the mailing list but can't find the reference now unfortunately

apt12:07:04

I’ve been using eglot for the past few months, it’s pretty great. The only issue I have is that find definition won’t work for external deps. Though, this might not be a problem if you use cider because it handles that. I use the xref functions as well (with its default bindings, M-. and M-?).

apt12:07:46

I only use this config:

(add-hook 'eglot-managed-mode-hook
	    ;; This displays full docs for clojure functions.
	    ;; See 
	    #'(lambda ()
		(setq-local eldoc-documentation-strategy
			    #'eldoc-documentation-compose

			    eldoc-echo-area-use-multiline-p
			    3)))

👀 1
pesterhazy18:08:04

Thanks @U976F1AR2, that fixes the truncated docstrings Here's a link to my init.el with my eglot keybindings, in case someone might find that helpful https://github.com/pesterhazy/emacs.d/blob/master/init.el#L357-L361

👍 1