Fork me on GitHub
#clj-kondo
<
2021-08-19
>
flowthing09:08:09

I'm helping out a colleague with Calva's clojure-lsp integration. They use Windows, and it's not working. Here's the error we see in clojure-lsp.out: https://github.com/BetterThanTomorrow/calva/issues/1050#issuecomment-824719118 The stack trace points to this bit in clj-kondo: https://github.com/clj-kondo/clj-kondo/blob/b6f76ea8371d2497ed14314cf99a679e41f5db4f/src/clj_kondo/impl/core.clj#L358-L360 Sadly, the error message doesn't show the path of the file it's trying to access, so we don't know what the problem is. In any case, I figured we could work around the problem by setting {:output {:canonical-paths false}} in the clj-kondo config, but that doesn't seem to make any difference. It's as though the config isn't getting picked up at all. At least I don't see how .getCanonicalPath is called if :canonical-paths is false. Would it be possible for clj-kondo to log the file it's trying to access so that it ends up in clojure-lsp.out? Any other ideas on how to troubleshoot the issue? The only thing I can think of is to create a custom clj-kondo version that logs the file path, then create a custom clojure-lsp version that includes the custom clj-kondo version, then try to set up Calva to use that. I guess we'll try that if nothing else pans out.

borkdude11:08:01

Currently afk but this might mean there is some weird path on your classpath

flowthing11:08:07

Could be, but inspecting the output of (System/getProperty "java.class.path") doesn't reveal anything.

flowthing11:08:38

Anyway, I guess that's sort of the problem. There's a weird path-related issue, but I can't figure out what it is. It doesn't seem to be Could clj-kondo maybe log those file paths using a debug/trace log level?

borkdude11:08:58

Normally you should get this as a lint warning but not if there is something wrong while producing the lint warning

borkdude11:08:09

I guess that’s what’s happening

borkdude11:08:57

You could try linting using clj-kondo on the command line or JVM and see if you can reproduce this

borkdude11:08:12

Also make sure you’re using the newest LSP

flowthing11:08:09

Hmm, is it possible to tell Calva to use a different clojure-lsp…? Need to look into it.

flowthing11:08:18

Yeah, could try reproing on the command line too. :thumbsup::skin-tone-2:

flowthing12:08:50

Can't repro on the command line.

borkdude12:08:26

Yes, it’s possible but perhaps they automatically use the newest already nowadays

borkdude12:08:49

Perhaps you can see which classpath LSP is using to call clj-kondo

borkdude12:08:04

In the logs

borkdude12:08:15

@UKFSJSM38 perhaps you have a hint?

ericdallo12:08:58

Clojure-lsp just pass the return of lein classpath to clj-kondo, you can try running that with the -NoProfile like said in the issue and get the output

ericdallo12:08:43

powershell.exe -NoProfile lein classpath

flowthing12:08:32

Hmm... actually, clojure-lsp.out has:

flowthing12:08:36

2021-08-19T12:41:42.131Z SOLPF2AKCBN INFO [clojure-lsp.crawler:20] - Finding classpath via `powershell.exe -NoProfile npx shadow-cljs classpath`
2021-08-19T12:41:51.213Z SOLPF2AKCBN INFO [clojure-lsp.crawler:118] - Analyzing classpath for project root C:\Users\foo.bar-baz\Coxa\piso
2021-08-19T12:41:51.214Z SOLPF2AKCBN INFO [clojure-lsp.kondo:125] - Analyzing 265 paths with clj-kondo with batch size of 6 ...
2021-08-19T12:41:51.214Z SOLPF2AKCBN INFO [clojure-lsp.kondo:131] - Analyzing 1/6 batch paths with clj-kondo...
2021-08-19T12:42:02.513Z SOLPF2AKCBN INFO [clojure-lsp.kondo:131] - Analyzing 2/6 batch paths with clj-kondo...

flowthing12:08:07

So maybe it's the classpath that npx shadow-cljs classpath returns that's the problem?

ericdallo12:08:29

yes, probably, if your project is both a lein and npx one, clojure-lso will scan the lein classpath first and then the npx one and merge it

ericdallo12:08:56

so try to run powershell.exe -NoProfile npx shadow-cljs classpath and check the output

flowthing12:08:54

Ha! Found the issue, finally! Thanks for the hints, they led us down the right track. 🙂

ericdallo12:08:11

Nice, what was the issue?

flowthing12:08:42

PS C:\Users\foo.bar-baz\Coxa\piso> powershell.exe npx shadow-cljs classpath

------------------------------------------------------------------------------

   WARNING: shadow-cljs not installed in project.

   See      

------------------------------------------------------------------------------

shadow-cljs - config: C:\Users\foo.bar-baz\Coxa\piso\shadow-cljs.edn    

dev/src;dev/resources;target/shadow-cljs;

flowthing12:08:08

npx shadow-cljs classpath prints a warning if shadow-cljs isn't installed locally (into the project node_modules.

flowthing12:08:20

The warning gets fed to clj-kondo.

flowthing12:08:37

npm install -D shadow-cljs fixes the issue.

ericdallo12:08:56

hummm, good to know, we can change the default on clojure-lsp adding that flag if that makes sense

flowthing12:08:21

(Our projects use the shadow-cljs Clojure API, so we don't typically install shadow-cljs via NPM.)

flowthing12:08:40

I don't know that there's a flag that hides the warning.

ericdallo12:08:25

hum, so I think we can improve the way clojure-lsp handles that, probably shadow-cljs prints to output that, but it's not the return of the command

ericdallo12:08:31

we should only listen to the return of the command

flowthing12:08:06

Yeah, I'm not sure whether shadow-cljs prints that warning into stderr or stdout.

ericdallo12:08:08

could you create a simple repro project where you confirm that happens so I can try debug it later?

ericdallo12:08:20

exactly, it should print to stderr I think

flowthing12:08:31

Yeah, if it does, then it should be a relatively easy fix.

flowthing12:08:34

Sure, I'll create an issue.

ericdallo12:08:34

anyway, we can improve clojure-lso to get only the last line of the output

ericdallo13:08:14

testing on a lein project, lein also prints a lot of thinks specifally if the :pedantic flag is :warn, but that work with clojure-lsp becaause probably lein prints only the classpath to stdout

flowthing13:08:41

No, shadow-cljs sadly prints the warning into stdout.

ericdallo13:08:10

yeah, I think the best option is to change clojure-lsp to only get the last line of the output as the classpath

flowthing13:08:10

λ npx shadow-cljs classpath 2>/dev/null
------------------------------------------------------------------------------

   WARNING: shadow-cljs not installed in project.
   See 

------------------------------------------------------------------------------
dev/src: <SNIP>

flowthing13:08:50

Yeah, that might be the quickest fix, although could maybe also ask Thomas to fix that in shadow-cljs.

ericdallo13:08:58

yeah, both sounds good 🙂

flowthing13:08:33

Heh, and just to emphasize that this only occurs on Windows, because at least macOS doesn't have a problem with a path that looks like ------------------------------------------------------------------------------\n\n WARNING….

flowthing13:08:53

λ clj -M -e '(.getCanonicalPath ( (first (clojure.string/split (slurp "/tmp/foo.txt") #":"))))'
"/private/tmp/lsp-shadow-cljs/------------------------------------------------------------------------------\n\n   WARNING"

borkdude13:08:53

ok, so there should be a try catch around canonical path as well in clj-kondo?

ericdallo13:08:35

a warning log from kondo when parsing those cases sounds good

borkdude13:08:17

@U4ZDX466T ok, can you raise an issue before I forget?

flowthing14:08:05

Yeah, can do, unless I forget first. 🙂

ericdallo01:08:48

Available on next clojure-lsp release

flowthing05:08:36

@UKFSJSM38 Awesome, many thanks! I also created an issue for shadow-cljs, so it'll probably get fixed there, too. Maybe it's enough to fix it in three places. 😛

👍 2
ericdallo20:08:02

I have an improvement suggestion that I'd like to know if that makes sense to be done on clj-kondo 🧵

ericdallo20:08:36

related to midje , there is a macro midje.sweet/provided that is used to mock function calls, and it's possible to pass a & anything to suppress any extra args, for example:

(fact "foo"
  (my-func) => true
  (provided
    (other-func & anything) => 2))
this should mock other-func call and it doesn't care about the args. clj-kondo reports the other-func as wrong arity, it's possible to make clj-kondo ignore that linter if the args is a & anything ?

ericdallo20:08:14

ATM we are always exlcluding this with {:linters {:invalid-arity {:exclude [(midje.sweet/provided [*])]}}} but that looks not ideal and needs extra configuration

borkdude21:08:41

if you remove the [*] it will exclude all calls within the provided macro

ericdallo21:08:48

yeah, but imagine there is a valid call that I want to clj-kondo lint as invalid arity:

(provided
  (foo & anything) => 1 ; don't care about the invalid-arity
  (bar 1 2 3) => 2)     ; I do want a invalid-arity here 

ericdallo21:08:52

does that makes sense?

borkdude21:08:20

it does make sense, you can support this by writing an analysis hook or macroexpand hook

ericdallo21:08:47

you mean, we could add a config on midje lib of:

{:linters {:invalid-arity {:exclude [(midje.sweet/provided)]}}}
but with a custom analysis hook to do lint the invalid-arity if not & anything ?

borkdude21:08:00

I would remove the invalid-arity config and just handle this from your hook. expand into a call to the function if it doesn't contain an ampersand, and expand into a non-call like [foo anything] in the case of an ampersand, so clj-kondo will still see that the function is being used, and the arguments, but it won't be treated as a call

ericdallo21:08:24

I don't thing I get it :thinking_face:

borkdude21:08:08

ok, well, this:

(provided
  (foo & anything) => 1 ; don't care about the invalid-arity
  (bar 1 2 3) => 2) 
would expand into:
(do 
  [foo anything] 1 ; don't care about the invalid-arity
  (bar 1 2 3) 2))
for example

ericdallo21:08:10

uhm, I see, looks a good idea

dominicm20:08:46

Using the clj-kondo api, how can I specify a hook to be used? It's already in a directory. I tried setting :config-paths, but had no luck.

ericdallo20:08:37

clj-kondo check your projec config and check for :config-paths first, if a config path of foo/bar is found, then it tries to find in your project a .clj-kondo/foo/bar/config.edn file

ericdallo20:08:55

if that folder doesn't exists it's because clj-kondo didn't import the folder, you can make that work with something like:

ericdallo20:08:11

clj-kondo --copy-configs --dependencies --lint $(lein classpath)

ericdallo20:08:41

then clj-kondo will analyze the classpath, copy the folders from other libs classpaths and then you just need to have your :config-paths configured correctly

ericdallo20:08:38

note that is recommended to commit that folder inside the .clj-kondo dir after clj-kondo copy the configs

dominicm20:08:27

I'm using the API, I want to analyze another project that I am not in the cwd of from the JVM.

ericdallo20:08:37

you can still use those flags via API, similar to how clojure-lsp uses https://github.com/clojure-lsp/clojure-lsp/blob/master/src/clojure_lsp/kondo.clj#L82-L89

ericdallo20:08:58

but you will need to pass the classpath manually like clojure-lsp gets manually the classpath

ericdallo20:08:09

or pass the path to your external lib if that makes sense to you

borkdude21:08:56

Let's take a step back. I don't think dominic is asking how you import a config. he already has the hook config

borkdude21:08:32

The way to enable that is in the :hooks config, as documented here: https://github.com/clj-kondo/clj-kondo/blob/master/doc/hooks.md

borkdude21:08:06

Or is the config imported from another library? Then adding it to the config-paths should work

borkdude21:08:42

Need more info. Show the directory structure for example

ericdallo21:08:04

> Or is the config imported from another library? Then adding it to the config-paths should work Unless there is no cache, so clj-kondo would not find the hook I think

borkdude21:08:44

.clj-kondo dir you mean

👍 3
borkdude21:08:00

this is why I need the directory structure

dominicm21:08:20

I have a .clj-kondo dir containing a "config.edn" and a "hooks". It belongs to projectA.

dominicm21:08:36

I'm in projectB (totally unrelated, but performing analysis on projectA). I'd like to utilize those hooks.

borkdude21:08:09

right and those configs are "closed source" let's say?

borkdude21:08:05

a mono-repo situation?

dominicm21:08:19

Yes, closed source. But not a mono-repo situation. ProjectB may be open sourced. I'm perfoming visualizations on namespaces using d3.

dominicm21:08:32

The hooks work, no problem there. I use them for linting.

borkdude21:08:50

you could put those hooks in a third directory with a config.edn + the hook code

borkdude21:08:59

and then refer to that from both projects using config-paths

dominicm21:08:15

They're already in a directory named .clj-kondo. I tried using :config-paths and didn't get the hooks factored in.

dominicm21:08:02

(kondo/run!
    {:lint ["/full/path/to/file/src/foo/bar/baz.cljs"]
     :config {:output {:analysis true
                       :config-paths ["/full/path/to/file/.clj-kondo"]}
              :config-paths ["/full/path/to/file/.clj-kondo"]}
     :config-paths ["/full/path/to/file/.clj-kondo"]})
I really tried to use it 😄

borkdude21:08:44

config-paths should go into your project's .clj-kondo/config.edn

borkdude21:08:02

not sure if it works with run! (could work, but not sure)

dominicm21:08:06

projectA or projectB?

borkdude21:08:22

well, as I said, I would put the (shared) config in a third directory

borkdude21:08:36

and then add that third directory to :config-paths in both projects

borkdude21:08:04

or you could use the import/export mechanism is projectA is a proper dependency

dominicm21:08:14

Oh, and it would be in config.edn :config-paths, rather than with run!

borkdude21:08:33

although it might work with run!, I've never really used it like that

dominicm21:08:41

It doesn't seem to so far.

borkdude21:08:49

you can use relative paths in config-paths as well

dominicm21:08:56

Maybe the more obvious solution, should I just slurp the config.edn in .clj-kondo/ ?

dominicm21:08:40

hm, no. Because the hooks won't be resolved relatively in the way I want.

dominicm21:08:06

Ideally this would work without needing cwds, so I could build something like https://next.github.com/projects/repo-visualization without cloning full repos.

borkdude21:08:25

hooks are resolved relative to their config.edn file

borkdude21:08:41

if you say: config-paths ["other-dir"]

borkdude21:08:02

and you have other-dir/config.edn + other-dir/hooks/code.clj

borkdude21:08:26

then you can refer from other-dir/config.edn to the code directly

dominicm22:08:35

Is there any other way to provide hooks? As literals somehow?

ericdallo22:08:16

yes, you can pass as code, there is a example on clj-kondo repo, let me check

ericdallo22:08:53

you can pass it as string

borkdude22:08:57

ehm, that's not really intended to be exposed to users, it's only done for testing ;)

☝️ 3
ericdallo22:08:36

yeah not sure why you would do that way

borkdude22:08:13

I'm going to sleep now

💤 6
dominicm10:08:32

Here's what I'm doing with kondo btw 🙂

dominicm10:08:52

I'd also like to do something similar for to tools.deps.graph

borkdude10:08:14

cool. so if you need additional help, I think it would be good to make a small minimal dummy repro so we have something concrete to talk about and debug instead of guessing

dominicm10:08:29

I think you've answered my question, which is, it's not possible to pull hooks in from another directory at the moment.

dominicm10:08:53

If I continue to be excited about this project, I'd like to explore changing that in kondo somehow.

dominicm10:08:50

Ideally I can load this data from github via http, and allow users to just type in "juxt/clip" or whatever, and get an analysis of namespaces. Similar to how https://next.github.com/projects/repo-visualization works

borkdude10:08:20

> it's not possible to pull hooks in from another directory at the moment I haven't said this isn't possible.

borkdude10:08:20

but your use case would be better off with a repro, so we can talk about code instead of guessing. it is possible to get hooks from other places

borkdude10:08:22

this may also work for your use case: https://github.com/clj-kondo/clj-kondo/blob/master/doc/config.md#exporting-and-importing-configuration but since your code is closed source, I can't really offer any help beyond guesswork which isn't very productive

borkdude10:08:18

you could also hire me for a few hours to do some consulting for your company if that's easier for you ;)