Fork me on GitHub
#cider
<
2019-10-08
>
r0man09:10:59

Hello, I'm having problems with alot of our projects at work when jacking in with cider. After jacking in, when I load a namespace with cider-load-buffer I sometimes get this clojure.lang.Compiler$CompilerException in the :compile-syntax-check phase saying some namespace can't be found. For example namespace 'taoensso.faraday' not found. I'm sure that this namespace exists on my classpath, because requiring this file manually by evaluating (require 'taoensso.faraday :reload) works. After loading random files in the project in a order I never remember I eventually can get the initial namespace loaded and can work from there. We do have a lot of dependencies and it might be that somewhere things are messed up. Any idea how to debug this?

dominicm10:10:18

@r0man I would be betting that your project is using namespaces without requiring them 🙂 clj-kondo is pretty good at picking up on that.

dominicm10:10:29

joker might be too actually

dominicm10:10:11

I'm lying, kondo doesn't catch this. Joker does though.

borkdude10:10:55

clj-kondo does report unused namespaces. can you give an example where it doesn't do this where it should?

borkdude10:10:43

oh you mean the other way around: you're using the namespace but you haven't required it. yeah

dominicm10:10:32

@borkdude just found the "wontfix" issue 🙂

dominicm10:10:57

@r0man in the past I've resolved this by writing a script to start a fresh JVM and require every namespace. That usually roots out any failures.

dominicm10:10:21

Sorry. Looks like the code is gone. It could be patched up with some clever grep + bash-fu though

dominicm10:10:14

grep -Poh '(?<=^\(ns )[a-z_\.]+$' **/*.clj will get you a list of namespaces

dominicm10:10:29

depending on lein, clj, boot, you'll have to differ what you do, but something like: while read -r namespace; do clj -e "(require '${namespace})"; done And then you pipe these together. So

dominicm10:10:47

grep -Poh '(?<=^\(ns )[a-z_\.]+$' **/*.clj | while read -r namespace; do clj -e "(require '${namespace})"; done

dominicm10:10:08

I'll see if I have the code anywhere.

borkdude10:10:21

might be worth reconsidering but I think it resulted in many false positives

dominicm10:10:36

I think it's the right stance until a solution is found which knows about the env.

dominicm10:10:52

actually, I do have an idea

dominicm10:10:32

you have the full list of namespaces from the initial analysis right? So if there's a use of a namespace from that list without an explicit require then it's a lint line.

borkdude10:10:07

There are still things like Java classes and goog/...

borkdude10:10:23

Of course we could whitelist goog

dominicm10:10:39

that doesn't matter though 🙂

dominicm10:10:56

you're blacklisting the namespaces you found in analysis.

dominicm10:10:06

(unless they are required)

borkdude10:10:13

you mean from the export?

dominicm10:10:28

analysis cache.

dominicm10:10:35

those two terms have too much overlap 🙂 too much ambiguity.

borkdude10:10:13

it could be a custom linter which works on the export if you want to do it like that. but clj-kondo should always be able to work without the initial cache as well, so it's not an option to use that for built-in linting

dominicm10:10:33

ah, but it will still work! 🙂

dominicm10:10:42

you just won't get that kind of lint.

dominicm10:10:04

if kondo doesn't know about any namespaces, then none of them will be on the blacklist, hence they will all be okay.

borkdude10:10:13

loading the entire cache isn't an option for built-in linting: it's too slow

dominicm10:10:33

I don't understand. Isn't the cache used for e.g. arity checks?

borkdude10:10:51

yes, but kondo carefully only loads what it needs from the cache

dominicm10:10:09

so, potentially, this could be carefully loaded?

borkdude10:10:02

it only loads namespaces from the cache which have been required by the user. it seems you want to know about all namespaces, even if they aren't required?

borkdude10:10:16

I have the same kind of problem with the private var linter. private vars might be validly used in other namespaces (e.g. foo.cljs + foo.cljc), but I can't really check for that while linting only one file. using the cache for this is too slow, because I'd have to scan all the previously linted namespaces as well

dominicm10:10:41

we only need to know if clojure.string exists or not, so the presence of the file is sufficient. The file doesn't need to be parsed necessarily. Alternatively you could spit out a tiny extra file with all the ns' in (assuming loading that doesn't take a long time that is)

borkdude10:10:44

if there is no cache yet, how can clj-kondo know about the existence of a namespace? it should be able to work with that

borkdude10:10:42

to prevent false positives

dominicm10:10:07

that's why you blacklist, not whitelist 🙂

dominicm10:10:06

(when (and (file/exists? "cache/clojure.string") (not (required? ana "clojure.string")) (add-lint-error!))

dominicm10:10:25

if there's no cache then file/exists? will not pass, and there will be no false positive.

borkdude10:10:32

scenario without cache: user types (foo/bar). clj-kondo doesn't know foo. so no error. scenario with cache: user types (foo/bar) and foo is in the cache. but hasn't been required. now you get an error?

borkdude10:10:54

that could work yeah

borkdude10:10:02

I'm finally getting it, thanks 😉

borkdude10:10:23

and this will be performant. yeah

dominicm10:10:46

seems like a very good compromise imo 🙂

dominicm10:10:36

@borkdude makes sense in a static sense because of the reasons you mentioned. 🙂

borkdude10:10:39

(which issue was this?)

r0man10:10:57

hey guys, thanks for the tips, trying joker then and see if it finds anything suspicious ...

dominicm10:10:49

better way to list all namespaces: echo "(require '[clojure.tools.namespace.find :as f]) (run! println (f/find-namespaces (map *command-line-args*)))" | clj -Sdeps '{:deps {org.clojure/tools.namespace {:mvn/version "0.3.1"}}}' - src

dominicm10:10:36

when I used joker it was a bit trigger-happy with this stuff. I'd go for the grep/require solution: https://clojurians.slack.com/archives/C0617A8PQ/p1570530047263100?thread_ts=1570529577.260100&amp;cid=C0617A8PQ not very sophisticated.

borkdude10:10:43

you might also be able to use the clj-kondo analysis export for this. right now it spits out:

:var-usages [{:filename "<stdin>", :row 1, :col 1, :from user, :to nil, :name foo/bar, :arity 0}]
:to nil = unknown, while the symbol does have a namespace

borkdude10:10:00

This is the output from:

clj-kondo --lint - --config '{:output {:analysis true :format :edn}}' <<< '(foo/bar)'

dominicm10:10:05

interesting.