This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-01-05
Channels
- # aleph (1)
- # announcements (18)
- # babashka (145)
- # beginners (70)
- # calva (34)
- # cider (3)
- # clj-kondo (98)
- # cljdoc (5)
- # cljs-dev (13)
- # clojure (134)
- # clojure-europe (57)
- # clojure-nl (4)
- # clojure-uk (4)
- # clojurescript (40)
- # code-reviews (3)
- # conjure (1)
- # core-async (5)
- # data-science (3)
- # datomic (8)
- # fulcro (9)
- # google-cloud (2)
- # inf-clojure (9)
- # jobs (1)
- # lsp (9)
- # malli (25)
- # polylith (4)
- # reitit (4)
- # releases (2)
- # remote-jobs (3)
- # rewrite-clj (8)
- # shadow-cljs (34)
- # tools-build (1)
- # tools-deps (67)
We bumped into an interesting issue with clj-kondo hooks vs tools.namespace/refresh. A one-liner repro case is:
clj -Srepro -Sdeps '{:deps {org.clojure/tools.namespace {:mvn/version "1.2.0"} seancorfield/next.jdbc {:git/url " " :git/sha "24bf1dbaa441d62461f980e9f880df5013f295dd"}}}' -M -e "((requiring-resolve 'clojure.tools.namespace.repl/refresh-all))"
result:
:reloading (next.jdbc.protocols next.jdbc.prepare next.jdbc.result-set next.jdbc.datafy next.jdbc.types next.jdbc.transaction next.jdbc.default-options next.jdbc.connection next.jdbc.sql.builder next.jdbc.sql-logging next.jdbc next.jdbc.sql next.jdbc.specs hooks.com.github.seancorfield.next-jdbc next.jdbc.quoted next.jdbc.date-time next.jdbc.optional next.jdbc.plan)
:error-while-loading hooks.com.github.seancorfield.next-jdbc
#error {
:cause "Could not locate hooks/com/github/seancorfield/next_jdbc__init.class, hooks/com/github/seancorfield/next_jdbc.clj or hooks/com/github/seancorfield/next_jdbc.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name."
:via
[{:type java.io.FileNotFoundException
:message "Could not locate hooks/com/github/seancorfield/next_jdbc__init.class, hooks/com/github/seancorfield/next_jdbc.clj or hooks/com/github/seancorfield/next_jdbc.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name."
:at [clojure.lang.RT load "RT.java" 462]}]
:trace
[[clojure.lang.RT load "RT.java" 462]
[clojure.lang.RT load "RT.java" 424]
[clojure.core$load$fn__6856 invoke "core.clj" 6115]
[clojure.core$load invokeStatic "core.clj" 6114]
[clojure.core$load doInvoke "core.clj" 6098]
[clojure.lang.RestFn invoke "RestFn.java" 408]
[clojure.core$load_one invokeStatic "core.clj" 5897]
[clojure.core$load_one invoke "core.clj" 5892]
[clojure.core$load_lib$fn__6796 invoke "core.clj" 5937]
[clojure.core$load_lib invokeStatic "core.clj" 5936]
[clojure.core$load_lib doInvoke "core.clj" 5917]
[clojure.lang.RestFn applyTo "RestFn.java" 142]
[clojure.core$apply invokeStatic "core.clj" 669]
[clojure.core$load_libs invokeStatic "core.clj" 5974]
[clojure.core$load_libs doInvoke "core.clj" 5958]
[clojure.lang.RestFn applyTo "RestFn.java" 137]
[clojure.core$apply invokeStatic "core.clj" 669]
[clojure.core$require invokeStatic "core.clj" 5996]
[clojure.core$require doInvoke "core.clj" 5996]
[clojure.lang.RestFn invoke "RestFn.java" 421]
[clojure.tools.namespace.reload$track_reload_one invokeStatic "reload.clj" 35]
[clojure.tools.namespace.reload$track_reload_one invoke "reload.clj" 21]
[clojure.tools.namespace.reload$track_reload invokeStatic "reload.clj" 52]
[clojure.tools.namespace.reload$track_reload invoke "reload.clj" 43]
[clojure.lang.AFn applyToHelper "AFn.java" 154]
[clojure.lang.AFn applyTo "AFn.java" 144]
[clojure.lang.Var alterRoot "Var.java" 308]
[clojure.core$alter_var_root invokeStatic "core.clj" 5499]
[clojure.core$alter_var_root doInvoke "core.clj" 5494]
[clojure.lang.RestFn invoke "RestFn.java" 425]
[clojure.tools.namespace.repl$do_refresh invokeStatic "repl.clj" 94]
[clojure.tools.namespace.repl$do_refresh invoke "repl.clj" 83]
[clojure.tools.namespace.repl$refresh_all invokeStatic "repl.clj" 162]
[clojure.tools.namespace.repl$refresh_all doInvoke "repl.clj" 147]
[clojure.lang.RestFn invoke "RestFn.java" 397]
[clojure.lang.Var invoke "Var.java" 380]
[user$eval1 invokeStatic "NO_SOURCE_FILE" 1]
[user$eval1 invoke "NO_SOURCE_FILE" 1]
[clojure.lang.Compiler eval "Compiler.java" 7181]
[clojure.lang.Compiler eval "Compiler.java" 7136]
[clojure.core$eval invokeStatic "core.clj" 3202]
[clojure.main$eval_opt invokeStatic "main.clj" 488]
[clojure.main$eval_opt invoke "main.clj" 482]
[clojure.main$initialize invokeStatic "main.clj" 508]
[clojure.main$null_opt invokeStatic "main.clj" 542]
[clojure.main$null_opt invoke "main.clj" 539]
[clojure.main$main invokeStatic "main.clj" 664]
[clojure.main$main doInvoke "main.clj" 616]
[clojure.lang.RestFn applyTo "RestFn.java" 137]
[clojure.lang.Var applyTo "Var.java" 705]
[clojure.main main "main.java" 40]]}
this is caused by a combination of: • referring a lib by git coordinates, which means that it will appear as a directory on the classpath • said lib exporting clj-kondo hooks • clj-kondo hooks themselves being .clj files • those files normally having a namespace inside them that does not appear in code proper (and this is totally fine)
I ended up writing a somewhat hacky helper function which tries to exclude clj-kondo.exports directories from the default refresh-dirs of tools.namespace:
(defn remove-clj-kondo-exports-from-tools-ns-refresh-dirs
"Opinionated helper that addresses the problem that happens when:
- a lib is referenced via a git dependency
- said lib exports clj-kondo hooks (which are clojure source files)
- and the namespace in those files is not something that also exists in a proper source file
(this is normally not the case at the time of writing this, see
for example)
- and clojure.tools.namespace.repl/refresh(-all) is used
Call this in your user.clj to (hopefully) fix the problem.
A potential issue from using this is that if the directory containing the clj-kondo.exports folder
also directly contains to-be-reloaded clojure source files, those will no longer be reloaded."
[]
(->> (clojure.java.classpath/classpath-directories)
(mapcat
(fn [^File classpath-directory]
(let [children (.listFiles classpath-directory)
directory? #(.isDirectory ^File %)
clj-kondo-exports?
#(= "clj-kondo.exports" (.getName ^File %))
has-clj-kondo-exports
(some (every-pred clj-kondo-exports? directory?) children)]
(if has-clj-kondo-exports
(->> children
(filter directory?)
(remove clj-kondo-exports?))
[classpath-directory]))))
(apply clojure.tools.namespace.repl/set-refresh-dirs)))
There was a similar issue recently with better-cond and cursive where cursive mistook exported hooks as application code
We could support some extension .cljk
or so to prevent other things from loading it... but then highlighting etc won't work as nicely. Tooling should probably have ways to exclude things from loading
Certainly a tooling problem. Whether the tool is clj-kondo or tools.namespace here, I'm not sure 🙂
I would actually vote for the latter, given hooks are already out there and you'd have to support the current .clj extensions
https://ask.clojure.org/index.php/11434/could-namespace-refresh-sophisticated-directory-filtering
So @U064X3EF3 commented on this raising the question of why are these files under the lib's (next.jdbc's) path? What are the location requirements for clj-kondo hooks?
The requirements are that these files are on the classpath under a clj-kondo.exports
directory so clj-kondo knows what to copy to the local config directory
so Alex recently commented on this again on ask.clojure, it appears a non-clj extension would be the "officially supported" solution: https://ask.clojure.org/index.php/11434/could-namespace-refresh-sophisticated-directory-filtering?show=11882#c11882
@U08BJGV6E I think there might be another workaround
We could make a clj-kondo hooks "shim" library so the namespaces are defined, but they don't do actually anything
although I think having a filter in that reloading library would be a better solution
For now I think I'd prefer my current workaround I detailed above tbh. While the tools.ns filter would be welcome, Alex seems to be on the position that distributing .clj files with a lib means you're distributing those namespaces
And as such if you give an instruction to load all namespaces from the lib, clojure isn't at fault for trying to load them
What if we supported .cljk
? But then we'd have to be sure there would be not other language in the future adopting that extension :/
That's not correct in the case of clj-kondo hooks. Did you explain this was used for clj-kondo hooks?
Anyway, perhaps you can PR your current workaround as documentation for clj-kondo hooks? At least we've got something
it would be a problem for tooling, an .edn file cannot contain language constructs that .clj can
or at least, not right now, but probably in the future when we would include such warnings
what about a file extension that is based off edn and not clj? Like ednk or something along those lines? Or even .clj-kondo or .hook
I have to agree with Alex on hooks themselves being only data from the library consumer's point of view
And since clj source files imply code to Clojure, perhaps this data shouldn't be in clj files
so I don't agree. A library could also bundle scripts that it would execute using load-file
in some special context
Would you mind commenting on the question there, then? I don't think I have enough weight in the community to have an effect on this either way
if we will support another extension, the other uphill battle would be to convince other hook authors to use that extension
linking to the ask issue + workaround in the docs seems like the best overall solution
Probably this section: https://github.com/clj-kondo/clj-kondo/blob/master/doc/hooks.md#tips-and-tricks
I think one of the problems here is that these files are not on the classpath where clojure would look for them if it tried to require one of these namespaces
; clj -Srepro -Sdeps '{:deps {org.clojure/tools.namespace {:mvn/version "1.2.0"} seancorfield/next.jdbc {:git/url " " :git/sha "24bf1dbaa441d62461f980e9f880df5013f295dd"}}}' -M -e "((requiring-resolve 'clojure.tools.namespace.repl/refresh-all))"
:reloading (next.jdbc.protocols next.jdbc.prepare next.jdbc.result-set next.jdbc.datafy next.jdbc.types next.jdbc.transaction next.jdbc.default-options next.jdbc.connection next.jdbc.sql.builder next.jdbc.sql-logging next.jdbc next.jdbc.sql next.jdbc.specs hooks.com.github.seancorfield.next-jdbc next.jdbc.quoted next.jdbc.date-time next.jdbc.optional next.jdbc.plan)
:error-while-loading hooks.com.github.seancorfield.next-jdbc
#error {
:cause "Could not locate hooks/com/github/seancorfield/next_jdbc__init.class, hooks/com/github/seancorfield/next_jdbc.clj or hooks/com/github/seancorfield/next_jdbc.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name."
:via
[{:type java.io.FileNotFoundException
:message "Could not locate hooks/com/github/seancorfield/next_jdbc__init.class, hooks/com/github/seancorfield/next_jdbc.clj or hooks/com/github/seancorfield/next_jdbc.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name."
:at [clojure.lang.RT load "RT.java" 462]}]
:trace
[[clojure.lang.RT load "RT.java" 462]
[clojure.lang.RT load "RT.java" 424]
[clojure.core$load$fn__6908 invoke "core.clj" 6161]
[clojure.core$load invokeStatic "core.clj" 6160]
[clojure.core$load doInvoke "core.clj" 6144]
[clojure.lang.RestFn invoke "RestFn.java" 408]
[clojure.core$load_one invokeStatic "core.clj" 5933]
[clojure.core$load_one invoke "core.clj" 5928]
[clojure.core$load_lib$fn__6850 invoke "core.clj" 5975]
[clojure.core$load_lib invokeStatic "core.clj" 5974]
[clojure.core$load_lib doInvoke "core.clj" 5953]
[clojure.lang.RestFn applyTo "RestFn.java" 142]
[clojure.core$apply invokeStatic "core.clj" 669]
[clojure.core$load_libs invokeStatic "core.clj" 6016]
[clojure.core$load_libs doInvoke "core.clj" 6000]
[clojure.lang.RestFn applyTo "RestFn.java" 137]
[clojure.core$apply invokeStatic "core.clj" 669]
[clojure.core$require invokeStatic "core.clj" 6038]
[clojure.core$require doInvoke "core.clj" 6038]
[clojure.lang.RestFn invoke "RestFn.java" 421]
[clojure.tools.namespace.reload$track_reload_one invokeStatic "reload.clj" 35]
[clojure.tools.namespace.reload$track_reload_one invoke "reload.clj" 21]
[clojure.tools.namespace.reload$track_reload invokeStatic "reload.clj" 52]
[clojure.tools.namespace.reload$track_reload invoke "reload.clj" 43]
[clojure.lang.AFn applyToHelper "AFn.java" 154]
[clojure.lang.AFn applyTo "AFn.java" 144]
[clojure.lang.Var alterRoot "Var.java" 308]
[clojure.core$alter_var_root invokeStatic "core.clj" 5535]
[clojure.core$alter_var_root doInvoke "core.clj" 5530]
[clojure.lang.RestFn invoke "RestFn.java" 425]
[clojure.tools.namespace.repl$do_refresh invokeStatic "repl.clj" 94]
[clojure.tools.namespace.repl$do_refresh invoke "repl.clj" 83]
[clojure.tools.namespace.repl$refresh_all invokeStatic "repl.clj" 162]
[clojure.tools.namespace.repl$refresh_all doInvoke "repl.clj" 147]
[clojure.lang.RestFn invoke "RestFn.java" 397]
[clojure.lang.Var invoke "Var.java" 380]
[user$eval1 invokeStatic "NO_SOURCE_FILE" 1]
[user$eval1 invoke "NO_SOURCE_FILE" 1]
[clojure.lang.Compiler eval "Compiler.java" 7194]
[clojure.lang.Compiler eval "Compiler.java" 7149]
[clojure.core$eval invokeStatic "core.clj" 3215]
[clojure.main$eval_opt invokeStatic "main.clj" 488]
[clojure.main$eval_opt invoke "main.clj" 482]
[clojure.main$initialize invokeStatic "main.clj" 508]
[clojure.main$null_opt invokeStatic "main.clj" 542]
[clojure.main$null_opt invoke "main.clj" 539]
[clojure.main$main invokeStatic "main.clj" 664]
[clojure.main$main doInvoke "main.clj" 616]
[clojure.lang.RestFn applyTo "RestFn.java" 137]
[clojure.lang.Var applyTo "Var.java" 705]
[clojure.main main "main.java" 40]]}
"Could not locate hooks/com/github/seancorfield/next_jdbc__init.class, hooks/com/github/seancorfield/next_jdbc.clj or hooks/com/github/seancorfield/next_jdbc.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name."
it's the same issue with a test runner, it loads a file to scan for tests. every sane test runner has a config option to skip certain namespaces / directories
Afair tools.ns looks for clojure source files on directory paths, then parses out the namespace name from their ns forms, then tries to load those namespaces
yes. it should have the option, is my opinion, but I'm not going to spend any time trying to convince the core team
I'll leave it at this for now: • document your workaround • if more related issues come up, we can consider the extension
One related issue in the past has been that Cursive also analyzes hook code and when you have same-named hooks then you end up navigating to the wrong file
The solution in Cursive was to just use a different ns for the hook code, but the extension would also save the day here I guess
Oh actually, one more reason for the custom extension: tools.build compilation: https://clojurians.slack.com/archives/C02B5GHQWP4/p1651685386479149
And here's the issue, I tried to collect everything without being too wordy: https://github.com/clj-kondo/clj-kondo/issues/1685