This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-11-19
Channels
- # announcements (3)
- # asami (3)
- # babashka (39)
- # beginners (65)
- # calva (13)
- # cider (4)
- # clj-kondo (69)
- # cljdoc (19)
- # cljs-dev (2)
- # clojure (90)
- # clojure-dev (10)
- # clojure-europe (61)
- # clojure-france (15)
- # clojure-nl (8)
- # clojure-uk (2)
- # clojurescript (28)
- # conjure (2)
- # core-logic (4)
- # cursive (8)
- # datalevin (5)
- # datascript (7)
- # datomic (14)
- # depstar (4)
- # events (1)
- # graphql (7)
- # holy-lambda (5)
- # jobs (5)
- # kaocha (1)
- # malli (14)
- # membrane-term (13)
- # missionary (13)
- # nextjournal (6)
- # off-topic (1)
- # polylith (15)
- # portal (10)
- # re-frame (35)
- # reitit (1)
- # remote-jobs (3)
- # schema (3)
- # sci (121)
- # spacemacs (6)
- # tools-build (8)
- # tools-deps (74)
- # xtdb (7)
I'm trying to debug a tools issue in CI. I'm seeing
clojure -Ttool list
Error building classpath. Bad coordinate for library , expected map: nil
The CLI is 1.10.3.933 The deps.edn
file in the project seems valid (`clojure -Stree` works). I've also tried updating to 1.10.3.1029. The failed CI is https://github.com/rm-hull/nvd-clojure/runs/4259660700?check_suite_focus=true -- you can see the output of clojure -Sdescribe
and the root deps.edn
just as a sanity check...I think that should be -Ttools ?
I added a check for unknown tool name with a better error message in a couple places
Er, yes, OK, that doesn't really explain what we were seeing, but I'll go back to the PR and see what's up.
Here's the earlier failure:
clojure -Ttools install nvd-clojure/nvd-clojure '{:mvn/version "RELEASE"}' :as nvd
Error building classpath. Bad coordinate for library , expected map: nil
and that definitely has -Ttools
@alexmillerI ended up "solving" it by writing the tools/tools.edn
file manually -- so it's something to do with clojure
not laying that down when it first runs...
so there was no tools.edn there?
Correct, here's another run:
+ clojure -Ttools list
Error building classpath. Bad coordinate for library , expected map: nil
+ ls -l /home/runner/.config/clojure/
total 4
-rw-r--r-- 1 runner docker 1490 Nov 19 04:07 deps.edn
+ ls -l /home/runner/.config/clojure/tools/
ls: cannot access '/home/runner/.config/clojure/tools/': No such file or directory
+ mkdir /home/runner/.config/clojure/tools
+ echo '{:lib io.github.clojure/tools.tools :coord {:git/tag "v0.2.2" :git/sha "e1febed7ddaa5be15721255c13eb68e11bbbb398"}}'
+ clojure -Ttools list
(works now)
TOOL LIB TYPE VERSION
tools io.github.clojure/tools.tools :git v0.2.2
+ clojure -Ttools install nvd-clojure/nvd-clojure '{:mvn/version "RELEASE"}' :as nvd
Installed nvd
+ clojure -Ttools list
TOOL LIB TYPE VERSION
nvd nvd-clojure/nvd-clojure :mvn 1.9.0
tools io.github.clojure/tools.tools :git v0.2.2
hmm, maybe something with using a special config dir
What makes clojure
decide whether to lay down the tools
stuff?
if it's missing or the one in the install is newer than the one there
Any suggestions for debugging why it isn't doing it (above)?
the logic is in the clojure script, so hack away
Hmm, OK. Will take a deeper look tomorrow, when I'm back at my main system...
what's your clj -Sdescribe?
if [ "$install_dir/tools.edn" -nt "$config_dir/tools/tools.edn" ]; then
mkdir -p "$config_dir/tools"
cp "$install_dir/tools.edn" "$config_dir/tools/tools.edn"
fi
is the whole of the relevant codemaybe that condition needs to explicitly check for missing, although I know I've tested this several ways
Just a sec... I had clojure -Sdescribe
in an earlier CI script...
Here's what appeared for 933 (which failed the same way):
{:version "1.10.3.933"
:config-files ["/opt/hostedtoolcache/ClojureToolsDeps/1.10.3-933/x64/lib/clojure/deps.edn" "/home/runner/.config/clojure/deps.edn" "deps.edn" ]
:config-user "/home/runner/.config/clojure/deps.edn"
:config-project "deps.edn"
:install-dir "/opt/hostedtoolcache/ClojureToolsDeps/1.10.3-933/x64/lib/clojure"
:config-dir "/home/runner/.config/clojure"
:cache-dir ".cpcache"
:force false
:repro false
:main-aliases ""
:repl-aliases ""}
Here's the output from 1029:
{:version "1.10.3.1029"
:config-files ["/opt/hostedtoolcache/ClojureToolsDeps/1.10.3-1029/x64/lib/clojure/deps.edn" "/home/runner/.config/clojure/deps.edn" "deps.edn" ]
:config-user "/home/runner/.config/clojure/deps.edn"
:config-project "deps.edn"
:install-dir "/opt/hostedtoolcache/ClojureToolsDeps/1.10.3-1029/x64/lib/clojure"
:config-dir "/home/runner/.config/clojure"
:cache-dir ".cpcache"
:force false
:repro false
:main-aliases ""
:repl-aliases ""}
This is on Ubuntu 20.04.3 LTS according to that CI log.
I repro'ed it when there is no tools dir
actually, no I didn't, it's fine
well I'm sure it's something in that area, can't really be much else
@alexmiller Should I create a JIRA for this or are you already on it?
if you have a repro for it....
otherwise not sure what I would do with it
I tested a few things, haven't been able to repro
Yeah, it's a weird one. I'd never seen it before and the only repro I have is that CI pipeline...
does it fail like that occasionally or always?
Always. And I think I know why:
+ ls -lR /opt/hostedtoolcache/ClojureToolsDeps/1.10.3-1029/x64/lib/clojure
/opt/hostedtoolcache/ClojureToolsDeps/1.10.3-1029/x64/lib/clojure:
total 12
-rw-r--r--+ 1 runner docker 512 Nov 19 14:17 deps.edn
-rw-r--r--+ 1 runner docker 1490 Nov 19 14:17 example-deps.edn
drwxr-xr-x+ 2 runner docker 4096 Nov 19 14:17 libexec
/opt/hostedtoolcache/ClojureToolsDeps/1.10.3-1029/x64/lib/clojure/libexec:
total 15752
-rw-r--r--+ 1 runner docker 16125828 Nov 19 14:17 clojure-tools-1.10.3.1029.jar
-rw-r--r--+ 1 runner docker 3636 Nov 19 14:17 exec.jar
There is no tools.edn
in the installation. So this is a bug with the DeLaGuardo/setup-clojure Action I think? @delaguardo?Looks like https://github.com/DeLaGuardo/setup-clojure/blob/master/src/cli.ts doesn't copy the tools.edn
file into the tool cache.
sounds right
Right, this is a problem of the action. Will fix it later today
@delaguardo Given the caching of the clojure
setup, will you be able to fix it for existing versions, or will this only affect new CLI versions going forward?
Should not be a problem. I will check it as well
Cool. I just read the issue that mentioned caching which was what made me think about it...
why isn't this using the bash script on linux and macos but it is using the powershell script on Windows?
I don't have much experience with windows and rely heavenly on premade script. For other platforms it seems like a better approach to control the resources because of few reasons: • Better control what should be cached (unfortunately this hit the problem of very strange cache behavior of GitHub actions, main reason why I don't yet reply in the corresponding issue) • I don't like the way how installation script works in general. It follows some "best practices" which i don't understand personally. Maybe it is the time to reconsider, but before that i would like to have a reply to the patch i proposed for packaging scripts.
(and I guess it will need to be conditional on tools.edn
existing -- that only came in with 1.10.3.933 right @alexmiller?)
@dominicm I have a project which still uses pack.alpha to create build artifacts; unfortunately the project has now acquired a dependency that uses the new :git/sha
key, and not the old :sha
key.
This causes pack.alpha to choke… I tried bumping the tools.deps.alpha version it uses to the latest; but it seems there have been some breaking changes for you in that dep. Specifically clojure.tools.deps.alpha.reader/merge-deps
install-deps
and slurp-deps
have now moved/vanished.
they're in clojure.tools.deps.alpha (that change happened long long ago)
I guess I’m probably better migrating this to use tools.build uber
instead; though I was using the skinny jar functionality to preserve the true classpath; rather than munge it
@alexmiller: so they’ve just moved ns? Or are they removed entirely now?
just trying to figure out what the quicker fix will be — migrating or patching a fork of pack.alpha
moved, and I think the install one changed names to root-deps
the merge stuff might be factored slightly differently too, don't remember now
the same stuff is all there, just might be slightly different api
yeah — I’ll see if I can patch it
thanks again for pointing me in the right direction 🙇
https://github.com/clojure/tools.deps.alpha/commit/3dec3cd5c47f3eb7ef0b7e3a77087dc3f19d11d5
(June 2020)
perfect that’s the commit I’m looking for 👌
@alexmiller thanks, that was exactly what I was looking for…
I’ve patched up the skinny feature of pack.alpha
here: https://github.com/juxt/pack.alpha/pull/94
when generating a jar, do all deps need to be available on maven as well and I can’t depend on :git/sha
deps? Seeing the following warning and indeed valuehash is missing from the jar:
$ clj -T:build jar
Producing jar: target/clerk-0.3.265.jar
Skipping coordinate: {:git/sha ff1d4b7f1260daf41c786a61cb45d02871b7baf9, :git/url , :deps/manifest :deps, :deps/root /Users/mk/.gitlibs/libs/io.github.arachne-framework/valuehash/ff1d4b7f1260daf41c786a61cb45d02871b7baf9, :parents #{[]}, :paths [/Users/mk/.gitlibs/libs/io.github.arachne-framework/valuehash/ff1d4b7f1260daf41c786a61cb45d02871b7baf9/src]}
I guess it makes sense that this doesn’t work. Any recommended way to bring this in (I can’t find a jar on clojars or central)? I guess https://jitpack.io should work.@mkvlr That message comes from tools.deps
when it is building the pom.xml
file -- and, yes, you can't have a JAR on Maven/Clojars that depends on git deps because consumers generally couldn't resolve that.
@seancorfield makes sense, would copying the sources into my jar be a bad idea?
Potentially a bad idea, yes. Users of your JAR would "unexpectedly" get all those other nses and that might cause conflicts.
jitpack doesn’t work, or only partially, master
seems to work but the lastest sha which master points to doesn’t. Seems to be bad to depend on a mutable name for sure.
This was a possibility with depstar
-- by default it copied local and git deps into the JAR (since consumers of JARs couldn't consume those as deps in the pom) but there were caveats in the docs about it, and an option to not do that. tools.build
by default does the latter -- not copying them (which is the better default behavior). Hard to say what you should do without knowing more about your project...
You're depending on Arachne but that doesn't have published artifacts?
In which case, anyone using your library would also have to depend on Arachne via git anyway -- so why publish your library at all? Why not let people depend on it via git, like they would have to do for Arachne?
(git deps are "contagious" in that respect: once you depend on them, your "customers" have to depend on them as well...)
I want consumers of our lib (http://github.com/nextjournal/clerk) to use it via maven as well (if they’re using lein or whatever). In general I’d like to make consumption easy. Can’t find a published artifact of https://github.com/arachne-framework/valuehash which I’d like to depend on, I’ll open a ticket with them.
I'm not sure Luke is still doing anything with that
so you might need to fork and publish if you want it out there
@alexmiller I’ll ask and do that if I don’t hear back, thank you. Might have been once published to http://maven.arachne-framework.org/artifactory/arachne-dev which shows up in some repos but http only is probably also not a good idea?
subject to MITM attacks