Fork me on GitHub
#tools-deps
<
2020-09-28
>
viesti08:09:37

humdum, I sometimes get this error in CirleCI while building project:

clojure -A:... -m kaocha.runner analysis
Picked up JAVA_TOOL_OPTIONS: -Dclojure.main.report=stderr -XX:-OmitStackTraceInFastThrow
Cloning: [email protected]:viesti/timbre-json-appender.git
Checking out: [email protected]:viesti/timbre-json-appender.git at 176a32bc5f7311c22b091b9700b5f98c11807774
Error building classpath. class java.util.HashMap$Node cannot be cast to class java.util.HashMap$TreeNode (java.util.HashMap$Node and java.util.HashMap$TreeNode are in module java.base of loader 'bootstrap')
java.lang.ClassCastException: class java.util.HashMap$Node cannot be cast to class java.util.HashMap$TreeNode (java.util.HashMap$Node and java.util.HashMap$TreeNode are in module java.base of loader 'bootstrap')
	at java.base/java.util.HashMap$TreeNode.moveRootToFront(HashMap.java:1882)
	at java.base/java.util.HashMap$TreeNode.treeify(HashMap.java:1998)
	at java.base/java.util.HashMap.treeifyBin(HashMap.java:767)
	at java.base/java.util.HashMap.putVal(HashMap.java:639)
	at java.base/java.util.HashMap.put(HashMap.java:607)
	at java.base/java.util.HashSet.add(HashSet.java:220)
	at org.apache.maven.model.validation.DefaultModelValidator.validateId(DefaultModelValidator.java:847)
	at org.apache.maven.model.validation.DefaultModelValidator.validateEffectiveDependency(DefaultModelValidator.java:659)
	at org.apache.maven.model.validation.DefaultModelValidator.validateEffectiveDependencies(DefaultModelValidator.java:583)
	at org.apache.maven.model.validation.DefaultModelValidator.validateEffectiveModel(DefaultModelValidator.java:368)
	at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:494)
	at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:440)
	at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:430)
	at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:292)
	at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:171)
	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.readArtifactDescriptor(DefaultRepositorySystem.java:255)
	at clojure.tools.deps.alpha.extensions.maven$eval813$fn__815.invoke(maven.clj:99)
	at clojure.lang.MultiFn.invoke(MultiFn.java:244)
	at clojure.tools.deps.alpha$expand_deps$fn__1230$fn__1231.invoke(alpha.clj:214)
	at clojure.tools.deps.alpha.util.concurrent$submit_task$task__500.invoke(concurrent.clj:33)
	at clojure.lang.AFn.call(AFn.java:18)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
error: Recipe `test` failed on line 17 with exit code 1
using cimg/clojure:1.10.1 as image, with Clojure cli version 1.10.1.536

3
viesti08:09:08

might be totally relared to this project only, but just thinking out loud, that might there be a concurrency issue in downloading dependencies

Alex Miller (Clojure team)13:09:31

You could try adding -Sthreads 1 and see if that helps

viesti08:10:12

thanks! 🙂

jjttjj15:09:09

I just updated my clj tools to the latest version and am now getting

CompilerException java.lang.RuntimeException: Unable to resolve symbol: requiring-resolve in this context, compiling:(cljfx/context.clj:37:32) 
which would seem to imply that an older version of clojure being used? Is there an obvious reason this would happen after the clj update? (Both cljfx and my project seem to explicitly depend on clojure 1.10.1)

jjttjj17:09:37

Ok I figured this out. I had a :local/root dependency which points to a standalone jar file which uses clojure 1.9, which seems like it causes the repl started from clj to be 1.9. Is this expected?

borkdude16:09:56

@jjttjj clojure -Stree is your friend here

borkdude16:09:08

(or whatever is the way of doing the tree now)

jjttjj16:09:40

Hmmm it looks like clj is giving me a clojure 1.9 repl even though my project only has 1.10 in the tree. might be a scoop issue going to try reinstalling clj Edit: ok I think this is a scoop issue

👀 3
nate16:09:57

a couple of things I've noticed that are different about the new cli tools: - Only the first -M is taken. I used to be able to do -A:one -A:two before and both :one and :two were used (was useful for a script I had that wrapped a call to clojure and I could add extra args if I wanted) - If I put -Spath at the end of my command, it's ignored if there's a -M.... Specifically clojure -Spath -M:repl prints a classpath, but clojure -M:repl -Spath runs the main specified in the :repl alias

Alex Miller (Clojure team)16:09:39

-A and -M have always supported multiple aliases so -M:one:two is what you need there. -M and -X should now be last as they grab all args afterwards

Alex Miller (Clojure team)16:09:18

So clojure -Spath -M:repl

nate16:09:05

right, I know that you can put them together, my point is that you used to be able to supply them separately

nate16:09:44

I definitely understand why the behavior changed

Alex Miller (Clojure team)16:09:00

Multiple -A should still work

nate16:09:49

interesting point about -A

nate16:09:03

deprecation warning is pushing me away from that, for good reasons

borkdude16:09:36

clojure -A:foo -M:bar should also work I think? brings in deps/paths from foo and bar but executes main from bar?

seancorfield16:09:26

There are a lot more changes in this version than I think are perhaps obvious on the surface -- it's probably always been "best practice" to specify :main-opts separately via a single -M option after using one or more -A options to accumulate aliases but that's never really been codified/written down. This new release sort of "enforces" (or at least "strongly encourages") what I think the core team have always considered best practice...

nate16:09:21

yeah, I agree with that

seancorfield16:09:53

-M used to only bring in :main-opts but now brings in paths/deps as well which should not cause any surprises -- but it is quite an important difference.

borkdude16:09:15

are you referring to a previous pre-release or release?

seancorfield16:09:04

I'm talking about this in the context of "previous stable" and "current stable" releases.

borkdude16:09:22

I didn't even know there was an -M in a previous release

borkdude16:09:07

A minor thing but -M -m still feels weird to me :)

Alex Miller (Clojure team)17:09:10

consider it encouragement to use -X :)

👍 3
borkdude17:09:52

-X is ok but I don't consider it a substitute for a command line interface for general usage

borkdude17:09:07

i.e. there will still be a place for tools.cli based tools

borkdude17:09:23

unless I'm missing something

borkdude17:09:49

But I guess the -m gets hidden in an alias mostly anyway :)

tvaughan17:09:25

I was doing something like clojure -A:alias-with-main-opts -m some.other.main With the new release I get a deprecation warning to switch to -M, but with -M -m some.other.main are passed as command-line options to the main entry point associated with alias-with-main-opts. Does this mean there's now no way to override a main entry point?

seancorfield17:09:32

@tvaughan I believe clojure -A:alias-with-main-opts -M -m some.other.main would be the equivalent now.

seancorfield17:09:42

(I haven't tried that...)

tvaughan17:09:17

Thanks @seancorfield. Interestingly, this clojure -A:toolbox:dxf-tools -M -m server.core results in an error message from my application about -m not -M being an unknown option. The main-opts associated with dxf-tools are still being used

seancorfield17:09:45

Hmm, sorry. Guess I'll have to do some digging (unless @alexmiller can give a quick answer off the top of his head).

tvaughan17:09:55

No worries. I appreciate the help. I can dig further too. I took a quick look at the changelog but didn't see anything about this specifically

seancorfield17:09:10

Ah, I think this is one of those cases where the advice is to split up :alias-with-main-opts into two aliases -- one with :main-opts (and any deps needed solely for that) and one with all the (other) deps.

seancorfield17:09:30

(the other option is to continue using the -R and/or -C options even though they are deprecated)

tvaughan17:09:19

Cool. Thanks. I thought that might be the case. Didn't want to change things without confirmation tho :)

Alex Miller (Clojure team)17:09:20

if you're including an uberjar in your classpath, then you've broken the ability of tools.deps to properly resolve libs

Alex Miller (Clojure team)17:09:44

b/c it can't "see" inside the uberjar to tell what libs are included there or their versions

jjttjj17:09:30

That makes sense

seancorfield18:09:00

@alexmiller Now that 1.10.1.697 is the stable release, there are four mentions of clj -m <some.ns> in https://clojure.org/guides/deps_and_cli that need to be updated to include -M if you want folks to avoid the clojure.main warnings (i.e., "WARNING: When invoking clojure.main, use -M").

Alex Miller (Clojure team)20:09:14

Yeah it’s on my list to update this week

3
seancorfield18:09:17

Those all need to be clj -M -m <some.ns> now, right?

seancorfield18:09:50

(also, I added a note on TDEPS-100 that indicates some confusion over -A since the reference page does not define clj-opt and clojure -h says that -A is an exec-opt which seems to be at odds with the CLI/deps reference text)

borkdude19:09:20

yes, now I feel the issue we discussed last weekend. I have this:

{;;:paths ["your-packages"]
 ;;:deps {your-org/your-packages {,,,}}
 :aliases
 {:glam {:extra-deps
         {borkdude/glam {:git/url ""
                         :sha "4599fb019deae9418d76a9996ae19b4003f3cc96"}}
         :main-opts ["-m" "glam.main"]}}}
and now I want to do:
bb -cp $(clojure -R:glam -Spath) -m glam.main --install clj-kondo/clj-kondo --global --verbose
but I have to make two aliases now, so from Clojure this becomes:
clojure -M:glam-deps:glam
:/

borkdude19:09:23

with -Spath it's pretty clear you don't want to run the main right

borkdude19:09:54

so why not ignore the main when doing this:

clojure -A:glam -Spath

borkdude19:09:36

that doesn't seem to be breaking

borkdude19:09:04

ah well, bash to the rescue:

{:aliases
 {:glam/packages {:extra-deps
                  {borkdude/glam {:git/url ""
                                  :sha "4599fb019deae9418d76a9996ae19b4003f3cc96"}
                   ;; your-org/your-packages {,,,}
                   }
                  ;; :extra-paths ["your-packages"]
                  }
  :glam {:main-opts ["-m" "glam.main"]}}}
alias glam='clojure -M:glam/packages:glam'
$ glam --install clj-kondo/clj-kondo org.babashka/babashka

seancorfield20:09:45

@borkdude If you put -Spath first it will not run the main.

seancorfield20:09:08

And that's supposed to be clear from the docs which show clj [clj-opt*] [exec-opt] as the syntax -- -Spath is a clj-opt, -M is an exec-opt

borkdude20:09:32

yep, makes sense, thanks for clearing that up

seancorfield20:09:59

(as I just noted to @alexmiller it's not clear whether -A is meant to be a clj-opt or an exec-opt at this point, esp. given that at some future date it will stop running :main-opts)

Alex Miller (Clojure team)20:09:24

It’s kind of in a gray area

Alex Miller (Clojure team)20:09:44

But I think I would call it a clj-opt

seancorfield20:09:26

At this point, I'm hoping that -A stops running :main-opts fairly soon and it can be clearly classified as a clj-opt and then -R and -C can be fully removed.

borkdude20:09:00

@seancorfield I'm still getting the warning:

$ clojure -Sforce -Spath -A:glam
WARNING: Use of :main-opts with -A is deprecated. Use -M instead.

borkdude20:09:32

I don't want main, I want path

borkdude20:09:43

I used to use -R -C for this

borkdude20:09:19

can I rely on this being a correct invocation and the warning begone next version?

seancorfield20:09:29

clojure -Sforce -Spath -M:glam should work.

seancorfield20:09:41

-Spath implies -P I believe.

borkdude20:09:43

it works, but I get the warning

seancorfield20:09:13

(! 696)-> clojure -Sforce -Spath -M:runner
src:resources:/Users/sean/.gitlibs/libs/com.cognitect/test-runner/f7ef16dc3b8332b0d77bc0274578ad5270fbfedd/src:/Users/sean/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/Users/sean/.m2/repository/org/clojure/tools.cli/0.3.5/tools.cli-0.3.5.jar:/Users/sean/.m2/repository/org/clojure/tools.namespace/0.2.11/tools.namespace-0.2.11.jar:/Users/sean/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar:/Users/sean/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar

seancorfield20:09:50

Without -Spath that runs the tests (or attempts to -- it's missing the :test alias)

borkdude20:09:01

aha, with -M it doesn't warn, but with -A it does... that's weird

borkdude20:09:28

I think -A is the preferred way for creating paths right

seancorfield20:09:39

(! 698)-> clojure -Sforce -M:test:runner

Running tests in #{"test"}

Testing tvaughan.example-test

FAIL in (a-test) (example_test.clj:7)
FIXME, I fail.
expected: (= 0 1)
  actual: (not (= 0 1))

Ran 1 tests containing 1 assertions.
1 failures, 0 errors.
(! 699)-> clojure -Sforce -Spath -M:test:runner
test:src:resources:/Users/sean/.gitlibs/libs/com.cognitect/test-runner/f7ef16dc3b8332b0d77bc0274578ad5270fbfedd/src:/Users/sean/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar:/Users/sean/.m2/repository/org/clojure/test.check/1.0.0/test.check-1.0.0.jar:/Users/sean/.m2/repository/org/clojure/tools.cli/0.3.5/tools.cli-0.3.5.jar:/Users/sean/.m2/repository/org/clojure/tools.namespace/0.2.11/tools.namespace-0.2.11.jar:/Users/sean/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar:/Users/sean/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar
(! 700)-> 
🙂

borkdude20:09:14

anyway, I'll be using -A (and advocating it in my README) and expect the warning to be gone some time in the future

borkdude20:09:01

clojure -Sforce -Spath -A:glam 2>/dev/null

borkdude20:09:23

Feel free to comment on this README for improved clojure usage in case I missed something: https://github.com/borkdude/glam