Fork me on GitHub
#tools-deps
<
2022-02-21
>
arohner10:02:04

I’m calling tools.deps as a library, and occasionally seeing this exception in CI:

Exception in thread "main" java.lang.ClassCastException: class java.util.HashMap$Node cannot be cast to class java.util.HashMap$TreeNode (java.util.HashMap$Node and java.util.HashMap$TreeNode are in module java.base of loader 'bootstrap')
	at java.base/java.util.HashMap$TreeNode.moveRootToFront(HashMap.java:1900)
	at java.base/java.util.HashMap$TreeNode.treeify(HashMap.java:2016)
	at java.base/java.util.HashMap.treeifyBin(HashMap.java:768)
	at java.base/java.util.HashMap.putVal(HashMap.java:640)
	at java.base/java.util.HashMap.put(HashMap.java:608)
	at java.base/java.util.HashSet.add(HashSet.java:220)
	at org.apache.maven.model.validation.DefaultModelValidator.validateId(DefaultModelValidator.java:848)
	at org.apache.maven.model.validation.DefaultModelValidator.validateId(DefaultModelValidator.java:825)
	at org.apache.maven.model.validation.DefaultModelValidator.validateEffectiveModel(DefaultModelValidator.java:341)
	at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:501)
	at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:437)
	at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:252)
	at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:297)
	at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:175)
	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.readArtifactDescriptor(DefaultRepositorySystem.java:255)
	at clojure.tools.deps.alpha.extensions.maven$fn__771.invokeStatic(maven.clj:132)
	at clojure.tools.deps.alpha.extensions.maven$fn__771.invoke(maven.clj:122)
	at clojure.lang.MultiFn.invoke(MultiFn.java:244)
	at clojure.tools.deps.alpha$expand_deps$children_task__464$fn__466$fn__467.invoke(alpha.clj:406)
	at clojure.tools.deps.alpha.util.concurrent$submit_task$task__187.invoke(concurrent.clj:35)
	at clojure.lang.AFn.call(AFn.java:18)
StackOverflow suggests a race condition is likely

borkdude10:02:42

It usually happens when you calculate more then one basis I think

arohner10:02:05

Oh, so then upgrading to TDEPS 0.12.1098 might fix it?

borkdude10:02:25

Worth a shot!

👍 1
arohner10:02:10

Hrm, that stacktrace doesn’t match the one I see

arohner10:02:47

0.12.1098 also contains a bump to maven-core 3.8.4, which contains a fix for https://issues.apache.org/jira/browse/MNG-7285 which might fix

Alex Miller (Clojure team)13:02:29

I believe, even on the latest version, this can still happen. Deep inside the bowels of Maven there is some unsynchronized state held in either the system or session that I'm sharing across expansion threads that clearly maven is not expecting to be shared.

Alex Miller (Clojure team)13:02:26

I have not had time to look at it more closely yet but I suspect the real solution is to not share stuff because Maven is dumb. Unfortunately, that means it will probably also be slower because I have to redo more work. So, this will need some time to figure out the tradeoff there. If you really want to avoid it, you can set -Sthreads 1 on the CLI to avoid multiple threads.

Alex Miller (Clojure team)13:02:58

This whole general area of concurrent use is an area of active work in Maven land. Maven core 4.x is redoing a lot of stuff, but it's not done yet so I've been holding off

Carsten Behring10:02:43

I was quite often bitten by a deps.edn and using somehow wrong (wrong spelled for example) git/sha coordinates. Example:

{:deps
 {org.clojure/data.csv {:git/url ""
                        :sha "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}}}
This result in a un-informative exception, NPE: It would be nice to have something more informative. (specially which coordinate is wrong, in case of many it saves some try-and-error)
Error building classpath. 
java.lang.NullPointerException
	at clojure.tools.deps.alpha.util.dir$canonicalize.invokeStatic(dir.clj:30)
	at clojure.tools.deps.alpha.extensions.deps$fn__1467.invokeStatic(deps.clj:32)
	at clojure.tools.deps.alpha.extensions.deps$fn__1467.invoke(deps.clj:32)
	at clojure.lang.MultiFn.invoke(MultiFn.java:244)
	at clojure.tools.deps.alpha$expand_deps$children_task__770$fn__772$fn__773.invoke(alpha.clj:406)
	at clojure.tools.deps.alpha.util.concurrent$submit_task$task__479.invoke(concurrent.clj:35)
	at clojure.lang.AFn.call(AFn.java:18)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

rickmoynihan11:02:12

yeah I’ve seen this too in dev after depending on a sha in a PR/branch for a dep which you later force push a fix to

Alex Miller (Clojure team)14:02:11

I think this particular problem is fixed in the latest release actually - what version of CLI are you on?

borkdude10:02:44

I think clj-kondo could help here as well if there is a way to see that it's an invalid SHA. Is that possible to check using a pure function?

Carsten Behring11:02:39

It can go wrong as well in other ways, even with a valid sha, example: (forgetting to add the :git/url gives as well a NPE (and no way to see which coordinate is not working)

{:deps
 {org.clojure/data.csv 
{                       :sha "e5beccad0bafdb8e78f19cba481d4ecef5fabf36"}}}

Alex Miller (Clojure team)14:02:20

Any time you find a bad error message, please file a question on https://ask.clojure.org with the repro and your version, happy to work on these as they come in

vlaaad11:02:49

I just had wild idea: for -X invocations, tdeps could provide an extra flag to fn to signify if this function is called as an entry point. Use case: changing the behavior depending on whether it’s known to be an entry point or whether it’s a part of a larger program, e.g.: • calling shutdown-agents at the end if it’s an entry point • calling (Platform/exit) for JavaFX UIs if it’s an entry point • rebinding System.{out,err} to special output if it’s an entry point Makes sense to create an ask?

Alex Miller (Clojure team)14:02:29

Put it on ask please. There are a lot cases here and we've made a bunch of delicate choices in this area already