Fork me on GitHub
#tools-deps
<
2021-07-30
>
Cam Saul01:07:32

I haven't really dug in to it much but was wondering whether other people run into this with the CLI. It occasionally gets into a weird state and barfs on launch when a deps.edn file (usually one from a :local/root dependency) was updated. Nuking all the .cpcache directories consistently fixes it. I've had maybe 4 people on my team run into this now (including just now with the latest 933 release)... just wondering if this is a known issue or maybe I've set things up the wrong way somehow

seancorfield02:07:39

Yes. Transitive deps in local deps don't get recalculated. Use -Sforce

πŸ‘ 3
hiredman02:07:53

I would expect that if you are both using local/root and a high churn of deps for those local/root projects

πŸ‘ 2
seancorfield02:07:03

It's a long standing known issue

πŸ‘ 3
cap10morgan17:07:45

Does the new prep feature need tools.build? What defines the :fn in the referenced :alias?

cap10morgan17:07:29

or is that just clojure.core/compile and you're telling it to invoke that as the main under the referenced alias?

cap10morgan17:07:36

(in the example in the docs)

Joshua Suskalo18:07:19

I believe the prep feature just runs an alias the same way -X does. It does not require tools.build.

cap10morgan18:07:34

Hmm, that leaves me somewhat confused what the :fn attribute is referencing then

Joshua Suskalo18:07:06

Some aliases specify a default ns, or otherwise don't provide a function and expect it to be provided on the command line. Since there's no user-controlled command line to prep a lib, that fn argument is what you want to pass on the command line.

cap10morgan18:07:01

oh ok I think that makes sense. will probably be clearer once I try it. πŸ™‚ thanks!

mpenet19:07:46

Are there workarounds to tdeps-174 now ?

mpenet19:07:54

I remember talks about allowing more control aver the "chain" of deps files or add something like aero tags to deps.edn reader but glancing at the changelogs I didn't see anything related

seancorfield20:07:31

@mpenet I've blogged about how we have restructured our monorepo/deps files (four parts in the series so far over several months) but where we've settled on now -- based on guidance from Alex and a bunch of experience with different approaches -- is to have a project deps.edn with dev/test aliases configured that treat the subprojects as :local/root libraries with their own deps.edn files, then we have a "project" subproject for each artifact we build which has a standalone deps.edn file that treats the parts it needs as :local/root libraries. That allows us to control a unified dev REPL/test environment but also have specific "build" environments for each "project" (artifact). We've also started to adopt Polylith which works much the same way but has a custom tool (that reads/merges deps.edn files) so it can perform incremental testing and do so in the context of each "project" as well as in the "dev" context.

seancorfield20:07:42

TL;DR: we no longer care about TDEPS-174 πŸ™‚

mpenet21:07:17

I d prefer to avoid using a custom tool to merge/read edn files, so it seems 174 is still relevant. I ll read the latest post, I might have missed it

seancorfield21:07:22

Well, Polylith's tool poly does a bunch of reading and merging, and it also does classpath isolation stuff to run tests in-process. In our own code at work, we're using tools.build and create-basis, java-command, and process to construct contexts to run tests (as subprocesses, rather than in-process).

mpenet21:07:38

I know about Polylith, but this is not something I could use in that context. I am also not sold on the approach. But maybe that changed recently too. I ll look again

mpenet21:07:27

Frankly if I had to impose some external tool/script to handle it I d rather have a simple "launcher" that allows more deps edn to the chain at "call" time. That seems less intrusive, but that's also not something I want to do.

mpenet21:07:56

I am curious to know if 174 & co is still considered as something to be improved or not

mpenet21:07:14

(by tools.deps)

seancorfield21:07:27

I get the impression that this is a problem the core team don't feel is worth solving in "core" since t.d.a exists and folks who need more deps.edn files can "easily" write their own in a few lines.

seancorfield21:07:20

(we use t.d.a in our build script to do stuff that's outside the CLI scope but pretty simple to write ourselves)

dpsutton23:07:52

i've seen this error show up in CI infrequently. anyone recognize it or know what i can do to mitigate it or lower the frequency?

Downloading: org/clojure/data.json/2.0.2/data.json-2.0.2.pom from central
Downloading: enlive/enlive/1.1.6/enlive-1.1.6.pom from clojars
Downloading: org/ccil/cowan/tagsoup/tagsoup/1.2.1/tagsoup-1.2.1.pom from central
Downloading: org/jsoup/jsoup/1.7.2/jsoup-1.7.2.pom from central
Error building classpath. class java.util.HashMap$Node cannot be cast to class java.util.HashMap$TreeNode (java.util.HashMap$Node and java.util.HashMap$TreeNode are in module java.base of loader 'bootstrap')
java.lang.ClassCastException: class java.util.HashMap$Node cannot be cast to class java.util.HashMap$TreeNode (java.util.HashMap$Node and java.util.HashMap$TreeNode are in module java.base of loader 'bootstrap')
	at java.base/java.util.HashMap$TreeNode.moveRootToFront(HashMap.java:1882)
	at java.base/java.util.HashMap$TreeNode.putTreeVal(HashMap.java:2061)
	at java.base/java.util.HashMap.putVal(HashMap.java:633)
	at java.base/java.util.HashMap.put(HashMap.java:607)
	at java.base/java.util.HashSet.add(HashSet.java:220)
	at org.apache.maven.model.validation.DefaultModelValidator.validateId(DefaultModelValidator.java:847)
	at org.apache.maven.model.validation.DefaultModelValidator.validateEffectiveDependency(DefaultModelValidator.java:659)
	at org.apache.maven.model.validation.DefaultModelValidator.validateEffectiveDependencies(DefaultModelValidator.java:583)
	at org.apache.maven.model.validation.DefaultModelValidator.validateEffectiveModel(DefaultModelValidator.java:373)
	at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:494)
	at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:440)
	at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:430)
	at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:292)
	at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:171)
	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.readArtifactDescriptor(DefaultRepositorySystem.java:255)
	at clojure.tools.deps.alpha.extensions.maven$eval1076$fn__1078.invoke(maven.clj:107)
	at clojure.lang.MultiFn.invoke(MultiFn.java:244)
	at clojure.tools.deps.alpha$expand_deps$children_task__790$fn__792$fn__793.invoke(alpha.clj:403)
	at clojure.tools.deps.alpha.util.concurrent$submit_task$task__505.invoke(concurrent.clj:34)
	at clojure.lang.AFn.call(AFn.java:18)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

Exited with code exit status 1

hiredman23:07:15

what version of tools deps? try adding -Sthreads 1 to your clj call

hiredman23:07:25

that error is due to unsafe concurrent modification of a hashmap, internally in maven, I believe it has been fixed (I haven't seen it in a while)

Alex Miller (Clojure team)23:07:01

That’s a known issue, hoping to work on it in a couple weeks

dpsutton23:07:37

ok. thank you much. i'll just kick the box when i see it. thanks

seancorfield23:07:33

I'm a bit surprised we've never seen that at work -- we don't use the -Sthreads option. Quite a few do seem to hit it. I wonder why it seems to affect some people more than others?