Fork me on GitHub
#tools-deps
<
2021-06-17
>
seancorfield01:06:47

I just want to go on record and say that my previous grumbling about needing to keep :test and :runner aliases separate to avoid accidentally triggering the :main-opts in :runner when trying to start a REPL with -A no longer applies if you switch over to -X for running tests (`:exec-fn cognitect.test-runner.api/test`). Thank you, Alex, for adding -X execution of tests to the runner!

seancorfield01:06:33

(another reason for preferring -X over -M unless you have a very specific need for -main-style command-line args, IMO)

borkdude08:06:47

@seancorfield The -A behavior should just not invoke the -main anymore, the behavior has been deprecated and will likely adopt to the REPL behavior in the future. Imo, it's got nothing to do with -X vs -M other than that the behavior for -A is currently inconsistent :)

seancorfield16:06:20

Right and the roots of my comment go back to the original -A behavior. I’m saying that just introducing -X alone is sufficient to negate my grumbling about splitting those aliases.

seancorfield16:06:06

I just didn’t realize that until recently (I’ve been on a hunt for :test:runner in my projects and merging them so -X:test is all I need).

👍 2
mkvlr08:06:11

ran into this today, nuking ~/.gitlibs fixed it:

clojure -Srepro -Sforce -Sverbose -Stree                                                                                                                                          ✘ 1 
version      = 1.10.3.855
install_dir  = /usr/local/Cellar/clojure/1.10.3.855
config_dir   = /Users/mk/.clojure
config_paths = /usr/local/Cellar/clojure/1.10.3.855/deps.edn deps.edn
cache_dir    = .cpcache
cp_file      = .cpcache/336449426.cp

Refreshing classpath
Error building classpath. 
java.lang.NullPointerException
	at clojure.tools.deps.alpha.util.dir$canonicalize.invokeStatic(dir.clj:30)
	at clojure.tools.deps.alpha.util.dir$canonicalize.invoke(dir.clj:25)
	at clojure.tools.deps.alpha.extensions.deps$eval1393$fn__1395.invoke(deps.clj:27)
	at clojure.lang.MultiFn.invoke(MultiFn.java:244)
	at clojure.tools.deps.alpha$expand_deps$children_task__790$fn__792$fn__793.invoke(alpha.clj:403)
	at clojure.tools.deps.alpha.util.concurrent$submit_task$task__505.invoke(concurrent.clj:34)
	at clojure.lang.AFn.call(AFn.java:18)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

mkvlr09:06:33

I guess this was the change that triggered it:

diff --git a/deps.edn b/deps.edn
index f44153b6572..602c70006b5 100644
--- a/deps.edn
+++ b/deps.edn
@@ -35,21 +35,17 @@
   enlive/enlive {:mvn/version "1.1.6"}
 
   io.pedestal/pedestal.route {:mvn/version "0.5.3"}
-  io.pedestal/pedestal.jetty {:mvn/version "0.5.7"}
+  io.pedestal/pedestal.jetty {:git/url "" :sha "bd567123cf78d0d11b82e3bab94473c239b13382"}
   image-resizer/image-resizer {:mvn/version "0.1.10"}
 
   stylefruits/gniazdo {:mvn/version "1.0.1"}
-  io.pedestal/pedestal.log {:git/url "" :sha "5ab1c9770472694f94d0835cda412ba753b28809"}
+  io.pedestal/pedestal.log {:mvn/version "0.5.9"}
   io.pedestal/pedestal.service {:mvn/version "0.5.3"}
   io.pedestal/pedestal.service-tools {:mvn/version "0.5.3"}
 
   co.deps/ring-etag-middleware {:mvn/version "0.2.0"}
 
-  ;; pinning jetty version to avoid incompatible changes with 9.4.36.v20210114 bundled with aws-api and other libs
-  org.eclipse.jetty/jetty-proxy {:mvn/version "9.4.18.v20190429"}
-  org.eclipse.jetty/jetty-http {:mvn/version "9.4.18.v20190429"}
-  org.eclipse.jetty/jetty-util {:mvn/version "9.4.18.v20190429"}
-  org.eclipse.jetty/jetty-client {:mvn/version "9.4.18.v20190429"}
+  org.eclipse.jetty/jetty-proxy {:mvn/version "9.4.35.v20201120"}
 
   ;; pedestal logging (from project template)
   org.slf4j/jul-to-slf4j {:mvn/version "1.7.25"}

mkvlr09:06:30

oh, just saw GITLIBS_DEBUG, I’ll try that next time I run into this

mkvlr09:06:09

also I was still on 10.0.3.814 when the error first occured:

clojure -Srepro -Sforce -Sverbose -Stree                                                                                                                                          ✘ 1 
version      = 1.10.3.814
install_dir  = /usr/local/Cellar/[email protected]/1.10.3.814
config_dir   = /Users/mk/.clojure
config_paths = /usr/local/Cellar/[email protected]/1.10.3.814/deps.edn deps.edn
cache_dir    = .cpcache
cp_file      = .cpcache/2770004842.cp
Refreshing classpath
Downloading: org/eclipse/jetty/jetty-proxy/9.4.35.v20201120/jetty-proxy-9.4.35.v20201120.pom from central
Downloading: org/eclipse/jetty/jetty-project/9.4.35.v20201120/jetty-project-9.4.35.v20201120.pom from central
Error building classpath. Cannot invoke "java.io.File.isAbsolute()" because "f" is null
java.lang.NullPointerException: Cannot invoke "java.io.File.isAbsolute()" because "f" is null
	at clojure.tools.deps.alpha.util.dir$canonicalize.invokeStatic(dir.clj:30)
	at clojure.tools.deps.alpha.util.dir$canonicalize.invoke(dir.clj:25)
	at clojure.tools.deps.alpha.extensions.deps$eval1390$fn__1392.invoke(deps.clj:27)
	at clojure.lang.MultiFn.invoke(MultiFn.java:244)
	at clojure.tools.deps.alpha$expand_deps$children_task__790$fn__792$fn__793.invoke(alpha.clj:403)
	at clojure.tools.deps.alpha.util.concurrent$submit_task$task__505.invoke(concurrent.clj:34)
	at clojure.lang.AFn.call(AFn.java:18)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
	at java.base/java.lang.Thread.run(Thread.java:831)

souenzzo13:06:49

Is it a know issue? Error building classpath. class java.util.HashMap$Node cannot be cast to class java.util.HashMap$TreeNode I'm having this error sometimes on CI (stacktrace on first comment)

souenzzo13:06:58

Error building classpath. class java.util.HashMap$Node cannot be cast to class java.util.HashMap$TreeNode (java.util.HashMap$Node and java.util.HashMap$TreeNode are in module java.base of loader 'bootstrap')
java.lang.ClassCastException: class java.util.HashMap$Node cannot be cast to class java.util.HashMap$TreeNode (java.util.HashMap$Node and java.util.HashMap$TreeNode are in module java.base of loader 'bootstrap')
	at java.base/java.util.HashMap$TreeNode.moveRootToFront(HashMap.java:1882)
	at java.base/java.util.HashMap$TreeNode.putTreeVal(HashMap.java:2061)
	at java.base/java.util.HashMap.putVal(HashMap.java:633)
	at java.base/java.util.HashMap.put(HashMap.java:607)
	at java.base/java.util.HashSet.add(HashSet.java:220)
	at org.apache.maven.model.validation.DefaultModelValidator.validateId(DefaultModelValidator.java:847)
	at org.apache.maven.model.validation.DefaultModelValidator.validateEffectiveDependency(DefaultModelValidator.java:659)
	at org.apache.maven.model.validation.DefaultModelValidator.validateEffectiveDependencies(DefaultModelValidator.java:583)
	at org.apache.maven.model.validation.DefaultModelValidator.validateEffectiveModel(DefaultModelValidator.java:373)
	at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:494)
	at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:440)
	at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:430)
	at org.apache.maven.model.building.DefaultModelBuilder.importDependencyManagement(DefaultModelBuilder.java:1299)
	at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:474)
	at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:440)
	at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:430)
	at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:292)
	at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:171)
	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.readArtifactDescriptor(DefaultRepositorySystem.java:255)
	at clojure.tools.deps.alpha.extensions.maven$eval1076$fn__1078.invoke(maven.clj:107)
	at clojure.lang.MultiFn.invoke(MultiFn.java:244)
	at clojure.tools.deps.alpha$expand_deps$children_task__790$fn__792$fn__793.invoke(alpha.clj:403)
	at clojure.tools.deps.alpha.util.concurrent$submit_task$task__505.invoke(concurrent.clj:34)
	at clojure.lang.AFn.call(AFn.java:18)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

Alex Miller (Clojure team)13:06:35

not familiar, looks pretty weird

Alex Miller (Clojure team)13:06:23

some googling seems to indicate this is likely a concurrency issue

Alex Miller (Clojure team)13:06:11

could I trouble you to make an ask.clojure question for this?

souenzzo13:06:05

Collecting data to report the only "non-standard" thing that I found is the zulu VM.

Alex Miller (Clojure team)14:06:13

yeah, I don't think the jvm matters

Alex Miller (Clojure team)14:06:41

can you post the deps.edn, that was the more interesting part

souenzzo18:06:17

Can't share, but it's a huge deps.edn, with a structure like seancorfield describes

projects/subproj1
projects/subproj2
projects/subproj3
projects/subproj-common
It uses CLJ_CONFIG="$(cat projects/subproj-common)", maven deps, git deps, local deps, datomic, etc...

Alex Miller (Clojure team)19:06:56

I'm pretty sure it's related to parallel expansion of maven based projects

Alex Miller (Clojure team)19:06:14

maven meaning git or local deps with pom.xml (not maven artifacts)

v3ga13:06:34

hmm I'm doing some experimenting with organizing here and I seem to have butchered my deps.edn. The repl starts but none of my dependencies are on the classpath. Can someone point me in the right direction? https://gist.github.com/v3gal0g/ed79584f09f03ffc996b1ce389d9a3ef

thheller13:06:39

@decim :aliases {:common {:extra-paths ["src/clj" "src/cljs] there is a " missing after src/cljs

v3ga13:06:42

@thheller oops, it's actually correct there in my actual file. I made a edit to keep up with a change I made after the initial gist.

v3ga13:06:09

ironically i'm setting up shadow-cljs now and started making some adjustments to get things working... I'm sure this is just a cluster of newbie mistakes.

practicalli-johnny13:06:01

@decim All your dependencies are within aliases. There is no top level :deps {,,,,}. So unless you include an alias when running a Clojure command then there wont be any dependencies on the class path.

practicalli-johnny13:06:38

For example, to run the backend in development environment, I assume you would do something like clojure -M:common:backend:dev

practicalli-johnny14:06:13

Personally I would move all the dependencies to a :deps {,,,} section before the aliases and only use aliases for optional things or to run supporting development tools like a test runner.

practicalli-johnny14:06:39

This page and video might help we it comes to Clojure CLI projects https://practical.li/clojure/clojure-tools/projects/create.html

👍 2
v3ga14:06:48

@jr0cket OH!! That's how they work...

practicalli-johnny14:06:47

🙂 There is also a reference page on defining aliases that might be of interest https://practical.li/clojure/reference/clojure-cli-tools/defining-aliases.html

borkdude14:06:24

@jr0cket I think @thheller advices to keep backend and frontend deps separate. I was already doing this before I heard him say that

borkdude14:06:39

But I guess one can also do that when there is a conflict

practicalli-johnny14:06:01

Which is cool, so long as you include the alias names on the command line 🙂

borkdude14:06:15

the :common alias is probably not needed, you can just move that to the top level deps

borkdude14:06:27

as those are the common deps :)

v3ga14:06:27

embarrassing. =P Thank you... yeah, I see whats going on now. I got a little organization happy without knowing what I was doing.

practicalli-johnny14:06:34

Thats the best way to learn - as well ask asking for help here :)

👍 2
v3ga14:06:05

Ok, appreciate it. It looks like i'm on the right path now.