Clojurians Log v2
leiningen
Hi there!
we are having some issues with the content that ends up in the final .jar file, when running lein uberjar. In particular, there is a lot of content in that .jar file that we don't expect to be there. Like some files that belong to dependencies that we think should only be used with the development profile (e.g., some clj-kondo files).
But also, and this is the core of our problem, some Java libraries are there, that we don't know here they are coming from. Those Java libraries don't appear anywhere when we run lein deps :tree, or if we explicitly ask about their "provenance" with lein deps :why xxx.
The Java libraries in question are com.google.guava/guava, and com.google.code.gson/gson.
The reason why we want to know where they are coming from, and why they are being included in the uberjar is that the versions that are included in the uberjar have some CVEs rated "high"[1]. And we would like to either remove them from the uberjar if they are not really needed. Or to use higher versions that have those CVEs fixed.
So our questions are:
• Is there a way to get the full dependency tree, as computed by lein uberjar(as opposed to what we get with lein deps)
• Is there a way to tell Leiningen to use higher versions of those libraries? We tried specifically adding them as top level dependencies (with the CVE fixed versions) in the :dependencies key in project.clj. But they were completely ignored.
P.D. We are using Leiningen 2.11.2, in case that makes a difference.
Thanks a lot in advance!
[1] The versios shipped in the ubjerar, that have the CVEs, are 31.0.1-jre (for com.google.guava/guava) and 2.7 for com.google.code.gson/gson ), in case they ring a bell to someone.
First of all, thanks a lot to all of you that provided very valuable answers to my question! Unfortunately, this time, lein with-profile uberjar deps :tree didn't provide any extra information that could help us.
But Alex Miller's trick gave us the key info:
dev> ( "com/google/gson/Gson.class")
#object[java.net.URL 0x2bd11a5 "jar:file:/home/hop/.m2/repository/com/google/javascript/closure-compiler-unshaded/v20220502/closure-compiler-unshaded-v20220502.jar!/com/google/gson/Gson.class"]
dev> lein with-profile uberjar deps :tree):
....
....
[org.clojure/clojurescript "1.11.132"]
[com.cognitect/transit-java "1.0.362" :exclusions [[org.clojure/clojure]]]
[javax.xml.bind/jaxb-api "2.3.0"]
[org.msgpack/msgpack "0.6.12"]
[com.googlecode.json-simple/json-simple "1.1.1" :exclusions [[junit]]]
[org.javassist/javassist "3.18.1-GA"]
[com.google.javascript/closure-compiler-unshaded "v20220502"]
[org.clojure/google-closure-library "0.0-20230227-c7c0a541"]
[org.clojure/google-closure-library-third-party "0.0-20230227-c7c0a541"]
[org.clojure/tools.reader "1.3.6"]
....
....com.google.javascript/closure-compiler-unshaded[1], so upgrading to ClojureScript 1.12 wouldn't fix this issue (in addition to potentially breaking our SPA, etc.).
Thus, using :managed-dependencies, as @pikariop suggested, seems to be our only viable option at the moment.
@alexmiller, given that the version shipped with ClojureScript (all 1.10.x, 1.11.x and 1.12.x) ship that vulnerable version of com.google.gson/gson, would it make sense to upgrade to a more recent version of com.google.javascript/closure-compiler-unshaded?
[1] https://github.com/clojure/clojurescript/blob/26dea31110502ae718399868ad7385ffb8efe6f6/project.clj#L18What surprises me a bit is that that dependency in particular (we still need to find out where the com.google.guava/guava one is coming from) is supposed to be only needed during the ClojureScript compilation step. And shouldn't be needed after that. Which means that it shouldn't be part of the final uberjar, should it?
Yes, maybe you could mark it as compile scope or put it in a profile, not sure what's common. Can you drop the cljs question in #cljs-dev for David Nolen to consider?
Sure, I'll try to make a detailed report of our findings and post the question there.
Hummm, maybe I have misunderstood the whole thing?. Upon further investigation, the problematic dependencies, flagged by the vulnerability scanner, are not the ones that have .class files in the uberjar.
It seems the scanner is finding the vulnerable versions details in META-INF/maven/com.google.code.gson/gson/pom.properties and META-INF/maven/com.google.guava/guava/pom.properties.
And even if I explicitly added:
:managed-dependencies [[com.google.code.gson/gson "2.8.9"]
[com.google.guava/guava "32.0.0-jre"]]A final note on this (in this channel), in case this is useful for other people in the future 😅
Adding the following line in project.clj finally made Leiningen (or maven, or whatever is in charge of building the dependency graph for the uberjar) use an updated version of the "problematic" libraries:
:managed-dependencies [[com.google.javascript/closure-compiler-unshaded "v20231112"]]com.google.javascript/closure-compiler-unshaded library that had those problematic dependencies with "recent enough" versions to avoid the CVEs. We tried the latest version first (v2026xxyy), but the ClojureScript compilation failed with a nice exception about a missing class or interface.
So we played the conservative card, and did the opposite (use the oldest one, with those CVEs fixed).
Because it turned out to be the Clojurescript compiler, I wonder if you could stick it into a separate profile or alias like Alex suggested, so it wouldn't end up in the uberjar in the first place, unless of course you do actually need it there for some reason
normally I would say to report a bug with the library in question when this happens
but when the maintainer is google, that's not likely to be fruitful =(
David forked this into the clojure org and took over maintenance for us, so actually is fixable
this could be happening if one of the dependencies is actually itself an uberjar including dependencies. that's usually bad on the part of a library, but it happens. you could try finding that by starting your repl and then searching for one of those Java resources via clojure.java.io/resource on a .class file you see in your uberjar
such as:
user=> ( "clojure/lang/Compiler.class")
#object[java.net.URL 0x78411116 "jar:file:/Users/alex.miller/.m2/repository/org/clojure/clojure/1.12.4/clojure-1.12.4.jar!/clojure/lang/Compiler.class"] you might find something surprising for guava or gson though
@alexmiller Thanks a lot for the tip! We will definitely follow that investigation path!
We would have never thought of that 😅
To the second question, you can define a more current version of a transitive dependency with https://codeberg.org/leiningen/leiningen/src/branch/stable/doc/MANAGED_DEPS.md
Of course this won't work if the dependency is embedded in another
technically possible they could be coming from an :uberjar profile too, (try lein with-profile uberjar deps :tree) but Alex's suggestion is a lot more likely, that a dependency is badly behaved. hard to say more without a repro case.