Clojurians Log v2

I can fix this issue by either remove user.clj from the classpath

;; or by adding this binding to the compile script in write-compile-script!
#'clojure.core/*loaded-libs* `(ref (sorted-set))

there’s a great thread touching on this in #clojure i think

https://clojurians.slack.com/archives/C03S1KBA2/p1657817988136859

ye looks like a similar issue, here clojure.main seems to cause the user ns to load.

it is anything

if you load the clojure runtime and a http://user.clj is on the classpath it will be loaded

https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/RT.java#L479-L486

ah cool

why not clear the loaded-libs cache in the compile script?

loaded-libs isn't everything

clearing loaded-libs will allow require to load things again, effectively forcing a reload-all, but that doesn't clear out the existing state of those namespaces before hand

all kinds of stuff can bleed over and cause further issues

ok

the most reliable thing is a nice clean state, fresh clojure process with nothing loaded

thanks for the info, I'll be sure to keep user.clj out of my classpath when compiling 😬

yea, seems the compile-clj is trying it's best to do that

there is this kind of thing too https://clojurians.slack.com/archives/C03S1KBA2/p1656690868855769

I’m a bit surprised that the lib itself doesn’t point to things it depends on :

classpath=> (get-in basis [:libs 'com.google.cloud.sql/postgres-socket-factory])
{:mvn/version "1.6.0",
 :deps/manifest :mvn,
 :parents #{[]},
 :paths
 ["/Users/dan/.m2/repository/com/google/cloud/sql/postgres-socket-factory/1.6.0/postgres-socket-factory-1.6.0.jar"]}

for example

com.google.oauth-client/google-oauth-client
 {:mvn/version "1.33.2",
  :deps/manifest :mvn,
  :dependents [com.google.api-client/google-api-client],
  :parents
  #{[com.google.cloud.sql/postgres-socket-factory
     com.google.cloud.sql/jdbc-socket-factory-core
     com.google.api-client/google-api-client]},
  :paths
  ["/Users/dan/.m2/repository/com/google/oauth-client/google-oauth-client/1.33.2/google-oauth-client-1.33.2.jar"]}

and the docstring of create-basis states that > :libs - lib map, per resolve-deps but resolve-deps doesn’t explain this relationship so I’m a bit lost

:dependents are things that depend on this lib

:parents should be considered implementation details right now

ah ok. So i can traverse bottom up but not top down (at least for the moment)

well, it's a graph

and it's somewhat tricky in that what is reflected in the final lib set may not correspond to what was in the original dep expansion

due to version selection, exclusions, etc

sure. i meant traverse by linear searching for dependents rather than direct access by parent (there’s no index for the direction i’m starting from)

you're trying to answer a question whose answer is not necessarily in the data you are looking at

oh that’s surprising to me. I guess this information is not available anywhere then? I can resort to getting (System/getProperty "java.class.path") from a repl started with clj -Sdeps {the-lib}. Is there no other way?

you can probably extract something close to what you by doing what you're doing (and that's what old versions of -Stree did - that code is still in tools.deps.alpha). The code in tools.deps.alpha.tree is probably better but really requires taking a trace of the dep expansion so duplicates the work done to calculate the basis

thanks. i’ll look at that. many thanks for answering questions

depending on whether you care, you could just call calc-trace then trace->tree to get an actual tree (specifically, the tree you see now with -Stree or -X:deps tree) with a lot of data (probably way more data than you want)

i’m ok with data. Ideally just want jars because i need to find out if i have duplicate classfiles on the classpath. So I just need an authoritative way to enumerate jars, then enumerate their files, and then intersect those

what you just said doesn't seem to require what you're asking for?

why not just do that directly off the classpath?

is that because you don't have the classpath, you're doing it from deps?

because we load other jars at runtime for extension. and adding the postgres-socket-factory to the main app breaks lazily loading the bigquery driver.

well rather than using the lib map, you could grab the classpath from the basis

specifically :classpath-roots from the basis

good point. It would be a little overinclusive (many more jars than what is brought in by the socket factory) but won’t be underinclusive. Thanks

I think if you find the conflict, should be easy to understand why there is one

agree. Thank you very much

and btw, I'd love to have this packaged as a tool I could run somehow :) people need this all the time

I just randomly looked for duplicated files and was surprised to see we had so many. Somehow have two copies of batik stuff for images

that batik stuff is a mess

they have some jars with circular dependencies

ouch

ask me how I know

:)

i’m assuming some very long days at work

broke tools.deps a long while back

i can’t imagine the edge cases it takes to get nice new tooling to play with every dark corner of the java ecosystem

such dark corners :)

and still headscratching. The jars are completely disjoint. We were pretty sure we had two different copies of com.google.cloud.ServiceOption on the classpath under different classloaders. But doesn’t seem to be the case

the same class loaded in different classloaders are two different classes

that is, class identity is based on classloader+name

I'm not sure I said that is the most understandable way so let me know if that doesn't make sense

I’m very handwavy right now. We seem to have found the offending class but somehow the longer I spend on this the less I feel i know where solid ground is 🙂

if classloader A loads class C, and classloader B loads class C, then instances of B are not instances of A

(C is the identical class on disk)

in general, the way to avoid this is to always have A or B in a parent-child relationship where the parent loads C