This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-07-14
Channels
- # aleph (3)
- # announcements (1)
- # babashka (36)
- # babashka-sci-dev (4)
- # beginners (62)
- # biff (2)
- # calva (13)
- # cider (4)
- # clj-kondo (6)
- # cljdoc (17)
- # clojure (142)
- # clojure-dev (6)
- # clojure-europe (62)
- # clojurescript (20)
- # core-async (26)
- # cursive (18)
- # data-oriented-programming (9)
- # data-science (1)
- # datahike (18)
- # events (4)
- # fulcro (4)
- # graalvm (2)
- # hyperfiddle (15)
- # interop (1)
- # jobs-discuss (8)
- # leiningen (2)
- # lsp (91)
- # malli (1)
- # missionary (11)
- # nbb (65)
- # off-topic (50)
- # practicalli (2)
- # programming-beginners (4)
- # re-frame (18)
- # remote-jobs (1)
- # shadow-cljs (53)
- # spacemacs (1)
- # specter (2)
- # sql (17)
- # tools-build (63)
- # web-security (1)
- # xtdb (15)
Hello, I've noticed an issue when using the compile-clj
function and I'm wondering if it's already known? Namespaces in user.clj
(if you mistakenly have it in your classpath) get loaded before compiling starts, seems they get cached in *loaded-libs*
, then when compiling the main namespace, if those same namespaces are required, they never get loaded and no .class
file is created for them.
I can fix this issue by either remove user.clj
from the classpath
;; or by adding this binding to the compile script in write-compile-script!
#'clojure.core/*loaded-libs* `(ref (sorted-set))
ye looks like a similar issue, here clojure.main
seems to cause the user
ns to load.
if you load the clojure runtime and a http://user.clj is on the classpath it will be loaded
https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/RT.java#L479-L486
ah cool
why not clear the loaded-libs
cache in the compile script?
clearing loaded-libs will allow require to load things again, effectively forcing a reload-all, but that doesn't clear out the existing state of those namespaces before hand
the most reliable thing is a nice clean state, fresh clojure process with nothing loaded
thanks for the info, I'll be sure to keep user.clj
out of my classpath when compiling 😬
yea, seems the compile-clj
is trying it's best to do that
there is this kind of thing too https://clojurians.slack.com/archives/C03S1KBA2/p1656690868855769
I’m trying to understand :parents
and :dependents
in a basis map. My ultimate goal is to get the transitive closure of jars required by a jar: in this case 'com.google.cloud.sql/postgres-socket-factory
.
I’m a bit surprised that the lib itself doesn’t point to things it depends on :
classpath=> (get-in basis [:libs 'com.google.cloud.sql/postgres-socket-factory])
{:mvn/version "1.6.0",
:deps/manifest :mvn,
:parents #{[]},
:paths
["/Users/dan/.m2/repository/com/google/cloud/sql/postgres-socket-factory/1.6.0/postgres-socket-factory-1.6.0.jar"]}
But several other libs point back at it, even though there seem to be parent
and descendants
keys which makes me think the graph should have edges on both sidesfor example
com.google.oauth-client/google-oauth-client
{:mvn/version "1.33.2",
:deps/manifest :mvn,
:dependents [com.google.api-client/google-api-client],
:parents
#{[com.google.cloud.sql/postgres-socket-factory
com.google.cloud.sql/jdbc-socket-factory-core
com.google.api-client/google-api-client]},
:paths
["/Users/dan/.m2/repository/com/google/oauth-client/google-oauth-client/1.33.2/google-oauth-client-1.33.2.jar"]}
Here google-oauth-client
has a parent relationship to the postgres-socket-factory
but the inverse of that doesn’t seem to be presentand the docstring of create-basis
states that
> :libs - lib map, per resolve-deps
but resolve-deps
doesn’t explain this relationship so I’m a bit lost
:dependents are things that depend on this lib
:parents should be considered implementation details right now
well, it's a graph
and it's somewhat tricky in that what is reflected in the final lib set may not correspond to what was in the original dep expansion
due to version selection, exclusions, etc
sure. i meant traverse by linear searching for dependents rather than direct access by parent (there’s no index for the direction i’m starting from)
you're trying to answer a question whose answer is not necessarily in the data you are looking at
oh that’s surprising to me. I guess this information is not available anywhere then? I can resort to getting (System/getProperty "java.class.path")
from a repl started with clj -Sdeps {the-lib}
. Is there no other way?
you can probably extract something close to what you by doing what you're doing (and that's what old versions of -Stree did - that code is still in tools.deps.alpha). The code in tools.deps.alpha.tree is probably better but really requires taking a trace of the dep expansion so duplicates the work done to calculate the basis
depending on whether you care, you could just call calc-trace
then trace->tree
to get an actual tree (specifically, the tree you see now with -Stree or -X:deps tree) with a lot of data (probably way more data than you want)
i’m ok with data. Ideally just want jars because i need to find out if i have duplicate classfiles on the classpath. So I just need an authoritative way to enumerate jars, then enumerate their files, and then intersect those
what you just said doesn't seem to require what you're asking for?
why not just do that directly off the classpath?
is that because you don't have the classpath, you're doing it from deps?
because we load other jars at runtime for extension. and adding the postgres-socket-factory to the main app breaks lazily loading the bigquery driver.
well rather than using the lib map, you could grab the classpath from the basis
specifically :classpath-roots
from the basis
good point. It would be a little overinclusive (many more jars than what is brought in by the socket factory) but won’t be underinclusive. Thanks
I think if you find the conflict, should be easy to understand why there is one
and btw, I'd love to have this packaged as a tool I could run somehow :) people need this all the time
I just randomly looked for duplicated files and was surprised to see we had so many. Somehow have two copies of batik stuff for images
that batik stuff is a mess
they have some jars with circular dependencies
ask me how I know
broke tools.deps a long while back
i can’t imagine the edge cases it takes to get nice new tooling to play with every dark corner of the java ecosystem
such dark corners :)
and still headscratching. The jars are completely disjoint. We were pretty sure we had two different copies of com.google.cloud.ServiceOption
on the classpath under different classloaders. But doesn’t seem to be the case
the same class loaded in different classloaders are two different classes
that is, class identity is based on classloader+name
I'm not sure I said that is the most understandable way so let me know if that doesn't make sense
I’m very handwavy right now. We seem to have found the offending class but somehow the longer I spend on this the less I feel i know where solid ground is 🙂
if classloader A loads class C, and classloader B loads class C, then instances of B are not instances of A
(C is the identical class on disk)
in general, the way to avoid this is to always have A or B in a parent-child relationship where the parent loads C