This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-06-03
Channels
- # babashka (17)
- # beginners (166)
- # calva (97)
- # cider (4)
- # clara (2)
- # clj-kondo (46)
- # cljsrn (5)
- # clojure (334)
- # clojure-canada (1)
- # clojure-dev (144)
- # clojure-europe (14)
- # clojure-germany (5)
- # clojure-nl (10)
- # clojure-spec (1)
- # clojure-uk (46)
- # clojurescript (50)
- # conjure (1)
- # core-async (52)
- # core-typed (5)
- # cursive (3)
- # datomic (3)
- # emacs (11)
- # figwheel (16)
- # figwheel-main (9)
- # fulcro (29)
- # graalvm (19)
- # graphql (14)
- # helix (46)
- # hoplon (4)
- # hugsql (2)
- # jobs (2)
- # jobs-discuss (1)
- # juxt (15)
- # kaocha (6)
- # off-topic (9)
- # pedestal (7)
- # portkey (7)
- # re-frame (10)
- # reagent (29)
- # shadow-cljs (13)
- # spacemacs (70)
- # sql (13)
- # tools-deps (26)
- # xtdb (23)
Tried this out in both java8 and java11 with clojure 1.10.1 and [clj-commons/pomegranate “1.2.0”] - and find the behavior hard to explain - in the repl -
using lein 2.9.3
. Wonder if anyone has any deeper insight on the classloader trickery happening:
----
These DO NOT work. hickory isn’t found on classpath.
(1)
(p/add-dependencies
:coordinates '[[hickory "0.7.1"]]
:repositories (merge {"clojars" " "}
pa/maven-central))
(require 'hickory.core)
(2)
(p/add-dependencies
:classloader (clojure.lang.RT/baseLoader)
:coordinates '[[hickory "0.7.1"]]
:repositories (merge {"clojars" " "}
pa/maven-central))
(require 'hickory.core)
(3)
(def cl (clojure.lang.RT/baseLoader))
(p/add-dependencies
:classloader cl
:coordinates '[[hickory "0.7.1"]]
:repositories (merge {"clojars" " "}
pa/maven-central))
(require 'hickory.core)
(4)
(let [cl (clojure.lang.RT/baseLoader)]
(p/add-dependencies
:classloader cl
:coordinates '[[hickory "0.7.1"]]
:repositories (merge {"clojars" " "}
pa/maven-central)))
;; NOTE: NOT INSIDE the `let`
(require 'hickory.core)
However, this DOES work:
(5)
(let [cl (clojure.lang.RT/baseLoader)]
(p/add-dependencies
:classloader cl
:coordinates '[[hickory "0.7.1"]]
:repositories (merge {"clojars" " "}
pa/maven-central))
;; NOTE: INSIDE the `let`.
(require 'hickory.core))
Also, after (5) happens in the repl, you can just use the new dependencies with no further let-bindings or references to that classloader. I think I somewhat understand why at this point since the dynamic classloaders are likely just sharing state of these clj loaded classes.it’s pretty weird to me. The case where the classloader is in a let-binding seems to matter. It makes it “used by require
”
I forget the exact details but when evaluating/compiling code the compiler will often push a new dynamic classloader binding, which is sometimes what RT/baseLoader returns
so my guess is what you are seeing is differences in behavior based on if the add-dependencies expression is compiled and run with the same classloader on that stack as the require expression
not 100% sure, wrapping in a let like that will make the two part of the same compilation unit, without all the explicit classloader stuff, so if that fixes the issue it is likely what is happening
the issue isn't likely with what you are explicitly passing to pomegranate, but what the compiler is implicitly doing behind the scenes
I've had to fiddle with requires with pomegranate in the past(the behavior changed around clojure 1.8 I think), but never dug into why
eg:
(let []
(p/add-dependencies
:classloader (clojure.lang.RT/baseLoader)
:coordinates '[[hickory "0.7.1"]]
:repositories (merge {"clojars" " "}
pa/maven-central))
(require 'hickory.core))
and
(p/add-dependencies
:classloader (clojure.lang.RT/makeClassLoader)
:coordinates '[[hickory "0.7.1"]]
:repositories (merge {"clojars" " "}
pa/maven-central))
(require 'hickory.core)
does not work; as you assumed above - I’m still not sure I can reason why it doesn’t workSo basically, if we can get “loading” (eg. require
) to happen in the same compilation unit as using clojure.lang.RT/baseLoader to add the deps; we’ll be dealing using that same mutated loader from pomegranate
I think once we do the load; it is safe to just use the stuff after that point within the repl
meaning, I don’t know of a reason I’d need to hold onto the loader and keep binding it
(def cl (clojure.lang.RT/makeClassLoader))
(with-bindings {clojure.lang.Compiler/LOADER cl}
(p/add-dependencies
:classloader cl
:coordinates '[[hickory "0.7.1"]]
:repositories (merge {"clojars" " "}
pa/maven-central))
(require 'hickory.core))
then again, using with-bindings
may be making them both in the same compilation unit still
so probably should break all the way into push thread bindings around multiple separate forms to see it in most raw
(def cl (clojure.lang.RT/makeClassLoader))
(with-bindings {}
(p/add-dependencies
:classloader cl
:coordinates '[[hickory "0.7.1"]]
:repositories (merge {"clojars" " "}
pa/maven-central))
(require 'hickory.core))
(with-bindings {}
(p/add-dependencies
:coordinates '[[hickory "0.7.1"]]
:repositories (merge {"clojars" " "}
pa/maven-central))
(require 'hickory.core))
(let []
(p/add-dependencies
:coordinates '[[hickory "0.7.1"]]
:repositories (merge {"clojars" " "}
pa/maven-central))
(require 'hickory.core))
doesn’t work though@mikerod This seems to be all accidental complexity introduced by pomegranate -- using add-lib
for this sort of thing "just works".
@seancorfield add-lib
being an unreleased fn right?
(! 504)-> clj -A:deps -r
user=> (require '[clojure.tools.deps.alpha.repl :refer [add-lib]])
nil
user=> (add-lib 'hickory {:mvn/version "0.7.1"})
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
true
user=> (require 'hickory.core)
nil
user=>
:deps ; to get access to clojure.tools.deps.alpha.repl/add-lib
{:extra-deps {org.clojure/tools.deps.alpha
{:git/url ""
:sha "19d197ab221d37db750423eb970880cb87a91100"}}
@seancorfield they use similar strategys. This doesn't just work with add-lib when using lein/add-lib
just trying to add to a classloader when it has the ability to be added to - via a protocol
but you are somewhat relying on the fact that clj require
(`load`) will use a RT/baseLoader
So the problem here is... what exactly? I'm confused. You show something related to Leiningen / Pomegranate that doesn't work in situations you think it should... and that's a bug/problem in... what? I'm not following...
But why does lein have particular issues with LOADER. I couldn't get it to trigger under clojure.
Right, this is feeling like a lein
/CIDER problem, not a Clojure problem.
so if you use something like pomegranate incorreclty - you may mutate the wrong loader
and not really have ability to use that loader for something like require
subsequently
So that's... a bug in lein
?
(sorry, I'm not trying to be difficult, I'm just trying to see how this relates to #clojure-dev )
I wanted to understand the underlying mechanism for why a given compile-context seemed to change my result
you can have different repl impl’s some may or may not end up the same - depends on how they manage classloaders etc
I’m not saying there is even a problem, was just baffled by how let
seemingly caused a difference for no reason
static public ClassLoader baseLoader(){
if(Compiler.LOADER.isBound())
return (ClassLoader) Compiler.LOADER.deref();
else if(booleanCast(USE_CONTEXT_CLASSLOADER.deref()))
return Thread.currentThread().getContextClassLoader();
return Compiler.class.getClassLoader();
}
so if you do some mutation to it - via pomegranate
and also end up doing a load
while still in that same eval - you are working against same loader at the right time
likely the reason lein's repl behaves different is the clojure.main/repl sets the context class loader to be a dynamic classloader
don’t have to rely on let
though - you can just use with-bindings
; you also don’t even need LOADER; you can set the USE_CONTEXT_CLASSLOADER and achieve similar
https://github.com/nrepl/nrepl/commit/c3ec4c699e4c9ac439fda822daeb0dda17370c9a was merged recently
FWIW, of your original five cases, (1) does work in the regular clj
REPL (and it also worked for me in a plain lein repl
) (2), (3), and (4) do not work, and (5) does work. -- @mikerod Does (1) not work for you in lein repl
?
and it does have middleware setting the ccl and LOADER etc - so things are being done
If (1) works for you in lein repl
, then I'm much clearer about what you're asking -- which is about the (2), (3), and (4) cases which don't work in either REPL and so that points to an isolated class loader issue (am I close / caught up now?)
@seancorfield all above was for lein repl
so whatever is happening in the repl
there - which really relates to nrepl
- didn’t end up sharing the loader across forms
Ah, so you have some middleware/plugin that is breaking (1) in lein repl
(as well as the three common cases)...
And again: I posted the examples trying to just get an idea of what mechanism was causing the the weird difference I saw when using let
.
good news is - it makes sense how to make pomegranate
work now in lein repl’s (to me at least)
I haven't used lein
for much of anything since late 2015, so I was just very puzzled when I tried your example (1) in a plain lein repl
and it worked... hence my confusion! 🙂
(! 513)-> lein version
Leiningen 2.9.3 on Java 14 OpenJDK 64-Bit Server VM
(! 514)-> lein repl
nREPL server started on port 52427 on host 127.0.0.1 -
REPL-y 0.4.4, nREPL 0.6.0
Clojure 1.10.0
OpenJDK 64-Bit Server VM 14+36-1461
Just so it's clear what version(s) I tried it on. No project.clj
present.Same as me @seancorfield besides I tried java8 & 11 (but those shouldn’t change the loader setup of the nrepl server)
@dominicm interesting; I’ll have to see - the current repl still can work as is - just have to be more careful about things, but would be nice to not have to
not for this case:
(p/add-dependencies
:coordinates '[[hickory "0.7.1"]]
:repositories (merge {"clojars" " "}
pa/maven-central))
(require 'hickory.core)
with that version of leinoh, guess I somewhat cut off too [cemerick.pomegranate.aether :as pa]
for the maven repo