Fork me on GitHub
#clojure
<
2022-04-13
>
borkdude11:04:47

I have a library which generates .class files on disk and references a type sci.impl.types.IReified as one of the implemented interfaces in those classes. But when using one of those classes in another library, I get:

(new babashka.impl.clojure.lang.Indexed nil nil nil)
Syntax error (ClassNotFoundException) compiling new at (babashka/impl/reify2.clj:11:1).
sci.impl.types.IReified
while
user=> sci.impl.types.IReified
sci.impl.types.IReified
does exist. Could this be some classloader problem or so?

borkdude12:04:34

When I compile sci.impl.types:

user=> (compile 'sci.impl.types)
sci.impl.types
user=> (require '[babashka.impl.reify2] )
Syntax error (VerifyError) compiling new at (babashka/impl/reify2.clj:11:1).
Illegal type at constant pool entry 65 in class babashka.impl.clojure.lang.Indexed
Exception Details:
  Location:
    babashka/impl/clojure/lang/Indexed.count()I @7: invokeinterface
  Reason:
    Constant pool index 65 is invalid
  Bytecode:
    0000000: 2ab4 002c b200 26b9 0041 0200 c000 434c
    0000010: 2bc6 0011 2b2a b900 5802 00c0 0045 b600
    0000020: 5bac 2ab7 005d ac
  Stackmap Table:
    append_frame(@34,Object[#67])

eskos12:04:26

Not sure if this’ll help, but you can get the classloader chain with this:

(loop [chain []
       cl  (.getContextClassLoader (Thread/currentThread))]
  (if (nil? cl)
    chain
    (recur (conj chain cl) (.getParent cl))))
To me it looks however that you’ve unearthed something larger than just a classloader error… 😅

gerritjvv13:04:03

Clojure Dart is coming !!! I can't wait for this!!!

borkdude14:04:12

Re-phrasing https://clojurians.slack.com/archives/C03S1KBA2/p1649850767629129 in a different way: Is it possible to define a protocol in Clojure (not compiled to disk), and use that protocol in Java classes (produced by ASM as .class files, so far that worked), but then use that .class file from Clojure again? That seems to be not working. Should I AOT-compile that protocol?

Alex Miller (Clojure team)14:04:46

"use that protocol in Java classes" == what? do you mean invoking protocol functions? implementing the protocol interface?

Alex Miller (Clojure team)14:04:19

the way we handle this (if I understand any of this) in the Clojure impl is to have a Java interface that Java classes participate in, then have a protocol that extends to the interface (see CollReduce and IReduce[Init] for example)

Alex Miller (Clojure team)14:04:03

CollReduce there is for Clojure extenders and IReduce[Init] is for Java extenders

borkdude14:04:23

I have a a couple of (instance? sci.impl.types.IReified ...) calls in SCI. IReified is a protocol. I implemented the interface of that protocol in some Java classes. But if I understand correctly, there should be a Java interface at the base. Which leads me to publish another mvn artifact, but if that's what it takes :).

Alex Miller (Clojure team)14:04:43

what I'm suggesting is fixing it from the other direction

Alex Miller (Clojure team)14:04:22

create a Java interface and have your Java stuff implement it, then extend the protocol to that

borkdude14:04:42

yeah, that's what I understood too

borkdude14:04:13

I didn't want those Java classes in the first place but it was another rabbit hole I fell into due not being able to call default interface methods in reify

borkdude15:04:12

@U064X3EF3 I refuse to write another line of Java if I can do it with Clojure :) https://github.com/babashka/sci/blob/master/types/build.clj#L14-L21

didibus15:04:34

Hum..., what if you compile the same with Java and javac and then compare the compiled classes? Logically to me, if the compiled result is the same it should work.

JohnJ17:04:07

nothing wrong with writing some Java here and there when situations demands it ;)

didibus18:04:27

I can see it maybe making sense for git deps. Though I guess you could include some .class and commit them to the git repo. But .class files are a bit trickier to manage for dependencies and versions.

borkdude18:04:33

For git deps there is the prep lib thing which could invoke the same thing as I did in build.clj

didibus19:04:30

Hum... can we assume javac will always exist when Clojure will?

borkdude19:04:46

gotta assume some things when building stuff

didibus19:04:14

Well, I normally don't assume I'm going to need to build my dependencies. So I feel it's bad practice for a git deps to make assumptions about my environment in the prep step. The prep step should be self-contained and have everything it needs I feel. Though maybe javac is an exception, given you need a JVM for Clojure and I think the JRE is no more, so I think it would always have javac with it.

Alex Miller (Clojure team)20:04:56

if you can't prep the lib for the classpath without compiling then it cannot be self-contained - that is a requirement

didibus22:04:03

Sorry Alex, I'm failing to parse that sentence. Are you saying it is a requirement for prep to be self-contained, or that prep cannot be self-contained?

Alex Miller (Clojure team)22:04:51

I'm saying if you have a lib that requires compiling during preparation, then it requires compilation during preparation and is not self-contained

Alex Miller (Clojure team)22:04:42

if you want to use that lib (without an artifact), then that's just a constraint you have to accept, or not use the lib

didibus22:04:10

Ah ok. I understand nothing guarantees preps are self-contained, I'm just saying, it seems like a really bad practice as a lib author to force consumers and transient consumer to figure out how to build your lib before using it. It would seem a big step back to me as opposed to maven

didibus22:04:38

And I really hope we don't get there, where I have to lookup documentation of every transient library to figure out how to prep them.

Alex Miller (Clojure team)23:04:44

Well, you don't - that's the whole point of having an explicit published prep recipe and support in the CLI

borkdude14:04:25

It's a bit unfortunate that I'm being pulled into this mvn-artifact-based programming again

p-himik14:04:34

Well, at the very least the perceived evolution of your questions now makes me imagine you as the guy on this pic.

p-himik14:04:47

Only 3 more heads to go.

Noah Bogart16:04:29

My project (which uses leiningen) depends on clj-http. clj-http depends on potemkin. I want to use potemkin; I should depend on it myself, right? I shouldn't implicitly depend on it just because one of my existing dependencies depends on it, right?

p-himik16:04:58

Right.

👍 4
seancorfield17:04:34

If you need to control the version of potemkin independently of clj-http, with Leiningen you'll need to add it as an exclusion on clj-http I think? (in deps.edn you would not need to -- top-level deps always take precedence but lein uses a more complex, and sometimes more surprising, algorithm).

Noah Bogart17:04:38

Yeah, we already do plenty of exclusions so this surprised me. Was honestly surprised to see that it worked at all

Wanja Hentze18:04:58

> top-level deps always take precedence Is this true? In my experience, newer versions of deps can still supersede older ones even when specified down in the tree.

seancorfield18:04:30

@U033F53LU0M That's the other rule in deps.edn (not project.clj): newest version wins if there is no explicit version. So an older top-level version will win over newer versions in the tree, but otherwise the newest version in the tree is used. clojure -Stree shows the full tree and how it decides on each repeated library.

Wanja Hentze18:04:13

top-level, or "transitive parent of the one with the newer version"?

Wanja Hentze18:04:32

yes, clojure -Stree is the thing that made me arrive at this conclusion

seancorfield18:04:56

Strictly top-level.

Wanja Hentze18:04:12

Ah, that explains it!

seancorfield18:04:56

Example from work (using the "modern" CLI version of -Stree that accepts aliases):

(! 533)-> clojure -X:deps tree :aliases '[:dev :everything]'
poly/crud-form /Developer/workspace/wsmain/clojure/components/crud-form
  . com.github.seancorfield/next.jdbc 1.2.780
    . org.clojure/java.data 1.0.95
      X org.clojure/tools.logging 1.2.1 :older-version
    . camel-snake-kebab/camel-snake-kebab 0.4.2
  . exoscale/coax 1.0.0-alpha19
  . phrase/phrase 0.3-alpha4
...
worldsingles/newrelic /Developer/workspace/wsmain/clojure/newrelic
  . com.newrelic.agent.java/newrelic-api 7.6.0
  . com.newrelic.telemetry/telemetry-core 0.13.1
    X org.slf4j/slf4j-api 1.7.30 :superseded
  . com.newrelic.telemetry/telemetry-http-okhttp 0.13.1
    . com.newrelic.telemetry/telemetry-core 0.13.1
    . com.squareup.okhttp3/okhttp 4.8.0
      . com.squareup.okio/okio 2.7.0 :newer-version
        X org.jetbrains.kotlin/kotlin-stdlib 1.3.70 :older-version
        X org.jetbrains.kotlin/kotlin-stdlib-common 1.3.70 :older-version
      X org.jetbrains.kotlin/kotlin-stdlib 1.3.72 :older-version
  . org.jetbrains.kotlin/kotlin-stdlib 1.6.20
    . org.jetbrains.kotlin/kotlin-stdlib-common 1.6.20
    . org.jetbrains/annotations 13.0
  . org.clojure/data.json 2.4.0
  . org.clojure/tools.logging 1.2.4
...

Wanja Hentze18:04:43

But I found this also quite surprising myself. Say you have (in the ASCII art style of https://clojure.org/reference/dep_expansion):

A
  B
    C 1.0.0
    D
      C 1.0.2
Does that mean we resolve C to 1.0.2?

Wanja Hentze18:04:20

But when the developers of B work on their project, they get 1.0.0?

seancorfield18:04:39

No, because B would be packaged as a JAR with a pom.xml that "locks" the dependencies it used.

p-himik18:04:14

What if it's a Git dep?

seancorfield18:04:18

I'd have to double-check how that works with git deps since I suspect they're treated like source deps...

seancorfield18:04:42

Here's an example of a lib at work where it starts with one version and discards it as it finds newer versions, then rejects later deps when they are older:

(! 535)-> clojure -X:deps tree :aliases '[:dev :everything]'|fgrep slf4j-api
  X org.slf4j/slf4j-api 1.7.25 :superseded
    X org.slf4j/slf4j-api 1.7.30 :superseded
    X org.slf4j/slf4j-api 1.7.32 :superseded
      X org.slf4j/slf4j-api 1.7.13 :older-version
    X org.slf4j/slf4j-api 1.7.30 :older-version
    . org.slf4j/slf4j-api 2.0.0-alpha1 :newer-version
      X org.slf4j/slf4j-api 1.7.22 :older-version
    X org.slf4j/slf4j-api 1.7.21 :older-version
      X org.slf4j/slf4j-api 1.7.7 :older-version
    X org.slf4j/slf4j-api 1.7.7 :older-version
    X org.slf4j/slf4j-api 1.7.7 :older-version
    X org.slf4j/slf4j-api 1.7.7 :older-version
    X org.slf4j/slf4j-api 1.7.7 :older-version
    X org.slf4j/slf4j-api 1.7.7 :older-version
    X org.slf4j/slf4j-api 1.7.7 :older-version
    X org.slf4j/slf4j-api 1.7.7 :older-version
(that is a bit of a surprise to me -- I didn't know we had a dependency bringing in that new alpha)

seancorfield18:04:30

I would hope that a deps.edn in a git dep would resolve based on what is top-level in that deps.edn file but I'd have to test that to be sure.

Wanja Hentze18:04:48

though all the clojure gitlibs I've seen have the former

seancorfield18:04:57

Right. And pom.xml is a "flat" set of fixed dependencies.

seancorfield19:04:59

Overall, I think the deps.edn rules are simpler and produce fewer conflicts and surprises.

Wanja Hentze19:04:08

Ah, I went back and looked at that actual -Stree output. The situation is more like

A
  B (:local)
    X 0.0.8 (:mvn)
  C (:git)
    D (:mvn)
      E (:mvn)
        X 0.2.0 (:mvn)
And t.d.a. resolves it to 0.2.0

Wanja Hentze19:04:25

What does it mean for a pom file to "lock" its deps' versions? i.e. if two poms in my tree are in conflict, does t.d.a. error out?

didibus19:04:27

Now I'm confused with this "locking" of pom.xml. I would have assumed that in the scenario described, 1.0.2 would be used even though B uses 1.0.0

didibus19:04:39

Or are you saying the POM xml is already flattened, so when you look inside the POM of B it already contains all the deps that C depends on? But even then, you'd then grab C and be able to see it's POM and see that if it has anything newer you could resolve to that no?

seancorfield20:04:59

I think I was reading too much into how the POM is produced -- but for the case I thought @U033F53LU0M was asking about, I expected the pom.xml for B to have X 0.0.8 and D a.b.c (based on the earlier ascii tree`) but if D depends on X 0.2.0 it's still going to come in as a transitive dep and be considered anyway I think, right?

seancorfield20:04:29

So you're still at the mercy of lein's tree-walking algorithm (which I should go read up on again) vs t.d.a. "prefer top or else prefer newest" strategy.

didibus22:04:41

For deps, my understanding was newest wins unless the application, aka, top top level explicitly has a different version. So the depth at which the newest version shows up doesn't matter, and it doesn't matter if something higher up uses an older version, the newest would win. At least, I hope this is the case, it makes most sense to me and seems easiest to deal with. Everything is latest, and if your app doesn't work with that, you pick what to pin down that is older or even newer.

1
Noah Bogart20:04:44

is there a way to make (set! clojure.test.check.clojure-test/*report-completion* false) work? i don't want the {:result true, :num-tests 100, :seed , :time-elapsed-ms 75, :test-var "example"} in my test output

Noah Bogart20:04:47

the error I'm seeing, which I forgot to post above: Can't change/establish root binding of: *report-completion* with set

Alex Miller (Clojure team)20:04:16

you can only set! something that has been thread bound at a higher stack level, so you would need to bind around this

Alex Miller (Clojure team)20:04:37

I'm not sure what your context is and whether you have the opportunity to do so (probably not)

Noah Bogart20:04:30

The context is I have a test file with some defspecs. I thought I could put that at the top of the file to disable it during the run. I'm using leiningen's test runner if that matters

hiredman20:04:27

You can use a fixture to wrap the running of the tests in a binding

Noah Bogart20:04:58

oh good call. that's probably better than using alter-var-root which works but feels fragile