Fork me on GitHub

I think the issue crinklywrappr was suggesting as a potential issue is a fractured/split eco-system - imagine if some lib authors for non-CLI packages begin relying on deps.edn gitlibs and only make their packages available in a similar manner - now those who have historically relied on maven/clojars/lein may not have an easy avenue to participating in said lib's usage without going through their own hoops to make accessible?


A coworker approached this from another angle I hadn't thought of. What if you want to publish your library to clojars, but need a dependency which isn't in a maven repo? Do you give up or fork the dep and pray you don't have to go further down the stack? It's not hard to see how this encourages movement away from maven.


> What if you want to publish your library to clojars, but need a dependency which isn't in a maven repo? This is already true in the Java world if you s/clojars/maven central/


The current solution is usually to bring the dependency directly into your project as shaded source code.


(in both Java and Clojure worlds)


Thanks Sean. I didn't know that.


Also, Leiningen has at least one plugin that has supported git dependencies (separate from Maven-style deps) for many years:

šŸ‘ 1

That puts me a little at ease


While I understand the reasoning for deps not looking inside jar files from maven for git deps, I'm not sure it would be a bad idea for that type of functionality to exist. Putting libraries as jars on clojars has other benefits besides just making the library available to leiningen users: ā€¢ clojars maintains a list of available libraries and versions ā€¢ clojars maintains stats on downloads which can be a useful metric for evaluating libraries ā€¢ packaging your dep in a jar allows you to choose which files belong or don't belong in the shipped library ā€¢ procuring a packaged jar allows you to avoid downloading unnecessary files ā€¢ clojars integrates with other infrastructure like cljdoc

Cora (she/her)01:03:45

I definitely use number of downloads to get an idea of popularity, and not just in clojars but npm and rubygems, too


Given that the CLI defers to Maven libraries to deal with the whole JAR/pom/dependencies thing, I suspect it would be non-trivial to additionally pull deps.edn from a downloaded JAR file and then re-run dependency analysis and fetching on that... and what if the dependency versions in pom.xml and deps.edn differ? That's probably the case for some of the Contrib libraries since pom.xml is the "system of record" for those, since they are tested/built via mvn. Some of them have project.clj files that are outdated since those were added as a "developer convenience" for the maintainer -- and the same is true of any Contrib libs that have deps.edn (as well as pom.xml).


(This discussion should probably move to #tools-deps since going deeper really belongs there)


I'm suspect that someone's seen something like this and can point me in the right direction. I'm using a Java class, Foodspot that I defined with deftype. (I need it in order to pass it to a Java class in a library.) I'm getting this error: "forage.mason.foodspot.Foodspot cannot be cast to forage.mason.foodspot.Foodspot". Huh?


Specifically I'm passing a Foodspot to a Clojure function. Ah, and when I remove the type hint ^Foodspot from the argument to the function, the error goes away. But that's still puzzling. And it's worked with the type hint in other cases.


Java classes are not entirely identified by name


They are identified by name and, uh, I guess defining class loader, so you can have multiple classes with the same name that are not the same, which results in errors like that


And every time you evaluate a given deftype it makes a new class with that name, so you easily have multiple classes with the same name if you are fiddling around in the repl


The type hint is referring to whatever the most recent class with that name was when the type hinted code was compiled


Oh, interesting. Thanks @hiredman. I did reload that source file. It sounds like if I recreate all of the deftype objects after I reload the source file containing that function as well as the deftype macro call, the error shouldn't occur.


I want to read the package name from a .class file without using external dependencies. Does anyone have an example of how this can be done? This should benefit clj-kondo analysis. I don't want to shell out to an external program either (too slow).


I guess I could just take on a dependency on ASM...


Seems like you can try and load the class file by its name (without .class and with . instead of /) and then watch for an exception:

$ pwd
$ cat
package aaa;

class Test {
    public static void main(String[] argv) {
$ jshell -c .
|  Welcome to JShell -- Version 11.0.14
|  For an introduction type: /help intro

jshell> import;

jshell> URL[] urls = new URL[]{new URL("file:///home/p-himik/tmp/java-classpath-test/")};
urls ==> URL[1] { file:/home/p-himik/tmp/java-classpath-test/ }

jshell> import;
jshell> URLClassLoader cl = new URLClassLoader(urls);
cl ==>

jshell> cl.loadClass("Test")
|  Exception java.lang.NoClassDefFoundError: aaa/Test (wrong name: Test)
|        at ClassLoader.defineClass1 (Native Method)
|        at ClassLoader.defineClass (
|        at SecureClassLoader.defineClass (
|        at BuiltinClassLoader.defineClass (
|        at BuiltinClassLoader.findClassOnClassPathOrNull (
|        at BuiltinClassLoader.loadClassOrNull (
|        at BuiltinClassLoader.loadClass (
|        at ClassLoaders$AppClassLoader.loadClass (
|        at ClassLoader.loadClass (
|        at ClassLoader.loadClass (
|        at (#5:1)



As you can see, I never mentioned aaa to jshell - but it does know about it.


Forgot one command above - of course, I also ran javac


Looking through JDK sources now - maybe there's a reasonable way to get to that aaa/Test without having to parse the exception.


@U2FRKM4TW clj-kondo should not be using any classloader stuff for this


it's supposed to be running in a graalvm native-image so you should just treat this as a random binary file without any relation to the classloader of the environment


Oh... makes sense.


I think ASM works that way


so maybe it's not so bad to use that, it seems to be a small library


@U037U6UBEAW helpful thank you!


you're welcome


excuse me I am a newbie


but here's a crude but working translation:

(defn get-class-name [path]
  (with-open [dis ( (io/input-stream path))]
    ;; skip first 8 bytes
    (.readLong dis)
    (let [constant-pool-count (dec (.readUnsignedShort dis))
          counter (atom 0)
          classes (atom {})
          strings (atom {})]
        (while (< @counter constant-pool-count)
            (case (.read dis)
              1 (swap! strings assoc @counter (.readUTF dis))
              5 (do (.readLong dis) (swap! counter inc))
              6 (do (.readLong dis) (swap! counter inc))
              7 (swap! classes assoc  @counter (.readShort dis))
              8 (.readShort dis)
              (.readInt dis))
            (swap! counter inc)))
        ;; skip access flags
        (.readShort dis)
           (get @strings (- (get @classes (.readUnsignedShort dis)) 2))


@U037U6UBEAW cool! I'm getting a NPE here:

(prn (get @classes (.readUnsignedShort dis)))


a nil I mean


can you send me the class you're running it against?


I just tried it only with a half dozen of classes I found on my laptop


This is clojure.lang.PersistentVector.class


I'll likely stick with ASM since I don't want to get a lot of problems with this :)


replace (get @strings (- (get @classes (.readUnsignedShort dis)) 2))


with (get @strings (- (get @classes (- (.readUnsignedShort dis) 1)) 1))


nice, it returns the package name now


(it also works in babashka I noticed :)


I've not found a more elegant way because of the need to increment the counter inside the loop a couple of times


I swapped your impl to use volatiles now, should be a little faster


That comment makes me want to stick with ASM since I won't have to deal with stuff that I don't really know a lot about (parsing class files)


I think ASM is safer


still cool that you managed to port it. I'm going to keep that example somewhere


a new class format eventually will break that simple parser


I just wanted to practice a little bit šŸ™‚


Double checked against counter need to start from 1 indeed: > A constant_pool index is considered valid if it is greater than zero and less than constant_pool_count and so there's no need to decrement indices for retrieving correct values


also, .skipBytes could be a better semantic choice since we are effectively throwing away those values


The need to skip constant pool entries is also explained here


> In retrospect, making 8-byte constants take two constant pool entries was a poor choice.


so ends up with


5 (do (.skipBytes dis 8) (swap! counter inc))
6 (do (.skipBytes dis 8) (swap! counter inc))


Have a nice day!


@U037U6UBEAW I converted the while and mutation to a loop and shared it here:


looks much better now, thanks šŸ™‚


Sorry @U04V15CAJ but I wasn't happy until having correctly parsed at least all 17k classes in my home... First there was an error in the gist at line 19 where the counter must be incremented by 2 (unfortunately, "8-byte constants take up two entries in the constant_pool table of the class file"), and some tags were not considered (most prominently CONSTANT_MethodHandle_info that consumes 3 bytes and caused some quirks). Also made the counter start from 1 ("constant_pool index is considered valid if it is greater than zero and less than constant_pool_count") so now indices on table are correct and there's no more need to decrease them when retrieving values...


(defn class->package
  "Implementation by Marco Marini."
  (with-open [dis ( (io/input-stream class-file))]
    ;; skip first 8 bytes
    (.skipBytes dis 8)
    (let [constant-pool-count (.readUnsignedShort dis)
          [strings classes]
          (loop [counter 1
                 classes {}
                 strings {}]
            (if (< counter constant-pool-count)
              (case (.read dis)
                1 (recur (inc counter) classes (assoc strings counter (.readUTF dis)))
                (5 6) (do (.skipBytes dis 8)
                          (recur (+ 2 counter) classes strings))
                7 (recur (inc counter) (assoc classes counter (.readUnsignedShort dis)) strings)
                (8 16 19 20) (do (.skipBytes dis 2)
                                 (recur (inc counter) classes strings))
                15 (do (.skipBytes dis 3) (recur (inc counter) classes strings))
                (do (.skipBytes dis 4)
                    (recur (inc counter) classes strings)))
              [strings classes]))]
      ;; skip access flags
      (.skipBytes dis 2)
      ;; (prn (get @classes (.readUnsignedShort dis)))
         (get strings (get classes (.readUnsignedShort dis)))


Thanks again and enjoy the evening šŸ™‚


Awesome, I'll update the gis!


I'm now trying @hiredmanā€™s javap-mode in emacs... Doesn't seem to work yet, but it would be a nice solution for when lsp-mode navigates us to a .class file


Very early effort, likely to have bitrotted, I seem to recall running into another package that did the same thing but better but I can't recall where


There are snippets like floating around


I also could have sworn I had a gist with some clojure code for parsing class files, but I couldn't find it


@UKFSJSM38 and I are now looking for a way to embed CFR (java decompiler) in clojure-lsp so it will automatically do it for every editor


Is there some clever way to trace into java method calls? Sadly I can't get visualvm up and running right now because Oracle is having some web problems.


what's the best way to handle clojure + openjfx? On Arch, they're shipped separately and to do the OpenJFX Hello World in pure java requires manually including --module-path and --add-modules or adjusting -cp to point there - I see I can set -Scp with clj, but that drops all the defaults clojure seems to need from it - I came up with this: clj -Scp '/usr/lib/jvm/java-17-openjdk/lib/javafx.swing.jar:/usr/lib/jvm/java-17-openjdk/lib/'$(clj -Spath 2>/dev/null) which seems to do the trick but feels like it isn't the best way to do it - also, how is the distribution story with clojure+openjfx for a gui (uberjar)? Would users have to deal wth all this hassle? Compared to a more self-contained clojure+swing jar


You could use a local deps for the javafx jars

thanks3 1

A question that was sparlkled from a conversation in #beginners: I'm a bit confused by this repl interaction:

Clojure 1.11.0
user=> (defn foo []
         (ns bar)
         (println *ns*)
         (defn baz [] (println "baz"))
         (ns user)
         (println *ns*))
user=> (foo)
#object[clojure.lang.Namespace 0x56f71edb bar]
#object[clojure.lang.Namespace 0x6fd1660 user]
user=> (bar/baz)
Syntax error compiling at (REPL:1:1).
No such var: bar/baz
user=> (user/baz)
Why is the symbol baz not defined in the bar namespace?

šŸ˜¦ 1

After reading @U054W022G explanation of ns related function in his book, I now understand that the proper way of writing what I was trying to write is:

(defn foo []
  (create-ns 'bar)
  (intern 'bar 'baz #(println "baz")))
but I still wonder why exactly the earlier version didn't work


Also note that ns is a macro, not a function. Meaning the ns call gets expanded when you define the function, not when you run the function. There's also in-ns, which might be doing what you're expecting: I thought it was a function, but I can't find its source in clojure.core.


During fixing I've encountered some strange behaviour related to symbol unmap in precompiled namespace. More in a thread.


EDIT: see below for a very minimal example (protocol doesn't matter here) Let's create projecta with two files: projecta.protocols

(ns projecta.protocols
  (:refer-clojure :exclude [abs]))

(defprotocol PAbs
  (abs [object])) 
(ns projecta.core
  (:require [projecta.protocols :as p]))

(deftype SomeType [^double v]
  (toString [_] (str v))
  (abs [_] (SomeType. (clojure.core/abs v))))

(ns-unmap *ns* 'abs)
(defn abs [v] (p/abs v))


Let's create projectb with lein project.clj

(defproject projectb "0.1.0-SNAPSHOT"
  :prep-tasks [["compile" "projecta.core"] "javac"]
  :dependencies [[org.clojure/clojure "1.11.0"]
                 [projecta "0.1.0-SNAPSHOT"]])


projecta.core/abs is not defined šŸ˜•

$ lein repl
Compiling projecta.core
nREPL server started on port 43847 on host - 
REPL-y 0.5.1, nREPL 0.8.3
Clojure 1.11.0
OpenJDK 64-Bit Server VM 18-ea+36-Ubuntu-1
user=> (require '[projecta.core :as pa])
user=> (def a (pa/->SomeType -3))
user=> (pa/abs a)
Syntax error compiling at (/tmp/form-init18231071095412228753.clj:1:1).
No such var: pa/abs


When projecta.core is constructed as following:

(ns projecta.core
  (:refer-clojure :exclude [abs])
  (:require [projecta.protocols :as p]))

(deftype SomeType [^double v]
  (toString [_] (str v))
  (abs [_] (SomeType. (clojure.core/abs v))))

(defn abs [v] (p/abs v))


Everything works as intended


$ lein repl
Compiling projecta.core
nREPL server started on port 45563 on host - 
REPL-y 0.5.1, nREPL 0.8.3
Clojure 1.11.0
OpenJDK 64-Bit Server VM 18-ea+36-Ubuntu-1
user=> (require '[projecta.core :as pa])
user=> (def a (pa/->SomeType -3))
user=> (pa/abs a)
#object[projecta.core.SomeType 0x3131b2ea "3.0"]


@U064X3EF3 maybe you can help here?

Alex Miller (Clojure team)21:03:37

Not going to look at it this weekend but I can check it out on Monday


Thanks! No rush, I just want to understand what is going on here and why this happens. Enjoy the weekend!


Struggling to parse the examples, it looks like everything working as intended


The protocol is defined in the namespace projecta.protocols so that is where the abs function that is part of the protocol is


Ah, I see the example in the middle with the face


No, that is correct too


Maybe protocol can be misleading, I just copied what I have in fastmath and clojure2d.


The difference is that in the first approach I call ns-unmap to remove abs from the namespace. In the second I use refer-clojure


The minimal example:

(ns projecta.core)
(ns-unmap *ns* 'abs)
(defn abs [v] (clojure.core/abs v))



(ns projecta.core
  (:refer-clojure :exclude [abs]))
(defn abs [v] (clojure.core/abs v))


Try printing out the value of *ns* before the unmap


Both work without precompilation.


But when the namespace is precompiled latter contains abs, former not


I just wonder if the value of *ns* is not what you expect when loading an aot compiled namespace, because compilation is not happening in that case


I kind of expect it should be, but don't recall


After adding (println *ns***)

Compiling projecta.core
#object[clojure.lang.Namespace 0x5d10455d projecta.core]


user=> (require '[projecta.core :as pa])
#object[clojure.lang.Namespace 0x2db705a7 projecta.core]
user=> (pa/abs -4)
Syntax error compiling at (/tmp/form-init1901708797312745991.clj:1:1).
No such var: pa/abs


hmmm... so compiled namespace is different object than namespace after require. Maybe that's why pa/abs is not visible? It belongs to a different namespace.


Oh, you know, I bet it is a constant pool thing


When you aot compile a namespace, all the code is turned into a static init method on the class file that is generated for the ns


Constants (like vars referenced) are lifted out into static fields


So for vars you only pay for var resolution once


But what is happening (my guess) is the def isn't looking up the var again, but reusing the one in the constant pool, which the ns-unmap previous removed from the ns


Try putting the def in an immediately invoked thunk ((fn [] (def ...))) Not sure if that would work around it if I have the issue right, but it might (the fn gets its own constant pool, but I am not sure when those constants would be initialized)


((fn [] (defn abs [v] (clojure.core/abs v))))


Yeah, and if you don't aot compile, top level forms are effectively each compiled in a think like that, so they don't share a constant pool


I need to digest it. I mean, I'm not familiar with the aot path (yet). Thanks a lot anyway.


ok, I've read it again, I think I got it.


Definitely an interesting divergence of behavior between aot and not

šŸ‘ 1