Fork me on GitHub
#clojure
<
2023-10-27
>
teodorlu13:10:08

Hi! I’m looking for a bit of a sanity check. I believe the following assumptions are true: 1. When I write Number in a Clojure REPL, “Number” refers to java.lang.Number (https://docs.oracle.com/javase/8/docs/api/java/lang/Number.html) 2. Numbers as used in clojure.core and clojure.math are all instances of java.lang.Number 3. Multimethods “know about” hierarchies when they dispatch 4. Java class hierarchies are also Clojure hierarchies. I believe that i can therefore write a multimethod that with type as dispatch-fn, and handle Number in a defmethod, and my own type in a different defmethod. Am I wrong? (I hope this question isn’t too vague to make sense)

p-himik13:10:11

You aren't wrong.

👍 1
Joshua Suskalo16:10:35

Keep in mind if multiple methods match for a given dispatch value, you'll need to manually set which is preferred.

👍 1
richiardiandrea15:10:58

Hi all, has anybody ever encountered this error when compiling http-kit?

clojure -T:build uber :jar-name setup.jar :target-dir \"out/dist\"
Checking out:  at 5316639d4e43a7b3ff6d48a90bbae0647e8150af
Execution error (ClassNotFoundException) at java.net.URLClassLoader/findClass (URLClassLoader.java:445).
org.httpkit.client.HttpClient

Full report at:
/tmp/clojure-7262190232603337376.edn
Execution error (ExceptionInfo) at clojure.tools.build.tasks.compile-clj/compile-clj (compile_clj.clj:114).
Clojure compilation failed, working dir preserved: /tmp/compile-clj5716707053231805684

Full report at:
/tmp/clojure-5252432327774702449.edn
make: *** [Makefile:77: out/dist/setup.jar] Error 1

hiredman15:10:24

What makes you think you are compiling http-kit there?

richiardiandrea15:10:34

it is a dependency of that project and the class https://github.com/http-kit/http-kit/blob/v2.7.0/src/org/httpkit/client.clj#L10 - Is that not the problem? I might be off

hiredman16:10:21

I don't see anything that indicates any compilation in the output you showed

hiredman16:10:46

(compilation of java source to class files)

richiardiandrea16:10:46

oh sorry, I'll post the full stack trace

hiredman16:10:06

There likely will not be any evidence in the stacktrace

hiredman16:10:09

And that is likely the problem, http-kit has java source code that needs to be compiled

richiardiandrea16:10:50

ah, yeah, but this used to work before?

richiardiandrea16:10:09

Did the clojure build tool change w.r.t. compiling?

richiardiandrea16:10:23

Ok it seems I just need to add clojure -X:deps prep before building the uberjar... https://clojure.org/guides/deps_and_cli#prep_libs Thank you, you comment made me realize a few more things

hiredman16:10:24

what is the rationale for consuming http-kit as a git dep instead of a maven dep?

richiardiandrea16:10:10

yeah, it's not a rationale but another error I faced

Execution error (ExceptionInfo) at clojure.tools.deps.extensions.maven/get-artifact (maven.clj:167).
Could not transfer artifact http-kit:http-kit:jar:2.7.0 from/to clojars (): status code: 416, reason phrase: Range Not Satisfiable (416)

hiredman16:10:33

% clj -Sdeps '{:deps {http-kit/http-kit {:mvn/version "2.7.0"}}}'    
Downloading: http-kit/http-kit/2.7.0/http-kit-2.7.0.pom from clojars
Downloading: http-kit/http-kit/2.7.0/http-kit-2.7.0.jar from clojars
Clojure 1.11.1
user=>
🤷

richiardiandrea16:10:17

$ clj -Sdeps '{:deps {http-kit/http-kit {:mvn/version "2.7.0"}}}'    
Error building classpath. Could not transfer artifact http-kit:http-kit:jar:2.7.0 from/to clojars (): status code: 416, reason phrase: Range Not Satisfiable (416)
Very strange

seancorfield17:10:17

Something in your user deps.edn perhaps? Does this work for you?

clj -Srepro -Sdeps '{:deps {http-kit/http-kit {:mvn/version "2.7.0"}}}'

richiardiandrea17:10:49

Uhm, good point - tried but nope, same error

$  clj -Srepro -Sdeps '{:deps {http-kit/http-kit {:mvn/version "2.7.0"}}}'
Error building classpath. Could not transfer artifact http-kit:http-kit:jar:2.7.0 from/to clojars (): status code: 416, reason phrase: Range Not Satisfiable (416)

hiredman17:10:12

I've seen some suggestions that if you get that error you should check for a partially downloaded jar in ~/.m2 and delete it

👍 2
1
richiardiandrea17:10:54

Hey, that worked!

Noah Bogart16:10:57

i have a top-level logging call in a namespace. when i call clojure -T:build uberjar, i see that logging call twice, which tells me it's being loaded twice. is that coming from calling b/compile-clj and then b/uber? is it something I should be worried about otherwise?

1
dpsutton17:10:56

one thing i like to do is add a form like

(require 'clojure.repl)
(clojure.repl/pst (Exception. "trace") 50)
and then you can see who is calling and under what circumstances. Other ways to do it but this does some moderate stack trace cleanup and lets me see what is going on

😍 2
Noah Bogart17:10:45

that's really clever

Alex Miller (Clojure team)17:10:54

Don’t think uber should load anything

Alex Miller (Clojure team)17:10:37

More likely getting loaded twice during compile but hard to say

👍 1
Noah Bogart17:10:30

i ran b/uber from the repl and nabbed the compile script from write-compile-script! . followed the stack traces as @U11BV7MTK suggested, and i see that at line 15, (compile 'crossbeam.routes.N) is called which (through 15+ namespaces of requires) loads the target namespace, and then at line 106, there's (compile 'crossbeam.target-namespace)

Noah Bogart17:10:13

given that lots of people use this and no one seems harmed by it lol, i'm guessing that this isn't worrisome for things like records?

Noah Bogart17:10:01

and for some reason, I thought compile was transitive and would compile everything in the require'd tree as well as the target namespace

dpsutton17:10:12

i would only compile things once. You can end up with maddening issues like type X cannot be cast to type X

Noah Bogart17:10:16

yeah, that's what i thought. i'm not sure how these two interplay, so maybe it's okay?

dpsutton17:10:33

why are you calling compile twice (lines 15 and 106?)

Noah Bogart17:10:52

i'm not, this is part of`tools.build`'s uberjar function

seancorfield17:10:07

Are you compiling just the main ns or all nses?

seancorfield17:10:00

(I generally recommend only compiling the main ns -- since it's transitive -- and then adding only nses you know are dynamically loaded which wouldn't be reached by transitive compilation from main)

👍 2
Noah Bogart17:10:47

this is our uberjar function:

(defn uberjar
  [_]
  (let [basis (b/create-basis {:project "deps.edn"})]
    (clean nil)
    (b/copy-dir {:src-dirs ["src" "resources"]
                 :target-dir class-dir})
    (b/compile-clj {:basis basis
                    :src-dirs ["src"]
                    :class-dir class-dir})
    (b/uber {:class-dir class-dir
             :uber-file uberjar-file
             :main 'crossbeam.core
             :basis basis
             :manifest {"Add-Opens" "java.base/java.nio"}
             :exclude [#"^META-INF/license/LICENSE.*"]})))

Noah Bogart17:10:17

your comment and reading the tools.build code makes me wonder if we should be passing in :ns-compile instead of :src-dirs

seancorfield17:10:36

Yes. Specify the main ns with that 🙂

Noah Bogart17:10:48

hmm that solved the double-load issue. i'll have to test the resulting jar to make sure we're not doing anything dynamic and naughty lol

seancorfield17:10:50

(`:src-dirs` is also needed I think? ... hmm, maybe not if you specify :ns-compile ... I think I do it just by default since :src-dirs is used for other stuff in the build pipeline and I pass options all the way through in general)

👍 1
Noah Bogart17:10:35

the docs say it only needs :src-dirs if there's no paths in the basis (which we are providing with our deps.edn)

seancorfield17:10:47

Ah, yeah, that would be it...

Noah Bogart17:10:54

maybe the tools.build guide should recommend :ns-compile instead of :scr-dirs. it looks like whoever wrote this copied the example directly and only lightly modified it lol

Noah Bogart17:10:08

thanks everyone, glad to solve this

seancorfield17:10:59

> maybe the tools.build guide should recommend :ns-compile instead of :scr-dirs. Post on http://ask.clojure.org about it 🙂

👍 1
Alex Miller (Clojure team)18:10:55

I can clean that up

😍 1
Alex Miller (Clojure team)18:10:11

src-dirs is actually not even needed (will use :paths from basis by default)

Noah Bogart18:10:51

thank you, i'll close the github issue i was in the process of writing lol

borkdude18:10:30

I'm also only using :ns-compile [main-ns] by default, weird things happen with graalvm native-image if you don't. lein's behavior same: :aot [main-ns], never :aot :all

seancorfield18:10:01

Yeah... "AOT ALL THE THINGS!!!" always made me nervous 🙂

Alex Miller (Clojure team)18:10:20

I've updated the tools.build guide and tweaked the tools.build docstring for compile-clj

🎉 5