Fork me on GitHub
#tools-deps
<
2019-06-28
>
vlaaad10:06:20

Can someone explain how do I build a classpath properly?

(deps/make-classpath
  (deps/resolve-deps {:deps {'re-find {:git/url ""
                                       :sha "c27af619a9d9e12832b7446e127d68ab916ce78c"}}}
                     nil)
  nil 
  nil)
produces this classpath: - .../tools.cli-0.4.1.jar - .../math.combinatorics-0.1.4.jar - .../clojure-1.8.0.jar" but there should also be .gitlibs/libs/refind/...sha/src, no?

vlaaad11:06:02

Apparently I had to add :paths ["src"] to map, which works, but feels a bit wrong... shouldn't it be the default? if I'll set :paths to ["src/clj"] in my ~/.clojure/deps.edn, will it break dependencies because I overridden their source path?

vlaaad11:06:11

Apparently I had to add :paths ["src"] to map, which works, but feels a bit wrong... shouldn't it be the default? if I'll set :paths to ["src/clj"] in my ~/.clojure/deps.edn, will it break dependencies because I overridden their source path?

Alex Miller (Clojure team)12:06:49

:paths ["src"] is in the install-level deps.edn, but when you're using the library directly, you're in full control of what you include

Alex Miller (Clojure team)12:06:47

setting to "src/clj" only affects your lib, not transitive deps - they set their own paths in their own deps.edn

vlaaad12:06:47

that's the thing, re-find does not set :paths explicitly, that's why I need to specify ["src"]

vlaaad13:06:14

it all works until everyone's system :paths property is the same

vlaaad13:06:54

setting "src/clj" affects dependent libs too

vlaaad13:06:05

(run! println 
      (str/split 
        (deps/make-classpath
          (deps/resolve-deps
            {:deps {'re-find {:git/url ""
                              :sha "c27af619a9d9e12832b7446e127d68ab916ce78c"}}
             :paths ["src/clj"]}
            nil)
          nil
          nil)
        #":"))
=> 
/home/vlaaad/.gitlibs/libs/re-find/re-find/c27af619a9d9e12832b7446e127d68ab916ce78c/src/clj
/home/vlaaad/.m2/repository/org/clojure/tools.cli/0.4.1/tools.cli-0.4.1.jar
/home/vlaaad/.m2/repository/org/clojure/math.combinatorics/0.1.4/math.combinatorics-0.1.4.jar
/home/vlaaad/.m2/repository/org/clojure/clojure/1.8.0/clojure-1.8.0.jar

vlaaad13:06:35

re-find sources are implicitly on src, not src/clj, but it gets overridden

Alex Miller (Clojure team)13:06:17

sorry, I don't understand what you're saying

Alex Miller (Clojure team)13:06:18

it looks like answer above is what I would expect to see

Alex Miller (Clojure team)13:06:24

it looks like answer above is what I would expect to see

vlaaad13:06:44

I am surprised that I have to specify :paths to get code from git dependency

vlaaad13:06:51

I expect gitlibs to have default :paths ["src"]

Alex Miller (Clojure team)13:06:23

oh, I see what you're saying

Alex Miller (Clojure team)13:06:25

it's up to libs to specify their own paths and re-find could (probably should) set :paths ["src"], and if it did, that would be used

Alex Miller (Clojure team)13:06:57

you're seeing src/clj there due to a bug, where if no paths are used, the paths in the root deps.edn are used

vlaaad13:06:09

ah, it's already reported, nice

Alex Miller (Clojure team)13:06:51

yeah, the cause is bound up in some other issues so I've had a long mull about how to fix, but ready to move forward on it when I have some time

vlaaad14:06:17

clojure.tools.deps.alpha.specs seem wrong

vlaaad14:06:25

(s/def :local/root string?)
(s/def :local/coord (s/keys :req-un [:local/root] :opt-un [::path]))

vlaaad14:06:39

shouldn't it be :req instead of :req-un?

Alex Miller (Clojure team)14:06:03

yes, I think you're right

Drew Verlee21:06:08

I'm attempting to build an uberjar using deps + uberdeps Everything works as expected until i try to run the uberjar and it tells me it cant find clojure.main.

āžœ  cat deps.edn 

{:aliases {
            :uberjar {:extra-deps {uberdeps {:mvn/version "0.1.4"}}
                      :main-opts ["-m" "uberdeps.uberjar"]}}}
āžœ   clj -A:uberjar
[uberdeps] Packaging target/frog.jar...
[uberdeps] Packaged target/frog.jar in 2 ms
āžœ   cat src/core.clj 
(ns frog.core
  (:gen-class))

(defn -main [& args]
  (println "HIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII"))
āžœ   java -cp target/frog.jar clojure.main -m frog.core                 

Error: Could not find or load main class clojure.main
Caused by: java.lang.ClassNotFoundException: clojure.main
As a starting point, im not sure why the instructions from uberdeps have me look for clojure.main, if its an uberjar isn't it included?

hiredman21:06:25

if you don't have clojure in your deps

hiredman21:06:50

some of it depends on how smart the uberdeps library is

hiredman21:06:30

if you don't include a dep on clojure explicitly, you will fallback to whatever is in the system (or whatever it is called) deps.edn file that was installed when you installed clj

hiredman21:06:03

but uberdeps might not be processing your deps.edn file using the tools.deps code, so it might not know to do that fallback

hiredman21:06:29

so if you don't get include a clojure version in your deps.edn, there wouldn't be clojure in your jar

hiredman21:06:02

it just slurps in the edn and reads it

hiredman21:06:54

a lot of tools built on top of deps.edn are super immature and very broken, and they largely duplicate each other

hiredman21:06:39

this is the second broken uberjar building library I have diagnosed today (the other tool seemed to end up aot compiling things twice)

hiredman21:06:54

also, if you are not aot compiling you don't need the :gen-class

Drew Verlee22:06:24

I thought it was necessary to for the rest of the tooling to convert the clojure code to java.

Drew Verlee22:06:30

Oh, your saying that if I don't specify that, the compiling still happens, just not ahead of time?

hiredman22:06:13

clojure code is not converted to java, it is compiled to jvm bytecode

hiredman22:06:04

it is always compiled to jvm bytecode, either when the code is loaded and right before it is run, or you load the code ahead of time and save the compiled bytecode and run the compiled bytecode (aot compilation)

hiredman22:06:20

:gen-class only does anything when you are aot compiling

hiredman22:06:45

and it is only needed in specific circumstances

hiredman22:06:57

you are not aot compiling so you do not need it

Drew Verlee22:06:06

Almost understand what your saying, I'll need to read a bit. Thanks!

Alex Miller (Clojure team)21:06:50

a new version of tools.deps is out now that probably makes it much harder to do the wrong thing with the install deps.edn

hiredman21:06:15

I dunno, this code literally just slurps the deps.edn and calls read-string on it

seancorfield21:06:23

@drewverlee Try using depstar instead of uberdeps -- unless there's some specific feature you need from the latter?

Drew Verlee21:06:45

There isn't! I did try both, I think I got a similar error. For one, It's possible I have an old version of deps. I'll have to check when I'm at my computer again. I'm always blown away by how fast everyone responds.

seancorfield21:06:57

Happy to help with any issues you have with depstar since I maintain it.

Drew Verlee23:06:03

So it resulted in a FileNotFoundException:

āžœ  cat deps.edn

{:deps {org.clojure/clojure {:mvn/version "1.10.0"}}
 :aliases {:depstar
           {:extra-deps
            {seancorfield/depstar {:mvn/version "0.2.1"}}}}}
āžœ  clojure -A:depstar -m hf.depstar.uberjar MyProject.jar

Building uber jar: MyProject.jar
āžœ  frog java -cp MyProject.jar clojure.main -m frog.core   
Exception in thread "main" java.io.FileNotFoundException: Could not locate frog/core__init.class, frog/core.clj or frog/core.cljc on classpath.
	at clojure.lang.RT.load(RT.java:466)
	at clojure.lang.RT.load(RT.java:428)
	at clojure.core$load$fn__6824.invoke(core.clj:6126)
	at clojure.core$load.invokeStatic(core.clj:6125)
	at clojure.core$load.doInvoke(core.clj:6109)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at clojure.core$load_one.invokeStatic(core.clj:5908)
	at clojure.core$load_one.invoke(core.clj:5903)
	at clojure.core$load_lib$fn__6765.invoke(core.clj:5948)
	at clojure.core$load_lib.invokeStatic(core.clj:5947)
	at clojure.core$load_lib.doInvoke(core.clj:5928)
	at clojure.lang.RestFn.applyTo(RestFn.java:142)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at clojure.core$load_libs.invokeStatic(core.clj:5985)
	at clojure.core$load_libs.doInvoke(core.clj:5969)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at clojure.core$require.invokeStatic(core.clj:6007)
	at clojure.main$main_opt.invokeStatic(main.clj:491)
	at clojure.main$main_opt.invoke(main.clj:487)
	at clojure.main$main.invokeStatic(main.clj:598)
	at clojure.main$main.doInvoke(main.clj:561)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.lang.Var.applyTo(Var.java:705)
	at clojure.main.main(main.java:37)
āžœ  clojure -A:depstar -m hf.depstar.uberjar MyProject.jar

Building uber jar: MyProject.jar
āžœ  java -cp MyProject.jar clojure.main -m frog.core      
Exception in thread "main" java.io.FileNotFoundException: Could not locate frog/core__init.class, frog/core.clj or frog/core.cljc on classpath.
	at clojure.lang.RT.load(RT.java:466)
	at clojure.lang.RT.load(RT.java:428)
	at clojure.core$load$fn__6824.invoke(core.clj:6126)
	at clojure.core$load.invokeStatic(core.clj:6125)
	at clojure.core$load.doInvoke(core.clj:6109)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at clojure.core$load_one.invokeStatic(core.clj:5908)
	at clojure.core$load_one.invoke(core.clj:5903)
	at clojure.core$load_lib$fn__6765.invoke(core.clj:5948)
	at clojure.core$load_lib.invokeStatic(core.clj:5947)
	at clojure.core$load_lib.doInvoke(core.clj:5928)
	at clojure.lang.RestFn.applyTo(RestFn.java:142)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at clojure.core$load_libs.invokeStatic(core.clj:5985)
	at clojure.core$load_libs.doInvoke(core.clj:5969)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.core$apply.invokeStatic(core.clj:667)
	at clojure.core$require.invokeStatic(core.clj:6007)
	at clojure.main$main_opt.invokeStatic(main.clj:491)
	at clojure.main$main_opt.invoke(main.clj:487)
	at clojure.main$main.invokeStatic(main.clj:598)
	at clojure.main$main.doInvoke(main.clj:561)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.lang.Var.applyTo(Var.java:705)
	at clojure.main.main(main.java:37)
āžœ   cat src/core.clj 
(ns frog.core)

(defn -main [& args]
  (println "Hello World"))
āžœ clj -Spath
src:/home/drew/.m2/repository/org/clojure/clojure/1.10.0/clojure-1.10.0.jar:/home/drew/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar:/home/drew/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar
I assume because as src/ is on my class path (which is default right?). Then it would pick up the core.clj file with the frog.core ns. But either it doesn't pick it up, or i'm not specifying it correctly. It seems to be in the jar file: ` āžœ jar xf ../MyProject.jar āžœ ls clojure core.clj META-INF

Drew Verlee23:06:03

āžœ  cat core.clj 
(ns frog.core)

(defn -main [& args]
  (println "Hello World"))

seancorfield00:06:37

Is that really src/core.clj or src/frog/core.clj?

seancorfield00:06:08

(it should be the latter: the file path must match the namespace segments!)

seancorfield00:06:50

A good sanity check is to see whether clj -m frog.core works before you try to build the JAR.

šŸ‘ 4
Drew Verlee00:06:11

ah ok. i guess i didn't realize the relationship between the files and the ns was more then convention.

seancorfield00:06:06

When you ask Clojure to run frog.core, how is it going to find it? The classpath is all file paths, so the ns has to be mapped to a file path to perform that search if it can't already find the compiled class in memory... so that's frog/core.clj (or frog/core.cljc) relative to each folder on the classpath, e.g., src.

Alex Miller (Clojure team)21:06:30

well, if you're not using tools.deps at all, then yes, that's prob broken

seancorfield21:06:57

Some of these packaging libs try to create a classpath which is not what you would have when running your code šŸ˜ž

Alex Miller (Clojure team)21:06:06

but I would encourage whomever to file tickets on these libs so they can be better, rather than raging about brokenness

seancorfield21:06:37

(I hadn't even heard of uberdeps before just now)

hiredman21:06:19

I could be wrong too, I am just guessing at the problem

hiredman21:06:26

I am just suggesting, like, be aware of that state of maturity of a lot of these libraries(tools.deps is still new, so tooling built on it is even newer), and maybe vet them a little more before using them.

andy.fingerhut21:06:30

Well, vetting includes using, at least a little bit šŸ™‚