Fork me on GitHub
#tools-build
<
2022-05-18
>
mpenet19:05:45

in create-basis: shouldn't classpath-roots be "canonicalize"d ?

mpenet19:05:00

I am hitting an issue related to this I think

mpenet19:05:29

it's a case where there's quite a bit of juggling going on between dirs at build time

Alex Miller (Clojure team)19:05:08

they are at different points, but a repro would sure help

mpenet19:05:58

I am not sure what happens. It's a case where if I run an uberjar task (with aot) from a module it just works but from a process that iterates over modules and run it for each deployable it then fails compiling the mains (just the mains). /tmp/compile-clj1025453397036198757/compile-clj/foo/main.class and the command starts with : java -cp /tmp/compile-clj1025453397036198757/compile-clj:... anyway I ll dig further

mpenet19:05:46

it's difficult to give a full picture in current state.

seancorfield20:05:38

@U050SC7SV Maybe you are skipping some binding/context when iterating over the project dirs? Here's what we do at work:

(doseq [p projects]
      (println "\nRunning: uber on" p)
      (let [project-dir (str "projects/" p)
            aliases     (with-dir (io/file project-dir) (get-project-aliases))]
        (binding [b/*project-root* project-dir]
          (bb/clean {})
          (bb/uber (assoc (:uberjar aliases)
                          :compile-opts {:direct-linking true}))
          (bb/clean {}))))
b = clojure.tools.build, bb = my build-clj wrapper to provide defaults to clojure.tools.build, and with-dir is [clojure.tools.deps.alpha.util.dir :refer [with-dir]]

mpenet20:05:19

no I do that religiously already

mpenet20:05:21

and everything is using absolute paths in the cp currently

mpenet20:05:12

do you aot in build-clj uberjar task?

mpenet20:05:46

no you don't with this snippet

mpenet20:05:57

without aot this works fine but aot is required in this context

mpenet20:05:10

the comment in compile-clj makes me think I either forget something or process/java-command is not running from the right dir

mpenet20:05:12

;; java-command will run in context of project-dir - basis, classpaths, etc ;; should all be relative to that (or absolute like working-compile-dir)

seancorfield20:05:31

Yes, we AOT. See the linked build-clj code.

seancorfield20:05:30

Are you explicitly building a basis? How/where? Note how we have to bind t.d.a's context separately to deal with reading/merging EDN files and extracting aliases. And we do all of the tools.build stuff inside the binding of *project-root*

mpenet20:05:44

yes, there's no merging going on, it's quite straightforward. we just run the command from a different project context than the one being compiled/uberjared

mpenet20:05:11

I ll step back and re-check the code that creates the basis, I was sure it was fine (it works for all the rest), but maybe I missed something

seancorfield20:05:27

https://github.com/seancorfield/build-clj/blob/main/src/org/corfield/build.clj#L234 is the code for the uber wrapper that does pom.xml creation, copying files, compilation, and uber all in one task.

mpenet20:05:40

that part is more or less identical as ours apparently

seancorfield20:05:46

The (:uberjar aliases) piece pulls in options like

:uberjar
  {:uber-file "../../../build/uberjars/api-1.0.0.jar"
   :main api.main}

seancorfield20:05:13

(and :main causes :ns-compile to be populated, as [api.main] for the example above)

seancorfield20:05:53

Feel free to share some of your build code via DM with me if you don't want to share it in the channel -- maybe I can spot what's wrong?

mpenet20:05:50

the error starts with one of the (compile 'foo.main) call in the generated script failing with a NPE (I guess it can't find the file)

seancorfield20:05:02

Or perhaps the output path for the generated classes is missing/`nil`?

mpenet20:05:13

but I can definitely see it under /tmp/compile-clj1025453397036198757/compile-clj/foo/main.class

mpenet20:05:36

no it's here (if you mean the binding in the generated script)

mpenet20:05:45

it fails at compile-clj step

mpenet20:05:32

hmm I said something wrong, should be the clj file, not the class file

seancorfield20:05:41

Hard for us to tell without seeing your scripts 🙂

Alex Miller (Clojure team)20:05:22

if the compile-clj fails, it does not delete the temp dir where it's working

mpenet20:05:31

I know! it's very useful 🙂

Alex Miller (Clojure team)20:05:33

so you can go look at the entrails if that's helpful

mpenet20:05:47

I have been doing that for a while

Alex Miller (Clojure team)20:05:55

you should be able to fully reproduce the step at that point

mpenet20:05:17

I can repro, but not fix it

mpenet20:05:44

somehow the http://clj.co/compile call fails to find the clj file that has the main

mpenet20:05:08

the src dir of the project/module in question is in the classpath

mpenet21:05:37

Figured it out :) It is a bit convoluted I will spare you the details, but it's not caused by t.build or t.deps.

mpenet21:05:05

It's a problem with that main.

😎 1
mpenet21:05:58

learned a bit about the internals for all this along the way, I guess we can file this one as "not a total waste of time"

mpenet19:05:09

it's actually not the root of my issue it seems