tools-build

mpenet 2022-05-18T19:31:45.657869Z

in create-basis: shouldn't classpath-roots be "canonicalize"d ?

mpenet 2022-05-18T19:32:00.737999Z

I am hitting an issue related to this I think

mpenet 2022-05-18T19:32:29.697589Z

it's a case where there's quite a bit of juggling going on between dirs at build time

Alex Miller (Clojure team) 2022-05-18T19:38:08.863669Z

they are at different points, but a repro would sure help

mpenet 2022-05-18T19:58:58.485499Z

I am not sure what happens. It's a case where if I run an uberjar task (with aot) from a module it just works but from a process that iterates over modules and run it for each deployable it then fails compiling the mains (just the mains). /tmp/compile-clj1025453397036198757/compile-clj/foo/main.class and the command starts with : java -cp /tmp/compile-clj1025453397036198757/compile-clj:... anyway I ll dig further

mpenet 2022-05-18T19:59:46.786029Z

it's difficult to give a full picture in current state.

seancorfield 2022-05-18T20:08:38.790199Z

@mpenet Maybe you are skipping some binding/context when iterating over the project dirs? Here's what we do at work:

(doseq [p projects]
      (println "\nRunning: uber on" p)
      (let [project-dir (str "projects/" p)
            aliases     (with-dir (io/file project-dir) (get-project-aliases))]
        (binding [b/*project-root* project-dir]
          (bb/clean {})
          (bb/uber (assoc (:uberjar aliases)
                          :compile-opts {:direct-linking true}))
          (bb/clean {}))))
b = clojure.tools.build, bb = my build-clj wrapper to provide defaults to clojure.tools.build, and with-dir is [clojure.tools.deps.alpha.util.dir :refer [with-dir]]

mpenet 2022-05-18T20:09:19.923399Z

no I do that religiously already

mpenet 2022-05-18T20:10:21.519849Z

and everything is using absolute paths in the cp currently

mpenet 2022-05-18T20:13:12.156949Z

do you aot in build-clj uberjar task?

mpenet 2022-05-18T20:13:46.464049Z

no you don't with this snippet

mpenet 2022-05-18T20:13:57.936419Z

without aot this works fine but aot is required in this context

mpenet 2022-05-18T20:15:10.604819Z

the comment in compile-clj makes me think I either forget something or process/java-command is not running from the right dir

mpenet 2022-05-18T20:15:12.444979Z

;; java-command will run in context of project-dir - basis, classpaths, etc ;; should all be relative to that (or absolute like working-compile-dir)

seancorfield 2022-05-18T20:16:31.491769Z

Yes, we AOT. See the linked build-clj code.

seancorfield 2022-05-18T20:18:30.749769Z

Are you explicitly building a basis? How/where? Note how we have to bind t.d.a's context separately to deal with reading/merging EDN files and extracting aliases. And we do all of the tools.build stuff inside the binding of *project-root*

mpenet 2022-05-18T20:19:44.247979Z

yes, there's no merging going on, it's quite straightforward. we just run the command from a different project context than the one being compiled/uberjared

mpenet 2022-05-18T20:20:11.762249Z

I ll step back and re-check the code that creates the basis, I was sure it was fine (it works for all the rest), but maybe I missed something

seancorfield 2022-05-18T20:20:27.060299Z

https://github.com/seancorfield/build-clj/blob/main/src/org/corfield/build.clj#L234 is the code for the uber wrapper that does pom.xml creation, copying files, compilation, and uber all in one task.

mpenet 2022-05-18T20:20:58.461609Z

thx

mpenet 2022-05-18T20:21:40.473669Z

that part is more or less identical as ours apparently

seancorfield 2022-05-18T20:21:46.332099Z

The (:uberjar aliases) piece pulls in options like

:uberjar
  {:uber-file "../../../build/uberjars/api-1.0.0.jar"
   :main api.main}

seancorfield 2022-05-18T20:24:13.512299Z

(and :main causes :ns-compile to be populated, as [api.main] for the example above)

seancorfield 2022-05-18T20:24:53.610489Z

Feel free to share some of your build code via DM with me if you don't want to share it in the channel -- maybe I can spot what's wrong?

mpenet 2022-05-18T20:25:50.609749Z

the error starts with one of the (compile 'foo.main) call in the generated script failing with a NPE (I guess it can't find the file)

seancorfield 2022-05-18T20:27:02.173879Z

Or perhaps the output path for the generated classes is missing/`nil`?

mpenet 2022-05-18T20:27:13.894689Z

but I can definitely see it under /tmp/compile-clj1025453397036198757/compile-clj/foo/main.class

mpenet 2022-05-18T20:27:36.144319Z

no it's here (if you mean the binding in the generated script)

mpenet 2022-05-18T20:27:45.356539Z

it fails at compile-clj step

mpenet 2022-05-18T20:28:32.898489Z

hmm I said something wrong, should be the clj file, not the class file

seancorfield 2022-05-18T20:28:41.208719Z

Hard for us to tell without seeing your scripts 🙂

Alex Miller (Clojure team) 2022-05-18T20:30:22.337919Z

if the compile-clj fails, it does not delete the temp dir where it's working

mpenet 2022-05-18T20:30:31.943119Z

I know! it's very useful 🙂

Alex Miller (Clojure team) 2022-05-18T20:30:33.919069Z

so you can go look at the entrails if that's helpful

mpenet 2022-05-18T20:30:47.656409Z

I have been doing that for a while

Alex Miller (Clojure team) 2022-05-18T20:30:55.696099Z

you should be able to fully reproduce the step at that point

mpenet 2022-05-18T20:31:17.591239Z

I can repro, but not fix it

mpenet 2022-05-18T20:31:44.457169Z

somehow the http://clj.co/compile call fails to find the clj file that has the main

mpenet 2022-05-18T20:32:08.854779Z

the src dir of the project/module in question is in the classpath

mpenet 2022-05-18T21:00:37.026559Z

Figured it out :) It is a bit convoluted I will spare you the details, but it's not caused by t.build or t.deps.

mpenet 2022-05-18T21:01:05.258639Z

It's a problem with that main.

😎 1
mpenet 2022-05-18T21:15:58.931939Z

learned a bit about the internals for all this along the way, I guess we can file this one as "not a total waste of time"

mpenet 2022-05-18T19:38:09.904529Z

it's actually not the root of my issue it seems