Fork me on GitHub
#tools-build
<
2022-02-23
>
Jungin Kwon07:02:12

Hello 🙂 Is there a way to build without the unused dependencies (or namespaces)?

delaguardo08:02:20

Unused dependencies is tricky to detect (not sure if this is even possible) because there are a lot of ways how dependencies can affect your program. Look at logging for example - you can add logging backend deps and even without explicit require anywhere it will be picked. For namespaces this should be done automatically as a side effect of compiling your code

đź‘Ť 1
FiVo16:02:16

Is there a way to get more verbose output from b/compile-clj. I am currently just getting. Clojure compilation failed .

FiVo17:02:18

I am currently trying to build a couple of subprojects with a build.clj at the root. My assumption was that I could do something like

(binding [b/*project-root* subproject]
   (do stuff with tools build))
but that does not seem correct. The things under the binding are similar to what is decribed here https://clojure.org/guides/tools_build#_compiled_uberjar_application_build. So my main question is what is affected by *project-root* and what not?

FiVo17:02:24

So under the binding I was assuming I could just do everything with tools.build as if I was in the subproject.

seancorfield17:02:57

@UL638RXE2 That definitely should work. That's how we deal with subprojects at work:

(binding [b/*project-root* project-dir]
          (bb/clean {})
          (b/copy-file {:src version-clj
                        :target "target/classes/ws/uberjar/release.clj"})
          (bb/uber (assoc (:uberjar aliases)
                          :compile-opts {:direct-linking true}))
          (bb/clean {}))
(`bb` is my build-clj wrapper around tools.build and b is tools.build itself)

seancorfield17:02:59

You might also need tools.deps.alpha.util.dir/with-dir if you're also working with t.d.a directly, for example:

aliases     (with-dir (io/file project-dir) (get-project-aliases))

seancorfield17:02:22

(that line is outside the binding BTW)

Alex Miller (Clojure team)17:02:15

you should probably not be messing with t.d.a's stuff - tools.build is setting that

Alex Miller (Clojure team)17:02:46

what are you actually trying to do?

seancorfield17:02:26

(we do it specifically to get the list of aliases in the subproject's deps.edn file outside of the binding context -- I don't remember the reason we ended up doing it that way @U064X3EF3)

Alex Miller (Clojure team)17:02:38

> So my main question is what is affected by `*project-root*` and what not? most paths in the build will be interpreted wrt the project-root

seancorfield17:02:51

FWIW, calling (get-project-aliases) via with-dir gives us the deps.edn data from the subproject, calling it inside the binding without with-dir gives us the top-level deps.edn data. Just tried that @U064X3EF3 So we do need with-dir here.

seancorfield17:02:19

That function is:

(defn- get-project-aliases []
  (let [edn-fn (juxt :root-edn :project-edn)]
    (-> (t/find-edn-maps)
        (edn-fn)
        (t/merge-edns)
        :aliases)))

seancorfield17:02:35

aliases-via-tda (:deps :test :uberjar)
aliases-inside-binding (:poly :cfml-admin :uploader :coverage :cfml-affiliate :build :+default :runner :+httpkit :everything :test :dev :deps)

seancorfield17:02:46

(didn't mean to derail the original thread)

FiVo08:02:55

> what are you actually trying to do? I am trying to create an uberjar for every subproject.

FiVo08:02:33

The project strucutre is as follows project/deps.edn

{:paths [...]
 :deps {...}}
project/subproject/deps.edn
{:paths [...]
 :deps {some-name {:local/root ".."}
        more deps ...}
and for every subproject I want to create an uberjar

seancorfield08:02:38

This is exactly what we do at work @UL638RXE2 -- but bear in mind your project/deps.edn will be completely ignored when you do this -- only project/subproject/deps.edn will be taken into account.

seancorfield08:02:56

(which is how Polylith is setup BTW)

seancorfield08:02:46

You haven't actually stated what goes wrong / what errors you get.

FiVo08:02:39

You mean even the deps from the root are ignored?

FiVo08:02:26

I saw your monorepo series, but haven't read everything

seancorfield08:02:52

There are three deps in play: root (system -- baked into t.d.a), user, and project. When you use the *project-root* binding, you are essentially changing what the project deps.edn is.

seancorfield08:02:13

So if you use project/subproject/deps.edn then project/deps.edn is not in play at all.

FiVo09:02:17

Yes but I am doing :local/root ".."

FiVo09:02:37

so would that not mean it's taking back into account?

seancorfield09:02:11

Oh, so you're trying to treat the top-level project as a dependency? Weird.

seancorfield09:02:36

Have you tried cd subproject && clojure -Stree to see what you get?

seancorfield09:02:00

Again, you haven't actually stated what goes wrong or what errors you are getting.

FiVo09:02:24

currently the root only has dependencies, but might get some code later on

FiVo09:02:47

these are shared across the subprojects

FiVo09:02:59

> Again, you haven't actually stated what goes wrong or what errors you are getting. Will do in a minute.

FiVo09:02:52

Here is the full build.clj

(ns build
  (:require [clojure.tools.build.api :as b]))

(def version "x.y.z")
(def class-dir "target/classes")
(def subprojects #{"foo" "bar"})

(defn- check-subproject [subproject]
  (when-not (subprojects subproject)
    (throw (IllegalArgumentException. (str "No such subproject " subproject)))))

(defn uber-file [subproject]
  (format "target/%s-%s-standalone.jar" subproject version))

(defn clean-subproject [subproject]
  (binding [b/*project-root* subproject]
    (b/delete {:path "target"})))

(defn clean [{:keys [subproject]}]
  (if subproject
    (clean-subproject (str subproject))
    (doseq [subproject subprojects]
      (clean-subproject subproject))))

(defn- uber-subproject [subproject]
  (check-subproject subproject)
  (clean-subproject subproject)
  (binding [b/*project-root* subproject]
    (let [basis (b/create-basis {:project "deps.edn"})]
      (b/copy-dir {:src-dirs ["src"]
                   :target-dir class-dir})
      (b/compile-clj {:basis basis
                      :src-dirs ["src"]
                      :class-dir class-dir})
      (b/uber {:class-dir class-dir
               :uber-file (uber-file subproject)
               :basis basis
               :main (symbol subproject)}))))

(defn uber [{:keys [subproject]}]
  (if subproject
    (uber-subproject (str subproject))
    (doseq [subproject subprojects]
      (uber-subproject subproject))))

FiVo09:02:25

Currently only trying with one subproject. clj -T:build uber can not locate the main entrypoint file/class during the compile step.

FiVo09:02:11

> Have you tried `cd subproject && clojure -Stree` to see what you get? Yes also shows the dependency of the root deps.edn

FiVo10:02:36

If I build an uberjar in the subproject with a build.clj in the subproject, everything works as expected.

FiVo10:02:40

I essentially don't want a build.clj for every subproject.

FiVo10:02:11

Maybe I have some missing knowledge about tools.build, but I would have expected the version with *project-root* to also work.

Alex Miller (Clojure team)13:02:22

Thanks for all the info, that helps a lot. So, when you make the basis in your subproject what's in it? Check the :classpath-roots of the basis for example

Alex Miller (Clojure team)13:02:46

(What you have seems very reasonable, and should work, but maybe there's a bug)

seancorfield15:02:57

@UL638RXE2 The problem is the relative path to the subproject -- I remember running into this and then forgetting about it. If you change the binding to this:

(binding [b/*project-root* (.getCanonicalPath (java.io.File. subproject))]
then it will all work as expected. @U064X3EF3 Is that expected?

seancorfield15:02:17

The relative path seems to work just fine for b/delete so it may only need the full path for the basis?

FiVo15:02:04

ok thanks, makes sense now. I thought I could set *project-root* with respect to (System/getProperty "user.dir"), but if it needs to be an absolute path than it makes sense that my stuff fails.

seancorfield15:02:50

We use user.dir -- it is an absolute path.

seancorfield15:02:25

(str (System/getProperty "user.dir") "/" subproject) should work.

seancorfield15:02:29

Yeah, that works.

FiVo15:02:37

Yes, I just wanted to say that my assumption before was that I could do

(binding [b/*project-root* "foo"] ...)
and that would be the same as with (str (System/getProperty "user.dir") "/foo" )

Alex Miller (Clojure team)16:02:57

so the problem is that setting a relative dir for project-root doesn't work?

seancorfield17:02:27

@U064X3EF3 Correct. Perhaps only for creating the basis though. delete seems to get the correct path.

Alex Miller (Clojure team)19:02:04

well this is definitely a bug, or maybe several, still working on it

1
đź‘Ť 1
Alex Miller (Clojure team)21:02:07

ok, give it a shot with v0.8.0 e3e3532

Alex Miller (Clojure team)21:02:16

the bugs were totally not where I thought they were when I started, but all had to do with how compile-clj and java-command make the classpath for compilation. I believe that is now all sorted out and consistent. I want to come back and leave some better tests for this stuff still but for now, I believe this should work for you.

FiVo21:02:40

Tested in the issue project above and it works. Will test the repo from work tomorrow.

Alex Miller (Clojure team)21:02:21

thanks so much for the report and the sample project - that made it 100x easier to work on and I'm glad this got sorted out

seancorfield21:02:14

I'll update our setup in an hour or so and verify it doesn't break things (and maybe I'll switch to relative paths 🙂 )

FiVo21:02:45

> thanks so much for the report and the sample project - that made it 100x easier to work on and I'm glad this got sorted out well, thanks for the quick fix

Alex Miller (Clojure team)21:02:12

yeah, to be clear, the basis all works just as it should afaict (I spent way too long with a faulty test trying to figure that out)

seancorfield21:02:49

Updated build-clj, updated our work repo, verified it works with the absolute path as before and also now with the relative path. Thanks, @U064X3EF3!

FiVo09:02:05

Seems to also work for the project at work.