Clojurians Log v2

Unused dependencies is tricky to detect (not sure if this is even possible) because there are a lot of ways how dependencies can affect your program. Look at logging for example - you can add logging backend deps and even without explicit require anywhere it will be picked. For namespaces this should be done automatically as a side effect of compiling your code

> what are you actually trying to do? I am trying to create an uberjar for every subproject.

The project strucutre is as follows project/deps.edn

{:paths [...]
 :deps {...}}

{:paths [...]
 :deps {some-name {:local/root ".."}
        more deps ...}

This is exactly what we do at work @finn.volkel -- but bear in mind your project/deps.edn will be completely ignored when you do this -- only project/subproject/deps.edn will be taken into account.

(which is how Polylith is setup BTW)

You haven't actually stated what goes wrong / what errors you get.

You mean even the deps from the root are ignored?

I saw your monorepo series, but haven't read everything

There are three deps in play: root (system -- baked into t.d.a), user, and project. When you use the *project-root* binding, you are essentially changing what the project deps.edn is.

So if you use project/subproject/deps.edn then project/deps.edn is not in play at all.

Yes but I am doing :local/root ".."

so would that not mean it's taking back into account?

Oh, so you're trying to treat the top-level project as a dependency? Weird.

Have you tried cd subproject && clojure -Stree to see what you get?

Again, you haven't actually stated what goes wrong or what errors you are getting.

currently the root only has dependencies, but might get some code later on

these are shared across the subprojects

> Again, you haven't actually stated what goes wrong or what errors you are getting. Will do in a minute.

Here is the full build.clj

(ns build
  (:require [clojure.tools.build.api :as b]))

(def version "x.y.z")
(def class-dir "target/classes")
(def subprojects #{"foo" "bar"})

(defn- check-subproject [subproject]
  (when-not (subprojects subproject)
    (throw (IllegalArgumentException. (str "No such subproject " subproject)))))

(defn uber-file [subproject]
  (format "target/%s-%s-standalone.jar" subproject version))

(defn clean-subproject [subproject]
  (binding [b/*project-root* subproject]
    (b/delete {:path "target"})))

(defn clean [{:keys [subproject]}]
  (if subproject
    (clean-subproject (str subproject))
    (doseq [subproject subprojects]
      (clean-subproject subproject))))

(defn- uber-subproject [subproject]
  (check-subproject subproject)
  (clean-subproject subproject)
  (binding [b/*project-root* subproject]
    (let [basis (b/create-basis {:project "deps.edn"})]
      (b/copy-dir {:src-dirs ["src"]
                   :target-dir class-dir})
      (b/compile-clj {:basis basis
                      :src-dirs ["src"]
                      :class-dir class-dir})
      (b/uber {:class-dir class-dir
               :uber-file (uber-file subproject)
               :basis basis
               :main (symbol subproject)}))))

(defn uber [{:keys [subproject]}]
  (if subproject
    (uber-subproject (str subproject))
    (doseq [subproject subprojects]
      (uber-subproject subproject))))

Currently only trying with one subproject. clj -T:build uber can not locate the main entrypoint file/class during the compile step.

> Have you tried `cd subproject && clojure -Stree` to see what you get? Yes also shows the dependency of the root deps.edn

If I build an uberjar in the subproject with a build.clj in the subproject, everything works as expected.

Here is a repo with the issue https://github.com/FiV0/tools.build-subproject-issue

I essentially don't want a build.clj for every subproject.

So also created a branch where I got it working https://github.com/FiV0/tools.build-subproject-issue/compare/working-version.

Maybe I have some missing knowledge about tools.build, but I would have expected the version with *project-root* to also work.

Thanks for all the info, that helps a lot. So, when you make the basis in your subproject what's in it? Check the :classpath-roots of the basis for example

(What you have seems very reasonable, and should work, but maybe there's a bug)

@finn.volkel The problem is the relative path to the subproject -- I remember running into this and then forgetting about it. If you change the binding to this:

(binding [b/*project-root* (.getCanonicalPath (java.io.File. subproject))]

The relative path seems to work just fine for b/delete so it may only need the full path for the basis?

ok thanks, makes sense now. I thought I could set *project-root* with respect to (System/getProperty "user.dir"), but if it needs to be an absolute path than it makes sense that my stuff fails.

We use user.dir -- it is an absolute path.

(str (System/getProperty "user.dir") "/" subproject) should work.

Yeah, that works.

Yes, I just wanted to say that my assumption before was that I could do

(binding [b/*project-root* "foo"] ...)

so the problem is that setting a relative dir for project-root doesn't work?

@alexmiller Correct. Perhaps only for creating the basis though. delete seems to get the correct path.

well this is definitely a bug, or maybe several, still working on it

ok, give it a shot with v0.8.0 e3e3532

the bugs were totally not where I thought they were when I started, but all had to do with how compile-clj and java-command make the classpath for compilation. I believe that is now all sorted out and consistent. I want to come back and leave some better tests for this stuff still but for now, I believe this should work for you.

Tested in the issue project above and it works. Will test the repo from work tomorrow.

thanks so much for the report and the sample project - that made it 100x easier to work on and I'm glad this got sorted out

I'll update our setup in an hour or so and verify it doesn't break things (and maybe I'll switch to relative paths 🙂 )

> thanks so much for the report and the sample project - that made it 100x easier to work on and I'm glad this got sorted out well, thanks for the quick fix

yeah, to be clear, the basis all works just as it should afaict (I spent way too long with a faulty test trying to figure that out)

Updated build-clj, updated our work repo, verified it works with the absolute path as before and also now with the relative path. Thanks, @alexmiller!

Seems to also work for the project at work.

👍

So under the binding I was assuming I could just do everything with tools.build as if I was in the subproject.

@finn.volkel That definitely should work. That's how we deal with subprojects at work:

(binding [b/*project-root* project-dir]
          (bb/clean {})
          (b/copy-file {:src version-clj
                        :target "target/classes/ws/uberjar/release.clj"})
          (bb/uber (assoc (:uberjar aliases)
                          :compile-opts {:direct-linking true}))
          (bb/clean {}))

You might also need tools.deps.alpha.util.dir/with-dir if you're also working with t.d.a directly, for example:

aliases     (with-dir (io/file project-dir) (get-project-aliases))

(that line is outside the binding BTW)

you should probably not be messing with t.d.a's stuff - tools.build is setting that

what are you actually trying to do?

(we do it specifically to get the list of aliases in the subproject's deps.edn file outside of the binding context -- I don't remember the reason we ended up doing it that way @alexmiller)

> So my main question is what is affected by `*project-root*` and what not? most paths in the build will be interpreted wrt the project-root

FWIW, calling (get-project-aliases) via with-dir gives us the deps.edn data from the subproject, calling it inside the binding without with-dir gives us the top-level deps.edn data. Just tried that @alexmiller So we do need with-dir here.

That function is:

(defn- get-project-aliases []
  (let [edn-fn (juxt :root-edn :project-edn)]
    (-> (t/find-edn-maps)
        (edn-fn)
        (t/merge-edns)
        :aliases)))

aliases-via-tda (:deps :test :uberjar)
aliases-inside-binding (:poly :cfml-admin :uploader :coverage :cfml-affiliate :build :+default :runner :+httpkit :everything :test :dev :deps)

(didn't mean to derail the original thread)