Suggest checking the value of :config-dir and other settings and see if it got set to /

❯ clojure -Sdescribe
{:version "1.11.1.1208"
 :config-files ["/usr/local/lib/clojure/deps.edn" "/home/practicalli/.config/clojure/deps.edn" ]
 :config-user "/home/practicalli/.config/clojure/deps.edn"
 :config-project "deps.edn"
 :install-dir "/usr/local/lib/clojure"
 :config-dir "/home/practicalli/.config/clojure"
 :cache-dir "/home/practicalli/.cache/clojure"
 :force false
 :repro false
 :main-aliases ""
 :repl-aliases ""}

Yes, I kinda forgot about -Sdescribe. I spotted a discrepancy between clojure -Sdescribe and bb clojure -Sdescribe which was really helpful. Thanks!

Ok I think that might explain why this:

(b/create-basis {:aliases [:overrides]
                :extra {:aliases {:overrides
                                   {:override-deps
                                   {'myns/module-a {:mvn/version version}
                                    'myns/module-b {:mvn/version version} 'myns/module-c {:mvn/version version}}}}}})

I wonder if I’d have more luck reading the deps.edn as data and replacing the :local/root s myself?

you can use :local/roots in the whole thing, consume with :local/roots and just do the replacements in a custom build script for deployment

yeah that’s basically what I’m trying to do, from a build.clj at the project root; but the internal deps are :local/root’s. First I tried it with with :extra :overrides, but it doesn’t seem to work — presumably because the application is shallow?! So now I’m looking at basically doing something like this, from the top:

(b/create-basis {:project (-> (edn/read-string (slurp "submodule-a/deps.edn"))
                                              (update-in [:deps 'myns/submodule-a] (constantly {:mvn/version version})))})
,,,

I'm not sure what :extra :overrides means there - :extra is an extra deps.edn (like -Sdeps on the command line)

:override-deps should apply throughout the tree though

yes I realised that — I did it because :project is set to the submodules “deps.edn” (which doesn’t have the :override-deps alias in it)… And it looked like :override-deps only has meaning in an :alias (not at the top of a deps.edn map) Though not 100% sure it’s right

as deps.edn becomes more complicated, things like slurp / read-string are going to be less likely to work and it will be better to lean on tools.deps apis (like https://clojure.github.io/tools.deps/#clojure.tools.deps/slurp-deps)

ah ok, thanks for the heads up there 🙇

> :override-deps only has meaning in an :alias correct

if you are "changing contexts" in a deps.edn, you'll want to use the tools.build apis for that

https://clojure.github.io/tools.build/clojure.tools.build.api.html#var-set-project-root.21

yes I’m already using set-project-root

I guess, I don't understand why the override thing "doesn't work"

though I wrote a macro to help with it:

(defmacro with-project-root [dir & forms]
  `(let [orig-root# @#'b/*project-root*] ;; yes this is correct, my cat didn't run over my keyboard I promise! :-)
     (try
       (b/set-project-root! ~dir)
       (let [ret# (do ~@forms)]
         ret#)
       (finally
         (b/set-project-root! orig-root#)))))

in particular, I don't understand exactly what you tried and what you saw and how it differed from what you expected

seems like rather than with-project-root, you could more directly just bind *project-root*

(defmacro with-project-root [dir & forms]
  `(binding [b/*project-root* ~dir] ~@forms))

Yes, I did actually have a question about that — I guess I was surprised to you setting the var directly, rather than suggesting users bind it (which was my first instinct); so I just assumed you had good reasons to use set. But thanks for clearing that up for me! 🙇

I was expecting people to change context as a 1-time kind of thing, not as a per-submodule change-and-reset, which in retrospect is an obvious thing. guess I should just add with-project-root to t.b

> in particular, I don’t understand exactly what you tried and what you saw and how it differed from what you expected So far everything I’ve tried has had the same effect, which is that the src paths for the independent module jars are being unioned… e.g. if I have in submodule-c:

{:deps {myns/submodule-c {:local/root "../submodule-a"}, myns/submodule-b {:local/root "../submodule-b"}}}

I haven’t yet tried just replacing those myself, by slurping and reading the file and munging the data, but that’s what I’m about to try next.

being unioned ... where?

I mean the jar contains the union of :paths from the submodules referenced via :local/root

(Not an explicit call to set/union or anything like that)

instead of just trying stuff, let's just be really clear about what you're doing

sure

so in submodule-c, you have a deps.edn like:

{:deps {myns/submodule-a {:local/root "../submodule-a"}
        myns/submodule-b {:local/root "../submodule-b"}}

yes

you're trying to create a jar for c that references a and b as maven artifacts, not as locals

yes, but submodule-c is not the parent (the parent also may shortly contain a submodule-d)

you control everything that is being used to make the jar

yes

you can copy the source from c to your jar making location, so that's fine

you control the basis used with write-pom which defines which deps the jar is published with in the pom

so if you provide an :extra deps that replaces the top-level deps, that's all you need to do. you don't care about the transitive deps - they don't affect what's in the jar

so when you create-basis, use :extra {yourgroup/artifact-a {:mvn/version v}, yourgroup/artifact-b {:mvn/version v}}

> you can copy the source from c to your jar making location, so that’s fine I could — but I’m not currently copying src files around myself.

why

I mean like (b/copy-dir {:src-dirs ["src" "resources"] :target-dir class-dir})

> so when you create-basis, use :extra {yourgroup/artifact-a {:mvn/version v}, yourgroup/artifact-b {:mvn/version v}} This was the first thing I tried…. I think it was actually (b/create-basis {:extra {:deps ,,,}}) before trying the form above with override-deps. Also another reason I moved to try again with override-deps was that I’m also building module-a and module-b, and I didn’t want to maintain separate overrides for each… i.e. I thought the semantics of override-deps were “if a dep uses this coordinate in its deps.edn override it; but if it doesn’t mention it don’t introduce it”

> why > I mean like (b/copy-dir {:src-dirs [“src” “resources”] :target-dir class-dir}) I could do that too; but I was avoiding that by doing it in situ by changing b/*project-root*

you still have to do something to get the sources

Does org.corfield.build/jar with the basis not handle that?

you have code - why maintain anything? just transform the data you have in the deps.edn

this is the first you've mentioned using that

> you have code - why maintain anything? just transform the data you have in the deps.edn yes that’s the approach I’m going to try next

> this is the first you’ve mentioned using that Sorry.

tbh, I don't like Sean's wrapper b/c it only works until something is different, then you have to fall back to using tools.build. the whole idea of tools.build is to start with something that can grow

this is a case where a lot of things are different

yeah I see it does the copy-dir

you're going to twist yourself in knots to create a context where his thing does the one thing it does, or you can just do it directly with tools.build. I think in the end you'll understand more and be able to do the next thing better with the latter

Yeah, that’s fair… I’ve used tools.build before, but also found seans wrapper useful on simpler builds. However I do also see that the copy-dir inside seans jar is causing my problem, because it doesn’t clean the :target-dir between runs.

So I think you’re right, that this is the root cause

Thanks for the help 🙇

👍

Another case of wanting simple not easy 🙂

@U06HHF230 A suggestion for you: look at the structure that Polylith uses. It has a development "project" for REPL usage where all code is on the classpath via :local/root (using a :dev alias) and then it has projects folders -- one for each buildable artifact -- with deps.edn that say how to build that specific artifact. It's primarily intended to build applications (so inlining :local/root deps is acceptable there) but I think it could also be used to build independent libraries, if you can figure out a model for how you version them all (and therefore build them in reverse dependency order, bottom-up, which should be reasonably straightforward?).

Thanks… I’ve been purposefully avoiding polylith at the minute; simply because it seems larger than I need, and more specialised towards application mono-repos.

> However I do also see that the copy-dir inside seans jar is causing my problem, because it doesn’t clean the :target-dir between runs. That is deliberate, so you can combine jar with other actions. As Alex says, that wrapper is fine for the simple stuff but breaks down in more complex settings -- at work we use a mixture of direct tools.deps, tools.build, and a few build-clj wrapper functions... and I'm planning to wean us off it "soon" as well as updating at least deps-new to use raw tools.build instead of the wrapper.

(well, Alex doesn't say the wrapper is "fine" but I hope you get my meaning)

yeah I realise it’s fine for bb/jar’s intended purpose… just not mine (here) 🙂

And, yes, the wrapper was born out of an apparent desire for "easy" over "simple" 🙂

as I said I’ve used the bb wrapper before, and found it helpful at cutting down boilerplate. I was also assuming when I set out that there might be an easy solution to this; which was partly why I started with your wrapper.

I think I’ve said this to you before though… I think a better way to slice “easy” is vertically; separating the needs of applications from the needs of libraries. Where as bb assumes it can add ease for both applications and libraries together.

Indeed. And it started out as a very "simple" wrapper to just cut down some basic boilerplate... but grew far too many knobs and dials in the pursuit of reducing boilerplate in more cases instead of "encouraging" people to just use tools.build... I'm far too accommodating in that area 🙂

haha well you do yourself a disservice there, there’s certainly a need for easier solutions to common problems… even if they do bring some complecting to the table. I think the problem is that easy solutions naturally grow knobs; and become a bit of a mess and hard to maintain overtime… Simple ones, more purposefully leave problems on the table, but are more maintainable over the long term. Anyway — I do very much appreciate everything you’ve done 🙇

I would have still liked an easy solution for this; as it’s taking far longer than I was expecting… and I was expecting it to take a while too! :rolling_on_the_floor_laughing:

I added with-project-root for the next t.b release btw

Heh, well, you've probably seen my long series of blog posts about our monorepo and the various approaches we've taken over the years, and the various things we've had to solve even with Polylith in the mix...

Yes, I’ve returned to them many times over the years