Fork me on GitHub
#tools-deps
<
2024-01-15
>
Adam Helins20:01:27

In the context of a monorepo, I have been successfully experimenting with the following when exposing "public modules": • There is a single deps.edn at the root of the repo, as commonly expected, where everything is actually described • A script magically generates module-specific deps.edn files in their respective dirs, by following the master deps.edn • These module-specific deps.edn files contain their own paths + any required deps external to the repo + deps to other modules (by requiring them as gitlibs) • Hence any public module is available as a gitlib with :deps/root The challenge of course is inherent to git: when generating the custom deps.edn files, we don't know the SHA of the next commit, hence the SHA that should be used for requiring other modules. So what I do is generate with previous SHA + commit... twice. Without changing anything else in between. The last SHA is what should be used by the user. This all actually works, so there is not much to complain about. However it unnecessarily puts some strain on version comparison and overall that double commit feels a bit like a hack. This could "easily" be solved if a gitlib could be expressed sort of relatively. "I need that :deps/root of the same repo at the same rev". A hybrid between a local dep and a typical gitlib. Does it make any sense? :thinking_face:

Alex Miller (Clojure team)21:01:35

It makes sense, although I’ve mostly thought about solving this in terms of local deps that also have awareness of being in a git context

Alex Miller (Clojure team)21:01:41

Although it depends a bit what consumption patterns you are targeting for users

seancorfield21:01:31

This is why Polylith uses :local/root for everything instead of trying to deal with git deps.

Adam Helins22:01:07

@U064X3EF3 That would be indeed more transparent. Are there any technical constraints preventing that sort of implementation? @U04V70XH6 But then what happens if you want to make some parts of the repo easily available for external users? This is my main concern with the setup above.

seancorfield23:01:40

> easily available for external users What do you mean by this? Who are the external users? Do you expect them to git-depend on your monorepo or via some other approach? With Polylith, the projects can be git-depended on -- that's how the various parts of the Polylith tooling are exposed...

Adam Helins23:01:00

Git-depending indeed. If a project requires :local/roots, wouldn't that be problematic when attempting to consume it as a gitlib?

seancorfield23:01:42

No. The whole repo is checked out but :deps/root determines which "project" to use so the local deps all work because they're relative.

seancorfield23:01:06

That's exactly how the poly and polyx tools work: one repo, multiple :deps/root values.

Adam Helins23:01:36

Ah, I might be misremembering but I remember encountering trouble when mixing gitlibs with :local/root. Experimenting a long time ago though, either things have changed or my memory is flawed.

seancorfield23:01:55

Happy to help if you try it out and get stuck. As far as I know, it's worked this way for at least a few years...

Adam Helins23:01:28

One last that should put my mind at ease 🙏 What happens with the version resolution of common :local/roots if a user depends on 2 projects but with different SHAs? I think that's the sort of things I might have had trouble with at the time.

seancorfield23:01:18

If the versions are comparable, tools.deps will pick the most recent version.

seancorfield00:01:57

(`:local/root` isn't really relevant to that -- any transitive path to the same lib with different versions is treated the same way)

Alex Miller (Clojure team)00:01:16

the place that can problematic is if you end up with transitive deps on the same sub-project, one via local root, one via git dep, which right now will not compare

Adam Helins00:01:56

Right, and so we circle back to your original answer I believe. Thank you both!

seancorfield00:01:29

But if the monorepo uses :local/root deps and everyone outside the repo uses :git deps that won't happen (at least with Polylith).

seancorfield00:01:01

Oh wait... I get it... tools.deps would not consider them needing to be compared because the libname would be different, even tho' multiple libs resolve to the same git repo?

seancorfield00:01:47

So tools.deps thinks they're different libs -- even tho' they are the same gitlib -- and wouldn't cause one to be treated as newer than the other... and so if those separate libs had overlapping :local/root deps, you could end up with an arbitrarily-chosen version of the common code due to classpath ordering...?

Alex Miller (Clojure team)00:01:58

that is also a possible problem, but I'm really referring to: foo/a {:git/url "..." :git/sha "..." :deps/root "subA"} foo/b {:local/root "../b"} ext/c foo/b {:git/url "..." :git/sha "..." :deps/root "subB"}

Alex Miller (Clojure team)00:01:28

can't compare the two foo/b references to select one

Alex Miller (Clojure team)00:01:23

foo/b needs to be able to act in both a local relative sense to foo/a but also have it's own git context. I think that's a solvable problem but it needs some thinking

seancorfield00:01:16

OK, so that specific problem "can't happen" with Polylith specifically because outside projects can depend on specific projects/foo/* parts of the monorepo and while those can have :local/root deps, nothing else in the repo can depend on the projects/foo/* parts via :local/root deps. But, just to confirm my understanding, foo/a and foo/b would not be considered for newer/older selection even if they both had identical :git/url values and different :git/shas? So you'd get the same repo, checked out at two different SHAs, and treated as completely separate libs.

Alex Miller (Clojure team)00:01:26

different lib names are never compared, they are treated as different libs

2