tools-deps

Adam Helins 2024-01-15T20:42:27.995649Z

In the context of a monorepo, I have been successfully experimenting with the following when exposing "public modules": • There is a single deps.edn at the root of the repo, as commonly expected, where everything is actually described • A script magically generates module-specific deps.edn files in their respective dirs, by following the master deps.edn • These module-specific deps.edn files contain their own paths + any required deps external to the repo + deps to other modules (by requiring them as gitlibs) • Hence any public module is available as a gitlib with :deps/root The challenge of course is inherent to git: when generating the custom deps.edn files, we don't know the SHA of the next commit, hence the SHA that should be used for requiring other modules. So what I do is generate with previous SHA + commit... twice. Without changing anything else in between. The last SHA is what should be used by the user. This all actually works, so there is not much to complain about. However it unnecessarily puts some strain on version comparison and overall that double commit feels a bit like a hack. This could "easily" be solved if a gitlib could be expressed sort of relatively. "I need that :deps/root of the same repo at the same rev". A hybrid between a local dep and a typical gitlib. Does it make any sense? 🤔

Alex Miller (Clojure team) 2024-01-15T21:23:35.705629Z

It makes sense, although I’ve mostly thought about solving this in terms of local deps that also have awareness of being in a git context

Alex Miller (Clojure team) 2024-01-15T21:24:41.867699Z

Although it depends a bit what consumption patterns you are targeting for users

seancorfield 2024-01-15T21:44:31.081249Z

This is why Polylith uses :local/root for everything instead of trying to deal with git deps.

Adam Helins 2024-01-15T22:34:07.236779Z

@alexmiller That would be indeed more transparent. Are there any technical constraints preventing that sort of implementation? @seancorfield But then what happens if you want to make some parts of the repo easily available for external users? This is my main concern with the setup above.

seancorfield 2024-01-15T23:05:40.677139Z

> easily available for external users What do you mean by this? Who are the external users? Do you expect them to git-depend on your monorepo or via some other approach? With Polylith, the projects can be git-depended on -- that's how the various parts of the Polylith tooling are exposed...

Adam Helins 2024-01-15T23:23:00.953769Z

Git-depending indeed. If a project requires :local/roots, wouldn't that be problematic when attempting to consume it as a gitlib?

seancorfield 2024-01-15T23:26:42.981889Z

No. The whole repo is checked out but :deps/root determines which "project" to use so the local deps all work because they're relative.

seancorfield 2024-01-15T23:28:06.673089Z

That's exactly how the poly and polyx tools work: one repo, multiple :deps/root values.

Adam Helins 2024-01-15T23:39:36.483389Z

Ah, I might be misremembering but I remember encountering trouble when mixing gitlibs with :local/root. Experimenting a long time ago though, either things have changed or my memory is flawed.

seancorfield 2024-01-15T23:45:55.438739Z

Happy to help if you try it out and get stuck. As far as I know, it's worked this way for at least a few years...

Adam Helins 2024-01-15T23:55:28.817219Z

One last that should put my mind at ease 🙏 What happens with the version resolution of common :local/roots if a user depends on 2 projects but with different SHAs? I think that's the sort of things I might have had trouble with at the time.

seancorfield 2024-01-15T23:59:18.143009Z

If the versions are comparable, tools.deps will pick the most recent version.

seancorfield 2024-01-16T00:00:57.136469Z

(`:local/root` isn't really relevant to that -- any transitive path to the same lib with different versions is treated the same way)

Alex Miller (Clojure team) 2024-01-16T00:11:16.958759Z

the place that can problematic is if you end up with transitive deps on the same sub-project, one via local root, one via git dep, which right now will not compare

Adam Helins 2024-01-16T00:13:56.961789Z

Right, and so we circle back to your original answer I believe. Thank you both!

seancorfield 2024-01-16T00:14:29.429829Z

But if the monorepo uses :local/root deps and everyone outside the repo uses :git deps that won't happen (at least with Polylith).

seancorfield 2024-01-16T00:19:01.594989Z

Oh wait... I get it... tools.deps would not consider them needing to be compared because the libname would be different, even tho' multiple libs resolve to the same git repo?

seancorfield 2024-01-16T00:20:47.257059Z

So tools.deps thinks they're different libs -- even tho' they are the same gitlib -- and wouldn't cause one to be treated as newer than the other... and so if those separate libs had overlapping :local/root deps, you could end up with an arbitrarily-chosen version of the common code due to classpath ordering...?

Alex Miller (Clojure team) 2024-01-16T00:24:58.985189Z

that is also a possible problem, but I'm really referring to: foo/a {:git/url "..." :git/sha "..." :deps/root "subA"} foo/b {:local/root "../b"} ext/c foo/b {:git/url "..." :git/sha "..." :deps/root "subB"}

Alex Miller (Clojure team) 2024-01-16T00:25:28.081999Z

can't compare the two foo/b references to select one

Alex Miller (Clojure team) 2024-01-16T00:28:23.031829Z

foo/b needs to be able to act in both a local relative sense to foo/a but also have it's own git context. I think that's a solvable problem but it needs some thinking

seancorfield 2024-01-16T00:34:16.214579Z

OK, so that specific problem "can't happen" with Polylith specifically because outside projects can depend on specific projects/foo/* parts of the monorepo and while those can have :local/root deps, nothing else in the repo can depend on the projects/foo/* parts via :local/root deps. But, just to confirm my understanding, foo/a and foo/b would not be considered for newer/older selection even if they both had identical :git/url values and different :git/shas? So you'd get the same repo, checked out at two different SHAs, and treated as completely separate libs.

Alex Miller (Clojure team) 2024-01-16T00:38:26.462749Z

different lib names are never compared, they are treated as different libs

👍🏻 1