Fork me on GitHub
#tools-deps
<
2022-01-22
>
pavlosmelissinos10:01:39

I think this belongs here. Please bear with the wall of text... I'm trying to understand how dependencies are resolved, so I'd like some help with this 🙂 (questions in thread)

pavlosmelissinos10:01:15

1. For obvious reasons you can't refer to a library twice using a single symbol in the same deps.edn. You can "trick" clojure.tools.deps to include different versions of the same library by using different symbols for each, as long as the namespaces are different (e.g. use honeysql v1 and honeysql v2 at the same time). Is my understanding correct? 2. How about different versions of the same library against transitive uses of it (top-level vs transitive or transitive vs transitive) when the same symbol is used? E.g. if dependency A has library X v1.4 in its deps.edn and dependency B has library X v1.5 is it possible to actually include both versions of library X in the deps tree? 3. If not, how does tools deps deal with the fact that libraries are generally tested against the specific versions of their dependencies? If, say, a top-level dependency of your project overrides every transitive use of it (like the docs mention, "top dep versions always win") then you might end up with unexpected results, even in the code of transitive deps. In a perfect world, this wouldn't be an issue because there would be no breaking changes but in our flawed reality how does Clojure handle this situation, especially in projects with large dependency graphs? 4. If Clojure doesn't try to handle it, what's driven that decision? 5. Is there a place (book, docs, whatever) where I can read more about the above? I've already read the very helpful https://clojure.org/reference/dep_expansion but I'm still not 100% sure how dep resolution works and why it's like that 🙂 Thanks!

Alex Miller (Clojure team)14:01:13

1. You could, except ultimately you're making a classpath that contains an ordered set of locations to look and whatever version is earlier in the classpath will be used as its found first. Having two different versions of the same lib in your classpath should generally be considered an error. 2. No - only one version of a lib will be selected 3. This is the world we live in - libs are tested against some version of a dep but the application using the lib has full control over what version is used and it might not work. The application developer has to handle this, ultimately by selecting a set of compatible versions at the top if the selected versions don't work together (either by expliciting pinning versions or excluding transitive deps). There is no guarantee that there is a compatible solution (although this happens surprisingly rarely) 4. Doesn't handle it, and we inherited this from the Java world. 5. Anything you read about the Java / Maven ecosystem is likely applicable, not sure I have a good pointer.

🙏 1
Alex Miller (Clojure team)14:01:35

Another factor to be aware of is that Maven/Leiningen use a different version selection algorithm than tools.deps/Clojure CLI. Maven uses the first version it encounters (top-down, breadth first walk) and tools.deps will generally select the newest version it can. In practice, this matters less than you'd expect.

pavlosmelissinos19:01:02

Alright, that was illuminating. I think I'll let the whole thing sink a bit before I ask any follow-up questions. Thanks so much for the authoritative answer!

seancorfield22:01:04

"(e.g. use honeysql v1 and honeysql v2 at the same time)." -- just to be clear here, HoneySQL v1 and v2 are essentially different libs: they use different Maven coords and different namespaces, precisely so you can depend on both and use them side-by-side when you are migrating. That's unusual -- most libraries that have v1 and v2 force you to choose, because they generally use the same coordinates (and usually the same namespaces too).

👍 1