Fork me on GitHub
#tools-deps
<
2020-03-26
>
daveliepmann10:03:28

Hi folks, I've got a head-scratcher with transitive dependencies. I found a workaround but it doesn't seem right. We have a new library which depends on a Java lib living in the BeDataDriven public repository: http://docs.renjin.org/en/latest/library/project-setup.html We wish to consume this library in a separate project. However, we get an error when starting a REPL via clojure CLI (or CIDER):

Error building classpath. Could not find artifact org.renjin:renjin-script-engine:jar:3.5-beta43 in central ()
This error goes away if I include the BeDataDriven repo to the consumer's :mvn/repos. Like I said, this is a fix but it doesn't feel correct. Is this by design? I've created a minimal reproduction (three tiny files) of the issue in this repo: https://github.com/daveliepmann/minimal-transitive-deps-example The workaround is commented out. I get the error above when running clojure from the command line in min-consumer directory. I am running the latest clojure from brew. The actual lib is at https://github.com/appliedsciencestudio/rdata/ and the consumer deps.edn fails with both git and local deps:
clojure
appliedsciencestudio/rdata ;; {:local/root "/Users/daveliepmann/src/forks/rdata"}
        {:git/url ""
         :sha "6dff85aa7b83a9ce85eb8d70b0672a3adf0dc7f2"}
Prior art: this issue looks like it could be related to https://clojure.atlassian.net/projects/TDEPS/issues/TDEPS-127?filter=allissues&amp;orderby=priority%20DESC&amp;keyword=transitive but that seems to have been fixed.

dominicm10:03:55

I don't think repos are conveyed across deps.edn projects. They're considered local like :aliases

daveliepmann10:03:07

Right, but the consumer lib doesn't know, or in my mind shouldn't care, that the lib it's requiring uses a repo that's not Maven or Clojars.

daveliepmann10:03:37

Yup that looks like it, thanks. I missed it in my Jira search.

dominicm10:03:54

I can think of a couple cases where that's not true. Banks being one, repos are not transparent there. There's also the risk of someone adding a repo that them causes shadowing. I'm not sure how maven works for this.

rickmoynihan11:03:42

I might be misunderstanding the problem here - but isn’t it a security risk for a transitive dep to specify a new :mvn/repo; if we’re not careful might it not cause tools deps to fetch other deps from there too? Granted 3rd party deps are always a security problem; but this suggested change in behaviour seems like it would obfuscate things further and potentially makes auditing harder? I think the usecase is interesting; but would it not be safer for libs containing :mvn/repos to cause tools.deps to fail with a message saying “add X to :mvn/repos”?

rickmoynihan11:03:29

Also couldn’t it open up new attack vectors? e.g. an attacker could possibly set it to a private repo you own; and cause you to inadvertantly package secret deps in published binaries etc? Kind of like a CSRF attack. Admitedly this is a fanciful attack — and it would require you to depend on an attackers library either explicitly or transitively -- but attacks usually are elaborate.

dominicm11:03:12

That's right. But maven allows this, so it might already be the expected model.

alexmiller12:03:26

Yes, this should work, just not handled right now

ghadi13:03:43

That attack vector sounds realistic

alexmiller13:03:34

it needs some thought. imo, this is an under-considered part of the Maven model in general. repos are identified by name, and those names are just totally arbitrary and unique solely by rarity/convention.

alexmiller13:03:42

but Maven does certainly let you add repos in transitive deps for this case. I'm not sure if their use is "scoped" to that dep and below though

rickmoynihan13:03:24

yeah was wondering about this too… seems less risky to scope down the subtree; than apply globally.

alexmiller13:03:41

they are somewhat shielded from this by using a top-down, breadth-first, first one found wins strategy as "parents" in the tree have already been resolved before the transitive repo is encountered

alexmiller13:03:39

that's kind of also true with tools.deps (resolution happens top-down), but selection can be modified during the traversal and the "source" is not tracked right now in any way