Fork me on GitHub
#tools-deps
<
2020-10-20
>
cch100:10:18

I’m getting up to speed with the deploy process using tools from the greater tools.deps ecosystem and I ran into a strange issue today. Essentially, I was finding that libs installed locally did not bring their transitive dependencies along with them. I used clojure.tools.cli.api/mvn-install to do the install.

cch100:10:05

A bit of investigation shows that mvn-install does not copy the pom file to the local repo -interestingly @seancorfield has observed that slipset/deps-deploy does install the pom file. So is that a bug or a limitation/design decision in c.t.d. mvn-install?

seancorfield00:10:29

@alexmiller I confirmed with @cch1 that running clojure -X:deps mvn-install omits the .pom file and when trying to use that dependency in a clean folder, t.d.a. reaches out to Clojars to look for the "missing" .pom file. When I re-ran the same scenario but using slipset/deps-deploy for the local install, it did copy pom.xml to <library>.pom in ~/.m2 and when trying to use that in a clean folder, t.d.a. did not reach out to Clojars to look for the .pom file. Definitely feels like a bug in t.d.a's mvn-install to me.

Alex Miller (Clojure team)00:10:32

can you back up to the full command you're running?

Alex Miller (Clojure team)00:10:07

are you supplying a pom? using the one in a jar file? supplying additional attributes?

seancorfield00:10:37

There is a pom.xml in the same directory. I ran just clojure -X:deps mvn-install :jar '"the-lib.jar"' -- do we need extra args to also copy the pom.xml file? It reads the pom.xml file to get the group/artifact/version data...

seancorfield00:10:42

The same pom.xml file is inside the JAR as well (so that's the one it reads, based on the reference docs).

seancorfield00:10:24

Ah, if I say :pom '"pom.xml"' as well, then it does copy that to <library>.pom -- is that an intentional difference @alexmiller?

seancorfield00:10:29

Here's a full repro session:

(! 692)-> rm -rf ~/.m2/repository/seancorfield/
(! 693)-> jar tvf next-jdbc.jar |fgrep pom
   110 Mon Oct 19 11:24:06 PDT 2020 META-INF/maven/seancorfield/next.jdbc/pom.properties
  1996 Mon Oct 19 11:24:06 PDT 2020 META-INF/maven/seancorfield/next.jdbc/pom.xml
(! 694)-> clojure -Sforce -X:deps mvn-install :jar '"next-jdbc.jar"'
Installing next-jdbc.jar 
Installed to /Users/sean/.m2/repository/seancorfield/next.jdbc/1.1.610
(! 695)-> ls -lR ~/.m2/repository/seancorfield/
total 0
drwxr-xr-x  4 sean  staff  136 Oct 19 17:51 next.jdbc

/Users/sean/.m2/repository/seancorfield//next.jdbc:
total 8
drwxr-xr-x  4 sean  staff  136 Oct 19 17:51 1.1.610
-rw-r--r--  1 sean  staff  305 Oct 19 17:51 maven-metadata-local.xml

/Users/sean/.m2/repository/seancorfield//next.jdbc/1.1.610:
total 96
-rw-r--r--  1 sean  staff    164 Oct 19 17:51 _remote.repositories
-rw-r--r--  1 sean  staff  44300 Oct 19 11:24 next.jdbc-1.1.610.jar
(! 696)-> rm -rf ~/.m2/repository/seancorfield/
(! 697)-> clojure -Sforce -X:deps mvn-install :jar '"next-jdbc.jar"' :pom '"pom.xml"'
Installing next-jdbc.jar and pom.xml
Installed to /Users/sean/.m2/repository/seancorfield/next.jdbc/1.1.610
(! 698)-> ls -lR ~/.m2/repository/seancorfield/
total 0
drwxr-xr-x  4 sean  staff  136 Oct 19 17:52 next.jdbc

/Users/sean/.m2/repository/seancorfield//next.jdbc:
total 8
drwxr-xr-x  5 sean  staff  170 Oct 19 17:52 1.1.610
-rw-r--r--  1 sean  staff  305 Oct 19 17:52 maven-metadata-local.xml

/Users/sean/.m2/repository/seancorfield//next.jdbc/1.1.610:
total 104
-rw-r--r--  1 sean  staff    188 Oct 19 17:52 _remote.repositories
-rw-r--r--  1 sean  staff  44300 Oct 19 11:24 next.jdbc-1.1.610.jar
-rw-r--r--  1 sean  staff   1996 Oct 19 11:16 next.jdbc-1.1.610.pom
(! 699)-> ls -l pom.xml
-rw-r--r--  1 sean  staff  1996 Oct 19 11:16 pom.xml

seancorfield00:10:50

I see Installing next-jdbc.jar in the first case and Installing next-jdbc.jar and pom.xml in the second case so I suppose it is telling you what it is doing...

seancorfield00:10:19

Without the .pom file in the repo, attempts to use the dependency will still reach out to Clojars/Maven for that .pom file and if it isn't found, it doesn't look like the pom.xml inside the JAR is sufficient for t.d.a to pick up the dependencies.

Alex Miller (Clojure team)01:10:04

reaching out to maven is surprising though, so I'll look at that tomorrow

seancorfield01:10:03

Downloading: seancorfield/next.jdbc/1.1.610/next.jdbc-1.1.610.pom from clojars
^ That's what I get trying to use the installed dependency without the .pom file in the repo. If I install the pom.xml file as well (or use slipset/deps-deploy install) so that I get the .pom file in the repo, that line does not appear when trying to use the dependency.

seancorfield01:10:27

Feel free to DM me tomorrow if you need me to run more test cases.

seancorfield01:10:48

(and there was no attempt to download the .jar file, only the .pom file)

Alex Miller (Clojure team)01:10:52

sorry, maven looking for the pom file is normal behavior, I thought you were saying it was reaching out for the pom file when installing

seancorfield01:10:45

Right, but that will fail if there is only a local install -- and the problem Chris was seeing was that if no .pom file existed, transitive dependencies were not being found.

Alex Miller (Clojure team)01:10:15

well that's to be expected

Alex Miller (Clojure team)01:10:24

the question is what you supply on install

Alex Miller (Clojure team)01:10:43

you have to either supply a pom, or use the pom in the jar, or is this the case where you did neither?

seancorfield01:10:03

There is a valid pom.xml inside the JAR -- I showed that above.

Alex Miller (Clojure team)01:10:27

sorry, too much reading, and I'm a drink into the night :)

seancorfield01:10:43

The problem is that the .pom file is not created in the local repo on the install. Unless you explicitly specify :pom '"pom.xml"'

Alex Miller (Clojure team)01:10:11

ok, that's surprising to me, should be using the one in the jar in that case

seancorfield01:10:12

OK, we can follow-up tomorrow. Feel free to DM me about it.

seancorfield01:10:44

See my console session above.

Alex Miller (Clojure team)01:10:10

yeah, that code is not there. it is using the pom in the jar to pull G/A/V but it's not actually installing it as a distinct artifact. not sure why I thought that would happen automatically, maybe tricked myself with some caching or something

seancorfield01:10:53

So, definitely a bug? Want me to create a JIRA?

Alex Miller (Clojure team)01:10:02

definitely a bug, sure on jira

seancorfield01:10:20

seancorfield/depstar {:mvn/version "1.1.132"} -- calls shutdown-agents at the end to account for "badly-behaved" code being AOT'd that has side-effecting top-level forms that start threads (e.g., Neanderthal) -- follow-up in #depstar

cch101:10:08

Sorry I’m late catching up here. I don’t have much more to offer than what Sean noted above. Let me know if I can help test.

Alex Miller (Clojure team)21:10:02

Clojure tools 1.10.1.723 (a prerelease) is now available: • Fix clj -X:deps tree adding tools.deps.alpha to tree • Fix clj -X:deps mvn-pom adding tools.deps.alpha to pom deps • Fix clj -X:deps git-resolve-tags not working • Fix clj -X:deps mvn-install on jar to also install embedded pom

🙂 6
24
Alex Miller (Clojure team)21:10:15

the first 3 were actually all a bug in the default :deps alias using :extra-deps rather than :replace-deps (duh)