Fork me on GitHub
#tools-deps
<
2021-06-21
>
dominicm07:06:55

Unclear from https://github.com/clojure/tools.deps.alpha/wiki/Tools is there a tool for tracking the licenses of dependencies in a deps.edn?

Cam Saul19:06:24

Sorry if this is a another dumb question. How can I programmatically get all the paths to a transient dependency with tools.deps? Example: org.apache.httpcomponents/httpclient is a transient dependency of both clj-http/clj-http and org.clojure/tools.deps.alpha. I tried resolve-deps , thinking maybe it would return both of those paths in :parents, but it doesn't:

(defn http-client-parents [deps]
  (-> (t/resolve-deps
       {:deps      deps
        :mvn/repos {"central" {:url ""}
                    "clojars" {:url ""}}}
       nil)
      (get-in ['org.apache.httpcomponents/httpclient :parents])))

(http-client-parents
 '{clj-http/clj-http {:mvn/version "3.10.3"}})
;; ->
#{[clj-http/clj-http] 
  [clj-http/clj-http org.apache.httpcomponents/httpmime]
  [clj-http/clj-http org.apache.httpcomponents/httpclient-cache]}

(http-client-parents
 '{org.clojure/tools.deps.alpha {:mvn/version "0.11.931"}
   clj-http/clj-http            {:mvn/version "3.10.3"}})
;; ->
#{[org.clojure/tools.deps.alpha org.apache.maven.resolver/maven-resolver-transport-http]}
What's the right way to do this?

Alex Miller (Clojure team)20:06:15

@camsaul I don't think the tools.deps api has a direct way to answer this question if I understand your question properly

Alex Miller (Clojure team)20:06:19

I think you're asking, given a set of root deps, why is a particular lib included in the expanded dep set?

Cam Saul20:06:16

Sort of. If a transient dep is include in the expanded dep set I want to know all of the root deps that would have caused that dep to be included

Alex Miller (Clojure team)20:06:27

the :parents keys are not part of the normative return of resolve-deps (that's an artifact of the expansion process)

👍 3
Alex Miller (Clojure team)20:06:14

but there are shortcuts in the expansion process that can cause the info in the return here to be incomplete

Alex Miller (Clojure team)20:06:41

it is probably better to derive this information from the tree expansion (the code that backs -Stree)

Alex Miller (Clojure team)20:06:18

if you call make-tree with the lib-map that will be an actual tree structure

Alex Miller (Clojure team)20:06:26

you should be able to find the leaves you want and trace their path back up

Cam Saul20:06:29

I actually did look at make-tree and -Stree but it does the same thing. The don't include httpclient as a dependency of clj-http/clj-http

borkdude20:06:51

-Stree can also print EDN right? you can also parse that on the command line I guess

Cam Saul20:06:08

Here's an example (gist because it's a lot of output): https://gist.github.com/camsaul/4e8c3977ce3f423e7bfd090d5c816da3 org.apache.httpcomponents/httpclient only appears once

borkdude20:06:12

clojure -X:deps tree :format :edn

Alex Miller (Clojure team)20:06:16

you can do that, he'd ask for programmatic

Alex Miller (Clojure team)20:06:33

sorry, I said make-tree above but that's the old tree printer code which is kind of vestigial now

Alex Miller (Clojure team)20:06:55

the newer stuff is under clojure.tools.deps.alpha.tree/trace->tree which importantly is fed by a trace log which comes from running resolve-deps with the trace flag set to true

Alex Miller (Clojure team)20:06:20

(require '[clojure.tools.deps.alpha :as deps] '[clojure.tools.deps.alpha.tree :as tree])
  (let [deps '{:deps      {org.clojure/tools.deps.alpha {:mvn/version "0.11.931"}
                           clj-http/clj-http            {:mvn/version "3.10.3"}}
               :mvn/repos {"central" {:url ""}
                           "clojars" {:url ""}}}
        libs (deps/resolve-deps deps {:trace true})
        tree (trace->tree (-> libs meta :trace))]
    (clojure.pprint/pprint tree))

Alex Miller (Clojure team)20:06:57

some sketch code to get the tree (this will give you a ton of output)

Alex Miller (Clojure team)20:06:43

the docstring for trace->tree gives you more info on the structure of the tree - basically it's a tree of map nodes keyed by lib, value is a map with a :children vector (which are more nodes)

👍 3
Cam Saul20:06:34

awesome, thanks! Looks like it shows up under both top-level libs there. I can use that to build what I need

Alex Miller (Clojure team)21:06:28

importantly, also check the :include flag! this tree includes nodes that are NOT included in the final deps (:reason tells you why)

👍 3
Cam Saul22:06:18

I have one more question that I couldn't seem to find an answer for on http://ask.clojure.org. What do people with Java source files do to approximate Leiningen's :java-source-paths functionality? Of course, it's easy enough to do find java -name '*.java' | xargs javac -cp $(clojure -Spath) or whatever before running stuff, or even put that logic in an alias (e.g. clojure -X:compile-java ), but it's a nicer dev experience if I don't have to run two commands instead of one for everything I want to do. I'm less worried about that tho, and more worried about the experience for contributors and other people using our project. I don't want to have to change our local dev server instructions from something relatively simple like lein run to something hairy like clojure -X:compile-java && clojure -X:dev-server . If there were a way to add custom tools.deps "extensions"https://github.com/clojure/tools.deps.alpha/tree/master/src/main/clojure/clojure/tools/deps/alpha/extensions I think I could get this working pretty easily with something like

{:deps {my-project/java-source {:com.my-company.java-source/local "/path/to/java/src"}}
But I'm guessing that sort of extension support isn't something that's likely to make it into the CLI any time soon. Other than doing something like setting up CI to compile the Java sources and publish them somewhere (e.g. by uploading it to an S3 bucket structured as a Maven repo), is there a way I can avoid requiring people to run a separate compile command before they are able to do anything else with the project?

Alex Miller (Clojure team)22:06:30

Coming very soon

🎉 4
2
Cam Saul22:06:33

Awesome, that's great to hear!