Fork me on GitHub
#tools-deps
<
2020-09-09
>
cfleming00:09:21

Just double checking - there is no way to get the deps of a subproject specified via :local/root including deps from an alias, correct? i.e. I cannot specify an alias together with :local/root in some way?

seancorfield00:09:56

Dependencies are opaque in that respect.

seancorfield00:09:06

Although... I guess you could read the basis (in the prerelease CLI) and figure out the actual :local/root path and see if it has deps.edn and then use t.d.a to read that and then inspect its aliases... 👀

seancorfield00:09:12

But a :local/root could point to a JAR, or a directory with pom.xml. Or project.clj 🙂

cfleming00:09:27

This is just specifying deps in a deps.edn file, so I can’t do anything funky or programmatic.

hugod01:09:26

What is the rationale for deprecating unqualified dependency symbols in :deps? (couldn’t see one in the release notes)

seancorfield01:09:46

(IOW it was deprecated in an earlier prerelease)

hugod01:09:25

Thanks - I missed that release 🙂

seancorfield01:09:21

I spent the first week of August, scurrying around, fixing all our unqualified lib names at work, and all my open source projects! 🙂

cfleming02:09:32

Is there any way to substitute values into a deps file that I’m not aware of, e.g. from env vars or properties? In my case I would like to be able to substitute a global version in several deps files. I’m aware of default-deps, but I’d need to be able to substitute it in a :local/root path.

seancorfield02:09:31

Nope. No substitutions.

seancorfield02:09:06

(I'm just a bundle of "nope" today, aren't I?)

cfleming02:09:51

No worries, that was what I expected, just makes my life a little more tedious 🙂

hugod13:09:41

Fwiw, having played with the pre-release, I still think the following CLI switches would be simpler, as they separate the selection of aliases from what to execute, and would simplify the doc. A given set of aliases can be used for main, exec or repl, so it makes it easier to compose a command line from other tools if the passing of options is distinct from the selection of what to invoke.

clj     [clj-opt*] [-A:aliases] [init-opt*]
clojure [clj-opt*] [-A:aliases] -x [a/fn] [kpath v]*
clojure [clj-opt*] [-A:aliases] [init-opt*] [main-opt] [arg*]
clojure [clj-opt*] -P [other exec opts]
or it could be
clojure [clj-opt*] [-A:aliases] -M [init-opt*] [main-opt] [arg*]
If that is really required, though I confess I don’t see the rationale for having to specify the -M given that you’re passing -m or have a :main-opts key. The (at least my perceived) lack of usage of the current -T, `-R`, `-C`, and -O options supports the above.

👌 3
hugod13:09:13

That’s clear 🙂

seancorfield16:09:15

Also, I think there's more usage of -R out there than you suspect @hugod

hugod16:09:04

I’m sure I don’t have a good perspective on that, but don’t think it really changes my view that just using -A would be conceptually simpler.

hugod16:09:30

Given the cli changes seem fixed, I’ll just go and update my nascent build tool invoker project to cope with multiple versions of the cli.

seancorfield16:09:13

I still think that a) given the expansion of -M to include more than just :main-opts b) the overlap between -A and -M (both run :main-opts now) c) the current uses of -R (doesn't run :main-opts but does "most" of what -A does -- based on resolve args being the most common; it would make more sense for -R to be kept and expanded as the "REPL" usage (and quietly undocumenting -A). So we'd have -R (REPL), -M (main), -X (exec) and they all three respect all arg types in deps.edn and there would be no overlap on :main-opts or :exec-fn.

hugod17:09:45

I do find it unfortunate that the deps.edn isn’t authoritative. By this I mean that whatever you specify in deps.edn you still have to know whether to invoke it with -M or -X.

seancorfield17:09:08

That works with -A today and either -M or -X tomorrow!

hugod17:09:58

I don’t think that is quite what I meant. As a deps.edn writer I would like to provide an abstract, say test, alias for doing something, and have the user be able to invoke that independent of how I chose to implement that alias. Writing it both ways seems like duplication.

Alex Miller (Clojure team)18:09:17

well the arg structures are different, not sure how that could work

seancorfield19:09:29

Exactly, as a tool writer you need to specifically support -M style invocation and/or -X style invocation as two separate functions in your code: -main taking & args which are strings and some-exec-fn taking a single hash map as its argument.

hugod13:09:41

main obviously takes strings and an exec-fn wants an edn value, but that doesn’t need to dictate the format of arguments passed on the cli. The cli could print edn to strings before passing it as a main argument, or read literal strings to edn strings before passing to an exec-fn. As a strawman, with intentionally hideous names, consider this: i) default to passing args as literal string values, or add a --string switch to explicitly specify this ii) add an --edn switch, that causes the next argument to be read as EDN and passed as a value to an exec-fn or as a string to a main. iii) add a --patch switch that takes an edn path vector and an edn value and patches the default value. if the value is prefixed with --string treat it as a string value rather than EDN. Allow main to have default arguments specified, in the same way as exec-fn. iv) add a --rest switch that collects the remaining arguments into an EDN vector. --rest-strings could build an EDN vector from string arguments rather than edn values. This could be used to pass glob values, for example. v) allow an exec-fn to take multiple arguments. As described this is unwieldy, but if there were volition, I’m sure it could be improved upon. The main idea would be to divorce the specification of the format of the arguments passed on the command line from how they are passed to the implementing function.

jeremys19:09:13

Hi, I have been working on a https://github.com/JeremS/mbt based on tools.deps for some time. It has a mix of functionality inspired from several other projects and it's at a stage where I use it to version, build and deploy my projects. Maybe someone other than me might find it useful! Cheers

seancorfield19:09:57

@alexmiller It looks like clojure -X:deps mvn-pom now adds the following dependencies to all pom.xml files:

<dependency>
      <groupId>org.clojure</groupId>
      <artifactId>tools.deps.alpha</artifactId>
      <version>0.9.782</version>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-nop</artifactId>
      <version>1.7.25</version>
    </dependency>

seancorfield19:09:04

That's going to cause a lot of projects to accidentally end up depending on tools.deps.alpha (and the slf4j-nop dependency is likely to be unwelcome in many cases I suspect).

☝️ 3
seancorfield19:09:45

These deps are in a freshly-generated pom.xml file -- and they are also added to any existing pom.xml, potentially overwriting the dependency for any projects that use a different version of t.d.a

seancorfield19:09:43

How will you fix it? For example, my clj-new project does depend on t.d.a (and has a dependency on slf4j-nop because of that) -- but I know that's an outlier -- so you'd need to also not remove any existing dependency on those?

Alex Miller (Clojure team)20:09:51

I'll fix it by calculating the basis, not using the injected basis

Alex Miller (Clojure team)20:09:02

basically what it did before

seancorfield20:09:56

Won't that break anyone currently using clojure -A:some:aliases -Spom ?

Alex Miller (Clojure team)20:09:12

it doesn't have to - the injected basis includes those argmaps

Alex Miller (Clojure team)20:09:36

the calculated basis can use the project deps.edn + the injected basis argmaps

seancorfield20:09:43

Re: basis -- so -A and -X are distinguishable via the basis? Or am I misunderstanding?

Alex Miller (Clojure team)20:09:57

oh, I see what you're asking. well let me think about it more

Alex Miller (Clojure team)20:09:30

another option would be to take those additional aliases as args to the -X pom function

Alex Miller (Clojure team)20:09:41

that would be an all new thing though

Alex Miller (Clojure team)20:09:24

very similar thing exists in the datomic ion-dev tool which I was just helping with a bit :)

Alex Miller (Clojure team)20:09:04

I am planning to reinstate the -Spom as well so looking at that right now too

seancorfield20:09:25

If it's just going to be identical to -X:deps mvn-pom I wouldn't have thought it was worth the effort?

Alex Miller (Clojure team)20:09:56

you were the one here arguing it was breaking people last week... :)

seancorfield20:09:20

Hahaha... yeah, but you didn't seem very sympathetic to that argument, and I'm not sure having duplicate ways to do the exact same thing is better 🙂

Alex Miller (Clojure team)20:09:59

I had some thoughts on the -Stree thing you ran into though - I think there are several better ways to detect changing deps tree rather than scraping the -Stree stdout. not sure what your requirements on that stuff are but might be useful to talk about

Alex Miller (Clojure team)20:09:47

from a clj pov, we're writing edn files (.libs in particular) to the cache dir (can see location with -Sdescribe), so you could be slurping and diffing Clojure maps rather than comparing strings

seancorfield20:09:46

True, but a) how can you easily get the correct .libs file from a given invocation without scraping the output and b) that gives no sense of depth of transitivity, which is specifically what I rely on in -X:deps tree

seancorfield20:09:54

"Because reasons"... we go one level down in the full tree (so we consider top-level deps and the first level of transitive deps, but we don't care about lower levels).

seancorfield20:09:14

We've found that we can safely ignore the 2nd-order transitive deps and if we have a problematic 1st-order transitive dep, we just promote it to the top-level (and then its transitive deps are promoted to 1st-order transitive deps and we start to track them separately).

Alex Miller (Clojure team)20:09:42

well the transitivity stuff is all broken in the -Stree output

Alex Miller (Clojure team)20:09:17

but maybe you're just not encountering the cases where it's bad (mostly with exclusions)

seancorfield20:09:43

We don't use exclusions, for the most part.

seancorfield20:09:13

This is our only exclusion:

com.walmartlabs/lacinia {:mvn/version "0.35.0"
                           :exclusions [clojure-future-spec/clojure-future-spec]}

Alex Miller (Clojure team)20:09:52

the main thing is, the code never actually builds the real expansion tree as a tree. -Stree is kind of faking from the libs map

seancorfield20:09:18

We decided that managing conflicts via :exclusions was a bit of a nightmare so we control conflicts by promoting transitive deps to the top level.

seancorfield20:09:45

Right, but I'd have to write that "faking" code myself, whereas -X:deps tree already does a "good enough" job.

Alex Miller (Clojure team)20:09:22

well, it exists as a function you could call (the one -Stree calls) :)

Alex Miller (Clojure team)20:09:48

but that stuff is due for some significant changes soon as I fix up some of this stuff

seancorfield20:09:55

Besides, I've had this shell script running with -Stree for ages and don't really want to rewrite it as a Clojure program right now. Maybe when I'm bored and want to go shave some more yaks...

Alex Miller (Clojure team)20:09:47

ok, just letting you know that I'm not considering stdout changes from -Stree (or its replacement) to be breaking :)

Alex Miller (Clojure team)20:09:35

but I am planning to add some more api fns to get tree-like deps data

6
Alex Miller (Clojure team)20:09:56

and from a tools.deps pov, you could probably make something that did this via the api instead, but not sure if that's worth the effort

Alex Miller (Clojure team)20:09:26

I really consider -Stree / -X:deps tree to be human consumer tools and ones likely to hopefully improve over time, changing format. I don't consider the current particular output to be something a script should rely on.