Fork me on GitHub
#tools-deps
<
2020-12-08
>
Alex Miller (Clojure team)04:12:51

I released clj 1.10.1.754 tonight, which has a bug fix for -Stree from yesterday and updates the windows scripts. If anyone wanted to give it a try I can promote it to stable. You should see new format for clj -Stree and same for clj -X:deps tree (see https://clojure.github.io/tools.deps.alpha/clojure.tools.cli.api-api.html#clojure.tools.cli.api/tree for more options)

seancorfield04:12:55

I updated my two dev systems and tested on a few projects, looks good. Updated the work system and ran -Stree against a few of our subprojects, looks good. @alexmiller

👍 3
onetom04:12:32

is it possible to run multiple functions via clojure, using multiple -X options?

Alex Miller (Clojure team)04:12:42

once you do -X, all args afterwards go to the function you're running

seancorfield04:12:43

@onetom You could always do -e "(do (invoke-fn1) (invoke-fn2))" 🙂

👍 3
seancorfield04:12:01

(well, using requiring-resolve for each function)

Alex Miller (Clojure team)04:12:55

A new version of the clojure tools is now available (https://clojure.org/releases/tools#v1.10.1.754) • New, more informative tree format for `clj -Stree` / `clj -X:deps tree` • Added https://clojure.github.io/tools.deps.alpha/clojure.tools.cli.api-api.html#clojure.tools.cli.api/tree for use with `clj -X:deps tree` • Use https://github.com/clojure/tools.deps.alpha/blob/master/CHANGELOG.md 0.9.857

Alex Miller (Clojure team)04:12:17

winding it's way through the homebrew tubes

onetom05:12:53

1st attempt via a Nix derivation:

: clj -X:deps tree
Downloading: org/clojure/tools.deps.alpha/0.8.677/tools.deps.alpha-0.8.677.pom from central
Downloading: org/clojure/tools.deps.alpha/0.8.677/tools.deps.alpha-0.8.677.jar from central
Unqualified function can't be resolved: tree
: which clj
/nix/store/g7cpwnmzfl9ikxnm7gbnq3qqgj0mynwk-clojure-1.10.1.754/bin/clj

onetom05:12:14

clj -Stree seems to work though. it has nice X-es and dots in front of the dependency names

seancorfield05:12:11

@onetom Sounds you have a :deps aliases that doesn't come from the system deps.edn file?

seancorfield05:12:56

That version of t.d.a is old.

seancorfield05:12:12

The new CLI uses 0.9.857

seancorfield05:12:00

seanc@DESKTOP-30ICA76:~/oss$ clojure -A:deps -Stree
org.clojure/clojure 1.10.1
  . org.clojure/spec.alpha 0.2.176
  . org.clojure/core.specs.alpha 0.2.44
org.clojure/tools.deps.alpha 0.9.857

onetom05:12:08

i know it's the old one, that's why im surprised and showed what happened.

seancorfield05:12:35

Like I said, you have a :deps alias in your user deps.edn I suspect.

onetom05:12:05

ah, indeed. i probably copied it from your ~/.clojure/deps.edn

seancorfield05:12:28

@alexmiller I notice that mvn-pom takes an argument argmaps as a vector of aliases -- I understand why it's called that but it's... less than ideal 🙂 aliases seems like a more intuitive name if we're going to standardize on a way to provide a list of aliases to tooling?

seancorfield05:12:09

(is the tree function possibly going to take an aliases (or argmaps) argument so we can see trees based on, say, test dependencies?)

Alex Miller (Clojure team)06:12:52

you can do this for now with -Stree

Alex Miller (Clojure team)06:12:52

or you can use -Strace and whatever to create a trace.edn and feed it to -X:deps tree with :file

seancorfield06:12:55

Yeah, I know what the workarounds are 🙂

seancorfield06:12:04

I'm asking about a "better future" 🙂

Alex Miller (Clojure team)06:12:27

well, this is the same conversation we've already had, not sure what more you're looking for

seancorfield06:12:07

I was at least hoping for a comment on the name of the argument, given our past conversations.

Alex Miller (Clojure team)06:12:31

I'm not sure how to convey "haven't worked on it yet" more than previously :)

seancorfield06:12:57

Fair enough. I was fishing for an opinion or a direction, that's all.

seancorfield06:12:56

I mean, I'm about to do this for depstar so calling it :argmaps because that's what mvn-install uses is one approach, but calling it :aliases seems a bit more intuitive.

seancorfield06:12:34

(one big "downside" to the exec stuff is that the names of arguments become part of the API so this is more important than it used to be)

onetom05:12:25

I saw in https://clojure.org/reference/deps_and_cli there is a :default-deps option for factoring out version info for common dependencies. What's the recommended way to provide that option from a central location? (Something like https://github.com/walmartlabs/shared-deps is for lein)

seancorfield05:12:08

@onetom There are only three deps.edn files: the system (that you can't touch), your user file (in ~/.clojure), and your project file.

onetom05:12:13

i thought about using CLJ_CONFIG to override the per-user default within project environments, but that would also prohibit developers to access their personal tools...

seancorfield05:12:50

We used to do that. It "works" but it introduces all sorts of problems regarding tools. We stopped doing that.

onetom05:12:03

so do you require your devs so define certain aliases in their ~/.clojure/deps.edn in order for the walmart projects to work or that's the thing you mean you stopped doing?

onetom05:12:54

i for example moved the ion-dev deps into the project's own deps.edn because we were accidentally using an older version on one of the machines and it failed in mysterious ways...

onetom05:12:41

i just don't want to keep copy-pasting a lot of "boilerplate" setup, while we are in a monorepo, where it could be just centralized.

seancorfield05:12:42

Unfortunately, there's no "good" solution for monorepos at this point. Or, more generally, for providing "shared project deps". Feel free to upvote this issue https://ask.clojure.org/index.php/9849/teams-common-dependencies-tooling-across-multiple-projects

👍 3
seancorfield06:12:02

We used to use the CLJ_CONFIG "hack". Now we're using a different "hack" that involves generating deps.edn from template files. Neither are good.

onetom06:12:12

good to hear that "it's not just me" 🙂 the problem is that i don't even know what would a great solution look like... otherwise i would hack together something

onetom06:12:56

thanks for sharing the problem statement!

seancorfield06:12:39

I think a reasonable solution is to be able to specify one-or-more additional deps.edn files that should be processed in this order: system -> user -> <specified deps.edn files in order> -> project deps.edn -> command-line -Sdeps

3
seancorfield06:12:53

but that touches a lot of parts of tools.deps.alpha

onetom05:12:16

i see and can probably do clj -Sdeps $(cat ../default-deps.edn) ..., but can i do that at some more fundamental level?

seancorfield05:12:50

:default-deps only works in an alias BTW, as does :override-deps

👍 3
cddr17:12:49

I'm trying to build a clojure project in circleci and it depends on the cognitect-labs test-runner. This all works fine locally but when circleci tries to build, I get an authentication error when it tries to download dependencies. I've seen other folks with this problem and the solution is usually to fix the entry in deps.edn to use the "https:" pointer to the test-runner rather than the "git@" pointer but my project already does that. https://app.circleci.com/pipelines/github/cddr/test-machine/6/workflows/05e75093-3a28-438c-97b0-7bd0f9bc5132/jobs/6 I'm using the "clojure -P -M:runner" trick to download dependencies and save the cache before actually trying to run the tests and it seems to be at this point where it fails. Would you expect that to work ok? The code for the project itself is here: https://github.com/cddr/test-machine

cddr17:12:14

Another question this has prompted is, where does clojure download repos it clones when a :git/url style dependency is used? Would be cool to delete previously downloaded repo in my local setup in order to test this locally without pushing to circleci each time I change something.

Alex Miller (Clojure team)17:12:03

You may have an insteadOf rule in your ci git config that is forcing ssh urls

cddr17:12:23

Ah interesting. I'll try to verify that. Thanks for the hint

Alex Miller (Clojure team)17:12:37

I have helped multiple people with the same problem

Alex Miller (Clojure team)17:12:27

This swaps before clj even sees it

cddr17:12:34

circleci@518e1f74cd0a:~$ cat .gitconfig
cat .gitconfig
[url ""]
	insteadOf = 
[gc]
	auto = 0
circleci@518e1f74cd0a:~$ 

cddr17:12:37

right enough

Alex Miller (Clojure team)17:12:23

Not sure if this is some recent change in default config

cddr17:12:19

Yeah I'm using one of their pre-built base images containing tools.deps. I'll report back with where it sets this ~/.gitconfig if I can find it 🙂

practicalli-johnny18:12:55

@cddr there is a discussion in #circleci on this very issue, with more details

borkdude18:12:04

@cddr yep, had exactly the same issue. "fixed" it by adding rm -f ~/.gitconfig to my circleci builds

👍 3
kirill.salykin18:12:45

Hi is there a way to use paths/deps from an alias but skip the main-opts for instance, for this test alias

:test {:extra-paths ["test"]
         :extra-deps {lambdaisland/kaocha           {:mvn/version "1.0.690"}
                      lambdaisland/kaocha-cloverage {:mvn/version "1.0.63"}
                      lambdaisland/kaocha-junit-xml {:mvn/version "0.0.76"}
                      ring/ring-mock                {:mvn/version "0.3.2"}
                      mockery                       {:mvn/version "0.1.4"}
                      http-kit.fake/http-kit.fake    {:mvn/version "0.2.1"}}
         :main-opts  ["-m" "kaocha.runner"]}
is there a way to skip the main-opts ? It seems that it is being used and this fails cider-jack-in I am looking for analog of R/C in old syntax

seancorfield18:12:03

@kirill.salykin The general recommendation is to separate that sort of thing into multiple aliases.

seancorfield18:12:59

So you have a :test alias with the deps/paths you need to access/run your test code in the REPL and a :runner alias that is the Kaocha deps :main-opts (and any additional deps needed)

kirill.salykin18:12:01

is there are reason for such change? -R -C seemed quite useful to me

seancorfield18:12:33

I suspect cider-jack-in should be adding its own :main-opts after the :test alias (so that it "wins" in the merge process)

kirill.salykin18:12:33

Indeed, this is exactly what happens

jack-in command: /usr/local/bin/clojure -Sdeps '{:deps {nrepl {:mvn/version "0.8.3"} cider/cider-nrepl {:mvn/version "0.25.5"}}}' -A:dev:test -m nrepl.cmdline --middleware '["cider.nrepl/cider-middleware"]'

seancorfield18:12:53

No, that's not what I meant.

seancorfield18:12:10

I literally meant :main-opts so that it is merged last and "wins"

kirill.salykin18:12:37

-A:test sets -m which seems conflicting with -m from the cider

seancorfield18:12:42

Ultimately, -A will stop running :main-opts at some point according to @alexmiller

kirill.salykin18:12:03

ah, so we need to migrate anyway

seancorfield18:12:08

clojure -Sdeps '{:deps ... :aliases {:cider/jack-in {:main-opts ["-m" ...]}}}' -M:dev:test:cider/jack-in --middleware ...

seancorfield18:12:32

Since there is no way to specify :main-opts without an alias (at least, not right now).

kirill.salykin18:12:44

probably I should reach to cider people to discss this

kirill.salykin18:12:48

thanks for help!

dpsutton18:12:11

that's a clever idea. offhand do you know if that works for the past several versions of t.d.a?

dpsutton18:12:24

(don't research, just if you know off-the-cuff)

seancorfield18:12:42

Which "that" are you asking about?

dpsutton18:12:06

the main opts in the inline -Sdeps and last merged wins

seancorfield18:12:20

That's always been the way :main-opts works in aliases.

seancorfield18:12:37

(I assume you mean "Clojure CLI" not "t.d.a"?)

dpsutton18:12:14

yes. i always blank on how to specifically refer to the command line stuff, not the language, not the library it is built on, etc

dpsutton18:12:44

i'll go with Clojure CLI from now on

seancorfield18:12:10

This is another case where I think -R should have been retained for "REPL" operations and expanded to include both resolve-args (old behavior) and classpath-args -- in the same way -M was expanded to include resolve-args and classpath-args -- in addition to just :main-opts (old behavior). And -A deprecated. That would have been an easier migration path for people. But that ship has sailed 😞

seancorfield18:12:12

@U11BV7MTK Mostly I nitpick on that because the versions of Clojure, the Clojure CLI, and the underlying t.d.a are all different.

kirill.salykin18:12:56

> This is another case where I think `-R` should have been retained for “REPL” operations and expanded to include both resolve-args (old behavior) and classpath-args -- in the same way `-M` was expanded to include resolve-args and classpath-args -- in addition to just `:main-opts` (old behavior). And `-A` deprecated. That would have been an easier migration path for people. But that ship has sailed that would be ideal indeed

dpsutton18:12:02

i appreciate it. i hope it didn't sound snarky. i always grasp for the right way to phrase it so thanks

seancorfield18:12:08

(I always start my REPL manually and then connect to it from my editor)

borkdude18:12:35

me too, never use cider-jack-in, don't want my editor to bring down the dev process, ever

practicalli-johnny11:12:49

This is a good point I had not considered. This point and given the different opinions on how to resolve cider-jack-in with clojure cli tools, I will switch to cider-connect and update the book to use that as the main approach for anything but a beginning. I have all the aliases I need to connect CIDER, Calva and Conjure to a repl run in a terminal, and they are updated monthly. Thank you.

seancorfield18:12:10

This is another case where I think -R should have been retained for "REPL" operations and expanded to include both resolve-args (old behavior) and classpath-args -- in the same way -M was expanded to include resolve-args and classpath-args -- in addition to just :main-opts (old behavior). And -A deprecated. That would have been an easier migration path for people. But that ship has sailed 😞

practicalli-johnny22:12:27

@kirill.salykin you can use an alias like :middleware/cider-clj in practicalli/clojure-deps-edn and a .dir-locals.el to switch off the auto-injection of cider dependencies, however it seems very strange to run two separate tools from one Clojure command, espcially as Kaocha does not interact with the REPL as I understand it, its only looking as the file system. This just seems weird I am afraid...

kirill.salykin07:12:29

The intention is not to run two separate tools - but to use the deps and paths from alias (ignoring the -m). Similar to what was possible to achieve using -c and -r opts

practicalli-johnny09:12:32

The simplest approach is to have two aliases, one for test and one for a test runner. That is what I do any what I have seen from others. Defining specific is also more efficient, as you are only loading in libraries that you are actually using. Loading in kaocha and it's great many dependencies adds to startup time and also more potential for library version conflicts, especially on larger Clojure projects.

kirill.salykin09:12:41

I understand that two separate aliases will work But I was looking for “-C” / “-R” analogs which allows you just pick paths/deps from the alias (now it is deprecating, so was asking how one can to do something similar with a new syntax)

practicalli-johnny09:12:48

The issue you raise is easily solved, just add an alias that runs the nREPL.middleware to the end of the chain of aliases. I used your :test alias with the :middleware/cider-clj from my practicalli/clojure-deps-edn and Cider jack-in works without any issues

practicalli-johnny09:12:57

Or you can just set cider variables to switch the cider auto-inject off Nd just use whatever aliases you wish, so long as a cider one is last.

practicalli-johnny09:12:09

On the command line, clojure -M:foo:bar:baz:lizard:spock would only run the main-opts from :spock even if all of the aliases have :main-opts configuration. Again this is simpler than having lots of different flags to remember.