Fork me on GitHub
#tools-deps
<
2020-10-08
>
rickmoynihan15:10:02

Does anyone know if https://clojure.atlassian.net/browse/TDEPS-163 is likely to be implemented any time soon?

Alex Miller (Clojure team)15:10:44

probably not, but I did think more about it - the time when it's read by clj is in tools.deps.alpha, which is the process that builds the classpath. at that point you don't have any deps on the classpath (b/c that's what you're making). so it is more complicated than it initially appeared.

rickmoynihan15:10:30

ahh so the data-readers aren’t even available yet?

Alex Miller (Clojure team)15:10:18

tools.deps could do what's in the ticket, and that would open that up for library use of tools.deps. but it won't help you via clj.

rickmoynihan15:10:31

not sure I understand. Would it not be possible to read it with a :default reader in the first pass, and then re-read it later when you have that info?

Alex Miller (Clojure team)15:10:09

no? you would need to start a whole new jvm to do that

borkdude15:10:20

@alexmiller With regards to env vars your reasoning was: the less tools have to know, the better it is. Doesn't that also go for reader tags?

borkdude15:10:09

when opening up reader tags, one could implement one which reads env vars (or wait, it was resolving your home dir), which brings the same trade-offs maybe

rickmoynihan16:10:35

Yeah I guess it makes sense not to do it then, thanks for considering it though 🙇

vlaaad16:10:47

what if reading tags is done with tagged-literal?

vlaaad16:10:22

and then basis can be read with proper reader tags when there is a classpath..

vlaaad16:10:19

What’s the problem reader tags are supposed to solve though?

rickmoynihan16:10:44

giving a data representation to otherwise un(print|read)able things.

rickmoynihan16:10:07

So my use case is essentially: I’m using integrant to define systems of components. Those systems are typically wired together in a base.edn file on the classpath. That file contains a map of integrant components. However it would be useful to override some parts (key/values) in the integrant system via the command line, and start the thing with clojure -X Or in prodution allow a user to specify their own profiles as :exec-args data in the deps.edn file directly. One of the things you may want to override though is an #ig/ref which basically states this component depends on this other component. My current example is a program that I want to communicate with two different deployments of the same system. One of which is authenticated via an auth0 provider, the other is identical but authenticated via a basic auth provider. I was hoping tagged reader support in the cli tooling would help with this; but it seems I’ll need to implement something to do this. It’s not too complicated; but it’s something I could use in a lot of projects, so I was hoping I could just leverage the cli tools rather than reimplement it each time; or require a library function to do it.

Alex Miller (Clojure team)16:10:14

You can read the deps.edn yourself and have any data you like in the aliases

rickmoynihan09:10:30

I’d be happy to do that, but it would also be useful if the cli tools didn’t choke on it:

{:aliases {:reader {;;:exec-fn foo/bar
                    :exec-data {:tag #foo [some stuff]}}
           }}
$ clj -X:reader
Error building classpath. Error reading edn. No reader function for tag foo (/Users/rick/tmp/deps.edn)
If they could set :default to clojure.core/tagged-literal this way would I think work for me. Is that what you mean by this? > A good problem would be to avoid any error like that while computing the classpath from the deps. > That I would be happy to do

rickmoynihan09:10:14

ahh also seen @U47G49KHQ appears to be suggesting this :thumbsup:

vlaaad09:10:22

yep, that’s what I suggested

👍 3
rickmoynihan09:10:08

Though I see now it wasn’t clear

Alex Miller (Clojure team)13:10:12

Oh, I didn’t change it for -X param reading though, I’ll get that too before the next release

vlaaad13:10:15

oh, that’s great!!!

vlaaad13:10:23

#clj/requiring-resolve foo.bar/baz here I come

3
rickmoynihan13:10:52

Ahh fab! Thanks @alexmiller 🙇

vlaaad13:10:28

Or bringing #= to edn with #clj/eval (some-code)

3
Alex Miller (Clojure team)16:10:04

A good problem would be to avoid any error like that while computing the classpath from the deps.

Alex Miller (Clojure team)16:10:24

That I would be happy to do

vlaaad16:10:46

Hmm, agree, supplying non-data args via -X is complicated. My api functions in Reveal are supposed to only accept functions as arguments, but to make them available from clj-exec, I have to accept symbols as well and introduce polymorphism just to satisfy clj-exec: https://github.com/vlaaad/reveal/blob/master/src/vlaaad/reveal.clj#L11

vlaaad16:10:37

If deps supported tagged literals, I wouldn't have to pollute my api

ghadi16:10:41

there is no straightforward way to do it. I wouldn't gate yourself on deps

dharrigan17:10:18

I'm curious - what is the caching strategy used for looking up dependencies - I know that a particular dependency has been updated, it shows on clojars, but referencing it as 'RELEASE' in the deps.edn (my main deps.edn, i.e., $HOME/.clojure/deps.edn) I can see that it it's not being pulled down.

hiredman17:10:51

use real versions

borkdude17:10:20

@dharrigan it's a simple string hash on the user deps.edn, args and local deps.edn. Use -Sforce to update

dharrigan17:10:17

Sure, I could do that - but I see for example in the popular seancorfield deps.edn, a lot of references to RELEASE as well. I mean it's not wrong to use RELEASE, it is supported. So surely, something must eventually attempt to find out if it is the latest release or not?

seancorfield18:10:58

@dharrigan The only reason that some things in there use "RELEASE" is so that folks can "guarantee" a recent version of development tools when they first lift aliases from that file (or use the file whole). @hiredman is correct that this is "wrong" insofar as it breaks CLI caching and as @alexmiller says, this is not officially supported by the CLI/t.d.a so it's certainly not a good idea for anything other than quick access to dev tooling. I stopped using it in all the templates baked into clj-new a while back, although it is still used when looking up templates (on Clojars/Maven) to try to get the most recent stable release of a template -- and boot-new and lein new before it all use that same approach, with the same flaws: you can always use the -V / --version option to fetch a specific version of a template if you want/need. I am planning to make a pass over depstar and clj-new to make -X the favored way to invoke those now, including in the generated app template from clj-new, and at that point I'll also make a pass over my dot-clojure repo and start pinning versions, instead of "RELEASE", to ensure folks get a "recent-enough" version of things that actually support -X. Unfortunately, this sort of change is a big ball of yarn that causes a lot of stuff to need unraveling...

👍 3
hiredman17:10:39

it is wrong

borkdude17:10:02

RELEASE is something that maven supports, but it's not encouraged by deps.edn, it just works because it works in maven. They will deprecate it.

hiredman17:10:10

RELEASE is a mutable reference

hiredman17:10:34

building caching on top of that is terrible, and clj relies on caching

dharrigan17:10:24

Okay, fairy enough. I will use the version, not release 🙂

hiredman17:10:54

RELEASE is an easy vs. simple thing where people consistently choose what is easy

dharrigan17:10:03

Indeed 🙂 We're all lazy 🙂

dharrigan17:10:23

$deity knows I didn't become a programmer to be active 🙂

Alex Miller (Clojure team)17:10:26

RELEASE is not officially supported by clj

Alex Miller (Clojure team)17:10:36

use at your peril (it breaks the cache)

dharrigan17:10:26

tbf, it (this library) was the only one in my deps.edn that has release for the coordinate

Alex Miller (Clojure team)17:10:33

imo you should only ever use it in -Sdeps on the command line, not in a deps.edn file. I reserve the right to enforce that opinion at an arbitrary future time :)

borkdude17:10:24

it's on the list for clj-kondo warnings here: https://github.com/borkdude/clj-kondo/issues/945

dharrigan17:10:03

Something else, does deps use the .m2/settings.xml at all?

Alex Miller (Clojure team)17:10:20

yes, for some things (like server creds, mirrors, proxies)

dharrigan17:10:26

right right, good. thanks 🙂

hiredman17:10:55

proxies for fetching maven deps, not for git deps?

Alex Miller (Clojure team)17:10:03

yes, that is still a todo

Alex Miller (Clojure team)17:10:17

there is a ticket for it, iirc

borkdude17:10:03

deps.clj has support for http / https proxy settings: https://github.com/borkdude/deps.clj - not sure if that's what you mean https://github.com/borkdude/deps.clj#proxy-environment-variables

borkdude18:10:27

I think it works in deps.clj because the proxy settings are also passed to the make-classpath invocation

dpsutton18:10:27

i can get the version from clojure by running clojure --help but it prints with lots of other text obviously. is there a way to get just the version?

dpsutton18:10:05

we have a script that invokes -m and wondering whether its better to omit the -M and get a warning or include it and break usage for people who are on older versions

Alex Miller (Clojure team)18:10:23

clj -Sdescribe will return edn content that includes a :version key

borkdude18:10:42

@dpsutton

$ clojure -Sdescribe | jet --query :version
"1.10.1.697"

👍 3
Alex Miller (Clojure team)18:10:03

note that the warnings are on stderr so you can redirect stderr to null too if you want to not see those

borkdude18:10:00

$ clojure -Sdescribe | clojure -M -e "(require '[clojure.edn :as edn]) (:version (edn/read *in*))"
"1.10.1.697"

borkdude18:10:37

$ clojure -Sdescribe | bb '(:version *input*)'
"1.10.1.697"
ok, I'm done now :)

Alex Miller (Clojure team)18:10:04

but I have all these cat skins lying about...

borkdude18:10:54

k, one more:

$ clojure -Sdescribe | grep :version | cut -d' ' -f2
"1.10.1.697"

lread18:10:43

@dpsutton, I’ve moved to the new clojure command line syntax for rewrite-cljc. I have its scripts fail with an error message if clojure version < 1.10.1.697. My scripts are all in babashka so the check was easy.

👍 3
dpsutton18:10:52

thanks everyone

seancorfield18:10:58

@dharrigan The only reason that some things in there use "RELEASE" is so that folks can "guarantee" a recent version of development tools when they first lift aliases from that file (or use the file whole). @hiredman is correct that this is "wrong" insofar as it breaks CLI caching and as @alexmiller says, this is not officially supported by the CLI/t.d.a so it's certainly not a good idea for anything other than quick access to dev tooling. I stopped using it in all the templates baked into clj-new a while back, although it is still used when looking up templates (on Clojars/Maven) to try to get the most recent stable release of a template -- and boot-new and lein new before it all use that same approach, with the same flaws: you can always use the -V / --version option to fetch a specific version of a template if you want/need. I am planning to make a pass over depstar and clj-new to make -X the favored way to invoke those now, including in the generated app template from clj-new, and at that point I'll also make a pass over my dot-clojure repo and start pinning versions, instead of "RELEASE", to ensure folks get a "recent-enough" version of things that actually support -X. Unfortunately, this sort of change is a big ball of yarn that causes a lot of stuff to need unraveling...

👍 3
dharrigan18:10:27

w00t. It will certainly help those coming fresh to Clojure and seeing your excellent deps.edn example.

seancorfield18:10:33

Or the Practicalli example (nicer web site, more descriptive names, much better documentation). I'm considering linking to it from my dot-clojure repo and encouraging new folks to use the Practicalli stuff instead -- so I'm under less pressure to keep dot-clojure maintained and documented 🙂

👍 3
practicalli-johnny21:10:48

@seancorfield I would be honoured to be linked from your repo. Thank you. The library versions used in the Practicalli aliases are updated at least once a month using the :project/outdated alias and version changes added to the changlog. Versions are updated sooner if I spot a new version or a library maintainer let's me know.

👍 3
seancorfield00:10:26

https://github.com/seancorfield/dot-clojure has been updated with caveats about it being my personal deps.edn file and to direct people to your repo if they want something well-documented and well-maintained!

👍 3
seancorfield21:10:18

seancorfield/depstar {:mvn/version "1.1.126"} -- https://github.com/seancorfield/depstar/releases/tag/v1.1.126 -- just three small bug fixes; follow-up in #depstar

seancorfield22:10:09

I was planning to make -X the preferred way of using depstar in that release but I realized that -X:uberjar ends up putting the Clojure CLI's version of tools.deps.alpha into the JAR file -- because depstar assumes the "current classpath" is what you want in the JAR (and it excludes itself, assuming it is the only dependency you don't want in the JAR). -X:jar works just fine (because it excludes JAR files). I'll need to modify depstar to use t.d.a directly to build a classpath from deps.edn files (per the discussion the other day) but that gets complicated if you need a "user" deps.edn file and you're using CLJ_CONFIG with a monorepo -- as we are at work -- so that will have to wait for "depstar 2". Given the discussions around that and t.d.a itself recently in this channel, I'll probably defer that work until t.d.a has better support for that (since CLJ_CONFIG is part of the clojure script, not t.d.a).

seancorfield22:10:23

BTW, CLJ_CONFIG is documented officially (I thought it wasn't):

User - cross-project configuration (typically tools)

Locations used in this order:

If $CLJ_CONFIG is set, then use $CLJ_CONFIG (explicit override)
https://clojure.org/reference/deps_and_cli#_deps_edn_sources

hugod22:10:32

Is it possible to ignore a project’s deps.edn when invoking clojure cli? It would sometimes be useful when passing -Sdeps, e.g. in a tool that bumps a project’s version, or builds docs.

seancorfield22:10:15

@hugod Yes, that's what :replace-deps is for.

hugod22:10:59

thank you - I was looking for a command line switch

seancorfield22:10:24

You can use it in -Sdeps but you'd need an alias I believe.