Fork me on GitHub
#tools-deps
<
2020-12-06
>
practicalli-johnny21:12:44

FYI: I noticed that the excellent antq project wasnt check for new library versions when added as a dependency via :replace-deps so I've created a PR (its a very simple change). I've been using :replace-deps specifically for tooling that doesn't need to use the project deps, such as clj-new. https://github.com/liquidz/antq/pull/45

seancorfield22:12:38

@jr0cket That served as a nice reminder that now we've abandoned the CLJ_CONFIG "hack" at work, we can get rid of our weird, custom "outdated deps" script and switch over to antq which works really well on our new setup!

👍 3
practicalli-johnny22:12:43

antq has some nice coverage, it even reported a newer version in my github action config for linting my user level configuration. Very impressive.

☝️ 3
deactivateduser00:12:14

I’ve been super impressed with antq too. Big 👍 from me!

seancorfield22:12:50

With the custom formatter, I even managed to exactly replicate the output of our custom shell script that it replaces. And it found a few more outdated deps than our script.

practicalli-johnny22:12:04

Nice. I've found the local/root config option very handy for hacking on an open source project that I've forked locally. I just added an alias so I could test my code before committing

:hack/antq
  {:replace-deps
   {antq/antq {:local/root "/home/practicalli/projects/clojure/community-tools/antq"}}
   :main-opts ["-m" "antq.core"]}

borkdude22:12:55

I use that as well (usually :clj-kondo/dev, :babashka/dev, etc) but I have not used or found the need to use :replace-deps with it.

borkdude22:12:20

:extra-deps works fine for this use case, it will override deps with the same qualified symbol I think, but maybe there's something subtle going on that I'm missing

ghadi22:12:05

:local/root is super useful in dev mode with multiple repos

ghadi22:12:34

I've been enjoying all the new knobs (I know that one isn't so new)

borkdude22:12:54

yeah, it's super useful

practicalli-johnny22:12:40

@borkdude :replace-deps was suggested to me for aliases that run tools which do not need the :deps from the project. The project deps wont be loaded, so don't add to execution time (although it probably has to be a large set of deps to notice). Its also in the official docs now https://clojure.org/reference/deps_and_cli#_replace_project_environment_tool

practicalli-johnny22:12:06

I still used extra deps for project related aliases

borkdude22:12:45

oh so because you normally use antq as a tool, you just copied that alias and made the local/root hack version?

borkdude22:12:03

that makes sense

practicalli-johnny22:12:23

copy paste is a wonderful excuse not to think 🙂 So i didn't really think about how I was pulling in the dependency. Using extra-deps probably is more approapriate for hack/antq

ghadi22:12:15

:replace-deps also useful for when you do want to read the project deps or construct the runtime basis

ghadi22:12:34

but not be in the project classpath

borkdude22:12:38

so what if you are combining multiple aliases for dev, like :clj-kondo/dev:cider-nrepl then I guess :replace-deps would still work in both aliases, combining the deps of both aliases?

practicalli-johnny22:12:32

Sound like something interesting to test...

borkdude22:12:40

@ghadi I need an example to be able to understand what you said

ghadi22:12:14

tools like depot or antq want to analyze the project dependencies

seancorfield22:12:28

clojure.basis will still be computed from the project deps.edn file (and user deps.edn etc).

ghadi22:12:42

those are good candidates for :replace-deps

ghadi22:12:37

without revealing much, I'm working on a tool right now that is installed as a :replace-deps tool. It need to read the project's deps.edn and produce a bunch of data derived from it

borkdude22:12:06

Where is clojure.basis documented, I haven't looked into it (I probably encountered it while porting the new clojure bash to clojure). I can't see it here: https://clojure.org/guides/deps_and_cli

Alex Miller (Clojure team)00:12:59

It is intentionally not yet documented

Alex Miller (Clojure team)00:12:11

As we are probably going to add some api around it

ghadi22:12:13

within my tool I remember :project-dir as (io/file ".") and call tools.deps as an API

ghadi22:12:26

loading in the deps.edn from the project and shredding it

borkdude22:12:52

shredding = rewriting?

ghadi23:12:09

deriving info from it for the purposes of my tool

ghadi23:12:30

(i need to produce several zip files with different partitions of the classpath)

borkdude23:12:53

it sounds like you're working on some caching tool?

seancorfield23:12:16

https://clojure.org/reference/deps_and_cli#_basis_and_classpath talks about the "runtime basis" but doesn't seem to clarify that you can read it via the JVM property 😐

borkdude23:12:53

Thanks, I missed that because I only searched for clojure.basis

ghadi23:12:28

you shouldn't actually rely on the JVM property

ghadi23:12:05

use tools.deps programmatically.

ghadi23:12:47

I assume there will eventually be some stable API for an application to discover its own runtime basis

seancorfield23:12:59

A stable API would be nice. I wrote a few questions on http://ask.clojure.org recently about providing easier ways to use t.d.a with higher-level APIs to avoid the boilerplate. A nicer API than (-> (System/getProperty "clojure.basis") (slurp) (edn/read-string)) would be appreciated 🙂

seancorfield23:12:04

Although right now it doesn't require you to depend on t.d.a so there's that small bonus (but I get the impression that Clojure itself will start to depend on t.d.a soon, or parts of it perhaps, maybe for the add-libs style functionality?).

ghadi23:12:05

most tools people write do not need to be within the project classpath

ghadi23:12:12

the depstar approach was wrong

ghadi23:12:35

most tools want access to the application's basis to analyze it

ghadi23:12:41

e.g. see what libs are outdated

ghadi23:12:07

I don't think clojure will ever depend on t.d.a., but I'm curious to know why you said that

seancorfield23:12:48

Well, now we've abandoned the CLJ_CONFIG hack at work, I'd be happy with a version of depstar that constructed the project basis instead of relying on the runtime classpath environment.

seancorfield23:12:30

But I don't want to break anyone's projects that are already depending on how depstar works today, so I'll have to give that some careful thought.

ghadi23:12:47

one advantage of running as a -X tool with :replace-deps is that you don't have to be frugal about which dependencies you allow yourself to use

ghadi23:12:08

you can access the application basis to produce an uberjar with whatever nice deps you want to use

seancorfield23:12:09

@ghadi Alex has talked several times about add-libs style functionality being part of Clojure itself and that's currently in t.d.a -- so my assumption is that at least part of t.d.a's functionality will be rolled into Clojure directly, or Clojure will come to depend on (a future variant of) t.d.a like it currently depends on Spec etc.

borkdude23:12:38

> A nicer API than (-> (System/getProperty "clojure.basis") (slurp) (edn/read-string)) would be appreciated It think having a generic way of storing properties in clojure would be nice. Instead of the Java properties String -> String. we could have something like clojure.core/*properties* with keyword -> parsed EDN, without re-inventing a new variable for every new thing we want to store.

ghadi23:12:10

I use the runtime basis for that borkdude. here is an ex:

borkdude23:12:44

(I'm also thinking of other use cases than tools.deps)

ghadi23:12:39

{:paths ["src"]

 :deps {org.clojure/clojure {:mvn/version "1.10.2-alpha4"}
        org.clojure/data.json {:mvn/version "1.0.0"}

        cognitect-labs/secret :local/root "/Users/ghadi/dev/secret"}}

 :aliases {:secret-deploy {:replace-deps {cognitect-labs/deploy {:local/root "/Users/ghadi/dev/deploy"}}
                           :exec-fn cognitect-labs.secret.deploy/magic
                           :exec-args {:compile-nses [hello-clojure
                                                      cognitect-labs.secret.runtime]}}}}

ghadi23:12:00

then I run clojure -X:secret-deploy

ghadi23:12:27

the code within cognitect-labs.secret.deploy/magic reads the local project's deps.edn and "does things" with it

ghadi23:12:06

but you can put arbitrary data in aliases now, like the :compile-nses stuff

borkdude23:12:43

yeah, you can store data in deps.edn, but it can be handy to have a general *properties* thing with the actual runtime data as well. e.g. instead of *command-line-args* we could have (:command-line-args *properties*) without ever introducing a new dynamic var for environment specific things again

ghadi23:12:58

yeah, that's the injected basis

ghadi23:12:27

just need a stable API 🙂 . I am 90% sure the clojure.basis property is going away

ghadi23:12:07

I see that clojure.basis is mentioned in Alex's blog

borkdude23:12:20

(afk now, sleeping)

3
seancorfield23:12:36

FWIW, clj-new uses the runtime basis right via the clojure.basis property but switching to a nice API that reads the deps and computes it would be nicer.

seancorfield23:12:29

Right now it still depends on being introduced via :extra-deps but it would make more sense to use :replace-deps and calc-basis -- but that goes to the "standard way to provide aliases to tools" Q I posted on http://ask.clojure.org @ghadi

ghadi23:12:03

yeah that's an interesting API design problem

ghadi23:12:36

how do you talk about the aliases to run the tool and then also about the tool's inputs?

seancorfield23:12:15

Yeah, and right now every tool does something different and ad hoc 😐

ghadi23:12:58

in my example above, I'd add :aliases as a parameter within the :exec-args map

ghadi23:12:16

(but I haven't the need for this tool)

seancorfield23:12:48

Yeah, maybe as everything catches up with t.d.a and switches over to the -X approach and :replace-deps we can standardize things.