my :depstar alias in deps.edn:

{:exec-fn    hf.depstar/uberjar  
 :exec-args  {:aot        true
              :jar        "target/server.jar"}
 :extra-deps {org.clojure/tools.namespace     {:mvn/version "1.0.0"}
              thheller/shadow-cljs            {:mvn/version "2.15.0"}
              com.github.seancorfield/depstar {:mvn/version "2.1.253"}}}

the error:

$ clojure -X:depstar
Exception in thread "main" Syntax error compiling at (user.clj:1:1).
        at ...
Caused by: java.io.FileNotFoundException: Could not locate clojure/tools/namespace/repl__init.class, clojure/tools/namespace/repl.clj or clojure/tools/namespace/repl.cljc on classpath.

Moving tools.namespace and shadow-cljs to my main :deps key solves the issue, but that's not ideal of course. Think I'm doing something silly here, would appreciate any insights

The depstar alias is for the dependencies of depstar

Adding dependencies there doesn't make them available to your code

Which is why adding them to your main deps key makes it work

Not sure why that is of course not ideal, if you depend on it, you need to specify it in your deps

that's right--I think I have other problems I need to figure out, actually... I don't think that I depend on shadow-cljs or tools.namespace anywhere in my code. My fried brain thought I depstar needed me to specify them but that can't be right obviously, it'd be pulling in whatever it needs on its own

total PEBKAC--was including my src/dev by mistake, which did use the dependencies i seemed to need to include 🙂 thanks @U0NCTKEV8

For future reference, there's a #depstar channel so we can keep #tools-deps for general t.d.a / deps.edn questions.

Sure, but I think I should call out then that the new tools feature is not useful to my team

And that as it is, it seems we'll just continue to use aliases

But, I don't want to speak too soon, I've yet to play with it fully. But I was really expecting it to be more in-line with the declarative style of deps.edn to define apps

I suspect it would be useful to individuals on your team who want to install and manage their own additional tooling 🙂

Ya, for that it would, but they might also find it better to add an alias so they can sync their tools install between desktop and laptop, and whenever they need to change laptop or rebuild their desktop instance

I also personally would have liked a way to mess with tools at a project level

I have ~/.clojure/ under version control and that's how I keep my laptop and desktop in sync.

The pain-point with alias was having to find the correct definition for it. Tools solve that, but it be nice to have something similar for alias. Like if I define an exported alias in my deps.edn for my lib

The same way you can define :tools/usage

I just tested on my laptop: I git pull'd the updated ~/.clojure repo from GH and ran clojure -Ttools list and the same tools/versions are installed there as on my desktop (without actually needing to install anything).

Ok, that's good, so it seems its just that tools don't pollute your deps.edn, but are also declarative.

Now I'm just concerned with reproducibility

But I guess those tools I'll just use aliases for

Well, the ~/.clojure/tools/<toolname>.edn file includes the git tag/sha so you can guarantee the version -- is that what you're concerned about?

https://github.com/seancorfield/dot-clojure/blob/develop/tools/new.edn for example

I'm concerned with like, pulling an older commit of some project/application, and having even the tools be the exact same version, so I can get a complete repro of the setup

Oh, well, for project-specific tooling you should still use aliases in the project's deps.edn definitely.

That's why we vendor the whole Clojure CLI into our repo at work so we can reliably have the exact same version of the CLI on every machine.

<repo>/build/clojure/bin/clojure is what we use in all of our scripts.

Ya, that's a reasonable practice. Though I wish there was first class support for this. Like I wish in my deps.edn I could declare a dependency on the Clojure CLI, tools.deps and tools.build versions to use. I know it sounds like a weird bootsrap issue, but basically you use the globally installed clojure cli and tools.deps to pull down the defined version of clojure cli and tools.deps in the current project.

And same for all tools and aliases and everything

there is a JVM version of the clojure CLI

deps.clj, non-official, by yours truly

Haha, of course, why am I not surprised!

Hum... but that might actually be useful. Maybe I could switch away from my custom launcher to using that.

clojure -Sdeps '{:deps {borkdude/deps.clj {:mvn/version "0.0.16"}}}' -M -m borkdude.deps -M -e '(+ 1 2 3)'

Though... couldn't the normal CLI also be bundled as a git lib or even in a Maven repo? As far as I know you are not restricted to java in them

It just needs to pull down the shell script

I guess it could, but running a shell script from a jar is a bit cumbersome

however, gitlibs might be more useful there

True, but with git libs, that shouldn't be a problem since it pull down the folders and files

Perhaps the new prep phase can install the script in a bin dir ;)

Yeah, that is definitely a new piece of OSS project's that we'll have to pay careful attention to -- since it could do "anything".

Yes, rather than install depstar as a "tool" and using -Tdepstar.

Ok, well at least there is that. So I guess one is convenient, the other solves edge cases. Which is good to have both

Ok, but what are you loving about it over aliases? Just that you don't need to manually edit your ~/.clojure/deps.edn and find the correct alias from some readme file? Or is it that you like to keep the deps.edn uncluttered?

It will allow me, over time, to remove a lot of stuff from my deps.edn file -- and the tools install stuff makes it easier to find the right versions of tools and upgrade/downgrade them: I can do it from the CLI instead of editing my deps.edn file.

Ok, I guess I just don't see how that could not all have been done for aliases as well, just providing a cli for it

Which would have naturally extended to project specific uses as well

A lot of people don't want a CLI tool editing their files.

I guess it comes down to the "brew install" vs "nixos" way of doing things. Personally I'm leaning towards the deps.edn approach to keep things reproducible but it's nice to have options, I guess.

The nice thing is that we have both approaches available now: locally-installed tools (with a manifest that can be sync'd across machines) and alias-bound tools, either in the project deps or your own personal deps.

Ya, I prefer the deps.edn approach as well "nixos" style. But I guess I'll deal with it 😛 I think it would have been fine to have like a convention of a aliases folder (or even the tools folder), but where you can just have more files that define more aliases.

Ya, but we don't have the thing I wanted the most, to have a way to install aliases from the CLI 😞

As I just said "A lot of people don't want a CLI tool editing their files." 🙂

@didibus That's just a script away

No its not, it requires full buy-in from the library authors

roll your own

You need to declare your alias in your lib in a way a cli could figure out what to install

That's why the tools approach pays attention to :tools/usage in your library's deps.edn file 🙂

I don't have a lot of churn in my deps.edn anyway, I edit it probably maybe once a month or less

Right, but so that's for tools, would have been nice for aliases as well

I don't see why I would want to install depstar as a system-wide tool. I want a reproducible version of depstar in my project in CI.

For any tool really

We've just cut our project aliases way, way down (now that we have a build.clj script instead of trying to use bash and the CLI and aliases instead of actual functions).

That's unrelated no? Its just all the functionality of these disparate aliases are all merged into tools.build

@U04V15CAJ With depstar you may be right. clj-new makes more sense as a global tool.

@didibus No, that change was unrelated to tools.build.

I usually write projects with my own bare hands without templates, but yes, that's a good example :)

Our build.clj existed before tools.build was released 🙂

Ok, but still confused, what does that relate to tools or aliases?

Your build.clj is a new tool?

Sigh. I'm commenting on how few aliases we have in our project deps.edn and that they don't change very much -- unless we do wholesale project-level changes. And we made some changes earlier this year (which I blogged about) specifically to avoid having developer tooling in the project, i.e., so developers could use any tooling they wanted.

No, build.clj is "just" a Clojure script. It replaced our build bash script (and a few auxiliary bash scripts).

I guess you've missed folks here talking about switching over to using Clojure programs to do "build stuff" with instead of various non-Clojure scripts...

Oh I see, all my Clojure build stuff are already Clojure programs 😛

I thought Shadow-cljs was primarily installed via npm or something?

it has a clojure part too. See: https://shadow-cljs.github.io/docs/UsersGuide.html#_high_level_overview

Right, but hardly anyone uses that directly I thought?

my deps.edn has

:main-opts  ["-m" "shadow.cljs.devtools.cli" "watch" "ui"]

OK, so it's a -m invocation if you do that -- which is not how -X/`-T` work.

Tools invocation assumes the "exec fn" model, where you invoke a specific function, passing in a hash map.

could that not be a wrapper around shadow-cljs?

Someone -- theller or someone else -- could write a -X-compatible entrypoint/wrapper for it I guess.

prior to tools.build, i was trying to do too much in deps.edn. task-like aliases, logical deps sets, version defaulting for subprojects. a lot of aliases use shadow-cljs but only as an entrypoint to start a process. that feels more like a tool to me?

I don't do any cljs so I'm not familiar with it (I dabbled with figwheel-main a bit last year).

When I saw that shadow-cljs was so node-centric, I went looking elsewhere. I don't do JS/node and don't have any of that tooling installed (and don't want to).

But you would need a -X-compatible entry point in a GitHub project to use it as a tool (or in a Clojars artifact).

i understand your npm concerns but it allows js lib access just as maven coordinates allows java lib access. a useful hack.

i'll see if theller is interested

Would be cool to make shadow-cljs a tool I think. But since it requires npm anyways, and npm already supports local installs, it might still be best to keep it using npm

shadow-cljs does not make sense as a tool. it is not standalone and needs to be part of the project classpath

shadow-cljs is two parts right now, unfortunately I named them the same name. the shadow-cljs CLI tools published as an npm package, which handles only the CLI. and then the real thing shadow-cljs the CLJ library, where all compilation and stuff happens

the npm package has always been optional, I wrote about it here https://code.thheller.com/blog/shadow-cljs/2017/11/18/the-many-ways-to-use-shadow-cljs.html

so a "tool" could technically replace the npm part but that doesn't make too much sense IMHO. maybe there could be a namespace for an :exec-fn-like API though

well maybe it could be a tool. by the nature of the CLJS compiler it needs the project classpath, which I guess it could construct via deps.edn itself and manage its own secondary JVM much like the npm shadow-cljs tool does now

What I gathered is that tools are expected to depend on tools.deps.alpha and use its api to build the classpath for the folder it is run in

So a tool which needs a project classpath would still get one by using tools.deps.alpha as a library for getting it from the deps.edn file

well or it could just run a secondary clojure -M process similar to what the current shadow-cljs CLI npm thing does

but since you'll still need npm anyways for actual JS packages I'll probably keep the CLI package too. Starts fast and does everything you'd want. For clj purists a :exec-fn namespace seems useful, more so than a tool

Ya I guess it could do that too.

tools.build like stuff is already fully supported and has been from the beginning so nothing to do there. meaning everything you can run from the command line you can also run from the CLJ REPL as regular functions

Ya, it does seem to possibly fit better as an alias with -X

Not sure that's even really useful, but if people are currently invoking it with clojure cli using -m as an alias, I guess it would probably be slightly better for them

most people using it directly today (without the npm CLI) use it in embedded mode though. so starting it as part of their user/start or so, together with their CLJ server

some have integrated it with libs like integrant, component or mount

What you said about tools.build I hadn't thought, ya I guess someone can just create themselves build tasks in build.clj that just call into shadow-cljs clojure API and then they can run shadow commands using the clojure cli with -T:build alongside their other commands they'd want

yeah many people do this today via either clj or https://shadow-cljs.github.io/docs/UsersGuide.html#clj-run

or lein of course. tools.build users will likely use the direct CLJ api a lot more than the CLI

Ya most likely

Hum, actually it seems now that I think of it, that all "build tool" or "build" related tooling will be better offered as a clojure lib to be used within tools.build. While pure "tools" would be the kind that don't care about the project you are in and are more like generic Clojure tools you'd want installed globally, like if I wrote a grep tool in Clojure and you want to "install" it so you can use it.

yeah, thats actually how shadow-build (predecessor to shadow-cljs) started. only functions, no command-line API. just got too tedious to create the same few functions over and over again in each project, so a proper CLI and thus shadow-cljs was born 😉

Interesting, I wonder if a similar thing will happen with tools.build, like if people will feel copy/pasting all the basic tasks will start to be tedious.

Though it has the advantage that each function becomes a "command" on the cli, even though a bit tedious: clj -T:build command

yeah, that is my concern with tools.build. just the same functions in every project, likely generated by some template. we will see I guess. doesn't have to be a bad thing, makes things easiert to customize (if you know what you are doing)

@U05224H0W just for clarity, would you be against adding the introspection keys to shadow-cljs's deps.edn? https://clojure.org/reference/deps_and_cli#_best_practices_for_tool_authors

not at all but as I described you really cannot run it as a tool currently. shadow-cljs needs full access to the project classpath which tools don't get (AFAIK)

so adding those things really doesn't do anything

also shadow-cljs uses lein and the deps.edn is not really up to date currently

Tools can use tools.deps.alpha or, now, tools.build to calculate the project basis and that includes a project classpath, which you can then load dynamically, if you need to. But I agree that if you have a "tool" that really needs to execute in the context of a project's classpath -- and test runners are a good example -- then making them a "tool" makes little sense.

i concede the point 🙂

but i do have a few apps which use the documented shadow-cljs deps setup, so it would be nice if that was up to date (which is a different discussion)

(and probably belongs in #shadow-cljs at this point 🙂 )

whats wrong with :main-opts? that doesn't go away 😉

:main-opts is great for some types of invocation (where argument processing needs to be *nix-compliant in some way or arguments can be kind of ad hoc -- and all derived from strings) but exec fn/args is nicer from a programmatic, composability p.o.v. and all exec fns take standardized arg formats (unrolled EDN hash maps, basically).

From a tool author's p.o.v., exec fn/args means that you don't have to write any argument parsing, and your entry points can (now) all have their help/documentation rendered from the command-line (via the new -X:deps help/doc stuff -- although I haven't quite gotten my head around how exactly it works).

shadow-cljs watch app is probably the most common command. kinda gets annoying with -X IMHO. so :main-opts looks better

but I'm open to adding stuff for CLJ purists so whatever makes sense I'll add

Also, from a user's p.o.v., exec args are great because they combine across multiple aliases -- :main-opts is always "last one wins" and does not combine across aliases.

So they both have their useful spheres.

(and, yes, simple commands like watch app do become more verbose with -X which is why some commands are simply better off with -M / -m)

I feel the word "tool" will confuse countless people going forward, that a "build tool" is not a Clojure "tool", but just an "alias".