Fork me on GitHub

Reading over the prerelease Deps/CLI reference again, and reviewing the discussions above, it seems the -A option is being promoted as the way to do everything-except-execution whereas before it used to include execution. Given that it's being referred to as the "REPL option" and, for the most part, -R used to be somewhat similar to -A without :main-opts, I wonder if keeping -R but making it also pull in the classpath stuff (but not :main-opts) would be the less disruptive path?


Then -A could quietly be undocumented, eventually deprecated, and finally removed.


This would be a minor expansion to what -R currently does (it would respect :extra-paths which only -A/`-C` did before) and it would still ignore :main-opts. I suspect -C is the least used option and almost all -A uses could be replaced by either -M (in the new world) or -R (in either world) with no changes in behavior.


And -R more closely relates to REPL. So we'd have -R/REPL, -X/execute, and -M/main.


Hmm, I guess -R would also have to subsume -O for that to work fully.


Going back to the :test:runner example, it seems like -A:test:runner -M:runner works identically on both stable and prerelease and produces no warning on prerelease (presumably because -M is present?).


Ugh! So... I guess instructions could change now to recommend -A... -M... and that would work now and tomorrow even though it's a bit redundant.


@seancorfield I like your suggestion of extending R to include C, I've always found it weird that these two were separated. Also the -A -M trick is something I tried yesterday, but somehow didn't work for me, since I got file not found -M, but now that I'm trying it again, it seems to work.


Yeah, having integrated CLI/t.d.a into our repo and started to update all our script to match this "new world", I'm very torn between "extending -R and undocumenting -A" and just "documenting -A... -M..." as the preferred approach.


I would much prefer a solution that works for today and tomorrow (which is the -A... -M... approach, especially since that suppresses the warning on -A).


Either way, our build script that previously just turned build aliases subproject options into cd subproject; CLJ_CONFIG=../versions clojure -A:aliases options needs to be way smarter about handling the options part...


Yeah, I guess clojure could override certain deprecated options when the more preferred ones are used, which provides a migration path


Well, it already does.


Yes, but also for other deprecated options?


If you use -A:test:runner -M:runner you don't get the warning, even when :runner includes both :extra-deps, :extra-paths, and :main-opts.


Which deprecated options?


-R, -T, and -O aren't deprecated, they're simply removed.


-A's use of :main-opts is deprecated.


I've never used -T or -O, so R is what I'm concerned about. So if you use -R ... -A ... in the new world, does that do the same as in the old world?


-R cannot be used in the new world. It does not exist.


but is it ignored?


as in, no error


It spits out an error and halts the script.


that's what I meant


(! 2172)-> clojure -R:test -A:test
-R is no longer supported, use -A for repl, -M for main, or -X for exec

Sat Sep 05 00:13:57
(! 2173)-> echo $?


but I guess -A would also override -R in the old world, so in the old case it also doesn't work as a polyfill for both


Right, the problem is that -A in the old world also executes :main-opts


But at least changing -A:... to -A... -M... works identically and doesn't issue a warning in either world.


that's already a win


But anything that uses -R today is plain ol' broken tomorrow.


There's no compatible way to run anything that uses -R today in a way that works regardless of clojure version I think.


So you pin to an old version?


I never thought about it really hard, just copied the install instructions and that seems to contain a version number


Fine for CI for you, but not fine for any library code or tools that have to live in a version where members could have any version of CLI installed...


I do have one custom installer script for mac CI as the linux script didn't work there:


But in that arena people see warnings and can react to them


which can still be confusing, no argument there


I have a solution for clj-new now so I'm somewhat mollified 🙂 but it doesn't help anything that uses -R and has to live in a multi-version world.


I guess clojure could print a link to a more elaborate blog post about the situation


I don't consider that a solution. "We broke your shit. Go read this to fix it!"


Seriously, that is not the right approach, esp. given Rich's whole pulpit-bashing about no breaking changes without renaming things.


I agree that it's far from ideal, but assuming that it's going to be like it is now, it provides people a place to read up if they are curious about why it changed.


Considering how many people learn Clojure by using lein and still trip over ~/.lein/profiles.clj stuff, I don't think we should just throw our hands up and accept this and just say "Well, read these docs about why we broke stuff". I really don't.


You don't have to convince me that avoiding breakage would be preferred. I think I'm going to take a break from this channel for the rest of the day :)


I can change all the -A uses in clj-new's docs and in the generated projects to be -A... -M... and at least I then say, "Hey, the latest version does recommend using both options so you're future-proofed" but it's still a fairly sucky experience for users.


Enjoy your weekend @borkdude 🙂


Proposal: * Recommend authors change -A:stuff to -A:stuff -M:stuff for compatibility with both versions where they want :main-opts executed. * Keep -R and expand it to include :extra-paths and :jvm-opts so it mnemonically matches REPL: -R/REPL, -X/eXecute, -M/Main. * Undocument -A for now, officially deprecate it later (and add that warning about using -M instead), and eventually drop it altogether.

Alex Miller (Clojure team)14:09:22

I think this is pretty good. I'll have to go back through my spreadsheets to evaluate more. I think the first bullet recommendation could even be for tool authors to do nothing, wait until most people are using a new version, then change their recommendation.


If you don't introduce the warning for -A -- per bullet 3 -- then, yes, bullet 1 isn't really necessary.

Alex Miller (Clojure team)14:09:28

the warning is what tells people how to move to the new world - without that, won't people be stuck on -A forever?


Bullet 3 suggests introducing the warning some time later, after everyone has had a chance to update.


People don't like warnings from their day-to-day development tools. Beginners in particular will complain to library authors about "incorrect instructions" if you introduce that warning before anyone has had a chance to update instructions/tutorials.

Alex Miller (Clojure team)14:09:31

I'm ok with that :) I'm not sure I'm in agreement with you on the scope of this as an issue

Alex Miller (Clojure team)14:09:02

you're right that they don't like warnings, so they'll change what they do, which is what we want

Alex Miller (Clojure team)14:09:07

library authors can either do nothing and situation is same or they can use dual instructions for a time or they can doc new instructions and say clj > X.Y.Z (which also gets people on new version)

Alex Miller (Clojure team)14:09:37

or they can start using -X style invocation (which also drives to new version)


> I'm not sure I'm in agreement with you on the scope I'm pretty sure we're just not in agreement on that 🙂

Alex Miller (Clojure team)14:09:43

there is a middle ground - no warning for -A on first stable release

Alex Miller (Clojure team)14:09:52

then (much later?) removal


Having been on the receiving end of complaints about incorrect instructions (or version-specific instructions that weren't absolutely clear on how to determine the version and how to "solve" the problem, on all platforms), I'm perhaps more sensitive to this whole issue than you... 🙂


Yes, bullet 3 is that middle ground.

Alex Miller (Clojure team)14:09:29

I'm on the receiving end of those complaints too


This provides a migration path for the most common use cases that is minimally disruptive (most uses of -R today won't care about paths or JVM options I suspect). It improves naming for the three desired use cases: REPL, eXecution, and Main opts. It breaks the least amount of code out there.


(question for Alex when he gets back: why are there two identical copies of exec.jar in the CLI distribution -- one at the top level, one in libexec?)

Alex Miller (Clojure team)14:09:14

I think there aren't two in the distribution, the install script copies the root one to libexec which is the structure brew expects, although I think maybe I didn't sync the clojure script up to that and it's using the root one. I'll check on that


FYI, confirmed that script uses the root one:

-classpath "$cp:$install_dir/exec.jar"

Alex Miller (Clojure team)14:09:14

Keep in mind that most Clojure users are on Mac and install via brew and that brew aggressively updates you to latest version (our stable) even if you're upgrading something else. So within some period of time (couple months maybe), many people are using newest version.


That only covers 55.36% of the survey responders, so this doesnt seem to be the strongest of arguments. Of course I am happy to help encourage upgrading to newer versions in any way I can, once I've understood the changes myself.


I wonder what percentage of the 35%-ish that are on Linux are using brew vs manually installing via the .sh file?


There are legion of package managers for Linux that brew competes against, maybe a question for the next survey? I find it easier to use the Linux script install.


I use brew on Ubuntu (on WSL) because it's easy to keep clojure up-to-date that way but that's recent for me -- I used to use the manual install script, but I tended to only upgrade when I remembered to do it, or I tried to do something that my older, installed version didn't support 😐


I've used Linux for many years, the brew stuff is extremely rare.


Yup, I'm just glad it exists for clojure. Getting stuff into the more standard installer repos on Linux seems very laborious? What relatively painless options exist for Linux package managers that the Clojure core team could leverage @U09LZR36F?


If only you knew the rabbit hole you'd just asked about


Every distribution has its own rules and timelines. Consider Ubuntu LTS. They don't take changes. Canonical pay people to backport security patches. Every now and then you get a system upgrade (like mac) and that's when you get changes. New features, bug fixes, etc.


There are, of course cutting edge distributions like arch and void. They'll update whenever the package maintainer feels like it. Usually regularly for important changes.


Some distributions care about reproducibility. They have other constraints I don't understand.


I am a void clojure package contributor, and my life is pretty easy. The installer script was "ported" to the void build system in ~10 lines and it's just worked since then. Maybe I'll need to make changes with the new system.


@seancorfield To package Clojure for Ubuntu, then I'd suggest a personal package archive (PPA), new release of Clojue can be pushed to this as soon as they are ready. There is a specific process for creating package to learn. Packages can be built for each release of Ubuntu (e.g. current long term support and intermediate releases) and published on the PPA. The relevant packages can be submitted for each new Ubuntu release (6 month cycle). There are also other options such as snaps and flatpack, but a proper *.deb package is much nicer in my opinion as it ensures there are no conflicts with other software installed via the package manager. There is a Leiningen package for Ubuntu, so some of the experience from that can be used. For Arch, I seem to remember that @U11EL3P9U packages up clj-kondo , so I assume his work can be of help for packing to the Arch User Repository (AUR).


That's right, I do maintain a few Clojure packages on Arch, like clj-kondo, babashka, clojure-lsp. A pleasure to do so 🙂 However, the main Clojure package is part of the Community Repo that is under a different maintainership. It is reguarly updated (last update was 2020-08-26, which brought in Clojure

Alex Miller (Clojure team)14:09:29

Also keep in mind that we are in work on a 1.10.2 Clojure cycle and we tie brew versions to Clojure versions, so that might be a natural point to either coax an upgrade or drop some kind of compatibility

👍 3

I have updated the depstar README to show usage with the prerelease -X option:


It works quite nicely that way...


I expect I'll update clj-new next, to support -X usage...


> Keep in mind that most Clojure users are on Mac and install via brew Have you considered that those Mac users will probably also use the linux scripts on CI? Do you get any stats from that usage? It may be less of an issue since the install script is versioned, but since people tend to develop CI scripts locally (I often do), it might become confusing.

👍 9

There might also be less Windows users using this because for some people it's harder to get going than lein. I got this report yesterday and it's not the first time I hear these kinds of issues. > Clojure until 1.6 used to work on Windows from too, without problems. Clojure Tools broke everything and it seems that OSX ( Homebew ) is the only "first class citizen" now. Don't shoot the messenger, I don't really have a stake in Windows support because I know how to work around it :)

Alex Miller (Clojure team)20:09:48

Oh I definitely think that’s a factor


Yeah, you have to be pretty persistent to use Clojure CLI/`deps.edn` on Windows, and willing to work with very "alpha" tooling. Which a lot of Windows users are not willing to do: they often have different expectations of the user experience than devs on macOS/Linux.


One personal data point: I found it difficult to install clojure on Windows CI because the powershell stuff you had to enable and whatnot. That's why I also went with lein to run tests there:


I do use clojure on the other two major OS CIs


I'll be using Windows for development a lot more going forward (Microsoft had a great deal on Surface Laptop 3 so I just ordered an i7 with 16GB RAM and 256GB SSD). But I'll be leaning on WSL2 very heavily, and using (Linux) brew for installation.


I have an odd case -- at least from my pure Clojure background (didn't start in Java.) I have a Java dependency that doesn't publish to maven. I can run ./gradlew build on it to get a library .jar. What's the best way to include this into deps.edn dependencies?


Can I publish just a .jar to maven under my own repo? I could try updating the library to properly publish to Maven & PR'ing it, but I'm just unfamiliar with the gradle ecosystem. The library author says he'll start publishing to maven soon...


You can use a local .jar file directly in deps.edn via a :local/root dependency. We use that for a couple of things that have to build locally because they aren't published anywhere.


oh man, I immediately get what you mean, but how did I utterly fail to find this in a doc somewhere? I spent 20 minutes searching around


anyway thanks Sean, you're a big help in many channels 🙂


It doesn't "scale" so well if you're part of a team or want to use it in a CI pipeline, but it's fine for local development and testing.


Ahh, I'll try just reading through the whole deps guide. I kept searching for "clojure deps include .jar library" and similar terms, never thought of "local library"


As far as "scaling" this approach, I was just going to git submodule the dependency library. I could see how many layers of builds would take a long time / cause odd failures. Thankfully still at "fun personal project" size for now


deps.edn didn't seem to pick up on the .jar (silent failure?) as I couldn't load any expected classes. I realized there's no pom.xml, probably why. At a minimum, I believe I need to tweak the library to use


@U22M06EKZ You shouldn't need a pom.xml for this usage -- but if that Java library has any dependencies, you need some way to tell clojure about them (if the JAR contains a pom.xml the t.d.a machinery would take care of that, else you could just add those deps directly to your deps.edn file).


@seancorfield Hmm, I could uptake them into my deps.edn. Do you know of a way to determine if the jar was loaded or not? I've just been trying to run an :import on a class that I expect to exist, but wondering it there's a better way to poke around


Use the -Spath option on the clojure CLI to see what the classpath contains.


Or even inside your REPL: (System/getProperty "java.class.path")


user=> (run! println (clojure.string/split (System/getProperty "java.class.path") #":"))
(example from a bare REPL)


^ I figured there was something from the REPL, but -Spath is a good tip. Appreciate the knowledge-share, you had a hand in me starting adoption of deps.edn a year ago too and I doubt I would've tried it otherwise :thumbsup:


Cool. We've never regretted switching to CLI/`deps.edn` in 2018 at work.


(and we're using the bleeding edge prerelease version right now -- as of yesterday)


For the up-coming release of Clojure CLI tools: I have been using clojure -R:cognitect-rebl -A:nrebl to run REBL with nREPL. The -R:congnitect-rebl load in dependencies from an alias that also has a :main-opts (which it not the main namespace to run) and -A:nrebl which has a :main-opts I do want to run. If I create a separate alias for cognitect-repl without the :main-opts and use clojure -M:rebl-deps:nrebl it works. Or I could just add all the rebl dependencies into the :nrebl alias. Does that seem the right approach?


@jr0cket I suspect if you used -A:cognitect-rebl:nrebl it would run the :main-opts from :nrebl since they overwrite each other -- last one wins.

☝️ 3

You could also use -A:cognitect-rebl:nrebl -M:nrebl which would work on both today's stable version and the current prerelease version without the -A warning about using -M to run the :main-opts (since the "winning" :main-opts here is using -M).


I think the guidance from Alex has fairly consistently been to separate out :main-opts from other stuff if you need to mix'n'match main opts from several aliases. But the expansion of both -M and -X to respect :extra-deps and other stuff in the latest prerelease feels like the pressure is in the opposite direction -- toward bundling more things into a single alias for convenience @jr0cket Not sure how closely you've been following the discussions generally in this channel?


I've read the whole conversation in here and reading through the docs Alex wrote. Still digesting what it all means. I'm currently testing all the aliases I have to understand the changes


I'm working on updating clj-new to work with -X, much as I've updated depstar. I'll probably add :new-x and :depstar-x aliases to my .clojure/deps.edn file so folks can use those tools via -X as examples. It'll be a while before that can become the default 🙂

Alex Miller (Clojure team)20:09:52

@jr0cket would love to hear your feedback too


@alexmiller so far very positive changes. I like that -M feels similar in concept to the :main in Leiningen, in part easing the move from Leiningen to CLI tools. My current thoughts are around edge cases where I have multiple aliases containing :main-opts and how to use them together with certainty rather than optomism. I have tried to elaborate some options in this gist


@jr0cket Multiple aliases containing :main-opts is an "issue" today (stable) as well as tomorrow (prerelease), and the answer is the same as it has always been -- per the documentation -- "If multiple maps with these keys are activated, only the last one will be used" -- see which explains how each different type of option is "merged" when multiple appear. The same wording is present in the prerelease reference docs, but spread out over multiple sections, rather than gathered together in one section.


Is that not "certainty" enough?


Certainly in my mind would be as part of the syntax rather than convention. If the convention changed in a release without requiring a change to the syntax of use. Convention also means more to remember, especially as some parts of slides are merged and others only take the last. I assume (hope) nothing takes just the first. Not saying it's an issue, but it adds to the learning curve.


As it's a current issue and there isn't anything in the up coming changes to resolve it, then this aspect seems a moot discuss point. I'll make a decision and move on.

Alex Miller (Clojure team)15:09:20

fwiw, I don't see this as an issue - the documented behavior is what it does and what I expect it to do in the future


(but of course, at this point, there's still some potential for changes in the prerelease that the -A/`-M` usage recommendations may change)


I happened to watch Spec’able before reading Alex’s post. Seems pretty ironic.


sorry, sepc’ulation


TL;DR: don't break stuff, only add stuff and fix bugs 🙂


at least once you decide you are out of alpha


The deps/cli reference page only talks about the library being in alpha. The guide doesn’t mention it at all, and besides “But, that is not to say just leave your thing 0.0.967. At a certain point, you are going to have users, and whether you change it to 1.0 or not, they are going to be depending on your stuff.”


Feedback on the "WARNING: When invoking clojure.main, use -M" message (for @alexmiller) -- this usage produces that warning: clojure -m my.entry.point -- is that intentional? Do you really want folks to write clojure -M -m my.entry.point instead?


Is this to discourage that usage in favor of -X? It seems to be another maybe unnecessary breaking thing that's been there for a long time

Alex Miller (Clojure team)20:09:31

It’s not to discourage it as much as that we are trying to open up the repl arg space for other things

Alex Miller (Clojure team)20:09:58

And that means making clojure.main usage explicit


Interesting choice. A lot of tutorials out there talk about running simple Clojure projects with just clojure -m my.entry.point


I think beginners will (quite rightly) ask "Why do we have to specify two m options?"...

Alex Miller (Clojure team)20:09:25

And if these tutorials transition into talking about -X...

Alex Miller (Clojure team)20:09:02

We want clj to do more. In some cases that means restructuring these options. It’s either that, or clj2 and clojure2 which do not seem better to me


Sure, if all the books and tutorials that exist today are updated to use -X instead of -m, it's not going to be a problem 🙂


Thanks for balancing all these things in a spreadsheet so lots of concerns are managed. On a Saturday no less. Looking forward to the previews


(I just want to be clear that I think all the new functionality is great -- all of my concerns are around how existing (stable) functionality is deprecated and/or removed over time)


And, ultimately, I'm not going to be the one complaining about the changes since I'm already on the latest prerelease version on every machine and I'm aggressively adopting the new functionality!


(I am somewhat concerned about users of my CLI/`deps.edn`-based projects, of course 🙂 )


There was a time when the clojure CLI wasn't driven by tools.deps.alpha right? Wasn't it non-alpha back then? Isn't only tools.deps alpha but the CLI / shell script now also got alpha?

Alex Miller (Clojure team)00:09:24

No, clj and tools.deps were developed together


(`depstar` added support for -X in its most recent 1.1.104 release, last week)

💯 3

I could be misremembering things, but I think there was a clojure package already years ago, before tools.deps.alpha? Not sure what it did back then, but -m was maybe already a part of it back then


I know there was an unofficial clojure script floating around which behaved differently than the official one, once it appeared.


maybe it just did java -jar clojure.jar $@


Yeah, I think that was pretty much it. My recollection of the official CLI was that it appeared in tandem with t.d.a during the 1.9 cycle once Spec was split out of the core clojure.jar since you could no longer just run a REPL with the JAR file alone so it was supporting machinery for that... but I'm having a hard time finding any of the original posts about it (maybe I should scan back through Inside Clojure since I bet Alex blogged about it).


ah right, thanks


Hmm, the pre-1.10 Inside Clojure posts are fairly sparse...


This is the first mention of t.d.a on the Clojure mailing list:


Been playing with the new features, and it's nice that you can have the following

:new {:extra-deps {seancorfield/clj-new {:mvn/version "RELEASE"}}
        :exec-fn clj-new/create
        :exec-args {:template lib} ; default
        :main-opts ["-m" "clj-new.create"]}
and it works with -A:new today (or -A:new -M:new) and, with the prerelease it works with -M:new (or -A:new -M:new without giving a warning) and it also works with -X:new so you can have a single alias provide for both styles of execution.

👍 6

Forgetting migration for a moment, would you see :exec-fn and :exec-args replacing :main-opts in the long term? Using these new keys seems a nicer approach for the design of the alias code itself. Would this be an example of edn over strings that Alex mentions?


I think we'll see a good uptake on the :exec-fn approach because you can avoid parsing strings to get your arguments and you can easily provide defaults -- configurable defaults, since each alias have have its own :exec-args map.


That's probably what I'll do in my dot-clojure repo by way of education for folks and as a way to encourage folks to transition (or at least see what the options available are).