Clojurians Log v2
clj-on-windows
One final opinion on installation of things on Windows in general: these days I mostly try to install things from the Microsoft Store (which mimics my "application" experience on macOS, iOS, and Android) but for non-store things I'd sort of expect an MSI-based installation. Until this thread here, I didn't even know there was a Powershell Gallery and until I first tried early versions of the Clojure PS Alpha install script, I'd actually never used Powershell -- I'd always used cmd (as limited as it was). My experiences with Powershell so far have been... underwhelming... and I still tend to open cmd by preference if I have to do something on the command-line of Windows that I can't do via WSL2 accessing /mnt/c. But I'm probably a very atypical Windows developer...
@chaos I'm interested in your thoughts on how Leiningen has handled this for years? I haven't heard too many complaints over the years about Windows users downloading lein.bat, and either putting it on their PATH or using a full path to it, and running it in cmd (I assume). It forgoes an "installer" completely and "self-installs" itself on first use. I'm wondering whether a clj.bat (or clojure.bat) that users downloaded and used "like Leiningen", and which "self-installs" the JARs and config etc on first use would be "better" than the status quo? @alexmiller is that what you had in mind for the .bat column or were you still thinking of an explicit install step that set up the .bat script and everything else?
There is also scoop install lein option
Yes, but that's not the "standard" way Leiningen recommends which is my point here.
There is no standard way
Download a Linux/macOS shell script or a Windows batch file. That's "standard" insofar as it's what the official Leiningen install docs say.
The question here is "what should the official CLI install docs say for Windows?"
That's probably terrible way. People just don't know there are better ways. That's all
If the official way works -- and I don't hear too many people complaining about Leiningen on Windows -- then that's what really matters. There are lots of package manager installers for lein (and the Clojure CLI) which may be subjectively "better" but they are not "official".
But that is probably lein maintainer's fault, isn't it?
Hardly a "fault" if what is provided and documented works. Which it does.
I think your Scoop setup is much better for the CLI (and other stuff) than the current ad hoc Windows instructions, but that's not really the point.
Unless the core team decide to endorse Scoop and say "If you want to use Clojure on Windows, first install this 3rd party tool that many Windows users have never heard of, and then use it to install Clojure" 🙂
There is no endorsement needee. It's kinda a point. But whatever.
The official Clojure core team's position could be "We don't support Windows Powershell/cmd, but here's a list of 3rd party solutions" -- that's one column on Alex's grid for evaluation. But they have to decide on something, one way or the other. There are lots of 3rd party approaches that aren't documented by core project maintainers but that's always up to users to figure out.
fyi scoop uses the provided ps script (or module or whatever it is). if they decide on .msi or .zip. scoop will use that until it is so bizzare that it brings more harm than value.
@alexmiller for your installer options there is also https://chocolatey.org/, not terribly familiar with it, but I think it is another popular alternative.
@borkdude, I suppose that maybe an nbb Clojure script compiled to an exe might also work? Is that possible and/or potentially interesting?
Also could list a babashka script. But then the runtime dependency would be babashka, which is probably not good. But could list it for completeness.
It might also be interesting to survey what other languages do for installers on Windows to see if there is any commonality.
Here's a start: https://forge.rust-lang.org/infra/other-installation-methods.html seems to support it all. Package managers like chocotaley or scoop or an msi. You pick. https://go.dev/doc/install seems to have gone with msi. https://www.python.org/downloads/release/python-3108/ has an exe installer, and a zip. https://nodejs.org/en/download/ has msi and zip. https://www.ruby-lang.org/en/documentation/installation/#winget seems to have gone with chocolatey and winget https://adoptium.net/installation/ docs suggest winget or msi, (I personally used scoop, probably a community supported effort) https://www.scala-lang.org/download/ uses an exe installer (which might be an installer to install scala? dunno) https://groovy.apache.org/download.html has binaries or a community supported msi installer
https://www.oracle.com/java/technologies/downloads/#jdk17-windows is available as msi, exe, and zip
That’s super helpful (and very confusing)
I'd like to rise one more issue that community members faced. False positives on malware detection (by Windows Defender). Sometimes, rarely, but it happened multiple times. Distributed binaries, even zip files were marked as malware 😞
I think, but I cannot confirm, having a .msx or .msi properly digitally signed can probably seen better. But I'm no expert here.
Solution to above problem was to raise a request to MSFT, sending them a file and location where it's distributed (was Github I believe) and they flag it as clean.
Hi @seancorfield, do you happen to know if the State of Clojure survey does account for prospective new Clojure users that tried or wanted to try Clojure perhaps only to be put off by the PS installer and the strange installation question you get right at the beginning (which directory to install it to) or the difficulty installing it on corporate windows environments? Maybe this number is significant (no data to support my claim).
I've started with Leiningen, which is still a breeze to use (it just works), and personally I am amazed with how much thought they put into it both the main program and the extra mile they took in creating the .bat file. One drops the bat file in their project, and they are just a lein away from using it on windows (exactly the same experience on unix). It nowadays faces the same hurdles with strict corporate firewalls policies on windows, one needs to know how to setup the proxy/certificates to bootstrap itself by downloading its main jar file from the internet.
I think there are two separate problems to be addressed, installation and usage, both super important, and the clojure team have addressed this very well on unix (brew installer, bash script invocation). I know none in person who uses PS, and it has its own operating concepts which are foreign to most people and hard to follow even for experience developers. I assume it is only used by devops and nobody else, so I would argue using it as the starting point for Clojure newcomers did not turn out to the the best of choices.
Perhaps if the clojure team still wans to go with the PS solution in the short/mid term, a streamlined installation process that brings it on par with Unix (the PowerShell gallery if it works?) addresses the installation point, while better instructions on how to get you started in Clojure with PS might address a big portion of the usability question (make documentation more prominent and beginners friendly, suggestion to use Windows Terminal which is very good looking and has ton of functionality and begins with a PS shell).
I would like to categorize the quoting issue into usability and treat it as a separate general problem (that has a worse impact on PS). I think there is a solution around it that solves it for good (program takes control of escaping). I hope to have something to present to you really soon, I still need to test the idea in practice, but could be a step towards the right direction regardless as food for thought. I hope to be able to share some results soon given that the conversation about better windows tools support has started in earnest here.
The standard windows installers, scoop and chocolatey do not work on corporate environments, this is something sysadmin avoid like the plague. These installers cover so many applications that they are a no-no for sysadmins from the start, and I haven't seen a way around using them in these environments.
Thanks
if you are not allowed to do anything than I'm afraid there is not much we can do. I've seen these "buy-sell-hold" processes in unnamed investment bank. But I guess since it's a productivity killer I believe they dismissed these some time ago.
Indeed, most organizations try to restrict as much as possible but is impractical to block everything as you said. Tools need to adapt to the times to take advantage of any window of opportunity that is still open. For example the Clojure team has introduced support for a new CLJ_JVM_OPTS variable very recently that will let the user pass additional properties to the clojure tools java process responsible for downloading dependencies. This can be useful for example to redirect the java process to use the windows certs store where important corporate certs are stored now a days for controlling traffic.
A third separate issue (1 is installer 2 is launcher) that I see us touching on is cli ease of use for all platforms.
This includes things like having to specify a string as '"imastring"'. This is the type of usability issue that https://github.com/babashka/cli addresses.
I think seeing this as a separate issue, if only to treat it as out of scope for the launcher, is helpful.
@chaos can you briefly recap the locked-down bigcorp acceptable installer options? msi? exe installer? winget? other?
Hi @lee, I don't think there is a standard, but bigcorps are likely to want to restrict everything that can come in and out of a user's workstation, but in practice is not possible to do so. If they are likely to allow an installer or package manager, that would be something that is endorsed by MS, such as the MSI installer and poweshell modules that I know off, which are both signed and the author can be trivially verified. Programs installed with these packages can be audited (a major point) and they can easily be included in a system setup in case they it is part of a workstation build. Compared this to standalone exes or unsinged scripts (they don't know what they are or where they are coming from) or open source package managers where there is a great range of packages that can be downloaded, but the author or maintainer of every package is difficult to verify. Thanks
Thanks @chaos!
for the non-locked down crowd, thanks to all of @ales.najmann's hard work I think Clojure's current Windows users are probably familiar with scoop.
also @jr0cket who was probably the first person to use it to introduce new devs to Clojure
It should be possible for a single msi installer artifact to be made available standalone and also republished into scoop/winget/chocolatey/etc
Cool, I guess the Clojure core team would decide which to publish to, or simply point to community supported works like @ales.najmann's scoops.
And in a corp environment the msi could be made available via group policy, SCCM, KACE, or whatever they use to package/deploy apps
I can confirm scoop can and will certainly use this when available
Just for my limited understanding of what an MSI installer is: can this be installed in CI without any GUI crap?
Yes. You can install msi’s using msiexec with options to suppress ui
Since Oracle only delivers x64 Java installers for Windows, I think we can remove the concern in the grid for supported architectures for graalvm binaries
Good point, I think it might still have an x86 windows installer for jdk8, but only see x64 past that.
Ah, so maybe Clojure 1.12 should just drop support for Java 8? ;)
I certainly would not object. But locked-down bigcorps might!
Clojure has dropped support before for eg. Java 1.7
Very tangental but are locked-down bigcorps the ones likely to adopt innovative tools like Clojure?
Could also be a constraint just for the Windows installer/launcher
The rest of those folks can use lein ;)
Let them eat lein!
or they can use the deps.clj uberjar
enough options
there are many alternatives, yes good point
I think x64 would be reasonable as a restriction for the launcher assuming it could still launch clojure under whatever the default JRE of the system is, e.g. ms publishes win/aarch64 JDKs, which I’m not holding my breath for support to make it into Graal, but do use as my default Java
Windows on ARM is an option for Graal I think, as they support ARM for all the other OSes, but they will only do it if there is demand, I think
FWIW I never had anyone ask me to provide a deps.exe for a different architecture
That's good info. It solves a pain point that does not exist on other OSes.
Side thought: Is memory footprint of the launcher something to care about?
And startup time (graalvm binaries do well in both regards)
graalvm binaries also support exec calls so the memory can be freed when launching the java process, if that is a concern
If I would have to pick an alternative binary creating solution I'd probably go for golang as it's easy to cross-compile to static binaries. But yeah, you would have to write golang. Might not be too bad compared to bash though.
I was thinking same for same reasons.
But having the tooling for a language in the same language definitely has benefits too
Yeah, it is definitely better for optics and marketing.
I don't think the core team cares much about marketing by building tooling to show off. One example of this is ask.clojure being a PHP website. Pragmatics for the win ;). But I think it has benefits of just being able to jump in and understand "the script" because it's in Clojure.
Yeah, probably true. I guess the marketing, if people would care to read between the lines, is that the core team is pragmatic.
we don't care about the marketing aspect of that. I personally care about what I have to maintain
in order, I care about: user's experience, being able to maintain it, clojure ftw
powershell was a reasonably close port from a maintenance pov and (I assumed) would result in a reasonable user experience, but I think experience has shown otherwise
It might be interesting to see what a launcher would look like in written in Go. Not necessarily to choose it, but just to see what that would look like. Any experienced Go devs in the crowd here?
I've done some go (some babashka pods are written in that) but I think you can imagine it would look similar to Java, but different, more if err != nil checks. Just look at any golang script to get a taste of it ;)
So how about a proof of concept MSI installer which bundles deps.exe and the clojure tools .jar , copies deps.exe to clojure.exe and clj.exe in a directory automatically added to the PATH, copies the tools jar to C:\Users\<user>\.deps.clj (see deps.clj code for expected location).
This should give you a feeling for how well the installation experience is, while having a taste of clj as an exe (`deps.exe` should already work as drop in currently, but deliberately isn't called clj.exe to prevent conflicts).
After that, we could continue thinking about what has to change about deps.exe or alternatives, if that doesn't tick all the boxes
By tools jar do you mean the compiled uber jar or the tools tool jar?
My assumption about msi being better than an exe is both familiarity and that it can include some kind of signing for verification
@alexmiller I mean all of these files that come from the tools zip. Normally deps.clj downloads that zip on demand and unpacks the files to a location where it searches it next time.
$ ls -la ~/.deps.clj/1.11.1.1165/ClojureTools
total 39912
drwxr-xr-x 6 borkdude staff 192 Sep 18 21:22 .
drwxr-xr-x 3 borkdude staff 96 Sep 18 21:22 ..
-rw-r--r-- 1 borkdude staff 20419005 Sep 18 21:22 clojure-tools-1.11.1.1165.jar
-rw-r--r-- 1 borkdude staff 1490 Sep 18 21:22 example-deps.edn
-rw-r--r-- 1 borkdude staff 3662 Sep 18 21:22 exec.jar
-rw-r--r-- 1 borkdude staff 126 Sep 18 21:22 tools.ednand i guess that msi can be run without a gui
I'm not sure why some folks provide both msi and exe. But if choosing only one, msi seems to make sense.
> My assumption about msi being better than an exe We're talking about the installer here right?
ya, I am anyway
yes
I have repressed experience with authoring MSIs, so I'd be happy to take a crack at the above sometime this weekend. I took a quick look at deps.clj, but would the correct way to support a global/all users installation be to set the DEPS_CLJ_TOOLS_DIR env var to somewhere multi-user accessible?
I think so
Updated version with better UI, supports per-user or per-machine install, and detects/warns if it finds the PowerShell module: https://github.com/casselc/clj-msi/releases/tag/clojure-1.11.1.1165-with-ui
Awesome! I'll try it in hour or so. Any recommendations how to delete the existing powershell install?
The installer warning will show the path it found the module in and you should be able to just delete that folder via explorer or shell. May want to close any PS windows first to be sure none of the module files are in-use.
I can add a custom action eventually to attempt to delete it from the installer. Just haven’t made it that far yet.
I’m heading to bed, but will check in later. I know you’d also asked about installing in CI, so fyi there’s some basic info in the readme about the switches for silent installation and overriding installation directory for the command line.
Nice!
@chuck.cassel This worked beautifully for me. I went from no clj to a working installation by downloading the binary, executing it, allowing the virus scanner to run it, clicking once or twice and then it worked in both cmd.exe and Powershell!
@chuck.cassel I think the README could be clearer about where to download the installer (if you want to pre-built one) and perhaps a link to deps.clj in the README could also help to clarify what happens under the hood.
I can definitely add some clearer instructions
Thanks for this! I’m traveling today, hope to check it out tomorrow!
Cool! Firing up my Windows 10 VM to try now!
I’ll move my experience to a separate thread to avoid polluting this one.
First attempt at this concept is available here: https://github.com/casselc/clj-msi I published a release, but since I'm just some guy from the internet, you can also use the .ps1 from the repo to build an MSI locally.
Oh yeah, good detail: if we move away from powershell, will need a migration guide, which would basically be, I guess, instructions on how to uninstall the powershell module.
But the existing scoop installer could probably handle this migration automagically.
As always, it's easier to add than remove :P
Ya true
Tangent: for making release available for various package managers (maybe in general, not just for Windows) might be worth looking at something like https://jreleaser.org/. Never used it, just stumbled upon it now.