tools-deps

borkdude 2023-10-07T09:11:55.147779Z

While I was trying to get to the root of a problem that @sibuna had in #clj-on-windows, we found that:

clojure -Ttools install-latest :lib io.github.seancorfield/clj-new :as new
clojure -Ttools install-latest :lib io.github.seancorfield/deps-new :as new
i.e., overwriting a tool by a different tool under the same name, results in an error. Could this be a bug in tools-deps?

seancorfield 2023-10-07T15:21:36.940879Z

What's the error?

borkdude 2023-10-07T16:19:37.766199Z

$ clojure -Ttools install-latest :lib io.github.seancorfield/deps-new :as new
Execution error (ExceptionInfo) at clojure.tools.deps.extensions.git/coord-err (git.clj:45).
Library io.github.seancorfield/deps-new has invalid tag: v1.2.404

Full report at:
/var/folders/j9/xmjlcym958b1fr0npsp9msvh0000gn/T/clojure-888812946633046028.edn
The issue is not specific to your tools, just happens when you install tool X under name N and then a completely different tool Y under the same name N

seancorfield 2023-10-07T16:45:26.825429Z

Ah, because it uses the existing alias EDN file to determine whether the newest version of the :lib it can find is actually older than the currently installed one. That's an interesting puzzle. I guess if it cannot compare the two versions (for any reason that leads to an exception) then you have two possible choices: • Report the old coordinates, say it can't be compared, and skip the install. • Go ahead and overwrite it, possibly with a message about the old coordinates. The former is probably safer and would, overall, be a better UX. As it stands, the behavior is "correct" but not very helpful to folks in terms of diagnosing what they actually did wrong.

Bastian 2023-10-07T18:01:23.856159Z

well, if one uses install + version number instead of install-latest it has no problem in overwriting the name. not sure if "folks did something wrong" according to the api docs, in any case a better error message could save some time

seancorfield 2023-10-07T18:18:45.279089Z

If you use an explicit version, it doesn't have to compare for latest: you can install any version, even older versions. If you want latest, it has to compare the current version to see whether to install or not. If you've already installed, say, some specific branch version that you're testing, you can overwrite it with another specific version, but you probably don't want some "random" latest version overwriting that specific version?

borkdude 2023-10-07T18:22:37.240049Z

why does it have to compare something if you want to install the latest. it could just decide to overwrite the installed one with the latest it can find?

borkdude 2023-10-07T18:22:50.031079Z

what does "install" even mean

Bastian 2023-10-07T18:22:58.387849Z

thats my point ( i just expected install-latest to install the latest )

seancorfield 2023-10-07T19:39:46.948649Z

This should be a safe, idempotent operation:

> clojure -Ttools install-latest
antq: Skipping, newest installed com.github.liquidz/antq 2.7.1133
clj-new: Skipping, newest installed com.github.seancorfield/clj-new v1.2.404
clj-watson: Skipping, newest installed io.github.clj-holmes/clj-watson v4.1.3
new: Skipping, newest installed io.github.seancorfield/deps-new v0.5.3
poly: Did not find versions for io.github.polyfy/polylith
tools: Skipping, newest installed io.github.clojure/tools.tools v0.3.1

borkdude 2023-10-07T19:41:32.627429Z

Do you mean, safe as in, when people remove the latest version from the internetz you will still have the latest version locally?

seancorfield 2023-10-07T19:42:04.889649Z

Strawman. "People" (Clojurians) don't randomly remove the latest version from the internetz.

borkdude 2023-10-07T19:42:25.212419Z

Then what do you mean?

seancorfield 2023-10-07T19:42:39.346309Z

I mean, safe in terms of not overwriting any special version I have installed as I already explained above.

seancorfield 2023-10-07T19:43:24.571619Z

For example, if you're working on a tool and have it installed via a :local/root dep, you don't want that overwritten by install-latest.

borkdude 2023-10-07T19:43:36.417559Z

Sorry, I didn’t understand that

borkdude 2023-10-07T19:43:41.329619Z

Got it

seancorfield 2023-10-07T19:43:46.946619Z

Or you're depending on a Pull Request, or non-merged branch...

seancorfield 2023-10-07T19:44:02.723019Z

You want all of those to be not comparable and therefore not overwritten.

seancorfield 2023-10-07T19:44:28.170859Z

I think we all agree that the error message could be improved in this case 🙂

borkdude 2023-10-07T19:44:37.432939Z

Right, fully agreed

Bastian 2023-10-07T19:47:31.055329Z

i dont't understand how the example of install-latest without any arguments relates to the above problem and think install and update are confused here.

borkdude 2023-10-07T19:48:29.954129Z

How does install-latest detect that you don’t want to upgrade a specific version?

seancorfield 2023-10-07T19:49:54.826179Z

I assumed you're asking Bastian their opinion since I've already explained why (and how) above?

borkdude 2023-10-07T19:52:53.477349Z

I don’t fully understand yet but I’m also ok with dropping this conversation for today

borkdude 2023-10-07T19:53:35.620679Z

I’m kinda tired from COVID so I’ll check back in later

borkdude 2023-10-07T20:31:17.159859Z

@seancorfield I guess what I mean is this scenario: This is in my ~/.clojure/tools/new.edn

{:lib io.github.seancorfield/deps-new, :coord {:git/tag "v0.5.3", :git/sha "c899135"}}
How does clj know that this was the result of install-latest or that I just put it there to stick to this specific tag + sha and I want to stay on those, thus install should skip in case there is a new version in the future. Is there a way to tell the difference? You might have already explained this, but then I'm just very sorry for not understanding and I'll just finally give up for today ;)

borkdude 2023-10-07T20:40:56.845329Z

When I change new.edn to {:lib io.github.seancorfield/deps-new, :coord {:git/sha "5d0e47d05dc828703c23253ed9fc57c2d61f7848"}} and then run install-latest, it happily changes it back to {:lib io.github.seancorfield/deps-new, :coord {:git/tag "v0.5.3", :git/sha "c899135"}}

borkdude 2023-10-09T10:30:11.617909Z

Aaaanyway, @alexmiller, if you want an issue for the problem "overwriting a tool with an existing name but from another project fails with cryptic error" on ask.clojure, I will create one

Alex Miller (Clojure team) 2023-10-09T12:12:22.249949Z

Yes

Alex Miller (Clojure team) 2023-10-09T12:18:56.845029Z

If you install-latest of a lib, it should install the latest of that lib and ignore the current tool named that

👍 1
borkdude 2023-10-09T12:19:55.066489Z

I'd say so too

seancorfield 2023-10-09T16:07:29.422309Z

@alexmiller would it still skip the download/install if the currently installed version looked the same as the latest version determined? (rather than trying to compare for greater than / less than - just use Clojure equality on the hash map)

Alex Miller (Clojure team) 2023-10-09T16:12:53.922179Z

yes, that's where it's getting tripped up, but it should just ignore in the case where they are different

borkdude 2023-10-09T16:14:43.168139Z

> would it still skip the download/install FWIW, I demonstrated above that install-latest went back to the latest tag even though I had a newer SHA than the tag specified in the tools/new.edn file. Not sure what logic is happening here, but also, it's not so important to me because for development I would probably just rely on :aliases + -X to test

seancorfield 2023-10-09T16:47:24.232039Z

point_up::skin-tone-2 That was unexpected to me. I had assumed the whole point of install-latest doing that version check was so that it would not overwrite a more recent SHA install of a tool with an earlier tag version, so that you could install from a PR and not worry about it getting overwritten until that PR was merged and a new version released? @alexmiller

Alex Miller (Clojure team) 2023-10-09T16:53:58.673499Z

no, that was not the point, it was to avoid doing an install if latest was already installed. If you say "install-latest", then it will install latest.

seancorfield 2023-10-09T17:04:01.280159Z

OK, as long as the docs make it clear that install-latest could overwrite an install of the latest SHA with an earlier tagged version... 🙂

Alex Miller (Clojure team) 2023-10-09T17:11:31.316079Z

tools only consider tags as versions. if you "install latest" version, then you are installing the latest tagged version. other shas are not considered

Alex Miller (Clojure team) 2023-10-09T17:12:20.543369Z

the https://clojure.github.io/tools.tools/clojure.tools.tools.api.html#var-install-latest imply all of this, but maybe but do not say it clearly enough

👍🏻 1