no, and yes

Cool, good to know. Thanks

Given how easy it is to run tests against multiple versions of Clojure, I'm surprised library maintainers don't do this as a matter of course anyway: https://github.com/seancorfield/next-jdbc/blob/develop/build.clj#L24-L36 I used it to do via the shell in a lot of my projects like this https://github.com/clojure-expectations/clojure-test/blob/develop/run-tests.sh (but now I use tools.build so I can write logic in Clojure instead of bash!).

Well, I was thinking the other way around. Someone develops a feature or bug fix or the core team wants to release a new version of Clojure, so they run "clojure-canary" on 80% of the projects on clojars and see what differences there are in the output

I think you're right that there's a lot of value in library authors maintaining backwards compatibility by testing against multiple released Clojure versions. My thought here was about the desire to change existing core code without affecting the whole ecosystem

yes, that is something I'm interested in - we sometimes do more targeted versions of this informally, but would be interested in doing it more broadly, automatically (esp for things like the var intern changes in 1.12.0-alpha1 where it's hard to predict unusual situations that have unexpected consequences)

Mike Fikes developed a test suite called canary for ClojureScript (I think it was Mike Fikes). You can ask him if they are still using it for ClojureScript development.

Excellent find, thank you

I'm doing a similar thing with babashka in CI: I run the test suites of a couple of dozen libraries

ClojureScript has a similar thing, I forgot the name but you can find it via the above link

https://github.com/cljs-oss/canary

borkdude’s link opened up slack in a browser tab so not exactly sure what it was linking to. But canary is largely what you were talking about noah

Here's the link again: https://clojurians.slack.com/archives/C07UQ678E/p1671210051263259

I guess the link turns into something different if I open it in another tab myself

thanks, i found the original message. and yes, canary looks right. Without the backing of a big corp I guess it's hard to run all of the tests locally/within something like canary directly, instead relying on the kindness/opt-in of library devs

why would that be?

I haven't asked any kindness of library devs to run those libs for bb

Github doesn't give unlimited CI minutes, so you're either limiting the number of libraries you check or you have to pay out of pocket

I've never hit the limitation with my own public projects, have you?

I'm not saying it doesn't exist, but I've never hit it

I think Cirrus CI doesn't limit minutes at all, instead they cap you to 1 CPU or so

sure, that's probably true. I'm coming from the crater perspective that checks almost every single crate and a very large number of rust github repos, going for a "blacklist faulty projects" approach instead of hand-selecting specific projects

Fwiw, I also do canary testing for rewrite-clj. Any project I find that directly uses rewrite-clj that also has a test suite and tagged releases gets its test suite run against rewrite-clj HEAD on GitHub Actions.