Fork me on GitHub
Noah Bogart15:02:51

In Rust, they use a tool called which allows them to detect regressions across the rust/cargo ecosystem by downloading, building, and running the test suites of "a large number of crates" (rust libraries) in two versions of the Rust compiler. Does the Clojure dev team have anything like that? Would there be any interest in such a tool?

Noah Bogart15:02:05

Cool, good to know. Thanks


Given how easy it is to run tests against multiple versions of Clojure, I'm surprised library maintainers don't do this as a matter of course anyway: I used it to do via the shell in a lot of my projects like this (but now I use so I can write logic in Clojure instead of bash!).

Noah Bogart19:02:58

Well, I was thinking the other way around. Someone develops a feature or bug fix or the core team wants to release a new version of Clojure, so they run "clojure-canary" on 80% of the projects on clojars and see what differences there are in the output

Noah Bogart19:02:52

I think you're right that there's a lot of value in library authors maintaining backwards compatibility by testing against multiple released Clojure versions. My thought here was about the desire to change existing core code without affecting the whole ecosystem

Alex Miller (Clojure team)19:02:48

yes, that is something I'm interested in - we sometimes do more targeted versions of this informally, but would be interested in doing it more broadly, automatically (esp for things like the var intern changes in 1.12.0-alpha1 where it's hard to predict unusual situations that have unexpected consequences)

👍 2

Mike Fikes developed a test suite called canary for ClojureScript (I think it was Mike Fikes). You can ask him if they are still using it for ClojureScript development.

👍 2
Noah Bogart15:02:03

Excellent find, thank you


I'm doing a similar thing with babashka in CI: I run the test suites of a couple of dozen libraries


ClojureScript has a similar thing, I forgot the name but you can find it via the above link


borkdude’s link opened up slack in a browser tab so not exactly sure what it was linking to. But canary is largely what you were talking about noah


I guess the link turns into something different if I open it in another tab myself

Noah Bogart15:02:51

thanks, i found the original message. and yes, canary looks right. Without the backing of a big corp I guess it's hard to run all of the tests locally/within something like canary directly, instead relying on the kindness/opt-in of library devs


why would that be?


I haven't asked any kindness of library devs to run those libs for bb

Noah Bogart15:02:24

Github doesn't give unlimited CI minutes, so you're either limiting the number of libraries you check or you have to pay out of pocket


I've never hit the limitation with my own public projects, have you?


I'm not saying it doesn't exist, but I've never hit it


I think Cirrus CI doesn't limit minutes at all, instead they cap you to 1 CPU or so

Noah Bogart15:02:42

sure, that's probably true. I'm coming from the crater perspective that checks almost every single crate and a very large number of rust github repos, going for a "blacklist faulty projects" approach instead of hand-selecting specific projects


Fwiw, I also do canary testing for rewrite-clj. Any project I find that directly uses rewrite-clj that also has a test suite and tagged releases gets its test suite run against rewrite-clj HEAD on GitHub Actions.

👍 2