Clojurians Log v2
polylith
A question about test runners...
I've been looking at https://github.com/seancorfield/polylith-external-test-runner/issues/5 (test selection/exclusion based on metadata) and https://github.com/seancorfield/polylith-external-test-runner/issues/6 (including the src folder when searching for tests) for the external test runner, and @tengstrand has considered :include-src-dir as a possible addition to the built-in test runner.
The other external test runner (kaocha) also has a whole bunch of functionality that could be exposed.
While it might be "obvious" to provide per-project test configuration via workspace.edn -- my test runner now has examples of including src directories and either including only tests with specific metadata or excluding tests with specific metadata https://github.com/seancorfield/polylith-external-test-runner/blob/main/workspace.edn (very much like Cognitect's test-runner) -- in reality, I suspect we would all be much more likely to want to control this on a per-test-run basis rather than statically per-project.
Especially in CI, I could see wanting to run ^:integration tests that you might have configured to be excluded by default in workspace.edn.
So the question is: how would you expect to be able to do this?
I'm currently leaning towards an environment variable (per test runner, since each will likely have a different set of options). The possibilities are an EDN string that is a hash map of test options that replaces the workspace.edn options, or is merged with those options, or perhaps identifies a configuration file that is read and then replaces/merges with the test options.
What would y'all prefer?
I have released 0.2.20-SNAPSHOT #13 to master. Includes https://github.com/polyfy/polylith/issues/457. Please check it out!
I'm unlikely to test this for quite a while since it doesn't address a problem we have at work, which is where I use Polylith.
I've gone with ORG_CORFIELD_EXTERNAL_TEST_RUNNER="{...}" and implemented that to try it out. Specifying "{:include-src-dir true :focus {}} overrides the workspace.edn settings and runs all tests in all namespaces (not just test namespaces).
I need a closer look at this, in the weekend.
I created issue https://github.com/polyfy/polylith/issues/457. Please have a look and see what you think.
Looks great! I added a comment on the issue -- I think that's a great, pragmatic compromise.
Okay, great! Will start working on it right away. Happy Easter!
Is it Easter? It's not generally celebrated here (unless you're religious I guess)
Here we eat easter eggs filled with candy, and we have Easter food (similar to what we eat at Christmas), and today is a day off (red day as we call it here) and almost all shops are closed.
I took this whole week off -- "just because" -- I bought some Cadbury Creme Eggs on sale a week or two back so I actually thought Easter was over 🙂
Something that might be nicer than an env var would be a way to pass some arbitrary EDN data into the poly test command, either on the command-line or within the shell:
clojure -M:poly test :settings "{:org.corfield/external-test-runner {:focus {:exclude [:integration]}}}"test-settings that is passed into the test runners.
This could be used to run tests with different test runners from the command-line too:
clojure -M:poly test :settings "{:create-test-runner org.corfield.external-test-runner.interface/create}"
...
clojure -M:poly test :settings "{:create-test-runner polylith-kaocha.test-runner/create}"