kaocha 2023-02-15 | Slack Archive

dergutemoritz09:02:42

Heya! Is there a builtin way of emitting / persisting which tests failed during a run for the purpose of feeding this list back into the runner for retry? Rationale: Some integrative tests in our test suite are sensitive to load on our CI runner and may flake out when a lot of other jobs are running concurrently. Retrying them usually makes them pass again. Only retrying the failed ones would greatly speed this process up. Other ideas for dealing with this welcome, too 🙂

frank13:02:00

there's this: https://github.com/lambdaisland/kaocha-junit-xml

frank13:02:32

alternatively, if your CI system has some sort of workflow system that allows rerun of failed jobs, you could tag the integration tests and run them in their own job

dergutemoritz12:02:17

Ah thanks, using the junit xml report (which we already have) for this purpose didn't occur to me, yet! 🙏

dergutemoritz12:02:20

Manually tagging the tests is an option but having it work automatically is even nicer since it's easy to forget to tag a test appropriately

dergutemoritz12:02:27

Thanks for your input!

Alys Brooks03:02:08

Sorry I didn't see this sooner, but if you're still looking for a solution, consider the community-created plugin kaocha-retry: https://github.com/AndreaCrotti/kaocha-retry

dergutemoritz13:02:07

Oh perfect, thanks!!

2023-02-15

Channels