kaocha 2023-01-04

Hi. Is there a way to run tests N times using kaocha? I need to run tests from command line without JVM restart.

@mike1452 Hmm, consider reporting it as an issue. Note that this plugin only retries tests when they fail fails.

Oh. I expected retries when all tests are passed and stop retry if any fails.

I'm curious why one would want to run the same tests N times 🤔

☝️ 1

you can do this with a fixture tho i don't recommend it: (defn n-times [n] (fn [f] (doseq [_ n] (f)))) and then (use-fixtures :each (n-times 100)) in the namespace of your choosing, then run kaocha selecting the single test

Personally I would use a shell script, a for loop that calls the test runner each time. A separate script avoids changing oradding complexity to existing code

👍 1

Could also be done with a tools.build function, which would use a single jvm instance to run the test multiple times

(doseq [_ n] (k/run 'kaocha.random-test/rand-ints-test))

👍 1

@frank I need to test I/O library via network with some complex state. Multiple test run helps me to find bugs in library implementation. Practically it helps to reorder network packets.

👍 1

@nbtheduke thanks. I think about the same solution.

👍 1

The solutions already mentioned are simpler, but you could do this with a plugin or a hook. There's an existing Kaocha plugin to rerun tests that you could probably use as a basis for a run N times plugin: https://github.com/AndreaCrotti/kaocha-retry

@actuallyalys_slack thanks. I tried it, but it has no effect - no retries. May be plugins interfere each other.