Fork me on GitHub
#kaocha
<
2023-01-04
>
mike_ananev14:01:50

Hi. Is there a way to run tests N times using kaocha? I need to run tests from command line without JVM restart.

frank15:01:48

I'm curious why one would want to run the same tests N times :thinking_face:

☝️ 2
Noah Bogart17:01:15

you can do this with a fixture tho i don't recommend it: (defn n-times [n] (fn [f] (doseq [_ n] (f)))) and then (use-fixtures :each (n-times 100)) in the namespace of your choosing, then run kaocha selecting the single test

practicalli-johnny17:01:04

Personally I would use a shell script, a for loop that calls the test runner each time. A separate script avoids changing oradding complexity to existing code

👍 2
Noah Bogart17:01:00

Could also be done with a tools.build function, which would use a single jvm instance to run the test multiple times

Noah Bogart17:01:37

(doseq [_ n] (k/run 'kaocha.random-test/rand-ints-test))

👍 2
mike_ananev19:01:32

@frank I need to test I/O library via network with some complex state. Multiple test run helps me to find bugs in library implementation. Practically it helps to reorder network packets.

👍 2
mike_ananev19:01:48

@UEENNMX0T thanks. I think about the same solution.

👍 2
Alys Brooks22:01:00

The solutions already mentioned are simpler, but you could do this with a plugin or a hook. There's an existing Kaocha plugin to rerun tests that you could probably use as a basis for a run N times plugin: https://github.com/AndreaCrotti/kaocha-retry

mike_ananev23:01:10

@U01FJUDL57C thanks. I tried it, but it has no effect - no retries. May be plugins interfere each other.

Alys Brooks19:01:33

@U097654L8 Hmm, consider reporting it as an issue. Note that this plugin only retries tests when they fail fails.

mike_ananev10:01:58

Oh. I expected retries when all tests are passed and stop retry if any fails.