kaocha

mike_ananev 2023-01-04T14:24:50.368909Z

Hi. Is there a way to run tests N times using kaocha? I need to run tests from command line without JVM restart.

Alys Brooks 2023-01-06T19:35:33.400479Z

@mike1452 Hmm, consider reporting it as an issue. Note that this plugin only retries tests when they fail fails.

mike_ananev 2023-01-07T10:29:58.652909Z

Oh. I expected retries when all tests are passed and stop retry if any fails.

frank 2023-01-04T15:47:48.732089Z

I'm curious why one would want to run the same tests N times 🤔

☝️ 1
2023-01-04T17:04:15.709029Z

you can do this with a fixture tho i don't recommend it: (defn n-times [n] (fn [f] (doseq [_ n] (f)))) and then (use-fixtures :each (n-times 100)) in the namespace of your choosing, then run kaocha selecting the single test

practicalli-johnny 2023-01-04T17:15:04.462459Z

Personally I would use a shell script, a for loop that calls the test runner each time. A separate script avoids changing oradding complexity to existing code

👍 1
2023-01-04T17:16:00.088109Z

Could also be done with a tools.build function, which would use a single jvm instance to run the test multiple times

2023-01-04T17:16:37.167279Z

(doseq [_ n] (k/run 'kaocha.random-test/rand-ints-test))

👍 1
mike_ananev 2023-01-04T19:43:32.582869Z

@frank I need to test I/O library via network with some complex state. Multiple test run helps me to find bugs in library implementation. Practically it helps to reorder network packets.

👍 1
mike_ananev 2023-01-04T19:45:48.897789Z

@nbtheduke thanks. I think about the same solution.

👍 1
Alys Brooks 2023-01-04T22:09:00.542539Z

The solutions already mentioned are simpler, but you could do this with a plugin or a hook. There's an existing Kaocha plugin to rerun tests that you could probably use as a basis for a run N times plugin: https://github.com/AndreaCrotti/kaocha-retry

mike_ananev 2023-01-04T23:31:10.593609Z

@actuallyalys_slack thanks. I tried it, but it has no effect - no retries. May be plugins interfere each other.