Turns out it is; HoneySQL is a library with such tests! https://clojurians.slack.com/archives/C03S1KBA2/p1606436180241300

Sure, we do this for instance in lambdaisland.uri , using kaocha and kaocha-cljs https://github.com/lambdaisland/uri

Cool, a different way to do it. How do you run them?

It’s probably obvious to people more well versed with clj and deps.edn - I’m still getting into it.

standard kaocha stuff, see tests.edn and bin/kaocha

Right you are 🙂 Thanks.

yeah well I didn't open the PR to merge it sorry (that's why it's just a draft)

just to have something to show to understand what I'm doing wrong

but yeah eventually when it works I can deploy it myself surely

but it's quite far from actually working atm

I'd just like to understand if you have any suggestions about how you would do that

sure, I would start by using a more fine grained hook. Do you absolutely want to run all tests first before retrying, or is it also fine to retry a failing test immediately?

you're using post-run which returns a full nested test plan, you'll need a lot more logically to correctly deal with that. If you can use post-test then you can deal with one test at a time

you might also be interested in kaocha.result/failed? or kaocha.result/failed-one?

ah yes I haven't tried post-test but that looks more suitable

and yeah I saw failed? but I needed to get the l ist of failed tests

which I could not get from failed?

(filter failed-one? (test-seq ...)) doesn't work?

ah good point I didn't try that

and the other question is how to make the reporting work (discarding the runs where the test failed)

but maybe if I use the other more granular hook it will work directly

that's harder... you'll have to intercept the reporting events first, and then replay them if necessary

ah ok yeah I see

and last question is if makes sense to use kaocha.repl/run in a plugin

you can use pre-report to intercept reporting events, change failing events to succeeding for instance, then replay the failure if all retries fail

alternatively you can use wrap-run to temporarily rebind the reporter...

ah cool yeah lots of possibilities

ok thanks I'll try a few more things

hmmm no I wouldn't use repl/run in a plugin. The repl namespace is really only meant for direct repl use, not as an API

ah yeah I thought it wasn't a great idea

if you want to re-run single test/testable testable/run-testable is your friend