Fork me on GitHub

yeah sorry; I meant outside of watch… e.g. in CI. Even printing a copy-pastable snippet of --focus args to rerun the last run (when run without watch) might be useful.


you mean like --plugin kaocha.plugin/print-invocations ? 🙂


Also in CI I’d quite like: 1. kaocha to be able to save the info you’re suggesting, cache it for the next run; and then execute the previously failing tests first. (So it will fail faster next time if that is still a failing test) 2. To be able to annotate selected tests with ^:flaky-test metadata, and get kaocha to retry those tests a set number of times if they fail. Ideally we wouldn’t need this one of course; but from time to time when a timing issue pops up in an integration test; it’d be useful to temporarily patch it up in this manner; rather than retry the whole build 😞 or be forced to debug a flakiness issue at a time when there other more pressing fires to put out.


Oh one other thing whilst I’ve been meaning to mention whilst I’m here. It’s great that you can use aero in config… however one thing I’ve found (both in using aero with kaocha and in numerous projects using aero elsewhere). Is that when trying to build profile cascades with aero; where you have a base config and a local.edn file of develper specific customisations, aero gets really messy and hard to DRY up. e.g. I have this base test.edn config:

#merge [{:tests [{:id          :static-data
                  :test-paths  ["test"]
                  :source-paths ["src"]
                  :ns-patterns ["-test$"]
                  :skip-meta [:cleans-database]}
                 {:id         :mutating-data
                  :test-paths  ["test"]
                  :source-paths ["src"]
                  :ns-patterns ["-test$"]
                  :focus-meta [:cleans-database]}]
         :plugins [:kaocha.plugin/instrument-specs
        #include "tests.user.edn"]
If a user wants to add their own extra plugin, e.g. :kaocha.plugin/notifier they have to copy the whole set of plugins from the base config into their own config. This can then be problematic when the base config changes without the user knowing, as they’re stomping the whole set of plugins, not just adding a new one. For these scenarios I’ve found meta-merge across whole files of config to be a far superior approach. I’ve not found a suitable solution to this with pure aero; however I thought I’d mention that on an internal project of ours which had this issue; we added meta-merge as a dependency, and exposed it through aero with a two-line aero/reader extension:
(defmethod aero/reader 'meta-merge [opts _tag value]
  (apply meta-merge value))
This one change then let replace almost all of aero with a single #meta-merge in place of #merge at the top, and aero #include


Those all sounds like great ideas to explore in plugins. I'm happy to make time to pair with you on that, or to answer questions and provide feedback through code review.


@plexus: thanks… I’ve been meaning to mention them, here first; incase there was something in the works or that did it already — and if not write an issue up for them, and then either write the plugin myself, or wait for someone else to do it (as happened with the spec check stuff 🙂 — which was a nice surprise; though I’ve yet to try it out) .


#meta-merge would be useful and welcome in core


That would be awesome


and is a PR I might even have time to implement 🙂


here's the code I mentioned, it creates a .kaocha-stats directory with timestamped files for each run, containing the test id, duration and result


this is two thirds of the way of implementing a --re-run-failed option, you'd need a cli-options and config hook to support the CLI flag, and a post-load hook to skip everything that passed in (latest-stats)


Thanks that’s interesting


thanks! not so sure about the k/meta-merge. Either make it kaocha/meta-merge, but I think just meta-merge is fine in this case.


also for next time please add an entry to the CHANGELOG. It's fine for now though, I'm happy to merge this and make these changes myself.