This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-04-17
Channels
- # ai (115)
- # announcements (8)
- # babashka (26)
- # beginners (7)
- # biff (8)
- # calva (1)
- # cider (10)
- # clerk (2)
- # clj-together (11)
- # clojure (26)
- # clojure-boston (1)
- # clojure-denmark (3)
- # clojure-europe (23)
- # clojure-nl (1)
- # clojure-norway (33)
- # clojure-uk (3)
- # clojurescript (14)
- # conjure (3)
- # cursive (65)
- # emacs (10)
- # events (18)
- # exercism (6)
- # honeysql (14)
- # hyperfiddle (11)
- # kaocha (6)
- # nbb (17)
- # off-topic (58)
- # pathom (5)
- # reagent (28)
- # reitit (1)
- # releases (1)
- # sci (3)
- # shadow-cljs (22)
- # xtdb (29)
In a codebase at work, I often struggle with the issue where I don't know which tests actually cover a certain function I am working on. For various reasons, it is not feasible for me to run the entire suite locally. Currently I manually find references and then keep going up the reference chain till I find tests that will indirectly/directly invoke the function I care about. I add focus
metadata to their namespaces and run the tests.
I was wondering given code coverage tools know which lines of code are covered by tests, can they (or other tools) also list which tests indirectly execute a certain function?
Some tools, including Conjure and CIDER can run relevant tests automatically, although I think they all just go off of the namespace name, e.g., if you activate the feature for namespace foo
, it tries to run tests in foo-test
.
I think you're right that running tests for a function should be possible using the same techniques as code coverage tools (or perhaps the coverage data directly), but I'm not aware of a current way to do it. A coverage tool could theoretically let you go to the granularity of a line or form, so you could, e.g., just run tests that exercise a particular branch.
Clojure-LSP can find all references to a symbol, so while that doesn't directly run them, that might save you a bit of time by finding tests that reference the relevant function. However, that would miss tests that call a function indirectly. That might be a strength in some cases: A core function might have both direct unit tests and also used indirectly to set up a large number of other tests—the "find symbol" approach would only find the tests that specifically validate its behavior.
I think this function could be implemented as a plugin that invokes an LSP or code coverage tool and identifies all tests that cover the specified funciton, skipping all other tests.
Thank you for articulating my approach better and in more detail, and listing the pro/cons. You described my issue perfectly 👍 Yeah, I am doing currently using LSP find references multiple times to manually create a call tree and eventually find interesting tests that indirectly call the function I care about. It gets tiresome very quickly. > as a plugin that invokes an LSP or code coverage tool and identifies all tests that cover the specified funciton, I think clojure-lsp uses clj-kondo under the hood to do the analysis and maintain references. I wonder whether I should look into using clj-kondo directly...
It would be super cool if, as a part of marking a line of code covered, cloverage also noted the test(s) that covered it. However, I'm also unaware of a tool that currently does this.
Glad to help, @U0A5B1LJU. I haven't looked into using clj-kondo as a library (as opposed to running it on my code), so I'd be curious how easy it is to use in that context. @U02E7A2NGKD, that is an interesting idea. I did find the coverage-testing function (https://github.com/cloverage/cloverage/blob/87fd10f82ea7c0f47c03354105e513b160d1e047/cloverage/src/cloverage/coverage.clj#L100), and it seems to just add each form to a list of covered forms. I suppose you could use a dynamic variable with the current test, and then record that test symbol as having invoked that form?
Clerk also has some pretty good tracking code that we could possibly steal. See here for an earlier write-up https://nextjournal.com/lambdaisland/kaocha-autotest