Clojurians Log v2
testing
How do people handle approaches to testing when your assertions start to get into the n >= 100,000 range? I have some tests that aren't particularly expensive to run, so I run them across a huge range of inputs to make sure the property holds over a realistic volume and variety of data, but naively wrapping each property check in clojure.test/is is pretty slow and makes my editor fall over when I try to run the tests from it.
It seems like I should probably just aggregate over the sequence of inputs I want to test and use an "expected true count" vs "actual true count" - but I'd still like to narrow in on the failing inputs more easily. Wondering what people do for tests like this.
This sounds like prime territory for test.check...?
I'm not using generators to produce inputs for these tests, which I think is the assumption made by prop/for-all - these are tests that use a lot of real data and verify properties of that data after transforming it.
https://clojure.github.io/test.check/clojure.test.check.properties.html
Right, I was suggesting switching from example-based tests to generative tests. It's fine to have a few boundary cases expressed as examples, but if you're trying to deal with large numbers of assertions across large numbers of inputs, I would definitely want to use property-based (generative) testing for that.
It may not be appropriate for your case -- but it sounds likely, given your description...
This is a context where "exactly right across all known inputs" is the kind of guarantee I need to offer rather than "doesn't fail on large numbers of randomly selected inputs"