testing 2023-05-04 | Slack Archive

respatialized14:05:30

How do people handle approaches to testing when your assertions start to get into the n >= 100,000 range? I have some tests that aren't particularly expensive to run, so I run them across a huge range of inputs to make sure the property holds over a realistic volume and variety of data, but naively wrapping each property check in clojure.test/is is pretty slow and makes my editor fall over when I try to run the tests from it. It seems like I should probably just aggregate over the sequence of inputs I want to test and use an "expected true count" vs "actual true count" - but I'd still like to narrow in on the failing inputs more easily. Wondering what people do for tests like this.

seancorfield17:05:20

This sounds like prime territory for test.check...?

respatialized17:05:45

I'm not using generators to produce inputs for these tests, which I think is the assumption made by prop/for-all - these are tests that use a lot of real data and verify properties of that data after transforming it. https://clojure.github.io/test.check/clojure.test.check.properties.html

seancorfield17:05:45

Right, I was suggesting switching from example-based tests to generative tests. It's fine to have a few boundary cases expressed as examples, but if you're trying to deal with large numbers of assertions across large numbers of inputs, I would definitely want to use property-based (generative) testing for that.

seancorfield17:05:38

It may not be appropriate for your case -- but it sounds likely, given your description...

respatialized18:05:51

This is a context where "exactly right across all known inputs" is the kind of guarantee I need to offer rather than "doesn't fail on large numbers of randomly selected inputs"

2023-05-04

Channels