Fork me on GitHub
#test-check
<
2016-08-02
>
rickmoynihan12:08:02

is there a good resource to help understand shrinking; how it works, what to consider etc...

rickmoynihan12:08:35

it seems like quite an involved topic

lucasbradstreet13:08:18

I’m interested in this. I have a reasonable intuition about it, but I’d like something in more detail

gfredericks14:08:04

well I can answer specific questions about it at least

gfredericks14:08:48

for the most part the shrinking strategy mirrors the structure of your generator

gfredericks14:08:21

e.g., tuple shrinks by shrinking its elements; collections shrink by deleting elements and shrinking elements

gfredericks14:08:39

fmap shrinks by shrinking the underlying element and running it through the function again

gfredericks14:08:55

bind is a little weird though, and therefore so is gen/let

gfredericks15:08:11

@alexmiller: regarding the test.check issue described in CLJ-1997, I'm considering pasting all the relevant code from riddley into test.check as the most practical option. I'm wondering if you see any problems with doing something like that for a contrib library.

mattly17:08:18

I've started moving away from generators like gen/frequency that don't shrink in favor of those that provide predictable shrinking behavior such as gen/one-of

mattly17:08:10

the idea being that when there are multiple possible values that require different generators, it will shrink along an axis of my choosing

mattly17:08:32

for example, I have a node on the tree structure I generate where most of the time the value is an element from a collection, but sometimes it's not – however for most of the problems I encounter, that doesn't matter

mattly17:08:38

so I setup the generator like so:

mattly17:08:37

instead of using gen/frequency to mimic the structure of my inputs

mattly17:08:09

it will definitely test the case where there's a thing, but will shrink towards there not being a thing

mattly17:08:20

which reduces the noise I have to contend with when troubleshooting a failure

mattly18:08:56

in some cases where things is a fixed size, I actually use gen/one-of to force a shrinking preference, since gen/elements is always random; you'd have to shrink the size of the input if you wanted predictability

mattly18:08:43

(and feel free to correct me if I'm wrong – I'm going off of observation from usage and cursory readings of the code)

gfredericks18:08:08

@mattly: what makes you say gen/frequency doesn't shrink?

gfredericks18:08:01

oh I see; I'm inclined to say that's a bug

mattly18:08:14

@gfredericks: my experience of using it, that example before, the frequencies didn't seem to matter when shrinking

gfredericks18:08:44

I would think it should shrink toward earlier generators like one-of does

mattly18:08:57

so f.e. [[3 (gen/return nil)] [1 (gen/elements things)]] it'll still pick gen/elements

mattly18:08:00

that'd be even better

gfredericks18:08:38

would you like to file the bug or shall I?

mattly18:08:09

go for it, i don't think I have an account on the jira

mattly18:08:14

I probably should

mattly18:08:23

and you'd know more about what to say about it 😄

mattly18:08:42

I'm planning to do a writeup on my usage of test.check and property-based testing on the system I'm working on

mattly18:08:19

there's a lot of material out there on what it is, but very little on how to use it effectively

mattly18:08:00

this has been such a success on my system, I'm teaching our QA guy how to write generators