Fork me on GitHub
#test-check
<
2022-11-03
>
Jared Langson22:11:12

Is there a way do this, but in a way that has shrinkage? I want to pull a random character from a random string and then do stuff. The random string is made from a generator. Is there a way to move the random character into the prop/for-all?

(defspec split-functionality-test
  (prop/for-all [s (gen/not-empty gen/string-alphanumeric)]
    (let [split-on (rand-nth s)                          ;can this be done in the for-all binding?
          pattern (re-pattern (str split-on))]
      (= (cp-str/split s pattern) (str/split s pattern)))))

hiredman22:11:09

it is complicated

hiredman22:11:55

you can, but the way to do it may make shrinking worse

hiredman22:11:56

I would try something like

(defspec split-functionality-test
  (prop/for-all [i gen/nat
                 s (gen/not-empty gen/string-alphanumeric)]
                (let [split-on (nth s (mod i (count s)))                         
                      pattern (re-pattern (str split-on))]
                  (= (cp-str/split s pattern) (str/split s pattern)))))

👍 1
Alex Miller (Clojure team)22:11:10

sometimes it's helpful to build up the things you need rather than draw from one random thing

Alex Miller (Clojure team)22:11:24

yeah, that's a decent way

hiredman22:11:46

potentially moving the order of i and s if it isn't shrinking well (I forget which way test.check throws away shrinking when composing with bind)

Jared Langson23:11:30

The other problem with my way is that because split-on is inside let, I don't know the value of split-on when the test fails. So even if the shrinkage is worse, moving split-on to the prop/for-all could be a good tradeoff

hiredman23:11:20

in theory the ideal would be something like

(defspec split-functionality-test
  (prop/for-all [s (gen/not-empty gen/string-alphanumeric)
                 i (gen/choose 0 (count s))]
                (let [split-on (nth s (mod i (count s)))
                      pattern (re-pattern (str split-on))]
                  (= (cp-str/split s pattern) (str/split s pattern)))))

hiredman23:11:06

so you only generate numbers from 0 to size of string, and those generated numbers also predictably shrink (throwing the mod in there can make things weird when shrinking)

hiredman23:11:06

but this is sort of like (gen/bind ..gen for s... (fn [s] .. gen for i ...))

hiredman23:11:15

and when you compose things with bind like that in test.check(and anything modeled after the original quickcheck) you lose shrinking information for one or the other, and I forget which choice test.check makes

Jared Langson23:11:59

When I ran your second version it complained about i (gen/choose 0 (count s))]. Said "Unable to resolve symbol: s in this context"

hiredman23:11:32

oh, good, I guess test.check's for-all is optimized to get around the issue I mentioned, but as a result the generators cannot refer to each other

hiredman23:11:05

not optimized so the issue isn't there, but optimized to try and avoid it

hiredman23:11:04

for-all* is the one that would let you I think

hiredman23:11:59

huh, for some reason I thought for-all turned into a bind

hiredman23:11:30

I guess the second isn't directly expressible using for-all

hiredman23:11:48

I was looking at an old version of the code from before > When there are multiple binding pairs, the earlier pairs are not > visible to the later pairs. was added to the docstring for for-all

hiredman23:11:49

http://clojure.test.check.ge/let is what I am thinking of that allows for the dependencies between generators

Jared Langson23:11:03

Are there pitfalls/gotchas with using gen/let?

hiredman23:11:20

the above mentioned lose of shrinkage, which is https://clojure.atlassian.net/browse/TCHECK-112