I'm trying to use spec.test/check to check all my functions, but I'm getting a couldn't satisfy such-that predicate exception. Is there any way to figure out which spec wasn't able to be satisfied?

I don't know of any way to debug that -- other than to s/exercise your specs as you write them...

You could call s/registry and go through all the keys in that (which are specs) I guess...

...I've just gotten kinda paranoid about exercising each spec in the REPL as I write it.

I think there's a request open to include the spec in that message. iterating the registry sounds like a good option in the meantime

@bfabry do you know which issue it is? I took a look in JIRA but couldn't see anything

How do I express, many strings and/or many vectors? And I don’t care which one it is, so I don’t need any branching

@danielcompton: I’ve had that same issue…. Supposedly the latest version of test.check exposes more information on that, infact I updated my test.check to the latest version which provides a bit more info, but still doesn’t tell you the spec that failed (as it knows nothing of spec).

😞

finding them is a bit of a PITA, lucky for you bisecting specs with exercise is logarithmic 🙂

but better to exercise them as you write them bottom up like seancorfield suggests

@danielcompton @rickmoynihan @bfabry what's needed is to find the spot[s] where clojure.spec calls gen/such-that and get it to pass that debugging info along; I have no idea how difficult that would be, might be super easy

the test.check change is that you can now pass an option (`:ex-fn`) that lets you customize the exception

Some of my spec generators seem to grow to pretty big values under the default 1000 generations used by clojure.spec.test.alpha/check. I don’t know how big they’re getting in numbers of items, but they’re OOMing. Is there an easy way to make specs flatten out at a certain size, but keep iterating over random generations?

obvs I could add an extra constraint e.g. #(> 1000 (count %)) to maps etc… but it feels like it’d be better done in the map/sequence generators

@rickmoynihan under default configurations your tests should max out their size at 200 trials and cycle back to small at that point; there should be a way to restrict the size further than that

See this option to quick-check that I expect spec provides a mechanism for passing through: https://github.com/clojure/test.check/blob/05a53600aab21576420984a54f3ff75714981bc3/src/main/clojure/clojure/test/check.cljc#L46

hmmm… not sure why I’m seeing so much heap then

the underlying problem here relates to collection sizing, which we have an open test.check issue for

but the workaround is to explicitly limit sizing via the option I just pointed to

yeah you can pass those through via (check ,,, {:clojure.spec.test.check {:max-size 10}})

All I was saying with 200 vs 1000 is that you should probably see the same issues with only 200 trials

I'd try 50 for starters

100 for num-tests seems to work but more than that GC seems to dominate the run time

will try reducing max-size

num-tests can be as high as you want as long as max-size is low enough

max-size defaults to 200, and the sizes used throughout the run are (cycle (range max-size))

cool

so num-tests can "fix" sizing issues only when num-tests < max-size

makes sense

cool. That max-size trick at 50 seems to work nicely.

@gfredericks: so does :max-size 50 mean maps have a maximum of 50 pairs, strings have a maximum of 50 chars, vectors have a maximum of 50 items etc?

or is it a weight/multiplier?

that tends to be true, but there's no true definition of size; it's an abstract thing that could theoretically be interpreted differently by any given generator

ok that’s what I thought

e.g., you can easily defy that by generating vectors with a specified length of 75

🙂

but is it true of the default generators?

yeah

presumably the fix for all this will be that lower-down nested structures will be substantially smaller

but that wouldn't contradict what you just said

lowering it to 50 seems to work nicely… it also means I can run more generations in the same time which seems like the right trade off for my data.

does anyone know a way to express “a map of keys of spec A or B, where A key points to spec A0 and B key points to spec B0”?

(s/def :foo (s/map-of #{:A :B} ...maybe (s/or :a :A0 :B0)

@lwhorton Sounds like a multi-spec to me.

heh, back to that ol’ zinger again

TBH, I'm not sure what you're actually trying to do, based on what you said. Can you give a concrete example?

@lwhorton You might also want to use tuples inside a coll-of. Something like (untested):

(s/coll-of (s/or :a (s/tuple ::A ::A0) :b (s/tuple ::B ::B0)) :into {} :kind map?)