I’m having trouble figuring out which generator is causing the error

@nwjsmith one option would be to walk the registry using s/registry and try and generate an example for each one until you fail

I believe there's a jira open to figure out a way to add the offending spec to the error message you're seeing

s/and is a common culprite

the way to add the offending spec to the error message is implemented in the latest alpha of test.check, but I doubt has been incorporated into spec. I assume there's an open ticket for that.

@bfabry @gfredericks thanks!

I’ve narrowed my current run-in with that error down to this:

(s/def ::header-name
  (s/with-gen
   (s/and string?
          (complement string/blank?)
          utilities.string/lower-case?
          utilities.string/ascii?)
   #(gen/fmap (comp string/lower-case (partial string/join "-"))
              (gen/vector (gen/not-empty (gen/string-ascii))
                          1
                          100))))

Ah, figured it out

my utilities.string/ascii? function is broken

phew

I’m specing a function that calls another function, stubbing out that second function with instrument. When I’m doing stest/check on the function, it’s really slow. Takes ~30 s with :num-tests 1000. With num-tests 100, it’s sub-second. What gives?

(require '[clojure.spec.alpha :as s])
(require '[clojure.spec.test.alpha :as stest])

(defn fetch-foo-from-somewhere
  [a]
  [,,,])

(defn get-foo
  [b]
  (fetch-foo-from-somewhere b))

(s/def ::foo string?)
(s/fdef fetch-foo-from-somewhere
        :args (s/cat :a any?)
        :ret ::foo)
(s/fdef get-foo
        :args (s/cat :b any?)
        :ret ::foo)

(stest/instrument `fetch-foo-from-somewhere {:stub #{`fetch-foo-from-somewhere}})

(stest/summarize-results (stest/check `get-foo {:clojure.spec.test.check/opts {:num-tests 1000}}))

@msolli how about {:num-tests 1000 :max-size 100}?

Yeah, that took the time down to ~4 s.

So I understand :max-size somehow controls the way generated values “grow” as the number of tests increase. How does this explain why larger sizes takes so much more time?

it's probably this: https://dev.clojure.org/jira/browse/TCHECK-106

probably stemming from the use of any? in this case

I suppose you could argue that the generator underlying any? should be designed to never get very big

Ah, I see. I’m just getting into Spec, and I’m following along to https://clojure.org/guides/spec#_combining_code_check_code_and_code_instrument_code. It seemed strange that the official docs endorse something this slow.

I can confirm that the values generated for any? really do become huge.

So I guess it’s the GC-ing of those large objects that is the bottleneck.

There are some tickets and work to do in this area still

@gfredericks the most critical fix was the bug in s/coll-of that went into the spec release last week actually. That was the cause of one set of problems (not the one here though).

@alexmiller this thing? https://github.com/clojure/spec.alpha/commit/739c1af56dae621aedf1bb282025a0d676eff713

I can't see anything that looks particularly relevant in the last 6 months of commits

oh was it generating a bunch of stuff and throwing it away?

The conj, well, enjoy yourselves! And thanks for the explanation and all the great work you’re doing.

@alexmiller after reading the ticket, I assume CLJ-2103 is what you were referring to Any thoughts (ha!) about any?? The underlying gen/any-printable in test.check could have its size scaled down. I feel like that would make it more useful for almost any conceivable use case.

Yeah that would be good

And yes 2103

although now that I think about it, making sizing better in collections and gen/recursive-gen might take care of the gen/any-printable problem

Is there anyway to get the context of a element being validated by a spec. I need to get hold of the overall datastructure that is being validate by a set of specs so that I can do some cross validation in my own predicate?

Hi. Wrote an suggestion of the spec coercion: https://dev.clojure.org/jira/browse/CLJ-2251

trying to spec a function that takes 2 params, a keyword and a set of string (which can be nil), but my spec always fails when a set is passed to the function, how should it be spec'd

(s/fdef cs! :args (s/cat :mis keyword? :table-names (or nil? (s/coll-of string?)))) is the spec I was using

you want (s/or (s/coll-of string? :kind set?) nil?) but really you want (s/nilable (s/coll-of string? :kind set?)