Fork me on GitHub

I don't understand why this causes "Couldn't generate enough distinct elements!"

(defn foo [gen]
  (gen/vector-distinct gen {:num-elements 10}))

(defspec foo-test
  (prop/for-all [x (gen/let [n (gen/resize 10 gen/nat)]
                     (foo (gen/vector gen/any n)))]

Works fine if I replace the gen/let wrapper with an inline constant.
(defspec foo-test
  (prop/for-all (fgen/foo (gen/vector gen/any 5))
Is there a better way to do this? or some way to work around the problem?


Does it happen when n is zero?


This was stripped way down to highlight the issue. Here is real code. unary, shown for easier explanation of the concept, takes gens for x and y, and generates a function f mapping x -> y, (returning all 3 in a map). n-aries is the same basic idea. But the number of xs is variable (i.e. generating n-arity of function instead of unary) and the number of xs -> y mappings is also variable, so the generated function can return different ys depending on specific inputs. The arity (or count of xs) should be determined by xs-gen (which is the problematic n generated in the spec). The count of xs-> y mappings is determined by count-gen. An empty xs should be a valid result from xs-gen. But xss needs to be a list of unique xs.

;; fgen.cljc

; unary

(defn- x-y->map [X Y]
  {:x X
   :y Y
   :f (fn [x]
        (if (= X x)
          (throw (ex-info "generated function called with incorrect arguments"
                          {:expected X
                           :actual x}))))})

(defn unary [x-gen y-gen]
  (gen/let [x x-gen
            y y-gen]
    (x-y->map x y))

; n-aries

(defn- xss-ys->map [xss ys] 
  (let [xss-ys (zipmap xss ys)]
    {:xss-ys xss-ys
     :f (fn [& xs]
          (if (some #(= % xs) xss)
            (xss-ys xs)
            (throw (ex-info "generated function called with incorrect arguments"
                            {:expected {:one-seq-of xs}
                             :actual xs}))))}))

(defn n-aries [count-gen xs-gen y-gen]
  (gen/let [n count-gen
            xss (gen/vector-distinct xs-gen {:num-elements n})
            ys (gen/vector y-gen n)]
    (xss-ys->map xss ys)))
; fgen_test.cljc

(defspec n-aries
  (prop/for-all [{f :f {xs :xs ys :ys} :xs-ys}
                 (gen/let [n (gen/resize 10 gen/nat)]
                    (gen/resize 10 gen/nat)
                    (gen/vector gen/any n)
    (or (util/empties? xs ys)
        (= ys (map f xs)))))


Why is uniqueness important?


Because the goal is to generate a function f, which has deterministic behavior y based on its input value(s) xs.


Then why is generating a minimum number of things important?


Additionally, I am representing that determinism internally as a map where xs is a key on a map of xs -> ys.


Who said anything about a minimum number of things?


Your gen-vector-distinct call did


The gen-distinct call asks for a specific number of things, where that number is generated by a user-provided generator. The user could choose to provide (gen/return 0) or gen/nat. That is not for me to decide.


What does n represent there, in the context of generating functions?


n in n-aries (not causing problems and replaced by 10 in the foo example) is the number of mappings from xs-> ys. n in the spec (which is causing problems) represents the arity of the target generated function. i.e (defn foo [x y z]) is (= n 3), and (defn foo []) is (= n 0). Again this will ultimately be provided by the user, when they define an xs-gen to pass into n-aries.


Do you at least agree that the error in your simplified example makes sense?


Yes. I see it now. In the case my test gens (= n 0) , foo cannot generate distinct data.


But that still leaves me at a loss how to put n-aries through its paces, when (= n 0) for should be a valid possibility. 😞


Hmm. I think with that in mind, the issue is not the test. Somehow n-aries needs to be smarter about comparing the result of count-gen with the results of xs-gen to avoid nonsensical attempts at uniqueness.


A 0-ary pure function can only have one input/output pair


It's effectively a constant


Yes. I don't know what that though. The fact a 0-arity pure function is a constant does not obviate the need for someone testing a higher-order function check how it handles being given a 0-arity function, among others.


I haven't internalized the details of the situation, but if you're getting a "can't generate enough distinct elements" error because of a trivial generator, then you should probably either question the need for distinctness or minimum elements, or else consider that your api allows users to ask for impossible things


The problem was just the location of let. It should be wrapping the generator passed to foo to keep generating new nvalues, rather than binding a single n ahead of calling foo.

(defspec foo-test
  (prop/for-all [x (foo (gen/let [n (gen/resize 10 gen/nat)]
                          (gen/vector gen/any n)))]
... instead of ...
(defspec foo-test
  (prop/for-all [x (gen/let [n (gen/resize 10 gen/nat)]
                     (foo (gen/vector gen/any n)))]


If I'm reading correctly, in that scenario you're asking it to generate ten distinct empty vectors


I can't tell what you're actually trying to do, so it's not clear what would count as a fix