Fork me on GitHub
#test-check
<
2022-12-13
>
skylize14:12:40

I don't understand why this causes "Couldn't generate enough distinct elements!"

``````(defn foo [gen]
(gen/vector-distinct gen {:num-elements 10}))

(defspec foo-test
(prop/for-all [x (gen/let [n (gen/resize 10 gen/nat)]
(foo (gen/vector gen/any n)))]
true))

(foo-test)``````
Works fine if I replace the `gen/let` wrapper with an inline constant.
``````(defspec foo-test
(prop/for-all (fgen/foo (gen/vector gen/any 5))
true))``````
Is there a better way to do this? or some way to work around the problem?

gfredericks14:12:58

Does it happen when n is zero?

skylize14:12:16

This was stripped way down to highlight the issue. Here is real code. `unary`, shown for easier explanation of the concept, takes gens for `x` and `y`, and generates a function `f` mapping `x` -> `y`, (returning all 3 in a map). `n-aries` is the same basic idea. But the number of `xs` is variable (i.e. generating n-arity of function instead of unary) and the number of `xs` -> `y` mappings is also variable, so the generated function can return different `y`s depending on specific inputs. The arity (or count of `xs`) should be determined by `xs-gen` (which is the problematic `n` generated in the spec). The count of `xs`-> `y` mappings is determined by `count-gen`. An empty `xs` should be a valid result from `xs-gen`. But `xss` needs to be a list of unique `xs`.

``````;; fgen.cljc

; unary

(defn- x-y->map [X Y]
{:x X
:y Y
:f (fn [x]
(if (= X x)
Y
(throw (ex-info "generated function called with incorrect arguments"
{:expected X
:actual x}))))})

(defn unary [x-gen y-gen]
(gen/let [x x-gen
y y-gen]
(x-y->map x y))

; n-aries

(defn- xss-ys->map [xss ys]
(let [xss-ys (zipmap xss ys)]
{:xss-ys xss-ys
:f (fn [& xs]
(if (some #(= % xs) xss)
(xss-ys xs)
(throw (ex-info "generated function called with incorrect arguments"
{:expected {:one-seq-of xs}
:actual xs}))))}))

(defn n-aries [count-gen xs-gen y-gen]
(gen/let [n count-gen
xss (gen/vector-distinct xs-gen {:num-elements n})
ys (gen/vector y-gen n)]
(xss-ys->map xss ys)))``````
``````; fgen_test.cljc

(defspec n-aries
200
(prop/for-all [{f :f {xs :xs ys :ys} :xs-ys}
(gen/let [n (gen/resize 10 gen/nat)]
(fgen/n-aries
(gen/resize 10 gen/nat)
(gen/vector gen/any n)
gen/any))]
(or (util/empties? xs ys)
(= ys (map f xs)))))``````

gfredericks14:12:02

Why is uniqueness important?

skylize14:12:29

Because the goal is to generate a function `f`, which has deterministic behavior `y` based on its input value(s) `xs`.

gfredericks14:12:11

Then why is generating a minimum number of things important?

skylize14:12:11

Additionally, I am representing that determinism internally as a map where `xs` is a key on a map of `xs` -> `ys`.

skylize14:12:48

Who said anything about a minimum number of things?

gfredericks14:12:11

skylize14:12:51

The `gen-distinct` call asks for a specific number of things, where that number is generated by a user-provided generator. The user could choose to provide `(gen/return 0)` or `gen/nat`. That is not for me to decide.

gfredericks14:12:21

What does n represent there, in the context of generating functions?

skylize14:12:35

`n` in `n-aries` (not causing problems and replaced by `10` in the `foo` example) is the number of mappings from `xs`-> `ys`. `n` in the spec (which is causing problems) represents the arity of the target generated function. i.e `(defn foo [x y z])` is `(= n 3)`, and `(defn foo [])` is `(= n 0)`. Again this will ultimately be provided by the user, when they define an `xs-gen` to pass into `n-aries`.

gfredericks14:12:27

Do you at least agree that the error in your simplified example makes sense?

skylize14:12:47

Yes. I see it now. In the case my test gens `(= n 0)` , `foo` cannot generate distinct data.

skylize14:12:39

But that still leaves me at a loss how to put `n-aries` through its paces, when `(= n 0)` for should be a valid possibility. ðŸ˜ž

skylize15:12:21

Hmm. I think with that in mind, the issue is not the test. Somehow `n-aries` needs to be smarter about comparing the result of `count-gen` with the results of `xs-gen` to avoid nonsensical attempts at uniqueness.

gfredericks15:12:28

A 0-ary pure function can only have one input/output pair

gfredericks15:12:44

It's effectively a constant

skylize15:12:34

Yes. I don't know what that though. The fact a 0-arity pure function is a constant does not obviate the need for someone testing a higher-order function check how it handles being given a 0-arity function, among others.

gfredericks15:12:39

I haven't internalized the details of the situation, but if you're getting a "can't generate enough distinct elements" error because of a trivial generator, then you should probably either question the need for distinctness or minimum elements, or else consider that your api allows users to ask for impossible things

skylize04:12:40

The problem was just the location of `let`. It should be wrapping the generator passed to `foo` to keep generating new `n`values, rather than binding a single `n` ahead of calling `foo`.

``````(defspec foo-test
(prop/for-all [x (foo (gen/let [n (gen/resize 10 gen/nat)]
(gen/vector gen/any n)))]
true))``````
``````(defspec foo-test