This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

## 2017-02-12

## Channels

- # bangalore-clj (1)
- # beginners (27)
- # boot (29)
- # clara (4)
- # cljs-dev (10)
- # cljsrn (2)
- # clojure (36)
- # clojure-austin (9)
- # clojure-brasil (4)
- # clojure-france (10)
- # clojure-nl (2)
- # clojure-quebec (1)
- # clojure-russia (10)
- # clojure-spec (48)
- # clojure-uk (6)
- # clojurescript (82)
- # clr (4)
- # core-async (5)
- # core-logic (3)
- # cursive (4)
- # datomic (1)
- # devcards (1)
- # figwheel (1)
- # leiningen (2)
- # lumo (6)
- # off-topic (23)
- # om (39)
- # onyx (3)
- # planck (2)
- # re-frame (5)
- # reagent (24)
- # specter (1)
- # test-check (3)

Does anyone have experience with using clojure.spec in a scenario where numerical precision becomes an issue? I would like to have generative testings for a function whose output would match the input (except for some numerical imprecision due to matrix calculations happening). I struggle to come up with a good `:fn`

spec which does not require crippling the generator. A simple `:fn #(m/equals (:ret %) (-> % :args :matrix) 1e-6)`

does not work, because the generator ends up using “large” doubles which break the max. diff of `1e-6`

. I tried a spec using ratios but this ends up with “Couldn’t satify such-that” issues:

```
#(-> (m/div (:ret %) (-> % :args :matrix))
(m/sub 1.0)
m/abs
m/maximum
(< 1e-8))
```

I am pretty sure I could constrain the generator in a way to make the simple check work but of course I would like to avoid doing that.

@nblumoe what does the function you're trying to spec do?

@gfredericks a simplified answer would be: Principal component analysis and then reconstruction of the original data from scores and loadings. Some special twists due to the specific domain of the problem though (chemistry).

@nblumoe so your problem is that the algorithm breaks down at extreme values, but it would be arbitrary and artificial to impose constraints on the values in the spec?

no, the algorithm works fine. the only issue is specing it reliably without making the generator too specific (with “specing” I mean including generative testing)

okay, so it's not that the algorithm doesn't work at extreme values, just that the arbitrary equality threshold of `1e-6`

breaks?

yes exactly. because this criterion is easy to break for example when using doubles in the range of `10e100`

or whatever large enough

can you make the equality threshold a function of the input data?

that might have the advantage of giving you a tighter threshold at the other end as well

I was trying that too but ended up with unsatisfied “such-that” issues. would also need to set the threshold element wise (e.g. if a matrix contains 1e100 as well as 1e1 those should have different thresholds then)

what does `such-that`

have to do with it?

quote from above: “I tried a spec using ratios but this ends up with “Couldn’t satify such-that” issues"

asserting the ratio should be close to 1?

are you using such-that directly, or is it generated by spec because of a `spec/and`

?

the `such-that`

issue basically means that the generator was not able to find valid data within 100 tries, which can happen when the spec is quite specific and the search space for data is rather large

I'm just not seeing why the "using ratios" idea entails modifying the input spec; I'd imagine it just means modifying the `:fn`

on the spec

⇑ edited

I'm 80% sure changes to `:fn`

should be independent of the probability of such-that errors; what does your `s/and`

look like?

I would have that that only `:args`

could have such effect. is anything generated for the `:fn`

itself maybe?!?

`:args`

is a bit complicated tbh:

```
:args (s/and (s/cat :matrix (s/and ::matrix
::range-transformable
::non-zero-row-sums
::non-zero-length-rows
#(s/valid? ::non-zero-length-rows (range-transformation %)))
:const-row-sum (s/int-in 1 100)
:num-eigenvectors (s/with-gen pos-int?
#(s/gen (set (range 0 20)))))
; upper limit of eigenvectors
; should not be hardcoded
#(= (:num-eigenvectors %)
(min (count (first (:matrix %)))
(count (:matrix %))))
#(s/valid? ::non-zero-row-sums (-> (:matrix %)
range-transformation
(initial-loadings (:num-eigenvectors %)))))
```

the such-that errors might be nondeterministic -- are you *sure* you don't get them with the non-ratio code?

usually that does not cause any `such-that`

issues on generation, works pretty flawlessly accept for the aforementioned changes to `:fn`

maybe your test fails before it's likely to encounter them?

I'm not sure what you're asking

I think just try `s/exercise`

on the args spec

`(s/exercise (s/and (s/cat ...) ...) 10000)`

probably wrap that in `(count ...)`

🙂

to avoid printing big things

indeed! it occasionally fails, so the issue is rather the `:args`

spec and the generator....

it could be either of the `s/and`

s, or both

the way `s/and`

works as a generator is to generate something from the first spec and then filter on the rest of them; so the generator of the first spec needs to be highly likely to generate something that passes all of the predicates, or you get the such-that error

sometimes you can simply restructure the `s/and`

to tailor to that, and it works out; other times the only way to get a good generator is to write a custom one for the `s/and`

since you have nested `s/and`

s it's not obvious which one is the problem

I'd try exercising the inner one and see what you get

ok thanks, that is valuable information! I already have a custom generator for `::matrix`

but that is not really tailored in any way to satisfy the following predicates. will do some `exercise`

s 🙂

great! the outer `s/and`

was the issue: the second predicate for `:num-eigenvectors`

was apparently failing too often

thanks a lot @gfredericks !

that specific predicate also does only cover one case where the number of eigenvector matches either the number of columns or number of rows. that is only one specific case though (in which the input data will be reproduced). I need to think about how to cover the usage with less eigenvectors than that… the invariant for the generative testing seems to be quite tricky though then