Hi, I just realized that spec/instrument is incompatible with event handler-type systems like re-frame (systems where functions are stored in a data structure prior to instrument being called). Since instrument wraps the original var, the function calls that are made through lookup in a data structure (e.g a map of event->handler-fn) aren't spec checked. Since this is a clear majority of the calls being made in for example re-frame, the value of spec:ing (with fdef) the handler functions is decreased. Is there any way around this at all? I guess you could add the vars themselves in the map instead of the functions they point to, but that seems like a big change to make in an existing system, and feels pretty unidiomatic.

I guess you could potentially spec the data structure and use fspec at the leaves where the event-handler functions are.

Seems complicated though

Not possible to def the functions outside the datastructure?

(defn my-fn [x y z] ...) (def handler {:event1 my-fn})

@danielstockton That's how I do it. The problem is that the functions, not the vars, are what are added to the datastructure. If you try it out, you'll notice that the function calls made from lookups in the map aren't checked. In my particular case the functions are added to a data structure behind the scenes, by re-frames reg-event.

I’m trying to instrument an fdef’d function with a stub and a custom generator but get an exception ClassCastException clojure.test.check.generators.Generator cannot be cast to clojure.lang.IFn clojure.spec.alpha/gensub (alpha.clj:268)

(defn foo [a]
    (+ a a))

  (s/def ::num number?)

  (s/fdef foo
          :args (s/cat :a ::num)
          :ret number?)

  (st/instrument `foo {:stub #{`foo} :gen {::num (g/elements [1])}})

the instrument call itself raises the exception

ahh I see the problem - I need to obviously gen the return value not the args… doh

changing it to :ret ::num works

Is it that (g/elements [1]) should be #(g/elements [1])?

ahh yeah sorry I’d fixed that locally too

np

actually that’s one thing I’m never really clear on… when to pass a generator and when to pass a 0-arg function returning a generator…

is there a resource somewhere explaining that?

I think you always need to give a function to something that comes from spec (and is deffed). As far as I worked out, it's to do with the way spec lazy-loads test.check. Not seen it written down anyway though.

I can't think of a place where spec requires a plain generator.

Only if you're calling test.check stuff directly

ok

at least I know the heuristic now 🙂

Is it just me or does instrument with a stub also try and generate conforming values for the :args spec; even though it’ll never need a generator for them - as it only needs to generate return values?

basically I’m getting a such-that error on instrument but if I replace the :args spec with something like (s/* any?) it works.

Hi folks! Is there a good approach to writing function specs and running clojure.spec.test.alpha/check in a way that allows functions to either return a value (checked with :ret) or throw an exception?

Or, simplified, is there a way to spec a function so that the fact that it throws under certain conditions is part of the spec?

@jannis I remember reading something here about spec not contemplating exceptional cases, I believe the idea is that specs are about data only

Yes, that’s what I was thinking.

I think I’ve found a solution for myself: have the spec verify the data is ok, then have the function only operate on valid data. That way generative tests only generate valid data but invalid data would still be caught at runtime if I use instrumentation.

That completely eliminates the need for exceptions. 🙂

You don’t want to use instrument at runtime, generally speaking @jannis

I think I do—but only while developing.

And while not testing performance perhaps 😉

yes, that’s what i meant and should have said “production” 😉

@joshjones No worries, I assumed that’s what you meant already.

Is there a clever way to get the merged form of a spec that uses spec/merge?

Just ran into this issue ( https://dev.clojure.org/jira/browse/CLJ-1989 ) and surprised that the lazy load coverage doesn’t extend to all of test.check.generators