So I am pretty sure this is absolutely not how spec was intended to be used, but has anyone gotten mileage from leaving stest/instrument turned on in production to use spec for runtime contract assertion? Even more so with jeaye/orchestra

One reason I can think of why this may not be such a great idea is that stest/instrument validates fspec using generated inputs

are there any other reasons not to leave stest/instrument turned on?

@djtango The generative validation of fspec is one reason. Performance is another. You'll also get bare AssertionError exceptions instead of anything meaningful you might normally catch and handle (assuming you try to (catch Exception e ...))

yeah, my understanding is that AssertionError is for things you expect to throw during development - the vm even has a flag to ignore them

I suppose, to give more context on my usecase, it is useful (for me) to declare some invariants about a function (e.g. a relationship between it's args and return value) but in a sufficiently large project, it's possible for inputs in runtime to fail that invariant and it's nicer when that failure comes at the point of the invariant failing rather than some arbitrary of steps later like a null pointer error

spec happens to be a great way for declaring properties about lots of things, and while generative testing is meant to give better guarantees about over your testing space, the associated complexity with them is a harder sell for the team

If that makes sense?

Coming from Racket, I'm kind of used to contracts always being on

the pragmatic thing might just be overriding fspecs for the instrumenting

it’d be great if there were a spec-impl that “merged” two s/ors or s/alts without having to reify the protocols (which is a dangerous game as i understand it).

for example, if you want to express that clojure.core/unquote-splicing forms are not permitted at the top level you’d have a union for your legal top level forms and then another union for your subforms in, say, a list that includes unquote-splicing forms in addition to what is legal at the top level.

to fully leverage all of spec, including explanations, it seems like i have to spell this out entirely in both cases if i want to avoid an additional layer of rettags.

@noprompt Or wrap s/or in s/nonconforming I guess...?

@seancorfield that’s not quite the ticket. lemme demonstrate.

(s/def :clj.form/top-level
  (s/or :list :clj.form/list
        :number number?))

(s/def :clj.form/list
  (s/coll-of
   (s/or :a :clj.form/top-level
         :b (s/or :unquote-splicing unquote-splicing-form?))
   :kind list?))

(s/conform :clj.form/top-level '(1 ~@[1]))
;; =>
[:list
 ([:a [:number 1]]
  [:a [:number 2]]
  [:b [:unquote-splicing (clojure.core/unquote-splicing [1])]])]

what i do not want is the :a and :b tags. i still want to retain the :number and :unquote-splicing tags.

by using s/nonconforming i lose the tags. 😕

essentially, i want to extend the :clj.form/top-level union without adding an additional layer of rettags.

of course, i can do this by using s/with-gen, s/conformer, etc. but i lose out on s/explain-data.

if there was an s/explainer this’d be a done deal. 🙂

in short, i think what i want is an or-spec-impl or an alt-spec-impl that does not require tagging the options.

when it conforms returns whatever the conformers it’s compose of returns.

i think there was previous discussion along same lines but for s/cat with an s/concat conformer..

Alex has talked several times about the possibility of adding a specific non-conforming or variant so you'd have that sort of level of control...

I have a big complicated s/keys, and I’m getting "Couldn't satisfy such-that predicate after 100 tries.". How do I discover which predicate is failing?

if you're using a new enough test.check, the ex-data should have the generator and predicate attached, at least