what’s the recommended way of actually running stest/check tests?

i’m in a cljs project, using lein-doo and cljs.test

should i be using (stest/summarize-results (stest/check `foo/bar)), and verifying that there’s no :check-failed key in the returned map? or is there a better way?

eg https://github.com/pieterbreed/ansible-inventory-clj/blob/9857f22ccd92e21b629d5d79907e4c266bdcbc52/test/ansible_inventory_clj/core_test.clj#L79

How can I duplicate a spec?

(s/def ::my-spec number?)
(s/def ::a ::my-spec)
(s/def ::b ::my-spec)

=> #error {
 :cause "Assert failed: k must be namespaced keyword or resolvable symbol\n(c/and (ident? k) (namespace k))"
 :via
 [{:type clojure.lang.Compiler$CompilerException
   :message "java.lang.AssertionError: Assert failed: k must be namespaced keyword or resolvable symbol\n(c/and (ident? k) (namespace k)), compiling:(ann.clj:63:1)"
   :at [clojure.lang.Compiler$InvokeExpr eval "Compiler.java" 3657]}
  {:type java.lang.AssertionError
   :message "Assert failed: k must be namespaced keyword or resolvable symbol\n(c/and (ident? k) (namespace k))"
   :at [clojure.spec$def_impl invokeStatic "spec.clj" 315]}]
 :trace
 [[clojure.spec$def_impl invokeStatic "spec.clj" 315]
  [clojure.spec$def_impl invoke "spec.clj" 312]
  [clojure.lang.AFn applyToHelper "AFn.java" 160]
  [clojure.lang.AFn applyTo "AFn.java" 144]
...

I can’t reproduce that - that should certainly work

what version are you using?

unform is not properly reversing vectors that have been conformed, e.g.

(spec/unform :clojure.core.specs/defn-args (spec/conform :clojure.core.specs/defn-args '(f [x] x)))
=> (f (x) x)

I checked that the spec "knows" that the args must be a vector.

I'm trying to make a defn-like macro by using spec to conform with the defn spec, then manipulate, then unform.

yeah, this is a known issue with the spec as it’s not enforcing the vector part

we are talking about creating an s/vcat as doing regex + vector is tedious right now

and it’s relatively common in macro syntax

this is a good example of it

you can make it work using something like (s/and (s/cat …) (s/conformer vec)) but like I said, it’s tedious

So you mean I should write my own more verbose version of the defn-args spec rather than use the one built into core?

well you could wrap defn-args too

part of being in alpha means things aren’t done

The problematic piece is :arg-list, which is a component of the :defn-args spec:

(s/def ::arg-list
  (s/and
    vector?
    (s/cat :args (s/* ::binding-form)
:varargs (s/? (s/cat :amp #{'&} :form ::binding-form))))

So could I just "patch" arg-list?

I guess if I s/def :clojure.core.specs/arg-list in my library, that will overwrite the spec for other consumers of my library as well, so maybe I shouldn't do that.

maybe you could make the patching a separate (temporary) step

@alexmiller Thanks for having a look. It's working now. I'm not sure, if it comes up again, I'll let you know

i’m having a lot of trouble getting spec’s generative testing to work

when i call (stest/check `foo/bar), and the output is [], how should i interpret that empty list?

i get [] from calling (stest/check) on each fdef’d function in this file, eg https://github.com/jrheard/voke/blob/spec/src/voke/input.cljs#L85 , and don’t really know where to go from here. has anyone seen this behavior? is there something obvious that i’m doing wrong?

example code:

(deftest generative
  (let [output (stest/check `input/intended-directions->angle)]
    (print "yo")
    (print output)
    (print (stest/summarize-results output))))

;; lein doo phantom test output:

Testing voke.input-test
yo
[]
{:total 0}

relevant specs are in https://github.com/jrheard/voke/blob/spec/src/voke/specs.cljs

Hm, going from -alpha11 to -alpha12/-alpha13 I get a pretty unhelpful NPE when calling explain on some data. clojure.spec portion of the stacktrace:

spec.clj:  864  clojure.spec/spec-impl/reify
                  spec.clj:  150  clojure.spec/conform
                  spec.clj:  782  clojure.spec/map-spec-impl/reify
                  spec.clj:  150  clojure.spec/conform
                  spec.clj:  731  clojure.spec/dt
                  spec.clj:  727  clojure.spec/dt
                  spec.clj: 1081  clojure.spec/explain-pred-list
                  spec.clj: 1118  clojure.spec/and-spec-impl/reify
                  spec.clj:  198  clojure.spec/explain-data*
                  spec.clj:  209  clojure.spec/explain-data
                  spec.clj:  202  clojure.spec/explain-data

I dunno if anyone’s seen anything like that or if it’s worth producing a minimal sample

which suggests that pred in spec-impl is nil

Did anything change about how registry-ref works?

(The weirdest thing that code path does is (with-redefs [s/registry-ref (atom @@#'s/registry-ref)] …), which it needs to because it’s generating specs so I don’t want to have tests fail because of stale state)

That does not look like it’s part of the problem.

I am dealing with strangest bug I cannot reproduce anywhere else: it works everywhere except Circle CI. Basically this spec:

(require '[com.gfredericks.test.chuck.generators :as gen']
         '[clojure.spec :as s])

(def iso-date-regex #"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}-\d{2}:\d{2}$")

(s/def ::date-iso (s/with-gen (s/and #(re-find iso-date-regex %) string?)
                   #(gen'/for [d (gen'/datetime)]
                      (.format (js/moment (js/Date. d))))))

ERROR in (test-datetime-string) (:)
Uncaught exception, not in assertion.
expected: nil
  actual: #error {:message "Couldn't satisfy such-that predicate after 100 tries.", :data {}}

first I thought it’s due to incompatible version of phantomjs. and then version of Leineingen, and then I disabled any parallelism on CI. and still can’t figure it out. I thought maybe I could replicate in a Docker container - still nothing

Anyone, any ideas?

hm, that’s weird: it would only make sense if the generator portion is being ignored

but why would that happen only in Circle CI and nowhere else?

maybe you should create a test PR that s/gen + gen/samples that spec

that could be a few reasons, including profile.clj files or whatever

s/gen + gen/samples will tell you what it thinks the generator is for that spec

how did you determine it wasn’t an incompatible version of phantomjs?

I am replacing phantom with the right version

LEIN_NO_USER_PROFILES=1 can be helpful for debugging things that only happen in ci

I’ve seen a lot of problems with Phantom where phantom versions insisting they were the same, although that regex does look innocuous

1.9.8 is like seventeen different webkits

Also have you rerun the ci? That "such-that" error could also just be highly improbable

Wouldn’t the such-that error never happen with with-gen? I thought that with-gen did not check that generated values match the spec?

(that may have changed in recent alphas, or not be true in JS, or I might be misremembering entirely of course)

I don't know where the such-that came from actually

well, I’m thinking of the spec by itself

(s/and #(re-find iso-date-regex %) string?)

IIRC s/and uses the first arg to determine type

so it won’t even be able to guess “string” let alone “string that incidentally matches this regex"

(Sorry, to be less obtuse: I think the such-that is coming from the s/and)

Oh I didn't see an and

It shouldn't matter though since it's wrapped in with-gen

exactly

@ag: Can you please dump the full test ns somewhere

ideally a reproducible sample