If I write a spec like:

(s/def :in/data (s/and
                  (s/keys :req-un [:in/id] :opt-un [:in/more])
                  (s/map-of #{:id :more} nil)))

(def req-keys [:in/id])
(def opt-keys [:in/more])

(defn unk
  "Returns ns unqualified keys for (possibly) qualified ones."
  [& ks]
  (map #(-> % name keyword) ks))

(s/def :in2/data (s/and
                   (s/keys :req-un req-keys :opt-un opt-keys)
                   (s/map-of (set (apply unk (concat req-keys opt-keys))) nil)))

I’ve found a case where conform -> unform -> conform leads to an invalid result. This is the case with the clojure.core.specs/defn-args spec. See https://gist.github.com/jeroenvandijk/28c6cdd867dbc9889565dca92673a531 Should I file a JIRA issue?

Sorry for asking such a basic question, but what is the recommended way to test spec'd functions in unit tests (i.e. by running lein test)?

I like the idea of combining unit and generative tests as per http://dpassen1.github.io/software/2016/09/07/from-repl-to-tests#a-better-way

But I'm not really sure how to get c.s.test/check to hook into the (deftest ... (checking ...)) style.

RTFM links welcome. 😉

jmglov: I'd be happy with figuring out that workflow too

Yeah, I was puzzled with this as well. I ended up writing a very small namespace for it and creating a lein alias for running the specs

Using enumerate-namespace?

Using clojure.tools.namespace.repl: https://gist.github.com/tgk/be0325e7b78bc692ad6c85ef6aca818d

The file is only on dev path by the way 🙂

Interesting. I didn't realise that test/check had a zero-arity form. Really useful!

Does it actually find all specs in all namespaces in your classpath, or something?

I don't see you specifying any namespaces under test, or requiring them in.

Yes, as long as they have been evaluated

Wow.

Ah yes, that’s where refresh comes into the picture 🙂

Would your approach actually run all the specs for dependencies as well?

Or just stuff in your src?

That's nifty!

All the same, it would be great to find an approach that would allow me to drop spec generative tests into my standard clojure.test files.

I haven't found anything with Google, but stemming is really fighting me on this one. 😉

Maybe I need to roll my own checking macro, like this guy did: http://blog.colinwilliams.name/blog/2015/01/26/alternative-clojure-dot-test-integration-with-test-dot-check/#the-alternative

Hmm yes, I don’t think specs for dependencies would be run. I can’t see how they would 🙂

That's good. 🙂

I found it very hard to find anyone who’d hooked it into tests

So refresh simply evals everything in your source directories?

using spec.test/check or spec.test/instrument will pick up any spec’ed fns that have been loaded and added to the registry, so it depends completely on what code you’ve loaded

Thanks, Alex!

Also, thanks for the spec Guide. I finished reading it, and it is really excellent!

@bret I personally would prefer your first spec (although looks like you missing the kw namespaces on the map-of and you probably want s/merge instead of s/and)

OK, switching gears for a second, I'm obviously doing something silly, but I'm not sure what. I'm trying to spec out the input coming in from some JSON, and I have some code like this:

(ns wtf
  (:require [clojure.spec :as s]
            [clojure.spec.test :as test]))

(s/def ::contract_type_id pos-int?)
(s/def ::product (s/keys :req-un [::contract_type_id]))
(s/fdef exclude-products
  :args (s/cat :products (s/coll-of ::product)
               :excluded-contracts (s/coll-of ::contract_type_id))
  :ret (s/coll-of ::product)
  :fn #(<= (-> % :args :products) (-> % :ret)))

(defn- exclude-products [products excluded-contracts]
  (letfn [(excluded? [product]
            (some #{(:contract_type_id product)} excluded-contracts))]
    (remove excluded? products)))

My exclude-products function should do something this:

wtf> (exclude-products [{:contract_type_id 1} {:contract_type_id 2}] [1])
({:contract_type_id 2})

Things look good with exercise-fn:

wtf> (s/exercise-fn `exclude-products 1)
([([{:contract_type_id 2} {:contract_type_id 2} {:contract_type_id 2} {:contract_type_id 1}] [2 2 1 1 1 1 1 2 1])
  ()])

But check completely rejects my entire worldview:

wtf> (test/check `exclude-products)
({:spec #object[clojure.spec$fspec_impl$reify__14244 0x2c221658 "clojure.spec$fspec_impl$reify__14244@2c221658"],
  :clojure.spec.test.check/ret {:result #error {
 :cause "clojure.lang.PersistentVector cannot be cast to java.lang.Number"
 :via
 [{:type java.lang.ClassCastException
   :message "clojure.lang.PersistentVector cannot be cast to java.lang.Number"
   :at [clojure.lang.Numbers lte "Numbers.java" 225]}]
 :trace
 [[clojure.lang.Numbers lte "Numbers.java" 225]
  [wtf$fn__29911 invokeStatic "wtf.clj" 11]
...
  [java.lang.Thread run "Thread.java" 745]]},
                                :seed 1473685015567,
                                :failing-size 0,
                                :num-tests 1,
                                :fail
                                [([{:contract_type_id 2}
                                   {:contract_type_id 1}
                                   {:contract_type_id 1}
                                   {:contract_type_id 2}
                                   {:contract_type_id 1}
                                   {:contract_type_id 2}]
                                  [2 1 1 2 2 2 2 1 1 2])],
                                :shrunk {:total-nodes-visited 18, :depth 16, :result #error {
...

Can anyone shed light on what I'm doing wrong?

@jeroenvandijk yes, please file a jira. this is where we are hurting for an s/vcat which Rich and I have talked about several times.

@jmglov it looks to me like your :fn spec is wrong and should be comparing count of each thing?

doh

@alexmiller Of course you are right. Thanks for pointing out my idiocy! 🙂

well I wouldn’t go that far. :) fwiw, I’ve done the same.

Oh, so much better! Now summarize-results is showing me a bug in my code or spec. 🙂

@alexmiller I guess my point/observation is that the second one is not allowed at all. That was puzzling at first. I've missed s/merge all this time. I’ll look at that and see if that changes anything.

@alexmiller thank, will do

@bret not sure what you mean by your point/observation, sorry

@alexmiller I’ve created the issue here http://dev.clojure.org/jira/browse/CLJ-2021?focusedCommentId=43842#comment-43842 Not sure what else to add

@alexmiller I probably shouldn't start writing at midnight on Sunday night. :) I guess it boils down to, is the reason this works

(s/keys :req-un [::k1 ::k2])
=> #object[clojure.spec$map_spec_impl$reify__13426 0x1b824394 "clojure.spec$map_spec_impl$reify__13426@1b824394”]

(def rks [::k1 ::k2])
=> #'onenine.core/rks
(s/keys :req-un rks)
CompilerException java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Symbol, compiling:(/Users/brety/dev/personal/onenine/src/onenine/core.clj:69:1) 

well, both

s/keys is a macro and expect a concrete list of keys

so that’s as intended

and the reason most of the spec creating fns are macros is to capture forms for reporting

Does anyone know how to make test/check work on private functions? I've tried #'full.namespace/foo, but I get java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Var.

Leaving off the #' tells me the function is not public, which I already know. 😉

we might in the future have a fn entry point for s/keys with the caveat that you may lose some of the explain reporting capability

That is what I suspected (the reporting aspect) but wanted to confirm.

@jmglov I don’t think that was considered in the design (and I’m not sure whether it should be)

Yeah, I'm a bit naughty, really.

On a related note: I find myself regularly wanting validation of runtime-defined specs, where I learn e.g. the structure of the JSON in this particular REST API at runtime. This is for presumably obvious reasons, not very convenient right now.

This is always a tough call. I want to use private functions to communicate to my clients that they are not part of the interface, but I also want to be able to unit test them. 😕

(Everything ends up being eval’d, which I guess is fine?)

It would be nice in some case, I think, to be able to define keys once and combine them in different ways ways when building specs. At least, as I think about repeating information in the spec(s) and trying to reduce that.

But I don’t have enough time in spec to be sure.

I might just go the foo.bar.internal route, where everything in the internal ns is public and can be tested, but is pretty clearly not for client consumption.

@lvh yeah, I understand that as a use case, not sure how common that will be in general though

Using a namespace-level docstring to warn off potential troublemakers, of course.

yeah

@bret well that’s exactly the point of having the registry

alexmiller: If you give Clojure programmers a feature it seems like a matter of time before they’ll try to express as much of it as data, and then it’s not a long stretch until that data isn’t available at compile time 😉

I might be able to get around it and just move more stuff into compile-time-land

specs are data in s-expr form

we haven’t released it yet, but I have a spec for spec forms

(which revealed a lot of bugs in s/form :)

nice; I would very much like that

since a hypothetical awful person might want to construct specs at runtime and have better feedback about why they don’t work 😉

oh, I don’t think you’re awful :)

it’s reasonable

just not the primary use case we were working to support

@alexmiller Ok, this will help me.

;; I want to check that an input map's keys are valid
;;   where the keys are unqualified, some required, some optional,
;;   and not allow keys outside that set.

; This is straight forward
(s/def :in/data (s/and
                  (s/keys :req-un [:in/id] :opt-un [:in/more])
                  (s/map-of #{:id :more} nil)))

; but I'm (kind of) repeating information.
;
; If I write

(def req-keys [:in/id])
(def opt-keys [:in/more])

(defn unk
  "Returns ns unqualified keys for (possibly) qualified ones."
  [& ks]
  (map #(-> % name keyword) ks))

(s/def :in2/data (s/and
                   (s/keys :req-un req-keys :opt-un opt-keys)
                   (s/map-of (set (apply unk (concat req-keys opt-keys))) nil)))

; I have not repeated the key values but s/key doesn't allow it.

I could be missing something fundamental.

I’d say generally that Rich believes in open maps and that’s why this is not a feature provided out of the box

and that I think your first example is what I would do if I was doing it

(although nil is not a valid spec there - you want any?)

and I would use s/merge instead of s/and

which I think would gen better

So, s/merge can be used for combining more than s/keys (`s/map-of` in this case)?

s/merge is used to combine (union) map specs

it differs in not flowing conformed results like s/and and also in being better at gen'ing

I get the open map approach and generally like it. One thought I had, that really relates to the reporting aspect, is that s/keys supports ‘and’/‘or’ combinations of key vectors. So, keeping the form used for reporting as close to literal boolean expressions of literal key vectors is not a bad thing. Ok, thanks, this helps.

what's the best way to spec that something satisfies? a protocol?

obviously you can just use the (partial satisifies? Protocol) predicate... but is there a way for implementers to hook in and extend the generator to generate types that satisfy it? I could imagine that if implementers spec'd their constructing functions, you could get this almost for free.

nothing built-in

Here's another fun one. Using the fixed version of the same spec as previously, I can use stest/check on it in my REPL:

kpcs.product-catalog.internal-test> (first (stest/check 'kpcs.product-catalog.internal/exclude-products))
{:spec #object[clojure.spec$fspec_impl$reify__14244 0x2c8e87a5 "clojure.spec$fspec_impl$reify__14244@2c8e87a5"],
 :clojure.spec.test.check/ret {:result true, :num-tests 1000, :seed 1473692045039},
 :sym kpcs.product-catalog.internal/exclude-products}

However, when I try to use it in a test, I get an exception. Here's what I'm trying to do:

(ns kpcs.product-catalog.internal-test
  (:require [clojure.spec.test :as stest]
            [clojure.test :refer [deftest is testing]]
            [kpcs.product-catalog.internal :as internal]))

(deftest exclude-products
  (testing "Specs"
    (let [res (first (stest/check 'kpcs.product-catalog.internal/exclude-products))]
      (is (= java.lang.String (type res)))))

And I get this:

java.util.concurrent.ExecutionException: java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn, compiling:(clojure/test/check/clojure_test.cljc:95:1)
 at java.util.concurrent.FutureTask.report (FutureTask.java:122)
    java.util.concurrent.FutureTask.get (FutureTask.java:192)
    clojure.core$deref_future.invokeStatic (core.clj:2290)
    clojure.core$future_call$reify__9352.deref (core.clj:6847)
    clojure.core$deref.invokeStatic (core.clj:2310)
    clojure.core$deref.invoke (core.clj:2296)
    clojure.core$map$fn__6856.invoke (core.clj:2728)
    clojure.lang.LazySeq.sval (LazySeq.java:40)
    clojure.lang.LazySeq.seq (LazySeq.java:56)
    clojure.lang.LazySeq.first (LazySeq.java:71)
    clojure.lang.RT.first (RT.java:682)
    clojure.core$first__6379.invokeStatic (core.clj:55)
    clojure.core/first (core.clj:55)
...

Any ideas?

To be clear, if I REPL into my kpcs.product-catalog.internal-test and run the code above, it works. If I run lein test, I get the exception.

I don't see what should be different between the two.

that’s a problem with lein test’s monkeypatching

it’s fixed in (not yet released) next version of test.check

but you can disable lein monkeypatching to fix

:monkeypatch-clojure-test false

Great, thanks!

that will disable lein retest but otherwise should not affect what you’re doing

Just tried it, and it works perfectly.

you are not the first person to encounter it :)

Thank goodness for that!

@otfrom @tgk Here's a cheap hack to fit check into my standard tests:

(deftest exclude-products
  (testing "Specs"
    (let [result (-> (stest/check 'kpcs.product-catalog.internal/exclude-products)
                     first
                     :clojure.spec.test.check/ret
                     :result)]
      (if (true? result)
        (is result)
        (is (= {} (ex-data result)))))))

jmglov: thx!

I'll make a checking function out of it and throw it in a test lib. The output is decent enough with the humane-test-output plugin. 🙂

If it's useful, you're welcome. Otherwise, I'm sorry for such a disgusting kludge! 😉

Hi there! Sorry for very dumb question. How to define spec for function with variable args?

Hi guys, I've just started using clojure.spec and was wondering how to use clojure.spec.test/check with clojure.test

sorry, missed the msg from @jmglov

@mike1452 (s/fdef myf :args (s/cat :map-params (s/? map?))) will take both 0 and 1 (map) arg

you can replace map? with something more specific too of course

it seems awkward that core/defn has a special syntax for arity dispatch but spec/fdef doesn't provide one for speccing/testing

doesn't really affect me as I never use arity dispatch but I could see it sucking if you previously used it a lot

they are doing different things. most multi-arity functions share param definitions across arities and merging them works very nicely for this in most cases.

sure, for the :args, but it doesn't make the :fn for say map less readable?

sure, although I think that’s an unusual case

true. I guess the common case would be reduce. smaller arity providing default value

Is there a reason why if-let and when-let can’t return :clojure.spec/invalid?

well that is problematic

(if-let [a 1] '::s/invalid) works if you need a work around

@hiredman cool, thanks. didn’t know that works too.

@ikitommi heh, that’s fun

there’s actually a ticket related to this

http://dev.clojure.org/jira/browse/CLJ-1966

I added the if-let example above to that ticket. I suspect people will run into this in more and more situations as they try to write conformers.

and as there are more spec’ed things

I need a spec that would generate vectors of values taking random elements from a predefined list, e.g: [:foo :bar] [:foo] [:baz :bar] []… etc.

how?

(s/coll-of #{:foo :bar :baz} :kind vector?)

you can also use the other options on coll-of to set :min-count, :max-count, :count constraints on the spec or :gen-max to cap what the generator will produce

right… http://clojure.org/guides/spec#_collections

thanks!