I've been following http://clojure.org/guides/spec, and trying to apply it to some maze-building code I'm toying with. The core data structure is a grid (a two-dimensional vector) of cells (x, y, and a set of open exit directions). I have some specs which seem entirely reasonable to me:

(spec/def ::x integer?)
(spec/def ::y integer?)
(spec/def ::direction #{::n ::ne ::e ::se ::s ::sw ::w ::nw})
(spec/def ::exits (spec/coll-of ::direction :kind set?))
(spec/def ::cell (spec/keys :req [::x ::y ::exits]))
(spec/def ::row (spec/coll-of ::cell))
(spec/def ::grid (spec/coll-of ::row))

(defn grid-contains?
  "True if the supplied x and y coordinates fall within the grid boundaries."
  [grid x y]
  (and (<= 0 y) (<= 0 x)
       (< y (count grid))
       (< x (count (nth grid y)))))

(spec/fdef grid-contains?
  :args (spec/cat :grid ::grid :x ::x :y ::y)
  :ret boolean?)

(require '[clojure.spec.test :as stest])

(stest/check `grid-contains?)

I did try this:

(stest/check `grid-contains? {:clojure.spec.test.check/opts {:num-tests 10}})

(spec/exercise ::grid) does fine, so I want to think that Clojure is able to generate correct test data without any issues. ¯\(ツ)/¯

Final note: when I first ran stest/check, it hit null errors which revealed bugs (yay?). Once I fixed those bugs, I ended up in this situation.

Can I use spec to spec out a protocol? Just fdef the protocol’s method separately??

@lvh I seem to recall a JIRA issue around protocol functions, but it may be only for instrument...?

@amacdougall my first thought is maybe your spec is generating enormous grids under test which is chewing up cpu walking around it

@amacdougall just added a print to your fn and ran (stest/check `grid-contains? {:num-tests 1}), getting values like 32174406 809896496 -13491 -1 -3980 -2856 193086473 -53862035

Would guess it's the (nth grid y) that's trying to walk along really long linked lists or something - out of time now but hope that helps!

http://dev.clojure.org/jira/browse/CLJ-1941 maybe

what's wrong with the above?

the generator works, the s/def works if I comment out the with-gen parts, but the whole fails with: "Couldn't satisfy such-that predicate after 100 tries."

sort turns maps into lists of map entries

@minimal thanks.

what would be the proper way?

are you trying to gen a sorted-map? @magnetophon

yes

i think this works (gen/sample (gen/fmap (fn [a] (apply sorted-map (flatten (sort a)))) (gen/map (s/gen int? ) (s/gen int?))))

({} {0 -1} {0 -1} {-1 -1, 0 -4, 7 -3} {-1 -4, 0 -5, 1 -1} {-2 -2, -1 0, 0 3, 3 7} {-1 6, 0 -4, 24 -12} {-16 -21, -7 0, -4 16, -1 0, 0 -1, 1 -10} {-32 -7, 0 -23, 92 -13} {-13 1, -3 -1, -2 0, -1 1, 6 51, 14 11})

@minimal It does! thanks! now I just need to decrypt that, but I'll manage.

Just turning it into what sorted map expects (sorted-map k1 v1 k2 v2 …)

@minimal I'm very new to clojure and lisp, so I'll have to study a bit to really get it, but I'll be fine.

@minimal Hmm, I sort of get what you're doing, and can use that with most types as the val of the map, but when I use (the type I need )[https://github.com/magnetophon/openloop/blob/master/src/openloop/spec.clj#L79] I get:

No value supplied for key: 0

the weird thing is: when I comment out the :src-index, it works. when I leave that in, but uncomment :length, it also works.

@brabster: Thanks for the pointer—maybe if I refine my spec to declare that the grid should never be more than, say, 10000x10000, and also specify that it must be made of vectors (for faster indexed access), test.check will be a bit less ambitious. On the other hand, should we expect such a system to generate arbitrarily large collections up to the literal limit of the integer type? I certainly see the benefit, but if a single test takes such a long time to run, how could you run a whole suite without a Bitcoin-mining level of resources?

Last night before giving up, I did change the grid spec to include :kind vector?:

(spec/def ::row (spec/coll-of ::cell :kind vector?))
(spec/def ::grid (spec/coll-of ::row :kind vector?))

To put it more generally: should developers expect to hand-tune specs to keep generative testing from trying to simulate the entire possible range of inputs? On reflection, the answer is yes: that's what generative testing is for. But in practice, generative testing on strings generates a bunch of garbage strings, it doesn't generate million-character strings. So why is this test trying to generate a 193,086,473-column grid? ...end of musing.

@minimal ah, found the problem: flatten flattens recursively, so when I have an uneven number of elements I get an error.

cool, sorry was afk for a bit

np, you've been very helpful.

np

I'm afraid that setting explicit bounds on my specs doesn't make a difference. I updated the relevant specs to limit both coordinates and grid dimensions:

(spec/def ::x (spec/and integer? (partial > 1000)))
(spec/def ::y (spec/and integer? (partial > 1000)))
(spec/def ::row (spec/coll-of ::cell :kind vector? :max-count 1000))
(spec/def ::grid (spec/coll-of ::row :kind vector? :max-count 1000))

#'user/grid-contains?
user=> (stest/check `grid-contains? {:num-tests 1})
"grid-contains? grid 0 0"

OutOfMemoryError GC overhead limit exceeded  clojure.lang.PersistentVector$TransientVector.persistent (PersistentVector.java:568)

The other main use of fdef is to instrument a namespace during development/test, which forces spec checking of all function calls, right? That seems like it would still be quite beneficial even without full-spectrum generative testing. 🌈

Hi guys, I want to do some checking of JSON data with spec, seems like it will be OK - any gotchas?

@minimal I found a solution ^^

when I s/conform some ok values it works as expected. when I try to conform values with a duplicate key, I don't get an error saying that, but instead I get:

Unmatched delimiter: )

should I report that as a bug in clojure, or am I doing it wrong?

Here is the code, and some sample data,if someone wants to try it out: https://github.com/magnetophon/openloop/blob/master/src/openloop/spec.clj#L79

@magnetophon cool! Yep (into (sorted-map)) is better. Not sure about the error

@minimal a clearer example of the error is:

(s/def ::test-type
  (s/map-of int? int?))
(s/conform ::test-type {41 1, 42 2}) ; ok
(s/conform ::test-type {42 1, 42 2}) ; unmatched delimiter: )

@amacdougall Keep in mind that instrument will only check :args, not :ret or :fn specs

@magnetophon you should be getting an exception from the reader creating a literal map with duplicate keys

@magnetophon you can’t have map literals with duplicate keys anyway. The unmatched delimiter must be a symptom of that

I mean.. you also can't have maps with duplicate keys either. if you try and construct one one of the keyval pairs will disappear

@bfabry @minimal thanks. It's just that I recently read the clojure spec page, and wanted to try out the promise of usefull error messages.

try (s/conform ::test-type {:a 2, 31 3})

the thing is spec is never coming into play when you're trying to detect duplicate keys in a map. the map data structure just doesn't support that, and the clojure reader also doesn't support that. so it gets chucked out way before spec ever checks it

I was hoping to get a nice sorry, the keys are not unique message from spec. oh well... 🙂

I'm kind of surprised the reader didn't give you that error

because that's what I get

user=> {1 1 1 1}

user=> clojure.lang.LispReader$ReaderException: java.lang.IllegalArgumentException: Duplicate key: 1
     java.lang.IllegalArgumentException: Duplicate key: 1

@bfabry yeah I get that too, but not when I do it in spec

ok but you're not doing it "in spec"

the error happens pre spec

you literally cannot create a map with duplicate keys, so there's no way for spec to look at a map that has duplicate keys and give you an error about it

@bfabry I mean I don't get a usefull error from the reader when the faulty map is inside a conform or explain function.

I get

(s/conform ::test-type {42 1 42 2}) 
     java.lang.IllegalArgumentException: Duplicate key: 42
clojure.lang.LispReader$ReaderException: java.lang.IllegalArgumentException: Duplicate key: 42
             java.lang.RuntimeException: Unmatched delimiter: )
clojure.lang.LispReader$ReaderException: java.lang.RuntimeException: Unmatched delimiter: )

yeah that's a bit odd, but I don't think it's related to spec

user=> (identity {1 1 1 1})

clojure.lang.LispReader$ReaderException: java.lang.IllegalArgumentException: Duplicate key: 1
     java.lang.IllegalArgumentException: Duplicate key: 1
clojure.lang.LispReader$ReaderException: java.lang.RuntimeException: Unmatched delimiter: )
             java.lang.RuntimeException: Unmatched delimiter: )
user=>

No it’s not spec

it's just because you've blown up inside the reader, so there's unconsumed input

Once the reader fails, all bets are off and it gets more errors

@minimal when I run that line, I get the ) error.

something about your environment is eating the first error, I guess

Yeah

It’s a distracting error but the duplicate key error is the useful one. So it's confusing if it’s missing from your output

@bfabry @minimal ah, I was looking in the wrong place. both errors are there. sorry for the noise.

np

no worries.

What is meant by

By default map-of will validate but not conform keys because conformed keys might create key duplicates that would cause entries in the map to be overridden. If conformed keys are desired, pass the option `:conform-keys true'.

afaik conform means to add metadata like the keys of a map. What do they mean by conforming keys? And why would that create duplicates?

conforming means modifying the data to fit the spec

boot.user=> (s/conform (s/map-of int? (s/or :k keyword? :i int?)) {1 :foo 2 3})
{1 [:k :foo], 2 [:i 3]}

so you see there the values were conformed

to tell you whether the value conformed to the :k branch or the :i branch of the spec

but you can use custom conformers, so hence the statement above

by default the keys in the conformed value of the map will be identical

but you can use :conform-keys true in the call to map-of to allow that to happen

user=> (s/conform (s/map-of (s/or :i int? :s string?) int?) {5 5, "a" 10})
{5 5, "a" 10}
user=> (s/conform (s/map-of (s/or :i int? :s string?) int? :conform-keys true) {5 5, "a" 10})
{[:i 5] 5, [:s "a"] 10}

and the reason not to do that automatically is that when you are altering the keys of the map, you might end up producing the same key and effectively erasing some of your transformed input, so we err on the side of safety

thanks. so it only makes sense if you have a branch in your key spec, and you want to know which branch was taken?

that is not clear (to me) from the guide.

It only matters if your key spec conforms to something other than the original value

Given the common map keys - strings, longs, keywords, symbols - those are all likely to conform the same and it's a nonissue

So in most cases you can ignore this entirely