This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-08-23
Channels
- # admin-announcements (1)
- # alda (1)
- # bangalore-clj (5)
- # beginners (17)
- # boot (392)
- # capetown (4)
- # cider (16)
- # cljs-dev (24)
- # cljsrn (33)
- # clojure (106)
- # clojure-berlin (1)
- # clojure-nl (1)
- # clojure-russia (168)
- # clojure-spec (85)
- # clojure-uk (137)
- # clojurescript (83)
- # clojutre (4)
- # component (10)
- # cursive (6)
- # datavis (9)
- # datomic (11)
- # defnpodcast (15)
- # dirac (4)
- # docker (1)
- # ethereum (1)
- # hoplon (27)
- # jobs (5)
- # jobs-rus (1)
- # lein-figwheel (2)
- # luminus (5)
- # off-topic (5)
- # om (13)
- # onyx (60)
- # parinfer (2)
- # planck (12)
- # proton (2)
- # re-frame (45)
- # rethinkdb (5)
- # ring-swagger (9)
- # spacemacs (9)
- # specter (49)
- # test-check (1)
- # untangled (104)
- # yada (10)
I've been following http://clojure.org/guides/spec, and trying to apply it to some maze-building code I'm toying with. The core data structure is a grid (a two-dimensional vector) of cells (x, y, and a set of open exit directions). I have some specs which seem entirely reasonable to me:
(spec/def ::x integer?)
(spec/def ::y integer?)
(spec/def ::direction #{::n ::ne ::e ::se ::s ::sw ::w ::nw})
(spec/def ::exits (spec/coll-of ::direction :kind set?))
(spec/def ::cell (spec/keys :req [::x ::y ::exits]))
(spec/def ::row (spec/coll-of ::cell))
(spec/def ::grid (spec/coll-of ::row))
And then I have this pretty basic function, with a pretty basic fspec:
(defn grid-contains?
"True if the supplied x and y coordinates fall within the grid boundaries."
[grid x y]
(and (<= 0 y) (<= 0 x)
(< y (count grid))
(< x (count (nth grid y)))))
(spec/fdef grid-contains?
:args (spec/cat :grid ::grid :x ::x :y ::y)
:ret boolean?)
But when I take the guide's advice and do this:
(require '[clojure.spec.test :as stest])
(stest/check `grid-contains?)
...I use 350% CPU for fifteen minutes and counting. What gives? Is there some unbounded complexity in my specs? Or is test.check just being very thorough?I did try this:
(stest/check `grid-contains? {:clojure.spec.test.check/opts {:num-tests 10}})
... which seems to be the right way to specify the options, according to http://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec.test/check, but it didn't make a difference.(spec/exercise ::grid)
does fine, so I want to think that Clojure is able to generate correct test data without any issues. ¯\(ツ)/¯
Final note: when I first ran stest/check
, it hit null errors which revealed bugs (yay?). Once I fixed those bugs, I ended up in this situation.
@lvh I seem to recall a JIRA issue around protocol functions, but it may be only for instrument
...?
@amacdougall my first thought is maybe your spec is generating enormous grids under test which is chewing up cpu walking around it
@amacdougall just added a print to your fn and ran (stest/check `grid-contains? {:num-tests 1}), getting values like 32174406 809896496 -13491 -1 -3980 -2856 193086473 -53862035
Would guess it's the (nth grid y) that's trying to walk along really long linked lists or something - out of time now but hope that helps!
what's wrong with the above?
the generator works, the s/def works if I comment out the with-gen parts, but the whole fails with: "Couldn't satisfy such-that predicate after 100 tries."
@minimal thanks.
what would be the proper way?
are you trying to gen a sorted-map? @magnetophon
i think this works (gen/sample (gen/fmap (fn [a] (apply sorted-map (flatten (sort a)))) (gen/map (s/gen int? ) (s/gen int?))))
({} {0 -1} {0 -1} {-1 -1, 0 -4, 7 -3} {-1 -4, 0 -5, 1 -1} {-2 -2, -1 0, 0 3, 3 7} {-1 6, 0 -4, 24 -12} {-16 -21, -7 0, -4 16, -1 0, 0 -1, 1 -10} {-32 -7, 0 -23, 92 -13} {-13 1, -3 -1, -2 0, -1 1, 6 51, 14 11})
@minimal It does! thanks! now I just need to decrypt that, but I'll manage.
@minimal I'm very new to clojure and lisp, so I'll have to study a bit to really get it, but I'll be fine.
@minimal Hmm, I sort of get what you're doing, and can use that with most types as the val
of the map, but when I use (the type I need )[https://github.com/magnetophon/openloop/blob/master/src/openloop/spec.clj#L79] I get:
No value supplied for key: 0
the weird thing is: when I comment out the :src-index, it works. when I leave that in, but uncomment :length, it also works.
@brabster: Thanks for the pointer—maybe if I refine my spec to declare that the grid should never be more than, say, 10000x10000, and also specify that it must be made of vectors (for faster indexed access), test.check
will be a bit less ambitious.
On the other hand, should we expect such a system to generate arbitrarily large collections up to the literal limit of the integer type? I certainly see the benefit, but if a single test takes such a long time to run, how could you run a whole suite without a Bitcoin-mining level of resources?
Last night before giving up, I did change the grid spec to include :kind vector?
:
(spec/def ::row (spec/coll-of ::cell :kind vector?))
(spec/def ::grid (spec/coll-of ::row :kind vector?))
But it didn't seem to make a difference. My understanding was that nth
on a vector is O(1), because it is effectively (v n)
, and although I don't understand all the voodoo at a glance, https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/PersistentVector.java seems to confirm that.To put it more generally: should developers expect to hand-tune specs to keep generative testing from trying to simulate the entire possible range of inputs? On reflection, the answer is yes: that's what generative testing is for. But in practice, generative testing on strings generates a bunch of garbage strings, it doesn't generate million-character strings. So why is this test trying to generate a 193,086,473-column grid? ...end of musing.
@minimal ah, found the problem: flatten flattens recursively, so when I have an uneven number of elements I get an error.
np, you've been very helpful.
I'm afraid that setting explicit bounds on my specs doesn't make a difference. I updated the relevant specs to limit both coordinates and grid dimensions:
(spec/def ::x (spec/and integer? (partial > 1000)))
(spec/def ::y (spec/and integer? (partial > 1000)))
(spec/def ::row (spec/coll-of ::cell :kind vector? :max-count 1000))
(spec/def ::grid (spec/coll-of ::row :kind vector? :max-count 1000))
...and added a prn
to grid-contains?
to see what test.check is trying to do... and I get this:
#'user/grid-contains?
user=> (stest/check `grid-contains? {:num-tests 1})
"grid-contains? grid 0 0"
OutOfMemoryError GC overhead limit exceeded clojure.lang.PersistentVector$TransientVector.persistent (PersistentVector.java:568)
Perhaps there's something I could do to work around this, but I think for now I'll skip the generative testing.The other main use of fdef
is to instrument
a namespace during development/test, which forces spec checking of all function calls, right? That seems like it would still be quite beneficial even without full-spectrum generative testing. 🌈
Hi guys, I want to do some checking of JSON data with spec, seems like it will be OK - any gotchas?
@minimal I found a solution ^^
when I s/conform
some ok values it works as expected. when I try to conform
values with a duplicate key, I don't get an error saying that, but instead I get:
Unmatched delimiter: )
should I report that as a bug in clojure, or am I doing it wrong?
Here is the code, and some sample data,if someone wants to try it out: https://github.com/magnetophon/openloop/blob/master/src/openloop/spec.clj#L79
@magnetophon cool! Yep (into (sorted-map))
is better. Not sure about the error
@minimal a clearer example of the error is:
(s/def ::test-type
(s/map-of int? int?))
(s/conform ::test-type {41 1, 42 2}) ; ok
(s/conform ::test-type {42 1, 42 2}) ; unmatched delimiter: )
@amacdougall Keep in mind that instrument
will only check :args
, not :ret
or :fn
specs
@magnetophon you should be getting an exception from the reader creating a literal map with duplicate keys
@magnetophon you can’t have map literals with duplicate keys anyway. The unmatched delimiter must be a symptom of that
I mean.. you also can't have maps with duplicate keys either. if you try and construct one one of the keyval pairs will disappear
@bfabry @minimal thanks. It's just that I recently read the clojure spec page, and wanted to try out the promise of usefull error messages.
the thing is spec is never coming into play when you're trying to detect duplicate keys in a map. the map data structure just doesn't support that, and the clojure reader also doesn't support that. so it gets chucked out way before spec ever checks it
I was hoping to get a nice sorry, the keys are not unique
message from spec. oh well... 🙂
user=> {1 1 1 1}
user=> clojure.lang.LispReader$ReaderException: java.lang.IllegalArgumentException: Duplicate key: 1
java.lang.IllegalArgumentException: Duplicate key: 1
@bfabry yeah I get that too, but not when I do it in spec
you literally cannot create a map with duplicate keys, so there's no way for spec to look at a map that has duplicate keys and give you an error about it
@bfabry I mean I don't get a usefull error from the reader when the faulty map is inside a conform or explain function.
I get
(s/conform ::test-type {42 1 42 2})
java.lang.IllegalArgumentException: Duplicate key: 42
clojure.lang.LispReader$ReaderException: java.lang.IllegalArgumentException: Duplicate key: 42
java.lang.RuntimeException: Unmatched delimiter: )
clojure.lang.LispReader$ReaderException: java.lang.RuntimeException: Unmatched delimiter: )
user=> (identity {1 1 1 1})
clojure.lang.LispReader$ReaderException: java.lang.IllegalArgumentException: Duplicate key: 1
java.lang.IllegalArgumentException: Duplicate key: 1
clojure.lang.LispReader$ReaderException: java.lang.RuntimeException: Unmatched delimiter: )
java.lang.RuntimeException: Unmatched delimiter: )
user=>
@minimal when I run that line, I get the )
error.
It’s a distracting error but the duplicate key error is the useful one. So it's confusing if it’s missing from your output
@bfabry @minimal ah, I was looking in the wrong place. both errors are there. sorry for the noise.
What is meant by
By default map-of will validate but not conform keys because conformed keys might create key duplicates that would cause entries in the map to be overridden. If conformed keys are desired, pass the option `:conform-keys true'.
in the spec guide?afaik conform means to add metadata like the keys of a map. What do they mean by conforming keys? And why would that create duplicates?
boot.user=> (s/conform (s/map-of int? (s/or :k keyword? :i int?)) {1 :foo 2 3})
{1 [:k :foo], 2 [:i 3]}
by default the keys in the conformed value of the map will be identical
but you can use :conform-keys true
in the call to map-of
to allow that to happen
user=> (s/conform (s/map-of (s/or :i int? :s string?) int?) {5 5, "a" 10})
{5 5, "a" 10}
user=> (s/conform (s/map-of (s/or :i int? :s string?) int? :conform-keys true) {5 5, "a" 10})
{[:i 5] 5, [:s "a"] 10}
and the reason not to do that automatically is that when you are altering the keys of the map, you might end up producing the same key and effectively erasing some of your transformed input, so we err on the side of safety
thanks. so it only makes sense if you have a branch in your key spec, and you want to know which branch was taken?
that is not clear (to me) from the guide.
It only matters if your key spec conforms to something other than the original value
Given the common map keys - strings, longs, keywords, symbols - those are all likely to conform the same and it's a nonissue
So in most cases you can ignore this entirely