when i use spec/exercise for things like string? and pos-int? i mostly get blank or 1 character strings and numbers 1 and 2

is there a way to get it to "try harder"

and give me longer strings more often, with more variety, like emojis and non english alphanumeric characteres

and give me random ints anywhere in the full range of available positive integers?

i really like the idea of using spec to help me generate tests, but it doesn't give me a lot of confidence to know that most of my tests are just passing "" and 1

oh hmmm

i just realised that if i (spec/exercise string? 1) i almost always get ""

but if i (spec/exercise string? 100) then the later strings are much longer

so i could do something like (ffirst (shuffle (spec/exercise pos-int? 100))) but this seems inefficient...

@thedavidmeister I'm a bit puzzled about how you're running tests here?

Are you using spec.test's check on functions? Or test.check with properties etc?

uh, nothing like that atm

i've just been working on creating specs for my existing codebase

and i've got existing tests that i've handrolled examples for

but i thought i could use spec/exercise as a "drop in" replacement for my examples

We have a couple of places where we do (rand-nth (map first (s/exercise ::my-spec 100))) which is an equivalent to what you have so it's a reasonable approach for getting a specific random piece of test data.

ok cool

just checking that i'm not crazy 🙂

I suspect rand-nth is going to be faster than first of shuffle (and (first (rand-nth ...))` is probably the fastest).

is there a way to get it to be lazy?

Lazy how? You only want one value and you want a random one out of 100...

yes but it's clearly doing something different for the 100th to the 1st item that it generates

i want the logic of the 100th iteration

but not the 99 values that come before it

Since we're talking about tests I wouldn't be too concerned about efficiency.

i know...

it just seems so wasteful

i'm not normally "that guy" but...

FWIW, (first (rand-nth (s/exercise string? 100))) takes between 6 and 30ms on my low-powered laptop

ok

well the other half of my q

is there a way to get string? to be less alphanumeric-y?

What do you want it to be?

i sort of expected utf-8

so :poop: emojis and japanese kanji, etc.

Some of the built-in generators are a bit conservative. Take a look at test.chuck and its regex generator.

We have a crazy email address regex and use test.chuck to produce random email addresses and it generates absolutely wild UTF-8 strings 🙂

oh yeah string-from-regex looks cool

We also have a wild password spec (with complex rules about UTF-8 character classes and length etc), but for testing we override to use a much simpler ASCII regex generator.

so how do i plug that into spec?

We're testing different things there.

s/exercise lets you specify overrides, or you can use s/with-gen to provide a custom generator for any spec.

ok cool, so can i do like (s/with-gen string? (partial string-from-regex #".*")

or is that not right?

It needs to be a zero arity function that returns a generator.

like: (s/def :nl.klijs.person/first-name (let [re #“^[A-Z]{1}[a-z]{2,20}“] (s/spec string? :gen #(sg/string-generator re))))

an example from our code

(s/def ::date-of-birth (s/with-gen age-18-120?
                         (fn [] (s/gen (s/inst-in (wdt/years-ago 120)
                                                  (wdt/years-ago 18))))))

We wrap test.chuck so we don't drag it in for production code but it can come in during testing so we have

(defn fn-string-from-regex
  "Return a function that produces a generator for the given
  regular expression string."
  [regex]
  (fn []
    (require '[com.gfredericks.test.chuck.generators :as xgen])
    (let [string-from-regex (resolve 'xgen/string-from-regex)]
      (string-from-regex regex))))

and then

(s/def ::password (s/with-gen (s/and string? password-valid?)
                    (wgen/fn-string-from-regex #"[a-zA-Z0-9 _!@#$%\^&*\(\)\).]{8,}]")))

(`password-valid?` is a gnarly predicate)

ok cool

and then once you've done that, exercise will know what to do with ::password

Yup, it will generate based on that regex.

that sounds much more thorough than alphanumeric strings

Which will satisfy our rules (`password-valid?`) but much more easily -- and still readable.

soz, i have another question while i go through all this

is there a predicate for date/times?

I needed to use it to be able to even generate maps containing an email

@gklijs yeah i'm running into limitations with the default string? generator pretty fast too

@thedavidmeister We have a set of specs and generators for date/times but it's a fair bit of code because we actually need to coerce from strings to dates, and we want to generate formatted dates.

But for basic stuff you can just use inst?

oh yeah, i'm slowly working through moving between js dates, goog dates, joda dates and iso8601 strings

just wanted something basic for now though 🙂

ah inst? looks great

although hmmm

it seems to strongly prefer 1970-01-01

even if i do (first (rand-nth (spec/exercise inst? 1000))) it rarely moves away from 1970

Yeah, the inst? generator is pretty conservative.

hmmm

know of anything like that regex lib?

i feel like even using a random int as a timestamp would be better 😕

maybe with dates you want one based on the current date and add/substract some random year/month/seconds etc?

Hence our age range spec / generator 🙂

yeah

i think i'm going to go for a jog and think about how i can adapt to my stuff 🙂

One thing I've found important with clojure.spec is to be careful not to over-specify things and to override generators to produce more "sane" data (for your problem domain) -- generating just a small subset of possible conforming values sometimes.

i think that will be ok for me atm

as i'm replacing hard-coded values in tests anyway

it will be an incremental process

as i gradually introduce more complexity of what is specced

@thedavidmeister spec generator have the notion of a size parameter (100 here). However, this is not a minimum length: (clojure.test.check.generators/generate (clojure.spec.alpha/gen string?) 100) and clojure.test.check.generators/resize

@thegeez wow that is great

thanks!

how do you express that all keys of a map should be keywords? I want to spec a structure that looks like this:

{vec {keyword {string {:some-known-key ::x
                         :another-known-key ::y}}}}

@martinklepsch map-of?

(map-of keyword? ::value-spec)

@seancorfield thanks, I’ll look into map-of

@martinklepsch map-of is the right answer, but your pseudo looks much like the thing that the data-specs are eating 😉 (and generating map-ofs & friends)

(require '[clojure.spec.alpha :as s])
(require '[spec-tools.data-spec :as ds])

(s/def ::x string?)
(s/def ::y int?)

(let [martin {vector? {keyword? {string? {:some-known-key ::x
                                          :another-known-key ::y}}}}
      value {[1 2 3] {:a {"b" {:some-known-key "nice"
                               :another-known-key 18}}}}
      spec (ds/spec ::martin martin)]
  (s/valid? spec value))
;; true

@ikitommi ah that indeed seems like a nice utility!

So I just discovered that (s/def :foo/bar :baz/quux) doesn't work if :baz/quux hasn't already been defined. Is this new?

Is it just s/keys that defers lookup?

And if so, does that mean that I can't redefine :baz/quux later?

almost everything defers lookup. this is a known (and unresolved) issue.

If I wrap it with s/spec, will it defer?

maybe? try it

apparently... more testing in half an hour or so