Fork me on GitHub
#clojure-spec
<
2016-09-16
>
jmglov09:09:31

I have a couple of timestamp string specs that require custom generators. I'm using a function to make the generators with the following function spec:

(s/fdef make-timestamp-gen
        :args (s/cat :min-year pos-int?
                     :max-year pos-int?
                     :formatter #(instance? DateTimeFormatter))
        :ret clojure.test.check.generators/generator?)
I'd really like to spec the function, but that would require adding org.clojure/test.check to my non-dev dependencies, which feels a little yucky. Is there another way to ensure that the return value of the function is a generator? clojure.spec.gen doesn't have a generator? function.

jmglov11:09:03

I have a data structure like this:

{:schema {:version "v1"}
 :payload {:type "v1" ...}}
The schema version can either be "v1" or "v2", and the contents of the payload vary based on this. Is there any good way to write a custom generator that will match up version and payload? I can write the conformer reasonably easily, but I can't think of a good way to do the generator.

jmglov11:09:36

OK, I came up with something that feels only mildly hackish:

(defn- basic-gen []
  (->> (s/gen (s/keys :req-un [::schema]))
       (gen/fmap (fn [{{:keys [version]} :schema :as event}]
                   (let [payload-spec (payload-spec-for-schema version)]
                     (assoc event :payload (gen/generate (s/gen payload-spec))))))))

(s/def ::event (s/with-gen
                 (s/keys :req-un [::schema
                                  ::payload])
                 basic-gen))

jmglov11:09:02

Is this OK, or is there a better way?

jmglov13:09:35

Writing a custom generator / conformer pair has led me down a merry path of learning a lot more about spec.

jmglov13:09:59

I have what I hope is the ultimate question in this particular journey:

jmglov13:09:00

Given a custom conformer, how to I make the s/explain output meaningful? At the moment, my conformer just says:

user> (s/conform ::event intentionally-bogus-event)
:clojure.spec/invalid
user> (s/explain ::event intentionally-bogus-event)
val: :kpcs.spec.event/basic fails predicate: :clojure.spec/unknown

hiredman16:09:10

re: the v1 vs v2 thing, that sounds sort of like what multi-spec does

hiredman16:09:22

re: :clojure.spec/unknown, my guess is it is something related to the Specize protocol, the fall through Object case for that protocol uses unknown as the name for the predicate

jmglov16:09:57

@hiredman I remember reading about multi-spec now that you bring it up. I think you're right there!

jmglov16:09:49

Any clue what I would need to do to solve the :clojure.spec/unknown problem?

jmglov16:09:13

Or maybe multi-spec would sort that for me anyway. 🙂

hiredman16:09:25

yeah, maybe

Alex Miller (Clojure team)16:09:05

@jmglov that’s a known issue that Rich and I are working on

Alex Miller (Clojure team)16:09:14

just hasn’t been finished yet

uwo17:09:26

I’ve instrumented the following spec and it catches errors except for the optional parameter. Any advice?

(s/def ::skip integer?)
(s/def ::limit integer?)
(s/def ::request-options (s/keys :un-opt [::skip ::limit]))
(s/fdef request!
  :args (s/cat :field-path ::field-path
               :query-data ::query-data
               :options (s/? ::request-options)))

uwo17:09:10

I can provide any value for skip/limit and it won’t complain

bfabry17:09:35

@uwo :opt-un not :un-opt

uwo17:09:54

jeeze, I’m on a roll with stupid questions. thanks!

bfabry17:09:39

haha it's all good. it'd be nice if spec caught those, but I imagine having spec spec'd would actually be a rather difficult problem

Alex Miller (Clojure team)18:09:48

I have spec specs but using them presents some problems

liamd19:09:04

does anyone know of any guides that go beyond explaining how spec works and show how to integrate it into a real project?

hiredman19:09:54

you know, there isn't a non-alpha version of clojure released with spec yet

liamd19:09:28

i know, but how is it intended to be used? what do i do with my specs in an actual codebase?

hiredman19:09:40

like where to put specs in relation to what they are specing?

liamd19:09:46

yeah i found this

(defn person-name
  [person]
  {:pre [(s/valid? ::person person)]
   :post [(s/valid? string? %)]}
  (str (::first-name person) " " (::last-name person)))

liamd19:09:47

and i see the instrument function

liamd19:09:08

is the idea that you have them as like accessories to your test suite or that they would run on deployed code?

hiredman19:09:43

I think spec can be split in to 1. descriptions of data 2. ways to use those descriptions

liamd19:09:55

yeah so 2 is where i'm foggy

hiredman19:09:24

#2 can be basically anything, but spec provides a few things out of the box

hiredman20:09:04

a. generating data that matches the description b. checking if data matches the description

hiredman20:09:22

and it actually does b. in a few different ways

hiredman20:09:08

one way it can check that data matches is you can call instrument

hiredman20:09:53

and instrument will instrument your functions with checks (and some of those checks rely on generating data, so you really shouldn't instrument out side of testing)

hiredman20:09:22

another way is using valid? like in that person-name function

hiredman20:09:22

instrument is very similar to sort of the classic idea of what you can do with assertions, assert all these expensive pre and post condition checks in development, then compile with assertions turned off for production

hiredman20:09:56

(not that I have ever seen a clojure build that turns off assertions, but I am sure they exist)

seancorfield20:09:01

@liamd I think it’s mostly too early for standard patterns of usage to have settled down (and I don’t think there’s One True Way to use specs anyway). Have you read the http://clojure.org Guide? And watched Stu Halloway’s webcast demos of it? Also the Cognicast with Rich talking about the motivations for it is good listening.

liamd20:09:22

i've listened to the cognicast and the generative testing was what sounded most useful to me

liamd20:09:26

and yeah i've read the guide

liamd20:09:29

i'll check out the webcast

bfabry20:09:37

as an aside, instrument doesn't do what would normally be considered :post checks. It only checks the :args section of an fdef

liamd20:09:56

maybe it's because i'm thinking of spec as an answer to the lack of typing? are those related concerns?

seancorfield20:09:28

At World Singles, we’re spec’ing data primarily and using conform as part of our validation and transformation across application layer boundaries. We’re using instrument to augment our testing. We’re starting to work with the generative side more at this point but we haven’t settled into a good flow for it yet.

hiredman20:09:01

maybe more like a database schema than what you would typically think of as a type

seancorfield20:09:08

@liamd Rich is pretty clear that spec != type system and should not be viewed the same way

liamd20:09:04

so basically my thinking is: - in a statically typed system you can make some assumptions about your passed in args since they wouldn't compile otherwise - without static typing we can just make those assumptions anyway but we might have to debug and get ugly errors down the line - or we could do some defensive programming and handle incorrect args gracefully (by some standard, a nice error or whatever)

richiardiandrea20:09:11

A side question: how folks use exercise?

bfabry20:09:44

it's not a type system, but it's clearly geared at solving some of the same problems in a different way, as well as solving some problems that type systems don't. documentation, correctness, better errors

liamd20:09:48

so does spec just make that third point easier? do a conform and then handle malformed inputs in every function without have to rewrite ugly validation logic everywhere

bfabry20:09:50

@liamd instrument and clojure.spec.test are attempts to make us more confident about point 2

hiredman20:09:55

I would say no

hiredman20:09:18

if you have a system where you are calling conform in every function on all your arguments, something has gone way wrong

seancorfield20:09:20

@liamd We’re finding spec to be a better way to write our existing validation and transformation code since it abstracts out the specification part "as data". We don’t view it as "defensive programming".

liamd20:09:36

something like:

(let [my-x (s/conform my-x-spec x)]
  (if (= my-x :clojure.spec/invalid)
        (do-something)
        (proceed)))

hiredman20:09:32

like, where are your functions getting passed all these malformed inputs from?

liamd20:09:46

it's a wild world out there

hiredman20:09:52

at an edge, like a rest api, sure

hiredman20:09:44

if all your functions are exposed an all passed arbitrary input from the outside world, sure

liamd20:09:54

i was just thinking "couldn't we try to analyze the specs at compile time" and then i clicked spectrum

liamd20:09:02

looks neat

liamd20:09:26

but i mean isn't that the strength of type systems

hiredman20:09:44

type systems are going to need help at the edge too

liamd20:09:44

we can make assumptions about the data we're handling

liamd20:09:05

right, parsing json in any type system is sticky

hiredman20:09:24

type systems can only analyze data they see, random data coming from the edge isn't going to be seen at compile time

arohner20:09:25

@liamd no, spectrum can’t handle that. It can prove that you haven’t validated the data though

seancorfield20:09:31

Spec can also enforce data "structure" in a way that type systems cannot. It can specify valid ranges of numbers, relationships between parts of a data structure, patterns in strings and so on.

arohner20:09:19

@seancorfield there are workarounds for that in spectrum, in some cases

hiredman20:09:21

@seancorfield: that is just opening the door and waiting for someone to step in and start talking about dependent types

arohner20:09:43

because spectrum can assert that the spec gets validated at runtime

bfabry20:09:51

@liamd spec isn't going to start down the path of compile time checking other than for macros. at the end of the day clojure is a dynamic language. spec is an attempt to give us stronger confidence about what we're passing around at test and dev time using instrument and check. It also includes tools to validating the shape of things at runtime in production but imo I think that's more meant for things that are at high risk of being invalid (boundaries) rather than things that are likely fine

seancorfield20:09:53

@hiredman Hahaha… true… but we don’t have those in very many languages and almost none that are in common production usage!

seancorfield20:09:26

Besides, if folks want a type system, then go use a statically typed language. Clojure is not that language.

hiredman20:09:46

I think the big thing about spec is the language of terms (regular clojure expressions) and the language of descriptions (specs) are the same, so you can easily get access to specs at runtime, and write some interpreter for them to do whatever you want

liamd20:09:10

i see. so it should be though about as "a set of tools that helps mitigate the downsides of a dynamic language during your dev work flow"

seancorfield20:09:31

I don’t see those as "downsides" 🙂

hiredman20:09:42

calling them downsides reflects a particular stance that is not universal

liamd20:09:01

well surely spec was developed as a solution to some perceived problem?

liamd20:09:16

maybe i'm wrong in thinking that that problem is rooted in lack of typing

bfabry20:09:26

it also gives you more confidence than any static type systems I'm aware of because predicates can be arbitrary

richiardiandrea20:09:36

A very nice talk about this was the last one by Felleisen at Clojure/West

hiredman20:09:15

@liamd: static typing is a solution to a similar set of problems

hiredman20:09:12

but it is sort of like saying you have a problem P, and solutions A and B, and saying solution B is there to fix a lack of solution A

hiredman20:09:22

no, B is there to fix P

arohner20:09:54

there’s significant overlap between spec and static types, but neither completely subsumes the other

arohner20:09:18

there are bugs that can be caught by static types that are hard to catch w/ spec, and vice versa

bfabry20:09:34

static type systems give a lot of confidence about correct calls (probably more than spec) but provide no tools for regular data validation. they're also limited in what they can describe regarding correctness

hiredman20:09:19

in most typed languages, the language of terms and the language of types are distinct

hiredman20:09:26

types end up being kind of their own langauge that exists largely at compile time

hiredman20:09:48

specs are more like a database schema, they exist in the system and you can poke and fiddle with them live

liamd20:09:49

so due to them being just plain old clojure we can ingest the specs and do something with them

liamd20:09:07

is there a way to get the spec of something at run time?

hiredman20:09:31

things don't inherently have a spec

hiredman20:09:48

(another different from types)

arohner20:09:50

yes, you can get spec definitions at runtime

liamd20:09:50

we just have things and specs and we can smash them into each otehr if we want

arohner20:09:05

(s/form (s/spec ::foo))

liamd20:09:49

would it make sense to use specs as part of your logic? like if this conforms to my Person spec do one thing else do some other thing

liamd20:09:06

or i could see in web dev using them to validate incoming json

bfabry20:09:18

convenient dsl for expressing that kind of logic

liamd20:09:39

it'd be nice if explain could map back to a json error

liamd20:09:48

i don't know if api consumers want to deal with ;; val: 42 fails spec: ::suit predicate: #{:spade :heart :diamond :club}

arohner20:09:03

s/explain-data

liamd20:09:21

ah, there it is

liamd20:09:23

very nice

seancorfield20:09:12

@liamd As I noted above, at World Singles, we are using spec to replace our custom validation and transformation logic so we can have separate, comprehensive specifications we can show to and discuss with product owners / stakeholders, and it removes a lot of bespoke code.

seancorfield20:09:07

We are actually using spec for three layers: input, domain, persistence and conforming between them. The validate-and-conform aspect of spec is very nice.

liamd20:09:11

is that running only during dev or when you deploy?

seancorfield21:09:57

It’s a core part of our production code, calling conform.

seancorfield21:09:21

We also use instrument in testing, and we’re using some generative testing too (`check`). As I said above, we haven’t yet settled onto a final workflow for the latter since generative testing can be slow and we want "unit tests" to be fast (so we can run them easily while writing code).