This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-09-16
Channels
- # alda (1)
- # aws-lambda (1)
- # bangalore-clj (1)
- # beginners (70)
- # boot (24)
- # cider (1)
- # cljs-dev (167)
- # cljsjs (8)
- # cljsrn (17)
- # clojure (224)
- # clojure-android (7)
- # clojure-austin (8)
- # clojure-russia (17)
- # clojure-spec (120)
- # clojure-uk (46)
- # clojurescript (68)
- # community-development (198)
- # conf-proposals (1)
- # core-async (7)
- # cursive (6)
- # datomic (27)
- # dirac (19)
- # events (9)
- # hoplon (2)
- # jobs (1)
- # luminus (9)
- # off-topic (1)
- # om (281)
- # om-next (5)
- # onyx (50)
- # pedestal (1)
- # re-frame (19)
- # reagent (11)
- # ring-swagger (14)
- # slack-help (2)
- # spacemacs (1)
- # untangled (72)
- # yada (30)
I have a couple of timestamp string specs that require custom generators. I'm using a function to make the generators with the following function spec:
(s/fdef make-timestamp-gen
:args (s/cat :min-year pos-int?
:max-year pos-int?
:formatter #(instance? DateTimeFormatter))
:ret clojure.test.check.generators/generator?)
I'd really like to spec the function, but that would require adding org.clojure/test.check
to my non-dev dependencies, which feels a little yucky. Is there another way to ensure that the return value of the function is a generator? clojure.spec.gen
doesn't have a generator?
function.I have a data structure like this:
{:schema {:version "v1"}
:payload {:type "v1" ...}}
The schema version can either be "v1"
or "v2"
, and the contents of the payload vary based on this. Is there any good way to write a custom generator that will match up version and payload? I can write the conformer reasonably easily, but I can't think of a good way to do the generator.OK, I came up with something that feels only mildly hackish:
(defn- basic-gen []
(->> (s/gen (s/keys :req-un [::schema]))
(gen/fmap (fn [{{:keys [version]} :schema :as event}]
(let [payload-spec (payload-spec-for-schema version)]
(assoc event :payload (gen/generate (s/gen payload-spec))))))))
(s/def ::event (s/with-gen
(s/keys :req-un [::schema
::payload])
basic-gen))
Writing a custom generator / conformer pair has led me down a merry path of learning a lot more about spec.
Given a custom conformer, how to I make the s/explain
output meaningful? At the moment, my conformer just says:
user> (s/conform ::event intentionally-bogus-event)
:clojure.spec/invalid
user> (s/explain ::event intentionally-bogus-event)
val: :kpcs.spec.event/basic fails predicate: :clojure.spec/unknown
re: :clojure.spec/unknown, my guess is it is something related to the Specize protocol, the fall through Object case for that protocol uses unknown as the name for the predicate
@hiredman I remember reading about multi-spec now that you bring it up. I think you're right there!
@jmglov that’s a known issue that Rich and I are working on
just hasn’t been finished yet
I’ve instrumented the following spec and it catches errors except for the optional parameter. Any advice?
(s/def ::skip integer?)
(s/def ::limit integer?)
(s/def ::request-options (s/keys :un-opt [::skip ::limit]))
(s/fdef request!
:args (s/cat :field-path ::field-path
:query-data ::query-data
:options (s/? ::request-options)))
haha it's all good. it'd be nice if spec caught those, but I imagine having spec spec'd would actually be a rather difficult problem
I have spec specs but using them presents some problems
does anyone know of any guides that go beyond explaining how spec works and show how to integrate it into a real project?
i know, but how is it intended to be used? what do i do with my specs in an actual codebase?
yeah i found this
(defn person-name
[person]
{:pre [(s/valid? ::person person)]
:post [(s/valid? string? %)]}
(str (::first-name person) " " (::last-name person)))
is the idea that you have them as like accessories to your test suite or that they would run on deployed code?
I think spec can be split in to 1. descriptions of data 2. ways to use those descriptions
a. generating data that matches the description b. checking if data matches the description
and instrument will instrument your functions with checks (and some of those checks rely on generating data, so you really shouldn't instrument out side of testing)
instrument is very similar to sort of the classic idea of what you can do with assertions, assert all these expensive pre and post condition checks in development, then compile with assertions turned off for production
(not that I have ever seen a clojure build that turns off assertions, but I am sure they exist)
@liamd I think it’s mostly too early for standard patterns of usage to have settled down (and I don’t think there’s One True Way to use specs anyway). Have you read the http://clojure.org Guide? And watched Stu Halloway’s webcast demos of it? Also the Cognicast with Rich talking about the motivations for it is good listening.
i've listened to the cognicast and the generative testing was what sounded most useful to me
as an aside, instrument
doesn't do what would normally be considered :post checks. It only checks the :args section of an fdef
maybe it's because i'm thinking of spec as an answer to the lack of typing? are those related concerns?
At World Singles, we’re spec’ing data primarily and using conform
as part of our validation and transformation across application layer boundaries. We’re using instrument
to augment our testing. We’re starting to work with the generative side more at this point but we haven’t settled into a good flow for it yet.
@liamd Rich is pretty clear that spec != type system and should not be viewed the same way
so basically my thinking is: - in a statically typed system you can make some assumptions about your passed in args since they wouldn't compile otherwise - without static typing we can just make those assumptions anyway but we might have to debug and get ugly errors down the line - or we could do some defensive programming and handle incorrect args gracefully (by some standard, a nice error or whatever)
A side question: how folks use exercise
?
it's not a type system, but it's clearly geared at solving some of the same problems in a different way, as well as solving some problems that type systems don't. documentation, correctness, better errors
so does spec just make that third point easier? do a conform and then handle malformed inputs in every function without have to rewrite ugly validation logic everywhere
@liamd: 2 interesting resources for the intersection of spec and typing https://www.indiegogo.com/projects/typed-clojure-clojure-spec-auto-annotations#/ https://github.com/arohner/spectrum
@liamd instrument
and clojure.spec.test
are attempts to make us more confident about point 2
if you have a system where you are calling conform in every function on all your arguments, something has gone way wrong
@liamd We’re finding spec to be a better way to write our existing validation and transformation code since it abstracts out the specification part "as data". We don’t view it as "defensive programming".
something like:
(let [my-x (s/conform my-x-spec x)]
(if (= my-x :clojure.spec/invalid)
(do-something)
(proceed)))
@shaun-mahood i'll take a look
if all your functions are exposed an all passed arbitrary input from the outside world, sure
i was just thinking "couldn't we try to analyze the specs at compile time" and then i clicked spectrum
type systems can only analyze data they see, random data coming from the edge isn't going to be seen at compile time
@liamd no, spectrum can’t handle that. It can prove that you haven’t validated the data though
Spec can also enforce data "structure" in a way that type systems cannot. It can specify valid ranges of numbers, relationships between parts of a data structure, patterns in strings and so on.
@seancorfield there are workarounds for that in spectrum, in some cases
@seancorfield: that is just opening the door and waiting for someone to step in and start talking about dependent types
@liamd spec isn't going to start down the path of compile time checking other than for macros. at the end of the day clojure is a dynamic language. spec is an attempt to give us stronger confidence about what we're passing around at test and dev time using instrument
and check
. It also includes tools to validating the shape of things at runtime in production but imo I think that's more meant for things that are at high risk of being invalid (boundaries) rather than things that are likely fine
@hiredman Hahaha… true… but we don’t have those in very many languages and almost none that are in common production usage!
Besides, if folks want a type system, then go use a statically typed language. Clojure is not that language.
I think the big thing about spec is the language of terms (regular clojure expressions) and the language of descriptions (specs) are the same, so you can easily get access to specs at runtime, and write some interpreter for them to do whatever you want
i see. so it should be though about as "a set of tools that helps mitigate the downsides of a dynamic language during your dev work flow"
I don’t see those as "downsides" 🙂
it also gives you more confidence than any static type systems I'm aware of because predicates can be arbitrary
A very nice talk about this was the last one by Felleisen at Clojure/West
but it is sort of like saying you have a problem P, and solutions A and B, and saying solution B is there to fix a lack of solution A
there’s significant overlap between spec and static types, but neither completely subsumes the other
there are bugs that can be caught by static types that are hard to catch w/ spec, and vice versa
static type systems give a lot of confidence about correct calls (probably more than spec) but provide no tools for regular data validation. they're also limited in what they can describe regarding correctness
in most typed languages, the language of terms and the language of types are distinct
specs are more like a database schema, they exist in the system and you can poke and fiddle with them live
so due to them being just plain old clojure we can ingest the specs and do something with them
would it make sense to use specs as part of your logic? like if this conforms to my Person spec do one thing else do some other thing
i don't know if api consumers want to deal with ;; val: 42 fails spec: ::suit predicate: #{:spade :heart :diamond :club}
@liamd As I noted above, at World Singles, we are using spec to replace our custom validation and transformation logic so we can have separate, comprehensive specifications we can show to and discuss with product owners / stakeholders, and it removes a lot of bespoke code.
We are actually using spec for three layers: input, domain, persistence and conforming between them. The validate-and-conform aspect of spec is very nice.
It’s a core part of our production code, calling conform
.
We also use instrument
in testing, and we’re using some generative testing too (`check`). As I said above, we haven’t yet settled onto a final workflow for the latter since generative testing can be slow and we want "unit tests" to be fast (so we can run them easily while writing code).