Clojurians
#clojure-spec
<
2016-10-03
>

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

gfredericks00:10:36

that sounds pretty weird

jrheard01:10:48

i think tomorrow i’m just gonna have to learn how to use checkout dependencies and start adding printfs to spec.test code so i can see what’s going on in there

bhagany02:10:08

@jrheard fwiw, I’m eager to hear what you learn. I tried getting doo to run my cljs spec tests a few weeks ago, and hit a wall.

jrheard04:10:42

well, for starters i think i’ve found a small issue with the cljs.spec documentation

jrheard04:10:10

https://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/spec/test.cljc#L210 says that options maps passed to (s/check) should look like {:clojure.spec.test.check/opts {:num-tests 100}}

jrheard04:10:20

but actually, they should look like {:clojure.test.check/opts {:num-tests 100}}

jrheard04:10:24

i haven’t managed to reproduce this [] return value issue using my local checkout of clojurescript, so maybe this behavior has been fixed on master? will poke at it some more in the morning

seancorfield04:10:46

Are you sure about the options map @jrheard ? When I was trying that on Clojure, it definitely needed to be :clojure.spec.test.check/opts (even tho’ no such namespace exists).

seancorfield04:10:20

If it really is the latter, as you say, then that’s a bug in my opinion — ClojureScript should follow Clojure there I think?

decoursin06:10:04

How to override keys in (s/merge ..)? My failing attempt:

(s/def :my-ns/a string?)
(s/def :my-other-ns/a int?)
(gen/generate (s/gen (s/merge (s/keys :req-un [:my-ns/a])
                              (s/keys :req-un [:my-other-ns/a]))))
=> ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.  clojure.core/ex-info (core.clj:4725)

misha07:10:51

@decoursin: what do you expect merged spec to be?

misha07:10:01

"Int or string"?

decoursin07:10:22

Either one, make a choice. Always take the first or always take the second.

decoursin07:10:01

probably the same way clojure.core/merge works which is the second (or last one.)

decoursin07:10:33

(merge {:a 3} {:a 4})
=> {:a 4}

misha07:10:26

Takes map-validating specs (e.g. 'keys' specs) and
returns a spec that returns a conformed map satisfying all of the
specs.  Unlike 'and', merge can generate maps satisfying the
union of the predicates.
does not sound like it can satisfy int? and string? with a same value on a single key

bfabry07:10:13

I think in terms of usefulness the way it's behaving atm is probably best, though I could see an argument for it behaving like core.merge

decoursin07:10:08

Yeah I think it should behave like core.merge. I'd rather see that what @misha described to be called like union or something

decoursin08:10:34

How could I do this then? Have one key override another

misha08:10:51

what is your use case? or rather "problem"

mpenet11:10:17

is there a way to validate against a (dynamic) subset of keys in a map? ex I define an exhaustive spec for a map (with mixed req/opt), and at runtime I want to be able to validate against a subset of it

mpenet11:10:43

a bit in the flavor of s/merge, I'd like s/select-keys (kind of)

misha11:10:38

you can give names to those subsets, and just merge those in uber-map-spec. no?

mpenet11:10:05

I dont want to define as many subsets are they are key combos no

mpenet11:10:59

ex with Schema you can just do (since map specs are just Maps) :

(defn validate-subset
  [schema value]
  (-> (select-keys schema (keys value))
      (s/validate value)))

mpenet11:10:22

seems like s/select-keys could be a nice addition actually, thoughts @alexmiller ?

mpenet11:10:41

just putting every key as optional is not an option either

misha11:10:14

given spec's "global" nature, I'd go and name all the combos. also multi-spec might be a good fit too

mpenet11:10:46

that's a lot of combos for a 20 key map for instance ... it'd be horrible

mpenet11:10:10

I'll check multi-spec, but my gut feeling it doesn't fit that problem either

misha11:10:44

depending on how you'd define combos out of those 20

misha11:10:32

you might have only 2-3 "domain-valid" combos.

mpenet11:10:34

doesn't matter

mpenet11:10:44

in my case the user can update any of them and I cannot pass all of it

mpenet11:10:09

(there are concurrent update issues for the "pass the whole thing" case)

misha11:10:27

not enough information then

mpenet11:10:48

it's a common enough problem imho, any REST api would hit it for partial update for instance

mpenet11:10:11

the least horrible (yet it sucks) imho would be to call valid? against every key in the set but yuk. Or just have a s/keys spec with all as optional for updates, but that's a lot of duplication I'd like to avoid

misha11:10:55

then probably I'd have 20 different specs defined, and constructed map spec on demand based on incoming map's keys

misha11:10:55

20 for possible keys, + maybe some for key combinations, like "if that one present, those 2 are required".

misha11:10:12

a collection of specs for possible keys should not necessarily be a ready to use map spec.

mpenet11:10:13

not going to do that, that's truly awful, not to mention for 20 fields (the example here but some of our records have more than this) that's a large number of specs (a lot more than 20)

misha11:10:07

not everything can be solved with oneliners :kappa:

mpenet11:10:38

it's not the point, you don't have a solution to this, no biggie

mpenet11:10:02

At the moment there's no elegant way to solve this with spec.

misha12:10:56

@mpenet how about this?

(s/def :foo/bar int?)
(s/def :foo/baz string?)

(def schema #{:foo/bar :foo/baz})

(defn validate-subset
  [schema m]
  (let [ks (filterv schema (keys m))]
    (s/valid? (s/keys :req `[[email protected]]) m)))
(validate-subset schema {:foo/bar 1 :foo/baz "y"})
=> true
(validate-subset schema {:foo/baz "y"})
=> true
(validate-subset schema {:foo/baz 1})
=> false

mpenet12:10:43

That's more or less what I mentioned earlier, but I dont like it. Having 1 separate spec with all opt keys is probably nicer

danielstockton13:10:26

Is there a way to instrument everything at once, rather than individual functions?

alexmiller13:10:36

Just call instrument with no args

danielstockton13:10:19

Is there a good way to hook this into the test runner? I imagine it can be quite helpful to know not only what tests failed but which functions got unexpected data.

mpenet13:10:25

I think you can just call instrument at top level of your test

danielstockton13:10:46

I suppose, i'd have to do it in every namespace

danielstockton13:10:33

Might also be useful to turn it on whenever starting lein repl or in a dev environment

mpenet14:10:03

@alexmiller (s/valid? #{false} false) => false that's what you mention on github?

mpenet14:10:35

it does look like a bug from the outside (without considering/knowing how it's implemented under the hood)

mpenet14:10:12

I guess its (#{false true} false)

alexmiller14:10:16

Yes that's what I meant. It's not a bug, just a consequence of using sets as specs

alexmiller14:10:38

Sets with falsey values won't work

alexmiller14:10:19

So don't do that

alexmiller14:10:52

You've used a function that returns false for what you consider to be a valid value

mpenet14:10:06

yep, got it

mpenet14:10:31

actually about my s/select-keys proposal earlier, why not making s/keys spec an c.l.Associative instance and allow us to compose this stuff with normal core fn?

mpenet14:10:18

maybe it's crazy talk

alexmiller15:10:27

the problem with that is that we capture the key set for describe and form, so you would lose that ability

alexmiller15:10:19

that’s the reason it’s a macro and not a function that takes data now

alexmiller15:10:06

@jrheard no, haven’t seen that

mlimotte15:10:58

hi. I'm just starting to play with clojure.spec. I'm ok with some basic specs and validation that I've tried. But having trouble with even a trivial example of specing a higher-order fn. Here's what I'm seeing:

(def my-spec (s/fspec :ret string?))
=> #'user/my-spec
(s/conform my-spec (fn [j] (str j)))
IllegalArgumentException No implementation of method: :specize* of protocol: #'clojure.spec/Specize found for class: nil  clojure.core/-cache-protocol-fn (core_deftype.clj:568)

alexmiller16:10:31

(s/fdef my-spec :ret string?) will be better for you right now

alexmiller16:10:23

sorry, I misread that first def as s/def, let me read again

alexmiller16:10:44

so the issue here is that to verify that the function you’ve passed is valid, it will generate args based on the :args spec for the function and invoke it

alexmiller16:10:50

but you’ve passed no args spec

mlimotte16:10:03

true. I was going with a simple example -- just validating that it is a fn that returns a string. I did try with args before, and was getting different errors, so I was trying to simplify the example.

mlimotte16:10:08

i'll try w/ args again

alexmiller16:10:17

although I am seeing some weird stuff on this path

mlimotte16:10:11

(s/def my-spec (s/fspec :args (s/tuple integer?) :ret string?)) => user/my-spec (s/conform my-spec (fn [j] (str j))) IllegalArgumentException No implementation of method: :specize* of protocol: #'clojure.spec/Specize found for class: nil clojure.core/-cache-protocol-fn (core_deftype.clj:568)

mlimotte16:10:41

:disappointed: same thing, even with args.

alexmiller16:10:00

ah, so conform takes a qualified keyword or symbol

alexmiller16:10:20

(s/conform `my-spec (fn [j] (str j)))

alexmiller16:10:39

will fully-qualify my-spec

alexmiller16:10:30

@jrheard the idea in both of those is to pass a map of options through to test.check - I think the underlying option keys differ in clj vs cljs test.check

alexmiller16:10:06

so there may be a disconnect between clj vs cljs and docs here

jrheard16:10:20

yeah, i think the cljs docs need to be updated to reflect the disconnect - https://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/spec/test.cljc#L211

jrheard16:10:38

i was using :clojure.spec.test.check/opts as the docs recommend, but nothing was happening

alexmiller16:10:41

I suspect so - prob should file a jira (and if you like a patch!)

mlimotte16:10:52

ahh! it does say that in the spec guide! Maybe it would be helpful for the conform and valid? docstring(s) could also highlight that requirement.

jrheard16:10:53

sounds good, it’ll be my first of both of those :slightly_smiling_face: time to go read the contributing guidelines again

alexmiller16:10:26

@mlimotte seems like there should be a better error message in this case

alexmiller16:10:39

I will log that and add a patc hfor it

mlimotte16:10:47

yep. that would help. here's another question. What's a recommended way to spec the requirement that the fn should be a one-arg fn? I tried:

(s/def my-spec (s/fspec :args (s/tuple identity) :ret string?))
(s/def my-spec (s/fspec :args #(= (count %) 1) :ret string?))

mlimotte16:10:27

which results in

ExceptionInfo Unable to construct gen at: [0] for: identity  clojure.core/ex-info (core.clj:4617)
ExceptionInfo Unable to construct gen at: [] for: [email protected]  clojure.core/ex-info (core.clj:4617)

alexmiller16:10:27

(s/def my-spec (s/fspec :args (s/cat :j any?) :ret string?)

jrheard16:10:40

(i’ve made a jira account, username jrheard - alex, are you the right person to bug to add whatever necessary permissions to that account, or should i go bother someone else?)

alexmiller16:10:05

you can add tickets with the starting permissions

mlimotte16:10:29

what is any?, AFAICT that is not in clojure.core nor clojure.spec

alexmiller16:10:35

it’s in core

lvh16:10:36

It’s new in Clojure

alexmiller16:10:42

it’s just (constantly true)

lvh16:10:08

hm; I figured it might take only 1 arg

alexmiller16:10:26

actually it does

alexmiller16:10:46

I was more sketching than saying the actual def :)

alexmiller16:10:54

(defn any?
  "Returns true given any argument."
  {:tag Boolean
   :added "1.9"}
  [x] true)

lvh16:10:26

ah, gotcha :smile:

mlimotte16:10:29

i see. I'm using clojure-future-spec w/ clojure 1.8, so this works: (s/def my-spec (s/fspec :args (s/cat :j clojure.future/any?) :ret string?))

mlimotte16:10:18

as does: (s/def my-spec (s/fspec :args (s/tuple clojure.future/any?) :ret string?)) Was your choice of s/cat over s/tuple just stylistic, or is there something more significant?

alexmiller16:10:59

@jrheard I added edit permissions for you as well in jira

jrheard16:10:49

opened CLJS-1808 , will submit a patch later today

alexmiller16:10:56

@mlimotte tuple is fine too. we usually use regex ops to describe args. Args are effectively syntax and that’s what regex ops in spec are the best match for.

alexmiller16:10:45

one benefit of cat is that you will get named parts for conforming the parts whereas you will get just indexes for tuple

alexmiller16:10:19

so that affects both the conform result (which you’ll see in the :fn spec) and the explain errors

alexmiller16:10:43

generally I find being able to tie the spec to the arg name helpful

mlimotte16:10:52

got it. thanks for the help.

jrheard17:10:52

@alexmiller - i just added a couple patches to the ticket. is there anything else i need to do, or should i just wait and expect to get a response at some point in the future? i’m asking you this not because i’m in an incredible hurry to have someone immediately look at this jira issue this very second; i just want to make sure i’m not missing some “notify the maintainers” step before i close all these tabs :slightly_smiling_face: thanks!

alexmiller17:10:33

For cljs-1808

mlimotte17:10:39

Could there be a problem with s/tuple? I have the code below, which bombs w/ the error below, but if I change the '(s/tuple integer?)` to (s/cat :a integer?) than it works.

(defn myfn [a] (str a))
(s/fdef myfn :args (s/tuple integer?) :ret string?)
(stest/instrument `myfn)
(myfn 1)
I get
=> #'user/myfn
=> user/myfn
=> [user/myfn]
ExceptionInfo Call to #'user/myfn did not conform to spec:
val: (1) fails at: [:args] predicate: vector?
:clojure.spec/args  (1)
:clojure.spec/failure  :instrument
:clojure.spec.test/caller  {:file "form-init1342783443404146057.clj", :line 4, :var-scope user/eval30918}
  clojure.core/ex-info (core.clj:4617)

bfabry17:10:58

@mlimotte s/tuple specifically expects its input to be a vector, the :args list is not a vector

bfabry17:10:28

s/cat just validates a sequence, which the :args list is

mlimotte17:10:42

i see. in retrospect, kind of obvious from the error message. i should have gotten that.

dnolen17:10:04

@jrheard just assign it to me I will look at it when I have time

jrheard17:10:11

great, will do, thanks!

tokenshift21:10:13

Can s/conform handle coercion? For example, could I have an s/def that defines a ::uuid to be either a (valid) string or a java.util.UUID, and always conforms it to a java.util.UUID?

lvh21:10:22

tokenshift Check out “conformer"

tokenshift21:10:50

I’d pass a conformer (the result of s/conformer) to s/def, e.g. (s/def ::foo (s/conformer #(do-stuff %)))?

tokenshift21:10:04

Answered my own question; yep, that works

tokenshift21:10:08

Now I have to decide if that’s a good pattern, or if it’d be better to keep input coercion completely separate from specs

lvh21:10:27

I think it’s fine to conform to a specific object

lvh21:10:53

conforming is not something that you want in your hot loop though IIUC; it’s a measurable performance impac

lvh21:10:04

but at the edges? sure; you’re gonna do that anyway, might as well have it be declarative

tokenshift21:10:22

Hmm, that’s a good point

tokenshift21:10:57

If it’s only at the edges, then I can assume the input will be a string (for a web service, at least)

tokenshift21:10:17

Though if the backend requires a UUID, I’d probably still want to coerce it as early as possible.

tokenshift21:10:56

I guess if the coercion is lossless, it’s safe; but if it’s lossy, I’d want to have an explicit coercion step.

tokenshift21:10:39

Thanks for the help!