Fork me on GitHub
#clojure-spec
<
2016-10-03
>
gfredericks00:10:36

that sounds pretty weird

jrheard01:10:48

i think tomorrow i’m just gonna have to learn how to use checkout dependencies and start adding printfs to spec.test code so i can see what’s going on in there

bhagany02:10:08

@jrheard fwiw, I’m eager to hear what you learn. I tried getting doo to run my cljs spec tests a few weeks ago, and hit a wall.

jrheard04:10:42

well, for starters i think i’ve found a small issue with the cljs.spec documentation

jrheard04:10:10

https://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/spec/test.cljc#L210 says that options maps passed to (s/check) should look like {:clojure.spec.test.check/opts {:num-tests 100}}

jrheard04:10:20

but actually, they should look like {:clojure.test.check/opts {:num-tests 100}}

jrheard04:10:24

i haven’t managed to reproduce this [] return value issue using my local checkout of clojurescript, so maybe this behavior has been fixed on master? will poke at it some more in the morning

seancorfield04:10:46

Are you sure about the options map @jrheard ? When I was trying that on Clojure, it definitely needed to be :clojure.spec.test.check/opts (even tho’ no such namespace exists).

seancorfield04:10:20

If it really is the latter, as you say, then that’s a bug in my opinion — ClojureScript should follow Clojure there I think?

decoursin06:10:04

How to override keys in (s/merge ..)? My failing attempt:

(s/def :my-ns/a string?)
(s/def :my-other-ns/a int?)
(gen/generate (s/gen (s/merge (s/keys :req-un [:my-ns/a])
                              (s/keys :req-un [:my-other-ns/a]))))
=> ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.  clojure.core/ex-info (core.clj:4725)

misha07:10:51

@decoursin: what do you expect merged spec to be?

misha07:10:01

"Int or string"?

decoursin07:10:22

Either one, make a choice. Always take the first or always take the second.

decoursin07:10:01

probably the same way clojure.core/merge works which is the second (or last one.)

decoursin07:10:33

(merge {:a 3} {:a 4})
=> {:a 4}

misha07:10:26

Takes map-validating specs (e.g. 'keys' specs) and
returns a spec that returns a conformed map satisfying all of the
specs.  Unlike 'and', merge can generate maps satisfying the
union of the predicates.
does not sound like it can satisfy int? and string? with a same value on a single key

bfabry07:10:13

I think in terms of usefulness the way it's behaving atm is probably best, though I could see an argument for it behaving like core.merge

decoursin07:10:08

Yeah I think it should behave like core.merge. I'd rather see that what @misha described to be called like union or something

decoursin08:10:34

How could I do this then? Have one key override another

misha08:10:51

what is your use case? or rather "problem"

mpenet11:10:17

is there a way to validate against a (dynamic) subset of keys in a map? ex I define an exhaustive spec for a map (with mixed req/opt), and at runtime I want to be able to validate against a subset of it

mpenet11:10:43

a bit in the flavor of s/merge, I'd like s/select-keys (kind of)

misha11:10:38

you can give names to those subsets, and just merge those in uber-map-spec. no?

mpenet11:10:05

I dont want to define as many subsets are they are key combos no

mpenet11:10:59

ex with Schema you can just do (since map specs are just Maps) :

(defn validate-subset
  [schema value]
  (-> (select-keys schema (keys value))
      (s/validate value)))

mpenet11:10:22

seems like s/select-keys could be a nice addition actually, thoughts @alexmiller ?

mpenet11:10:41

just putting every key as optional is not an option either

misha11:10:14

given spec's "global" nature, I'd go and name all the combos. also multi-spec might be a good fit too

mpenet11:10:46

that's a lot of combos for a 20 key map for instance ... it'd be horrible

mpenet11:10:10

I'll check multi-spec, but my gut feeling it doesn't fit that problem either

misha11:10:44

depending on how you'd define combos out of those 20

misha11:10:32

you might have only 2-3 "domain-valid" combos.

mpenet11:10:34

doesn't matter

mpenet11:10:44

in my case the user can update any of them and I cannot pass all of it

mpenet11:10:09

(there are concurrent update issues for the "pass the whole thing" case)

misha11:10:27

not enough information then

mpenet11:10:48

it's a common enough problem imho, any REST api would hit it for partial update for instance

mpenet11:10:11

the least horrible (yet it sucks) imho would be to call valid? against every key in the set but yuk. Or just have a s/keys spec with all as optional for updates, but that's a lot of duplication I'd like to avoid

misha11:10:55

then probably I'd have 20 different specs defined, and constructed map spec on demand based on incoming map's keys

misha11:10:55

20 for possible keys, + maybe some for key combinations, like "if that one present, those 2 are required".

misha11:10:12

a collection of specs for possible keys should not necessarily be a ready to use map spec.

mpenet11:10:13

not going to do that, that's truly awful, not to mention for 20 fields (the example here but some of our records have more than this) that's a large number of specs (a lot more than 20)

misha11:10:07

not everything can be solved with oneliners kappa

mpenet11:10:38

it's not the point, you don't have a solution to this, no biggie

mpenet11:10:02

At the moment there's no elegant way to solve this with spec.

misha12:10:56

@mpenet how about this?

(s/def :foo/bar int?)
(s/def :foo/baz string?)

(def schema #{:foo/bar :foo/baz})

(defn validate-subset
  [schema m]
  (let [ks (filterv schema (keys m))]
    (s/valid? (s/keys :req `[~@ks]) m)))
(validate-subset schema {:foo/bar 1 :foo/baz "y"})
=> true
(validate-subset schema {:foo/baz "y"})
=> true
(validate-subset schema {:foo/baz 1})
=> false

mpenet12:10:43

That's more or less what I mentioned earlier, but I dont like it. Having 1 separate spec with all opt keys is probably nicer

danielstockton13:10:26

Is there a way to instrument everything at once, rather than individual functions?

Alex Miller (Clojure team)13:10:36

Just call instrument with no args

danielstockton13:10:19

Is there a good way to hook this into the test runner? I imagine it can be quite helpful to know not only what tests failed but which functions got unexpected data.

mpenet13:10:25

I think you can just call instrument at top level of your test

danielstockton13:10:46

I suppose, i'd have to do it in every namespace

danielstockton13:10:33

Might also be useful to turn it on whenever starting lein repl or in a dev environment

mpenet14:10:03

@alexmiller (s/valid? #{false} false) => false that's what you mention on github?

mpenet14:10:35

it does look like a bug from the outside (without considering/knowing how it's implemented under the hood)

mpenet14:10:12

I guess its (#{false true} false)

Alex Miller (Clojure team)14:10:16

Yes that's what I meant. It's not a bug, just a consequence of using sets as specs

Alex Miller (Clojure team)14:10:38

Sets with falsey values won't work

Alex Miller (Clojure team)14:10:52

You've used a function that returns false for what you consider to be a valid value

mpenet14:10:06

yep, got it

mpenet14:10:31

actually about my s/select-keys proposal earlier, why not making s/keys spec an c.l.Associative instance and allow us to compose this stuff with normal core fn?

mpenet14:10:18

maybe it's crazy talk

Alex Miller (Clojure team)15:10:27

the problem with that is that we capture the key set for describe and form, so you would lose that ability

Alex Miller (Clojure team)15:10:19

that’s the reason it’s a macro and not a function that takes data now

mlimotte15:10:58

hi. I'm just starting to play with clojure.spec. I'm ok with some basic specs and validation that I've tried. But having trouble with even a trivial example of specing a higher-order fn. Here's what I'm seeing:

(def my-spec (s/fspec :ret string?))
=> #'user/my-spec
(s/conform my-spec (fn [j] (str j)))
IllegalArgumentException No implementation of method: :specize* of protocol: #'clojure.spec/Specize found for class: nil  clojure.core/-cache-protocol-fn (core_deftype.clj:568)

Alex Miller (Clojure team)16:10:31

(s/fdef my-spec :ret string?) will be better for you right now

Alex Miller (Clojure team)16:10:23

sorry, I misread that first def as s/def, let me read again

Alex Miller (Clojure team)16:10:44

so the issue here is that to verify that the function you’ve passed is valid, it will generate args based on the :args spec for the function and invoke it

Alex Miller (Clojure team)16:10:50

but you’ve passed no args spec

mlimotte16:10:03

true. I was going with a simple example -- just validating that it is a fn that returns a string. I did try with args before, and was getting different errors, so I was trying to simplify the example.

mlimotte16:10:08

i'll try w/ args again

Alex Miller (Clojure team)16:10:17

although I am seeing some weird stuff on this path

mlimotte16:10:11

(s/def my-spec (s/fspec :args (s/tuple integer?) :ret string?)) => user/my-spec (s/conform my-spec (fn [j] (str j))) IllegalArgumentException No implementation of method: :specize* of protocol: #'clojure.spec/Specize found for class: nil clojure.core/-cache-protocol-fn (core_deftype.clj:568)

mlimotte16:10:41

😞 same thing, even with args.

Alex Miller (Clojure team)16:10:00

ah, so conform takes a qualified keyword or symbol

Alex Miller (Clojure team)16:10:20

(s/conform `my-spec (fn [j] (str j)))

Alex Miller (Clojure team)16:10:39

will fully-qualify my-spec

Alex Miller (Clojure team)16:10:30

@jrheard the idea in both of those is to pass a map of options through to test.check - I think the underlying option keys differ in clj vs cljs test.check

Alex Miller (Clojure team)16:10:06

so there may be a disconnect between clj vs cljs and docs here

jrheard16:10:20

yeah, i think the cljs docs need to be updated to reflect the disconnect - https://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/spec/test.cljc#L211

jrheard16:10:38

i was using :clojure.spec.test.check/opts as the docs recommend, but nothing was happening

Alex Miller (Clojure team)16:10:41

I suspect so - prob should file a jira (and if you like a patch!)

mlimotte16:10:52

ahh! it does say that in the spec guide! Maybe it would be helpful for the conform and valid? docstring(s) could also highlight that requirement.

jrheard16:10:53

sounds good, it’ll be my first of both of those 🙂 time to go read the contributing guidelines again

Alex Miller (Clojure team)16:10:26

@mlimotte seems like there should be a better error message in this case

Alex Miller (Clojure team)16:10:39

I will log that and add a patc hfor it

mlimotte16:10:47

yep. that would help. here's another question. What's a recommended way to spec the requirement that the fn should be a one-arg fn? I tried:

(s/def my-spec (s/fspec :args (s/tuple identity) :ret string?))
(s/def my-spec (s/fspec :args #(= (count %) 1) :ret string?))

mlimotte16:10:27

which results in

ExceptionInfo Unable to construct gen at: [0] for: identity  clojure.core/ex-info (core.clj:4617)
ExceptionInfo Unable to construct gen at: [] for: clojure.spec$spec_impl$reify__1451@4e6417a0  clojure.core/ex-info (core.clj:4617)

Alex Miller (Clojure team)16:10:27

(s/def my-spec (s/fspec :args (s/cat :j any?) :ret string?)

jrheard16:10:40

(i’ve made a jira account, username jrheard - alex, are you the right person to bug to add whatever necessary permissions to that account, or should i go bother someone else?)

Alex Miller (Clojure team)16:10:05

you can add tickets with the starting permissions

mlimotte16:10:29

what is any?, AFAICT that is not in clojure.core nor clojure.spec

lvh16:10:36

It’s new in Clojure

Alex Miller (Clojure team)16:10:42

it’s just (constantly true)

lvh16:10:08

hm; I figured it might take only 1 arg

Alex Miller (Clojure team)16:10:46

I was more sketching than saying the actual def :)

Alex Miller (Clojure team)16:10:54

(defn any?
  "Returns true given any argument."
  {:tag Boolean
   :added "1.9"}
  [x] true)

lvh16:10:26

ah, gotcha 😄

mlimotte16:10:29

i see. I'm using clojure-future-spec w/ clojure 1.8, so this works: (s/def my-spec (s/fspec :args (s/cat :j clojure.future/any?) :ret string?))

mlimotte16:10:18

as does: (s/def my-spec (s/fspec :args (s/tuple clojure.future/any?) :ret string?)) Was your choice of s/cat over s/tuple just stylistic, or is there something more significant?

Alex Miller (Clojure team)16:10:59

@jrheard I added edit permissions for you as well in jira

jrheard16:10:49

opened CLJS-1808 , will submit a patch later today

Alex Miller (Clojure team)16:10:56

@mlimotte tuple is fine too. we usually use regex ops to describe args. Args are effectively syntax and that’s what regex ops in spec are the best match for.

Alex Miller (Clojure team)16:10:45

one benefit of cat is that you will get named parts for conforming the parts whereas you will get just indexes for tuple

Alex Miller (Clojure team)16:10:19

so that affects both the conform result (which you’ll see in the :fn spec) and the explain errors

Alex Miller (Clojure team)16:10:43

generally I find being able to tie the spec to the arg name helpful

mlimotte16:10:52

got it. thanks for the help.

jrheard17:10:52

@alexmiller - i just added a couple patches to the ticket. is there anything else i need to do, or should i just wait and expect to get a response at some point in the future? i’m asking you this not because i’m in an incredible hurry to have someone immediately look at this jira issue this very second; i just want to make sure i’m not missing some “notify the maintainers” step before i close all these tabs 🙂 thanks!

mlimotte17:10:39

Could there be a problem with s/tuple? I have the code below, which bombs w/ the error below, but if I change the '(s/tuple integer?)` to (s/cat :a integer?) than it works.

(defn myfn [a] (str a))
(s/fdef myfn :args (s/tuple integer?) :ret string?)
(stest/instrument `myfn)
(myfn 1)
I get
=> #'user/myfn
=> user/myfn
=> [user/myfn]
ExceptionInfo Call to #'user/myfn did not conform to spec:
val: (1) fails at: [:args] predicate: vector?
:clojure.spec/args  (1)
:clojure.spec/failure  :instrument
:clojure.spec.test/caller  {:file "form-init1342783443404146057.clj", :line 4, :var-scope user/eval30918}
  clojure.core/ex-info (core.clj:4617)

bfabry17:10:58

@mlimotte s/tuple specifically expects its input to be a vector, the :args list is not a vector

bfabry17:10:28

s/cat just validates a sequence, which the :args list is

mlimotte17:10:42

i see. in retrospect, kind of obvious from the error message. i should have gotten that.

dnolen17:10:04

@jrheard just assign it to me I will look at it when I have time

jrheard17:10:11

great, will do, thanks!

tokenshift21:10:13

Can s/conform handle coercion? For example, could I have an s/def that defines a ::uuid to be either a (valid) string or a java.util.UUID, and always conforms it to a java.util.UUID?

lvh21:10:22

tokenshift Check out “conformer"

tokenshift21:10:50

I’d pass a conformer (the result of s/conformer) to s/def, e.g. (s/def ::foo (s/conformer #(do-stuff %)))?

tokenshift21:10:04

Answered my own question; yep, that works

tokenshift21:10:08

Now I have to decide if that’s a good pattern, or if it’d be better to keep input coercion completely separate from specs

lvh21:10:27

I think it’s fine to conform to a specific object

lvh21:10:53

conforming is not something that you want in your hot loop though IIUC; it’s a measurable performance impac

lvh21:10:04

but at the edges? sure; you’re gonna do that anyway, might as well have it be declarative

tokenshift21:10:22

Hmm, that’s a good point

tokenshift21:10:57

If it’s only at the edges, then I can assume the input will be a string (for a web service, at least)

tokenshift21:10:17

Though if the backend requires a UUID, I’d probably still want to coerce it as early as possible.

tokenshift21:10:56

I guess if the coercion is lossless, it’s safe; but if it’s lossy, I’d want to have an explicit coercion step.

tokenshift21:10:39

Thanks for the help!