Fork me on GitHub
#clojure-spec
<
2016-09-01
>
sattvik00:09:11

Curious. When working with higher-order functions, instrumentation will result in the execution of a function argument repeatedly. Which, isn’t great when the function has side effects.

gfredericks00:09:12

you probably need to stub things?

sattvik00:09:42

Eh… it’s kind of a weird situation. I have a function that performs some assertions for testing. It is invoked by the function under test. Under normal circumstances, my function with assertions is only invoked once per test. However, as the function under test has its parameters specified, the conformance checker ends up running my test function scores of times.

sattvik00:09:49

I think one workaround is to instrument the function under test with less demanding specification (something like fn?). In the end, I just use a mutable cell to hold the data I want to check and have my test code update that cell. I can count on the last invocation of the function to be the one I care about.

gfredericks00:09:15

you also get to override specs

gfredericks00:09:18

so maybe you could test it in two ways

gfredericks00:09:40

once where you have the function doing the assertion and you make sure it gets called only once, and another test without that where the spec gets exercised

sattvik00:09:54

Well, doesn’t overriding specs only be done when using check? I am not doing that. I am only instrumenting for input argument validation.

gfredericks00:09:07

that might be true

seancorfield01:09:14

Is there any sort of standard pattern for spec’ing a function that curries its arguments?

(defn my-fn
  ([a] (fn [b] (my-fn a b)))
  ([a b] … a … b …))

seancorfield01:09:01

Specifically, how to handle the fact that :ret depends on whether you provided one or two arguments...

sattvik01:09:48

Hmm… good question. My guess is that you would use alt for the different return types and then an fn to tie them together to the passed args. I’m not sure if there is a better way, though.

seancorfield02:09:00

@gfredericks all of a sudden, I’m getting this exception from test.check — any pointers? clojure.lang.Compiler$CompilerException: java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn, compiling:(clojure/test/check/clojure_test.cljc:95:1)

seancorfield02:09:43

@sattvik yeah… that feels kinda ugly tho’… I’m hoping there’s a cleaner way...

sattvik02:09:21

Hmm… I wonder if it is possible to do (s/or :unary (s/fspec …) :binary (s/fspec …))

gfredericks02:09:38

@seancorfield that smells like this one thing hold on

seancorfield02:09:32

Perfect! Thanks… I’ll add that Leiningen setting!

seancorfield02:09:49

(I’m expanding the clojure.spec coverage for java.jdbc and ran into that for the first time)

gfredericks02:09:22

this is all my fault for not having a new test.check release ready yet

seancorfield04:09:29

Don't sweat it. test.check is already great!

seancorfield04:09:49

Making progress with spec'ing java.jdbc. Some good, some frustrating simple_smile

sveri07:09:09

Hi, I have trouble specing & args. What I want is (f [a1 a2 & ax]) where ax can be zero or more (s/*) and it must have an even number of elements. This is what I thought should work: (s/* (s/and any? #(even? (count %)))) But it does not, instead it works for (f 1 2 3) but not for (f 1 2 3 4)

sveri08:09:28

Is it because spec takes the args vector apart and checks every element of the vector?

Alex Miller (Clojure team)12:09:25

The s/and there is in the spot for each element of the arg vector so that doesn't make sense

Alex Miller (Clojure team)12:09:14

(s/& (s/* any?) #(even (count? %)))

Alex Miller (Clojure team)12:09:53

Is one solution - s/& applies a predicate in addition to a regex match

Alex Miller (Clojure team)12:09:14

Or you could make the pairs explicit in the regex if that makes sense

Alex Miller (Clojure team)12:09:14

(s/* (s/cat :x1 any? :x2 any?))

Alex Miller (Clojure team)12:09:05

Or actually since you require at least 2 args that should be + not *

Alex Miller (Clojure team)12:09:10

That's probably preferred, esp if you can give better names to x1 and x2

sveri12:09:14

Zero / 2 / 4 / ...args are fine

sveri12:09:16

I just wonder, if I look at: (s/ (s/cat :x1 any? :x2 any?)) I would assume it should have two arguments, not more, not less. But, I think its the s/ around it, that makes it work for more inputs, true?

Alex Miller (Clojure team)12:09:19

Yes - it's just like regex

Alex Miller (Clojure team)12:09:31

It's a repetition of pairs

Alex Miller (Clojure team)12:09:45

@sattvik: you can override specs in instrument to use a simpler spec

Alex Miller (Clojure team)12:09:47

@seancorfield re currying, you can use a :fn spec if you need to make an assertion about the relationship between :args and :ret

sveri13:09:06

@alexmiller Thank you very much 🙂

mschmele13:09:07

Is it generally considered bad practice to allow functions to have nilable return values?

mschmele13:09:42

It makes having functions that pass spec tests quite a bit easier, but I feel like it kinda defeats the purpose to some extent

sattvik13:09:22

@mschmele It’s perfectly reasonable, if nil means something like ‘not found’ or ‘empty’.

sattvik13:09:49

@alexmiller That’s true. It’s just that I was a little surprised (but perhaps I shouldn’t have been) that conforming a function argument involved repeated invocations of the function.

mschmele13:09:38

That’s what I’ve been using it for in simpler functions like get-from-id for example. I guess my real question (and I probably should have been more specific), applies to more complex functions like transfer which would take two accounts that can’t be nil

mschmele13:09:11

Despite having validation in the function, I’m having trouble getting it to pass spec check

seancorfield16:09:08

@alexmiller Yeah, I know I can deal with currying via :fn but that still makes for a fairly complex spec — Has there been any discussion of making it easier to spec multi-arity functions?

Alex Miller (Clojure team)16:09:50

My experience has been that it is comparatively rare for the ret spec to rely on the args arity

Alex Miller (Clojure team)16:09:28

And when it does :fn is the way to talk about that constraint

seancorfield16:09:42

Well, with a curried function, the :ret will either be a function (of the remaining argument(s)) or a result...

seancorfield16:09:53

So

(defn foo
  ([a] (fn [b] (foo a b)))
  ([a b] (* a (inc b))))
=> :ret would be int? in the two arg case but some fspec in the one arg case

seancorfield16:09:24

If fspec / fdef supported multi-arity, this could be neater...

Alex Miller (Clojure team)16:09:45

Well I'd say curried functions are not idiomatic in Clojure :)

Alex Miller (Clojure team)16:09:34

But the main place I've run into this is with the core seq functions with transducer arity

seancorfield16:09:38

What about multi-arity functions in general?

Alex Miller (Clojure team)16:09:22

Usually most arities call into a canonical arity and have the same ret spec

seancorfield16:09:23

They’re still pretty messy to spec out, even if :ret is fixed.

Alex Miller (Clojure team)16:09:44

I have not found multiple arities at all difficult to spec

Alex Miller (Clojure team)16:09:59

They're often quite easy to talk about in regex

seancorfield16:09:31

As long as the arities all extend the base case in order, yes… but there’s quite a bit of real world code out there which doesn’t follow that model (or is that also non-idiomatic?).

Alex Miller (Clojure team)16:09:02

Even if not in same order, regex can easily describe with ?

Alex Miller (Clojure team)16:09:02

I'm not discounting what you're saying, but I have not found that to be an issue in my own experience

Alex Miller (Clojure team)16:09:11

In general, I am more commonly surprised at how well regex specs work for args

robert-stuttaford16:09:03

you guys are having a ball, aren't you. can't wait to get stuck in myself!

mschmele16:09:18

@sattvik copied from above, because I forgot to @ you earlier 😝 That’s what I’ve been using it for in simpler functions like get-from-id for example. I guess my real question (and I probably should have been more specific), applies to more complex functions like transfer which would take two accounts that can’t be nil Despite having validation in the function, I’m having trouble getting it to pass spec check

seancorfield16:09:47

@alexmiller Fair enough. I held off spec’ing some of the multi-arity java.jdbc stuff at first since it didn’t seem "easy" (although it may yet prove to be "simple"). I’ll take another run at it soon (next week probably) and report back.

seancorfield16:09:48

@alexmiller My last Q for the morning (I promise!): is there an idiomatic way to spec something that is treated as truthy / falsey, when it might not be strictly true or false. I tried #{true false nil} before realizing won’t work facepalm and (s/nilable #{true false}) "works" but seems a bit … I guess that treating an any? argument as a pseudo-boolean is probably a bit sketchy but ...

Alex Miller (Clojure team)17:09:01

I don’t think (s/nilable #{true false}) works either for false (for same reason as the set)

Alex Miller (Clojure team)17:09:57

so you need some kind of fn to do it so pick your favorite function that matches those 3 values :)

Alex Miller (Clojure team)17:09:52

I guess (s/nilable boolean?) works

donaldball17:09:38

(s/def ::boolish (s/or :truthy (complement #{false nil}) :falsey #{false nil})) ?

Alex Miller (Clojure team)17:09:51

no set with falsey values in it is going to be useful :)

Alex Miller (Clojure team)17:09:42

s/nilable has been made significantly better performing this week from Rich’s commits in master btw

seancorfield17:09:00

Ah, yes, of course false won’t work either. facepalm again 🙂

seancorfield17:09:14

(it hadn’t failed in testing yet but…)

seancorfield17:09:15

The specific case is for ::as-arrays? in java.jdbc where it’s intended to be true / false or :cols-as-is but in reality nil is acceptable and common when passing defaulted options around (and, of course, it really accepts any? and just treats it as a boolean).

sattvik17:09:50

For the multi-arity stuff, the following doesn’t work, but it might be nice if it did:

(defn foo
  ([a] (fn [b] (foo a b)))
  ([a b] (* a (inc b))))

(s/def foo
  (s/or :unary (s/fspec :args (s/cat :arg int?)
                        :ret (s/fspec :args (s/cat :arg int?)
                                      :ret int?))
        :binary (s/fspec :args (s/cat :arg1 int?
                                      :arg2 int?)
                         :ret int?)))

seancorfield17:09:30

(s/fdef foo
  (:args (s/cat :a int?) 
   :ret  (s/fspec :args (s/cat :b int?) 
                  :ret int?))
  (:args (s/cat :a int? :b int?) 
   :ret  int?))
That was along the lines of what I was thinking...

seancorfield17:09:46

Which matches defn for multi-arity functions.

Alex Miller (Clojure team)18:09:33

(s/fdef foo
  :args (s/cat :a int? :b (s/? int?))
  :ret (s/or :val int? :fun (s/fspec :args (s/cat :arg int?) :ret int?))
  :fn (fn [m]
        (= (-> m :ret key)
           (if (-> m :args :b) :val :fun))))

Alex Miller (Clojure team)18:09:56

I think Rich would say about :ret here that it should simply state the truth - it can either be a number or a function

Alex Miller (Clojure team)18:09:43

and :fn can add an arg-dependent constraint

Alex Miller (Clojure team)18:09:24

I think it’s unlikely we would extend to either of the two suggestions above

Alex Miller (Clojure team)18:09:03

this kind of dependent constraint is the whole reason to have :fn

Alex Miller (Clojure team)18:09:48

if you felt the need, I think you could create a macro that automatically created a spec for a curried fn

seancorfield18:09:52

Hmm, yeah, that might well be worth doing… I’ll have a think about that...

seancorfield18:09:31

A variant of fdef for which you give :args, :ret, and :fn -- and then also an indication of which curried variants you need… and then it automatically generates the fdef spec from that...

Alex Miller (Clojure team)18:09:39

I guess you’d need to specify the shape of every curried result

seancorfield18:09:07

That would be the hard part since you can’t use s/? for all those args, only for the last one.

seancorfield18:09:28

(s/? (s/cat …))

Alex Miller (Clojure team)18:09:05

yeah, it’s gross looking - you’d want to generate it out of an s/cat or an s/cat with noted optional parts or something

Alex Miller (Clojure team)18:09:36

I don’t think curried fns are common enough to do any of this :) but if you’re looking for a puzzle to play with …

seancorfield18:09:44

I tend to curry functions quite a bit — to avoid partial all over the place — but it is almost always currying just a two arg function.

Alex Miller (Clojure team)19:09:40

well maybe that’s a simplifier

Alex Miller (Clojure team)19:09:57

the shape I had above could be made generic

seancorfield19:09:28

I may just remove the currying in java.jdbc since I’d be shocked if anyone actually leverages it in client code (given your comment about it being non-idiomatic). java.jdbc doesn’t use the curried form of as-sql-name internally and I don’t know why anyone would externally. And I wouldn’t expect the non-curried form of quoted to be used by anyone either (the one-argument form is actually the common, useful arity).

seancorfield19:09:46

I hadn’t thought hard about either of those until I sat down to try to spec them out.

seancorfield19:09:41

clojure.spec definitely makes you question your design choices 🙂

patrkris19:09:42

Any ideas on how to organize a project's specs? For instance, if I have a domain model for my application that describes a customer entity, would I then create the spec in a com.example.customer namespace and use ::name for defining a spec for the customer's name? Or would I benefit from centralizing specs in a dedicated namespace, and in that spell out the fully namespaced keyword, i.e. com.example.customer/name?

Alex Miller (Clojure team)19:09:30

I’d say both are fine :)

patrkris19:09:37

Yeah. That's what I thought you'd say. 😉 But then I am imagining scenarios where a customer means different things in different contexts. It may be one thing in the domain model and another thing in a HTTP request handler. So it might be okay to have :com.example.customer/name and :com.example.resources.customer/name? Does that look wrong?

Alex Miller (Clojure team)19:09:28

you can alias specs (s/def :com.example.customer/name :com.example.resources.customer/name)

Alex Miller (Clojure team)19:09:02

has to be done explicitly of course so ymmv

Alex Miller (Clojure team)19:09:44

we’ve talked about a version of s/keys that would separate the map keys from the specs rather than requiring them to be the same keyword, not sure if that will pan out

manderson20:09:00

+1 to separating map keys from specs ^^^

Alex Miller (Clojure team)20:09:54

the spec part would still be a qualified keyword (not inline specs), just to be clear

Alex Miller (Clojure team)20:09:39

this was considered as an alternative to :req-un too

seancorfield21:09:53

It took me a while to internalize that :req-un could have :my.foo/bar as a spec for the key :bar independent of the namespace you define the keys spec in.

seancorfield21:09:00

(Because I didn't read the docs closely enough apparently)

seancorfield21:09:11

So I'm not sure why you'd want to separate the specs from the keys at this point?

seancorfield21:09:29

(Or am I still misunderstanding the issue?)