Fork me on GitHub
#clojure-spec
<
2016-06-30
>
jjcomer00:06:27

I’m trying to use the new stub feature, but I can’t seem to get the stub to take hold. The println still happens for me:

(defn ding
  [x]
  (println "Hello World"))

(s/fdef ding
        :args (s/cat :x pos-int?)
        :ret nil?)

(defn adding
  [x y]
  (ding x)
  (+ x y))

(s/fdef adding
        :args (s/cat :x pos-int? :y pos-int?)
        :ret pos-int?
        :fn (fn [{:keys [args ret]}]
              (= ret (+ (:x args) (:y args)))))

(defn test-adding
  []
  (try
    (st/instrument `adding
                   {:stub #{`ding}})
    (adding 2 3)
    (finally (st/unstrument `adding))))

puzzler00:06:32

What happened to instrument-all and instrument-ns in the new alpha?

jjcomer00:06:07

@puzzler: All is instrument with no params and ns is passing an ns to instrument

puzzler00:06:17

Is there any way to keep the instrumenting on during development? It seems that when you recompile a namespace it blows away the instrumentation.

seancorfield01:06:14

Since instrument updates the Vars in place, I’d expect that (so, "no", I think is the answer).

seancorfield01:06:40

If you recompile the namespace, you’re replacing all the Vars, so you’d need to re-run instrument.

gfredericks01:06:22

puzzler: one option is to put (s/instrument ...) at the bottom of relevant namespaces

gfredericks01:06:33

options that don't modify the source code would depend on how you're loading your code

puzzler01:06:51

In theory, defn could be rebound to automatically instrument functions that have specs.

seancorfield01:06:49

That would only help if the spec was defined ahead of the function...

puzzler01:06:33

I'm not saying this should be the general definition of defn, I just mean that when you choose to instrument all, a new version of defn (and fdef to handle the case of function before spec) would ensure the instrumentation never gets out of sync at the REPL, which I'm finding to be an annoyance. Right now, you change the spec and the instrumented function doesn't see it, or you change the function and the spec-checking goes away. Too easy to forget to re-run instrumentation.

seancorfield01:06:10

I have my specs in a different namespace...

puzzler01:06:28

@seancorfield: Are you saying that having specs in a different namespace alleviates this issue? Seems to me it would just make it even more complicated.

seancorfield01:06:46

I’m saying your suggestions wouldn’t address that problem.

seancorfield01:06:58

i.e., it’s not a general solution

seancorfield01:06:46

(and I don’t know what the general solution is)

seancorfield01:06:38

For the specific case of java.jdbc, I have the test namespace — which loads the specs — also run instrument.

seancorfield01:06:09

So the code is separate from the specs, but the tests depend on them.

puzzler01:06:36

The first time you call instrument-all, it's calling your spec name space which registers the specs in a global registry. So a defn that was aware of the specs in the registry would help that direction. Going the other direction, if you edit the spec in the separate file, that spec refers to the var in the implementation namespace, so that function could be re-instrumented. So I think it would probably work. What am I missing?

seancorfield01:06:54

I think it’s very dependent on workflow. I’m waiting to see what Clojure/core come up with (based on the discussion earlier about what’s coming for specs in the test tooling area).

seancorfield01:06:25

What you raise is certainly an issue that can make working with spec in the REPL pretty annoying.

puzzler01:06:58

I'm only imagining this would be helpful in a mode where all instrumentation is turned on for everything. I don't have a good handle on the workflow for spec yet, so I agree with your point that we'll see how this evolves.

seancorfield01:06:12

Right now I’m only instrumenting on a per-namespace basis as I work with / test a particular namespace. I ran into some early weirdness with instrument-all (and no longer remember the details) and backed off that… may have been back when it caused generative testing to occur on higher-order function specs?

seancorfield01:06:42

So it may be that separating the generative part of function specs from instrumentation has addressed the issues I ran into...

puzzler01:06:39

The only issue I have with putting specs in a separate namespace is that it diminishes their value as a form of documentation. (Actually, this is also an issue with putting tests in a separate namespace, which I've gotten used to, so maybe I'll get used to putting specs elsewhere as well).

puzzler01:06:43

In your split-namespace approach, do you see specs in the doc strings of your functions?

seancorfield01:06:47

If you’re working in the test namespace (where the spec has been loaded), yes.

seancorfield01:06:17

I haven’t settled on it as a normal way to work yet. We’re still exploring several options

puzzler01:06:25

But users of your library generally wouldn't see that if they call doc on a function at the REPL, right?

seancorfield01:06:38

If they load the specs, yes.

seancorfield01:06:50

This allows the code to be used by pre-1.9 clients.

seancorfield01:06:07

For java.jdbc, it supports all the way back to 1.4

seancorfield01:06:19

and even in 1.9, the specs are optional but available.

puzzler01:06:24

Ah, yes, I can see how that would be the most critical thing to keep in mind for java.jdbc

seancorfield01:06:46

when I run the java.jdbc tests on 1.9, it loads the specs and instruments the ns (not all, just java.jdbc)

seancorfield01:06:36

So, for library code that might have pre-1.9 users, separate specs is pretty much the only possibility.

puzzler01:06:44

Could the main sourcecode namespace do that test on version number, and automatically load the specs for docstring purposes?

seancorfield01:06:15

For application code that’s is bound to 1.9, I can imagine having specs intermingled — maybe.

seancorfield01:06:55

Yeah, I guess java.jdbc’s source namespace could auto-load the specs on 1.9. I’m just not sure that would be what all its users would expect.

seancorfield01:06:37

After all, if you (instrument) your code would you expect all the external libraries to slow down due to specs?

seancorfield01:06:13

Some specs can have a really high performance overhead so I don’t think the library should dictate that.

puzzler01:06:43

I'm thinking that my number one use for specs will be to get good error messages when working with other libraries, so yes, I think that's what I personally would want. The biggest source of errors in my code is having a false expectation about how someone else's code should work.

seancorfield02:06:45

We’ve jumped on the Alpha builds at work (in production) in preparation for using clojure.spec fairly heavily but we’re still in the early exploration phase. We’re already leveraging the new predicates in production code tho’.

seancorfield02:06:03

We haven’t fully decided what we’ll spec out yet vs what we won’t bother with. I’ve been exploring using java.jdbc with namespaced keys as part of this work.

seancorfield02:06:34

We have over 30Kloc production code and over 11Kloc tests so whatever we do will need to be a gradual process...

seancorfield02:06:39

…expanding what we can do with generative testing alone might bring us a good "bang for our buck".

Tim02:06:32

@seancorfield: at world singles?

seancorfield02:06:58

Yup. Just pushed a build to QA based on Clojure 1.9.0 Alpha 8 so that’ll go to production in our next build.

seancorfield02:06:13

(we already have Alpha 7 in production)

cap10morgan03:06:30

I'm getting Don't know how to create ISeq from: spec_demo.core$full_name when running (stest/instrument full-name) in an ns where I have a fn full-name and have (s/fdef full-name ...). It works if I leave the argument off (so it instruments everything), but I thought it was supposed to take symbol args now too in alpha8?

seancorfield04:06:27

(stest/instrument 'full-name) <-- needs to be a symbol

seancorfield04:06:49

To explain the error message, (stest/instrument full-name) will evaluate full-name, since it's a function it evaluates to its class name which is a class full_name inside the package spec_demo.core because the Clojure is spec-demo.core/full-name.

puzzler04:06:30

I'm having trouble ascertaining the purpose of the retag in multi-spec. Can someone clarify when you'd actually see the retag?

cap10morgan11:06:27

@seancorfield: ah, yes, thanks. I was trying #'full-name and getting a var but totally forgot about 'symbol.

cap10morgan11:06:02

I'm giving a presentation tonight on clojure.spec at Den of Clojure (Denver meetup). So putting together as many live demoes as I can.

cap10morgan11:06:10

Hmm, but it still doesn't work. (stest/instrument 'full-name) doesn't seem to turn on instrumentation (I get an exception when an invalid arg blows up in the fn body rather than spec telling me it's invalid) and (stest/test 'full-name) just returns (). These all worked with just (stest/instrument) and (stest/test).

Alex Miller (Clojure team)11:06:50

If you use back tick it will resolve the symbol to fully qualified

Alex Miller (Clojure team)11:06:08

The symbol has to be fully qualified

cap10morgan11:06:22

OK, that fixed it. Thanks, @alexmiller. I think that's the first time I've used back tick outside of a macro. Makes sense that it would need to be fully-qualified since it's going into a central registry.

cap10morgan11:06:09

Did clojure.spec.test.check go away or change in alpha8? Trying to change how many tests get run in clojure.spec.test/test and test's docstring says to pass a map with a :clojure.spec.test.check/opts keyword.

Alex Miller (Clojure team)12:06:13

That's just a keyword namespace, not an actual namespace

cap10morgan12:06:40

huh, OK. not sure how to alias it, then. but I guess I don't strictly need to.

jjcomer12:06:18

Does anyone have an example of stub or replace? Neither seem to be working for me, the original fns still execute.

puzzler12:06:34

@alexmiller I thought multi-specs couldn't auto-generate data since the possibilities can't be enumerated. And if you have to write your own generator, I don't see how the retag comes into play.

bhauman13:06:18

I'm sure that this has probably been discussed before, a bunch, but I am late to the game and curious about and flowing conformed values to successive predicates. Initially, it seems very surprising that a successive predicate is operating on a different__ value. This seems semantically more like and->. Is this in the Guide? It's also curious that the underlying implementation of merge is named and-preds. I know there is a reason for this and admit that I haven't looked closely enough at it.

fenton14:06:02

@puzzler: you can auto-generate data for multi-specs.

Alex Miller (Clojure team)15:06:10

@puzzler did you read the doc-string for multi-spec? I think it answers all these questions.

Alex Miller (Clojure team)15:06:47

@bhauman: and vs and-> is under consideration. for now, and flows conformed values.

bhauman15:06:35

cool, i didn't know and-> was under consideration

Alex Miller (Clojure team)15:06:33

Rich mentioned it in the back chat

bhauman15:06:56

oh and a big big + 💯 on every

flipmokid15:06:18

Hi, I'm looking to write a spec for a map in clojurescript which is used to check user input. One of the keys requires that the input be an int and be in a set of allowed values which comes from a server (so it's an async call). What is the best way of achieving this?

bhauman15:06:32

if its deep in the spec, make the call to the server first, and then make a binding, and have your predicate read the binding

bhauman16:06:21

but otherwise I would separate this concern from the spec

bhauman16:06:51

and do a second pass

Alex Miller (Clojure team)16:06:23

you don’t have to make it a fn spec to use spec - you can explicitly call s/valid? to make that check

flipmokid16:06:45

@bhauman: Thanks, that make sense (having that separate from the spec). BTW thanks for figwheel! @alexmiller I wanted to originally do something like (s/and integer? (call-service)) where call-service returns a channel which eventually returns a set of ints. I wasn't sure of the best way of doing this as that function isn't a predicate I'm fairly new to clojure btw so please bear with me!

Alex Miller (Clojure team)16:06:51

have you considered just writing some Clojure code? :)

Alex Miller (Clojure team)16:06:08

I’m not sure spec is buying you anything for what you’re describing

flipmokid16:06:09

I will give it a go. Time to crack open Clojure Applied again 🙂

Alex Miller (Clojure team)16:06:59

well nothing about spec in there :)

Alex Miller (Clojure team)16:06:12

those idiots were too dumb to write about it

flipmokid16:06:07

Lol. The book is great, thank you! I meant using the book to write some clojure code. I thought I should do the server check separately from the spec but just wanted to make sure.

flipmokid16:06:14

Much appreciated

Alex Miller (Clojure team)16:06:35

you shouldn’t register a spec that’s calling a server, but you could use s/validate? to verify a value conforms to a spec (which has a predicate, which is based on a set obtained from elsewhere)

Alex Miller (Clojure team)16:06:05

however, that “check” is so simple, that you might as well just check if the value is in the set (without using spec)

flipmokid16:06:51

With the first option, what would be the best way of waiting for the set to come back out of the channel in clojurescript inside the predicate? The best my mind could do was poll the channel until there was something on it. The current map spec I have does have a lot of keys which are much more involved than this one, but this is the only one that needs to check against a remote resource.

fenton17:06:56

where do we report issues found with spec?

fenton17:06:07

but change :matching-p to :matching and it works okay... seems hyphens don't work for matching values.

fenton17:06:23

hmm... hold on that... maybe dirty repl

blance18:06:49

I just read about clojure.spec and it seems like a great tool for testing with test.check. To utilize it for unit testing, should I specify a spec for every functions using fdef in my application so that I just call clojure.spec.test/test to do unit testing? I've never used schema before so I'm not sure which function/args/data I should specify those and which not to

seancorfield18:06:52

@blance: Have you read the Rationale for clojure.spec? It talks about when to use spec and what sort of stuff it’s designed for.

bhauman18:06:38

So here is another thing, the :in path on key value errors in (s/every-kv ..) end withs a 0 1 presumably to disambiguate between the key and the value, but this renders the :in path pretty tough to use to look up a value in a nested structure. I'm thinking that intention was to have the :in path terminate with the map key and the 0 or 1 should be at the end of :path only

blance19:06:16

Yea I've read both.. It must be that I read through them too fast. I'll read them again.

blance19:06:08

I mean I know how to use them, but I'm not sure on where to use it. Does it make sense to say have a fdef for each single function in my app?

eggsyntax19:06:20

A reasonable shorthand might be: any function that you would otherwise have hand-written a test for.

eggsyntax19:06:59

(and of course for fns where spec is valuable in other ways, eg runtime validation of inputs)

eggsyntax19:06:24

That's just my take on it, though, and I'm a spec beginner too. As I guess are we all 😉

blance19:06:36

you would want your unit test to cover as much functions as possible right? so write spec for as much functions as possible as well?

blance19:06:23

I feel like some fns would take as long time to write a spec to validate its input&output as to write the actual function itself

eggsyntax19:06:53

Matter of taste. Personally I don't shoot for complete test coverage, just coverage of fns that a) are tricky, b) are at risk for regression bugs, c) could use a test as usage documentation.

eggsyntax19:06:59

And yeah, it's definitely arguable that hand-written tests are a better tool to reach for in some cases than spec is. It'll be interesting to watch as different ideas about best practices for spec emerge.

eggsyntax19:06:41

(although the new stub/mock functionality makes it a tool I'm likely to reach for a lot more often)

blance19:06:40

that make sense. and I would assume you only need to use clojure.spec.test/test for unit testing? Do you still hand write test to cover corner cases?

blance19:06:56

what's the stub/mock functionality? is that also in spec?

blance19:06:20

sorry I have lots of questions as I'm new to testing as well:sweat_smile:

eggsyntax19:06:47

:stub replaces a fn with a stub that checks :args, then uses the :ret spec to generate a return value. https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/test.clj

eggsyntax19:06:46

To me it seems like generative testing is, in general, more likely to catch corner cases than hand-written tests, unless there's a specific corner case you want to test for that isn't likely to be generated.

eggsyntax19:06:17

(oh, :stub is an optional arg to instrument btw)

blance19:06:25

that's cool! didn't see that in the doc

eggsyntax19:06:43

It's new as of alpha8. I think the guide isn't quite caught up yet.

blance19:06:23

Ill take a look at it. Thanks!

bfabry19:06:25

@fenton: there's no way and could combine the generators. there's a new function s/merge specifically for that scenario though

fenton19:06:50

@bfabry: okay, will check that.

seancorfield19:06:42

@blance: If you specify "every" function then your system will be brittle: changing anything will likely break specs and you’ll have to constantly rewrite them — it’ll be hard to do refactoring. I think specs make sense on "module" boundaries and on "library APIs".

seancorfield19:06:25

As for generative testing, you can test a function against a spec (in the test suite) without needing to fdef it in the main code. That seems like a reasonable approach to me.

fenton19:06:30

how to spec for a string that should be an alpha-numeric?

seancorfield19:06:59

Use a predicate with a string regex?

fenton19:06:42

@seancorfield: sorry not understand, example?

seancorfield19:06:32

#(re-find #"^[a-zA-Z0-9]+$" %)

seancorfield19:06:44

(there’s sure to be a shortcut for that pattern

seancorfield19:06:48

So you’d have (def ::alphanumeric (s/and string? #(re-find #"…" %)))

fenton19:06:33

@seancorfield: okay that makes sense...thx.

seancorfield19:06:58

boot.user=> (s/def ::alphanumeric (s/and string? (partial re-find #"^\w+$")))
:boot.user/alphanumeric
boot.user=> (s/exercise ::alphanumeric)
(["Q" "Q"] ["7" "7"] ["4" "4"] ["6z6" "6z6"] ["RjN" "RjN"] ["r05" "r05"] ["9" "9"] ["o8m" "o8m"] ["5j7YFcm" "5j7YFcm"] ["RFNe5Xj3" "RFNe5Xj3"])

eggsyntax19:06:54

It'd be nice to do something like (s/def ::alphanumeric (s/and string? #(Integer/parseInt %))), but that'll throw an exception rather than failing...

blance19:06:50

@seancorfield: I'm actually working on clojurescipt. most of my fns are api calls, transition db to new states, and UI stuffs. No need to write spec for any of them I guess?

seancorfield19:06:10

@eggsyntax: you would need both a predicate to match numeric strings and a conformer to convert to Long I think?

bhauman19:06:13

I looked into the :in path problems for maps that I mentioned above, it seems like ::kfn isn't enough to eliminate the following tuple key.

eggsyntax19:06:44

(& at that point the regex solution is nicer)

seancorfield19:06:54

@blance: Hard to say in absolute terms. Try a few approaches and see what works best for you?

blance19:06:42

sounds good. Thanks!

seancorfield19:06:55

I’d probably spec the API itself (between client and server) and then maybe some of the bigger components within each side. But I suspect you have a lot of "trivial" functions which wouldn’t be worth spec’ing?

eggsyntax19:06:22

New motto: "Don't spec the small stuff"

blance19:06:36

api calls are async, or rather returning a core.async channel

blance19:06:42

so not sure how to spec that

bhauman19:06:53

its expressive enough for me to fix it in use though

bfabry19:06:25

I figure fdefs probably live at a similar level to unit tests. if you unit test every function in your application it'll be annoying to change, but unit testing still holds value at some "right" level. same with fdefs

bfabry19:06:56

fdefs have the added advantage that they also give you runtime contract checking and easier to understand documentation than unit tests

puzzler19:06:07

@alexmiller I read the doc-string about multi-spec but (sg/generate (s/gen my-multi-spec)) throws ExceptionInfo Unable to construct gen at: [] for: [email protected] clojure.core/ex-info (core.clj:4718) so I must be misunderstanding the point about generation.

fenton20:06:42

@puzzler: did u see the example i made for u?

puzzler20:06:26

@fenton no, didn't see that. Where?

puzzler20:06:48

@fenton, ah, I see it now.

puzzler20:06:11

@fenton, hmm, not sure why it isn't working for me. Maybe one of the individual multimethods can't generate but the error message is misleading, making it seem like the overall multispec is failing.

fenton20:06:46

@puzzler: did my example work for you?

puzzler20:06:49

@fenton: no, I copied and pasted your code and tried it at the REPL but I'm getting NullPointerException clojure.test.check.generators/call-gen (generators.cljc:41) This is on the latest alpha. Not sure what's going on.

puzzler20:06:40

@fenton, oh I called gen and generate in the wrong order on your code. So yours is now working for me. But I had it right on my example which wasn't working. So still trying to figure that out...

puzzler20:06:27

@fenton, ok figured it out. I wasn't calling gen on the keyword associated with the multispec, I was calling it on the actual multispec.

puzzler20:06:44

@fenton confusing error messages

puzzler20:06:54

In some cases the docstring info provided automatically by spec isn't terribly useful, especially in the case of multi-specs. Is there any way to manually add info to the spec docstring?

fenton21:06:37

@puzzler glad u got it work....dunno about ur last comment tho. 🙂

blance23:06:15

Would you put all spec defination at one place? say my.appns.spec? If so, any trick to refer to a namespaced keyword?

blance23:06:20

I assume nobody want to type {:my.appname.is.very.long/key1 "foo" :my.appname.is.very.long/key2 "bar"} `

blance23:06:17

Also i'm wondering if clojure.spec can spec arbitrary nested vector? for example, geoJSON coordinates can be [1 1] or

[
      [ [100.0, 0.0], [101.0, 0.0], [101.0, 1.0], [100.0, 1.0], [100.0, 0.0] ],
      [ [100.2, 0.2], [100.8, 0.2], [100.8, 0.8], [100.2, 0.8], [100.2, 0.2] ]
      ]
or [ [100.0, 0.0], [101.0, 1.0] ]

Alex Miller (Clojure team)23:06:37

You have optionality and recursion, so without knowing more than those examples, yes

Alex Miller (Clojure team)23:06:57

@blance if you are making a spec namespace, I would propose the plural (specs) rather than spec. We will be doing clojure.core.specs for example.

Alex Miller (Clojure team)23:06:32

For namespaces, you can alias with :as in require then autoresolve with :: keywords

blance23:06:05

like my.appname.specs :as specs then ::specs/key1 right?

blance23:06:15

cool thanks!

blance23:06:53

On a side note, I feel like clojure team does not want us to put all specs at one place, otherwise we would have the option to just describe un-namespaced key which is more convenient.