I’m trying to use the new stub feature, but I can’t seem to get the stub to take hold. The println still happens for me:

(defn ding
  [x]
  (println "Hello World"))

(s/fdef ding
        :args (s/cat :x pos-int?)
        :ret nil?)

(defn adding
  [x y]
  (ding x)
  (+ x y))

(s/fdef adding
        :args (s/cat :x pos-int? :y pos-int?)
        :ret pos-int?
        :fn (fn [{:keys [args ret]}]
              (= ret (+ (:x args) (:y args)))))

(defn test-adding
  []
  (try
    (st/instrument `adding
                   {:stub #{`ding}})
    (adding 2 3)
    (finally (st/unstrument `adding))))

What happened to instrument-all and instrument-ns in the new alpha?

@puzzler: All is instrument with no params and ns is passing an ns to instrument

thanks

Is there any way to keep the instrumenting on during development? It seems that when you recompile a namespace it blows away the instrumentation.

Since instrument updates the Vars in place, I’d expect that (so, "no", I think is the answer).

If you recompile the namespace, you’re replacing all the Vars, so you’d need to re-run instrument.

puzzler: one option is to put (s/instrument ...) at the bottom of relevant namespaces

options that don't modify the source code would depend on how you're loading your code

In theory, defn could be rebound to automatically instrument functions that have specs.

That would only help if the spec was defined ahead of the function...

I'm not saying this should be the general definition of defn, I just mean that when you choose to instrument all, a new version of defn (and fdef to handle the case of function before spec) would ensure the instrumentation never gets out of sync at the REPL, which I'm finding to be an annoyance. Right now, you change the spec and the instrumented function doesn't see it, or you change the function and the spec-checking goes away. Too easy to forget to re-run instrumentation.

I have my specs in a different namespace...

@seancorfield: Are you saying that having specs in a different namespace alleviates this issue? Seems to me it would just make it even more complicated.

I’m saying your suggestions wouldn’t address that problem.

i.e., it’s not a general solution

(and I don’t know what the general solution is)

For the specific case of java.jdbc, I have the test namespace — which loads the specs — also run instrument.

So the code is separate from the specs, but the tests depend on them.

The first time you call instrument-all, it's calling your spec name space which registers the specs in a global registry. So a defn that was aware of the specs in the registry would help that direction. Going the other direction, if you edit the spec in the separate file, that spec refers to the var in the implementation namespace, so that function could be re-instrumented. So I think it would probably work. What am I missing?

I think it’s very dependent on workflow. I’m waiting to see what Clojure/core come up with (based on the discussion earlier about what’s coming for specs in the test tooling area).

What you raise is certainly an issue that can make working with spec in the REPL pretty annoying.

I'm only imagining this would be helpful in a mode where all instrumentation is turned on for everything. I don't have a good handle on the workflow for spec yet, so I agree with your point that we'll see how this evolves.

Right now I’m only instrumenting on a per-namespace basis as I work with / test a particular namespace. I ran into some early weirdness with instrument-all (and no longer remember the details) and backed off that… may have been back when it caused generative testing to occur on higher-order function specs?

So it may be that separating the generative part of function specs from instrumentation has addressed the issues I ran into...

The only issue I have with putting specs in a separate namespace is that it diminishes their value as a form of documentation. (Actually, this is also an issue with putting tests in a separate namespace, which I've gotten used to, so maybe I'll get used to putting specs elsewhere as well).

In your split-namespace approach, do you see specs in the doc strings of your functions?

If you’re working in the test namespace (where the spec has been loaded), yes.

I haven’t settled on it as a normal way to work yet. We’re still exploring several options

But users of your library generally wouldn't see that if they call doc on a function at the REPL, right?

If they load the specs, yes.

This allows the code to be used by pre-1.9 clients.

For java.jdbc, it supports all the way back to 1.4

and even in 1.9, the specs are optional but available.

Ah, yes, I can see how that would be the most critical thing to keep in mind for java.jdbc

when I run the java.jdbc tests on 1.9, it loads the specs and instruments the ns (not all, just java.jdbc)

So, for library code that might have pre-1.9 users, separate specs is pretty much the only possibility.

Could the main sourcecode namespace do that test on version number, and automatically load the specs for docstring purposes?

For application code that’s is bound to 1.9, I can imagine having specs intermingled — maybe.

Yeah, I guess java.jdbc’s source namespace could auto-load the specs on 1.9. I’m just not sure that would be what all its users would expect.

After all, if you (instrument) your code would you expect all the external libraries to slow down due to specs?

Some specs can have a really high performance overhead so I don’t think the library should dictate that.

I'm thinking that my number one use for specs will be to get good error messages when working with other libraries, so yes, I think that's what I personally would want. The biggest source of errors in my code is having a false expectation about how someone else's code should work.

Good to know.

We’ve jumped on the Alpha builds at work (in production) in preparation for using clojure.spec fairly heavily but we’re still in the early exploration phase. We’re already leveraging the new predicates in production code tho’.

We haven’t fully decided what we’ll spec out yet vs what we won’t bother with. I’ve been exploring using java.jdbc with namespaced keys as part of this work.

We have over 30Kloc production code and over 11Kloc tests so whatever we do will need to be a gradual process...

…expanding what we can do with generative testing alone might bring us a good "bang for our buck".

@seancorfield: at world singles?

Yup. Just pushed a build to QA based on Clojure 1.9.0 Alpha 8 so that’ll go to production in our next build.

(we already have Alpha 7 in production)

I'm getting Don't know how to create ISeq from: spec_demo.core$full_name when running (stest/instrument full-name) in an ns where I have a fn full-name and have (s/fdef full-name ...). It works if I leave the argument off (so it instruments everything), but I thought it was supposed to take symbol args now too in alpha8?

(stest/instrument 'full-name) <-- needs to be a symbol

To explain the error message, (stest/instrument full-name) will evaluate full-name, since it's a function it evaluates to its class name which is a class full_name inside the package spec_demo.core because the Clojure is spec-demo.core/full-name.

Hope that helps @cap10morgan ?

I'm having trouble ascertaining the purpose of the retag in multi-spec. Can someone clarify when you'd actually see the retag?

In gen

@seancorfield: ah, yes, thanks. I was trying #'full-name and getting a var but totally forgot about 'symbol.

I'm giving a presentation tonight on clojure.spec at Den of Clojure (Denver meetup). So putting together as many live demoes as I can.

Hmm, but it still doesn't work. (stest/instrument 'full-name) doesn't seem to turn on instrumentation (I get an exception when an invalid arg blows up in the fn body rather than spec telling me it's invalid) and (stest/test 'full-name) just returns (). These all worked with just (stest/instrument) and (stest/test).

If you use back tick it will resolve the symbol to fully qualified

The symbol has to be fully qualified

Symbol, not var

OK, that fixed it. Thanks, @alexmiller. I think that's the first time I've used back tick outside of a macro. Makes sense that it would need to be fully-qualified since it's going into a central registry.

Did clojure.spec.test.check go away or change in alpha8? Trying to change how many tests get run in clojure.spec.test/test and test's docstring says to pass a map with a :clojure.spec.test.check/opts keyword.

That's just a keyword namespace, not an actual namespace

huh, OK. not sure how to alias it, then. but I guess I don't strictly need to.

Does anyone have an example of stub or replace? Neither seem to be working for me, the original fns still execute.

@alexmiller I thought multi-specs couldn't auto-generate data since the possibilities can't be enumerated. And if you have to write your own generator, I don't see how the retag comes into play.

I'm sure that this has probably been discussed before, a bunch, but I am late to the game and curious about and flowing conformed values to successive predicates. Initially, it seems very surprising that a successive predicate is operating on a different__ value. This seems semantically more like and->. Is this in the Guide? It's also curious that the underlying implementation of merge is named and-preds. I know there is a reason for this and admit that I haven't looked closely enough at it.

@puzzler: you can auto-generate data for multi-specs.

@puzzler did you read the doc-string for multi-spec? I think it answers all these questions.

@bhauman: and vs and-> is under consideration. for now, and flows conformed values.

cool, i didn't know and-> was under consideration

Rich mentioned it in the back chat

oh and a big big + 💯 on every

Hi, I'm looking to write a spec for a map in clojurescript which is used to check user input. One of the keys requires that the input be an int and be in a set of allowed values which comes from a server (so it's an async call). What is the best way of achieving this?

if its deep in the spec, make the call to the server first, and then make a binding, and have your predicate read the binding

but otherwise I would separate this concern from the spec

and do a second pass

you don’t have to make it a fn spec to use spec - you can explicitly call s/valid? to make that check

@bhauman: Thanks, that make sense (having that separate from the spec). BTW thanks for figwheel! @alexmiller I wanted to originally do something like (s/and integer? (call-service)) where call-service returns a channel which eventually returns a set of ints. I wasn't sure of the best way of doing this as that function isn't a predicate I'm fairly new to clojure btw so please bear with me!

have you considered just writing some Clojure code? :)

I’m not sure spec is buying you anything for what you’re describing

I will give it a go. Time to crack open Clojure Applied again 🙂

well nothing about spec in there :)

those idiots were too dumb to write about it

Lol. The book is great, thank you! I meant using the book to write some clojure code. I thought I should do the server check separately from the spec but just wanted to make sure.

Thanks guys

Much appreciated

you shouldn’t register a spec that’s calling a server, but you could use s/validate? to verify a value conforms to a spec (which has a predicate, which is based on a set obtained from elsewhere)

however, that “check” is so simple, that you might as well just check if the value is in the set (without using spec)

With the first option, what would be the best way of waiting for the set to come back out of the channel in clojurescript inside the predicate? The best my mind could do was poll the channel until there was something on it. The current map spec I have does have a lot of keys which are much more involved than this one, but this is the only one that needs to check against a remote resource.

where do we report issues found with spec?

but change :matching-p to :matching and it works okay... seems hyphens don't work for matching values.

hmm... hold on that... maybe dirty repl

I just read about clojure.spec and it seems like a great tool for testing with test.check. To utilize it for unit testing, should I specify a spec for every functions using fdef in my application so that I just call clojure.spec.test/test to do unit testing? I've never used schema before so I'm not sure which function/args/data I should specify those and which not to

@blance: Have you read the Rationale for clojure.spec? It talks about when to use spec and what sort of stuff it’s designed for.

https://clojure.org/about/spec

So here is another thing, the :in path on key value errors in (s/every-kv ..) end withs a 0 1 presumably to disambiguate between the key and the value, but this renders the :in path pretty tough to use to look up a value in a nested structure. I'm thinking that intention was to have the :in path terminate with the map key and the 0 or 1 should be at the end of :path only

@blance: & lots more detail in the guide: http://clojure.org/guides/spec

Notably http://clojure.org/guides/spec#_testing

Yea I've read both.. It must be that I read through them too fast. I'll read them again.

I mean I know how to use them, but I'm not sure on where to use it. Does it make sense to say have a fdef for each single function in my app?

A reasonable shorthand might be: any function that you would otherwise have hand-written a test for.

(and of course for fns where spec is valuable in other ways, eg runtime validation of inputs)

That's just my take on it, though, and I'm a spec beginner too. As I guess are we all 😉

you would want your unit test to cover as much functions as possible right? so write spec for as much functions as possible as well?

I feel like some fns would take as long time to write a spec to validate its input&output as to write the actual function itself

Matter of taste. Personally I don't shoot for complete test coverage, just coverage of fns that a) are tricky, b) are at risk for regression bugs, c) could use a test as usage documentation.

And yeah, it's definitely arguable that hand-written tests are a better tool to reach for in some cases than spec is. It'll be interesting to watch as different ideas about best practices for spec emerge.

(although the new stub/mock functionality makes it a tool I'm likely to reach for a lot more often)

that make sense. and I would assume you only need to use clojure.spec.test/test for unit testing? Do you still hand write test to cover corner cases?

what's the stub/mock functionality? is that also in spec?

sorry I have lots of questions as I'm new to testing as well:sweat_smile:

:stub replaces a fn with a stub that checks :args, then uses the :ret spec to generate a return value. https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/test.clj

To me it seems like generative testing is, in general, more likely to catch corner cases than hand-written tests, unless there's a specific corner case you want to test for that isn't likely to be generated.

(oh, :stub is an optional arg to instrument btw)

that's cool! didn't see that in the doc

It's new as of alpha8. I think the guide isn't quite caught up yet.

Ill take a look at it. Thanks!

@fenton: there's no way and could combine the generators. there's a new function s/merge specifically for that scenario though

@bfabry: okay, will check that.

@blance: If you specify "every" function then your system will be brittle: changing anything will likely break specs and you’ll have to constantly rewrite them — it’ll be hard to do refactoring. I think specs make sense on "module" boundaries and on "library APIs".

As for generative testing, you can test a function against a spec (in the test suite) without needing to fdef it in the main code. That seems like a reasonable approach to me.

how to spec for a string that should be an alpha-numeric?

Use a predicate with a string regex?

@seancorfield: sorry not understand, example?

#(re-find #"^[a-zA-Z0-9]+$" %)

(there’s sure to be a shortcut for that pattern

So you’d have (def ::alphanumeric (s/and string? #(re-find #"…" %)))

@seancorfield: okay that makes sense...thx.

boot.user=> (s/def ::alphanumeric (s/and string? (partial re-find #"^\w+$")))
:boot.user/alphanumeric
boot.user=> (s/exercise ::alphanumeric)
(["Q" "Q"] ["7" "7"] ["4" "4"] ["6z6" "6z6"] ["RjN" "RjN"] ["r05" "r05"] ["9" "9"] ["o8m" "o8m"] ["5j7YFcm" "5j7YFcm"] ["RFNe5Xj3" "RFNe5Xj3"])

@seancorfield: even nicer!

It'd be nice to do something like (s/def ::alphanumeric (s/and string? #(Integer/parseInt %))), but that'll throw an exception rather than failing...

@seancorfield: I'm actually working on clojurescipt. most of my fns are api calls, transition db to new states, and UI stuffs. No need to write spec for any of them I guess?

@eggsyntax: you would need both a predicate to match numeric strings and a conformer to convert to Long I think?

I looked into the :in path problems for maps that I mentioned above, it seems like ::kfn isn't enough to eliminate the following tuple key.

@seancorfield: makes sense.

(& at that point the regex solution is nicer)

@blance: Hard to say in absolute terms. Try a few approaches and see what works best for you?

sounds good. Thanks!

I’d probably spec the API itself (between client and server) and then maybe some of the bigger components within each side. But I suspect you have a lot of "trivial" functions which wouldn’t be worth spec’ing?

New motto: "Don't spec the small stuff"

😉

api calls are async, or rather returning a core.async channel

so not sure how to spec that

its expressive enough for me to fix it in use though

I figure fdefs probably live at a similar level to unit tests. if you unit test every function in your application it'll be annoying to change, but unit testing still holds value at some "right" level. same with fdefs

fdefs have the added advantage that they also give you runtime contract checking and easier to understand documentation than unit tests

@alexmiller I read the doc-string about multi-spec but (sg/generate (s/gen my-multi-spec)) throws ExceptionInfo Unable to construct gen at: [] for: clojure.lang.MultiFn@334f517 clojure.core/ex-info (core.clj:4718) so I must be misunderstanding the point about generation.

@puzzler: did u see the example i made for u?

@fenton no, didn't see that. Where?

@fenton, ah, I see it now.

@fenton, hmm, not sure why it isn't working for me. Maybe one of the individual multimethods can't generate but the error message is misleading, making it seem like the overall multispec is failing.

@puzzler: did my example work for you?

@fenton: no, I copied and pasted your code and tried it at the REPL but I'm getting NullPointerException clojure.test.check.generators/call-gen (generators.cljc:41) This is on the latest alpha. Not sure what's going on.

@fenton, oh I called gen and generate in the wrong order on your code. So yours is now working for me. But I had it right on my example which wasn't working. So still trying to figure that out...

@fenton, ok figured it out. I wasn't calling gen on the keyword associated with the multispec, I was calling it on the actual multispec.

@fenton confusing error messages

In some cases the docstring info provided automatically by spec isn't terribly useful, especially in the case of multi-specs. Is there any way to manually add info to the spec docstring?

@puzzler glad u got it work....dunno about ur last comment tho. 🙂

Would you put all spec defination at one place? say my.appns.spec? If so, any trick to refer to a namespaced keyword?

I assume nobody want to type {:my.appname.is.very.long/key1 "foo" :my.appname.is.very.long/key2 "bar"} `

Also i'm wondering if clojure.spec can spec arbitrary nested vector? for example, geoJSON coordinates can be [1 1] or

[
      [ [100.0, 0.0], [101.0, 0.0], [101.0, 1.0], [100.0, 1.0], [100.0, 0.0] ],
      [ [100.2, 0.2], [100.8, 0.2], [100.8, 0.8], [100.2, 0.8], [100.2, 0.2] ]
      ]

You have optionality and recursion, so without knowing more than those examples, yes

@blance if you are making a spec namespace, I would propose the plural (specs) rather than spec. We will be doing clojure.core.specs for example.

For namespaces, you can alias with :as in require then autoresolve with :: keywords

like my.appname.specs :as specs then ::specs/key1 right?

Yeah

cool thanks!

On a side note, I feel like clojure team does not want us to put all specs at one place, otherwise we would have the option to just describe un-namespaced key which is more convenient.