I’m trying to make a patch to spec.alpha, but I’m running into problems testing my changes (with provided tests) and building a local jar (so I can include in my own projects for further testing). I’m used to doing everything with lein, so I’m less familiar with “bare” clojure development. Does anyone know of a good guide? My googling is failing me.

mvn test maybe?

@gfredericks yes, that works! thank you so much. I’m realizing I never bothered to learn much about the underlying java foundations before jumping straight into lein for managing clj projects.

Does anyone see what I'm doing wrong here?

user> (defn foo [x] :bar)
#'user/foo
user> (clojure.spec/fdef foo :args (clojure.spec/coll-of int?) :ret int?)
user/foo
user> (clojure.spec.test/instrument `foo)
[user/foo]
user> (foo :fail)
clojure.lang.ExceptionInfo: Call to #'user/foo did not conform to spec:
                            In: [0] val: :fail fails at: [:args] predicate: int?
                            :clojure.spec/args  (:fail)
                            :clojure.spec/failure  :instrument
                            :clojure.spec.test/caller  {:file "form-init7754260867026619201.clj", :line 230, :var-scope user/eval98211}
                            
user> (foo 1)
:bar
user> *clojure-version*
{:major 1, :minor 9, :incremental 0, :qualifier "alpha12"}

you need to call check

@bmaddy instrument does not check :ret specs

instrument only checks args (sadly)

I think everyone hits that one once. Not really intuitive I guess

arguably it can be considered a naming issue at least

Interesting.... Is there a reason for instrument not looking at :ret or is it just not implemented?

it's intentional

I think it's to allow granularity with gen overrides and such, in theory bundling the 2 in a single function isn't really difficult, it's just not provided

Ok. Thanks for the help you two!

@bmaddy https://groups.google.com/forum/#!msg/clojure/jcVnjk1MOWY/UwP5bc1oCAAJ https://groups.google.com/d/msg/clojure/JU6EmjtbRiQ/uND70kAFBgAJ

Just went through the spec guide today... and looking forward to heavily using it for better code and better testing workflow. How do you control whether to instrument, and whether to run test code reliant chiefly on stest.check, using leiningen? What would a leiningen workflow look like?

Interesting, thanks @misha. Those comments were very helpful.

@matan We're mostly using spec for defining and validating data structures, rather than testing. I've suggested to people that they call instrument either in their test fixtures or directly in each test namespace as appropriate (to instrument functions in the namespace under test).

I think there are lots of ways to go about using clojure.spec in tests -- and no "best practices" have arisen yet because it's still alpha and early days.

As Rich and others have said, generative testing -- via test.check and clojure.spec.test/check -- should probably be viewed as separate from your "unit testing", since it can take a while and you want "unit tests" to run very quickly (for fast feedback).

So it's probably a good idea to separate those out into "tests" that don't run as a normal part of your "unit" test suite, but can be run via a separate Leiningen/Boot task (so they can still be run automatically as needed).

I will say that with clojure.java.jdbc, which has optional specs, the "unit tests" run much, much slower with instrumentation in place so even that probably needs to be something you optionally enable for testing.

I wish there was a good way of specifying different profiles for test.check properties

you want to do different things with it at different times

For example, clojure.java.jdbc tests on Clojure 1.8 take about 30 seconds (user; 12.5s real) whereas tests on Clojure 1.9 with instrumentation of all java.jdbc functions take 1 minute 30 seconds (user; 1 minute real). So that's a huge overhead.

@seancorfield thanks for clarifying that! I have been trying to add specs to an internal tool at where I work and seems like a massive uphill battle to get them to work in the way that I was expecting. Having the generative tests be a separate thing all together make so much more sense

We use s/conform a lot for validating (and coercing) input to our REST API and also for user input validation, where we can pick apart s/explain-data to generate informative error messages. That stuff's all in production.

We use test.check and stest/check for a small handful of what would otherwise be "unit tests" where they don't introduce a drag on our tests. We use s/exercise and rand-nth to get random, conforming input in some of our tests.

So far we've generally kept instrument and stest/check primarily for manual, isolated test runs. We'll probably integrate that based on environment variables or Boot task flags at some point.

@royalaid One thing to be careful of: don't try to spec everything! Use spec where it provides the most leverage, at system boundaries, and for key APIs, and/or key arguments to those APIs. One of the really nice aspects of spec (compared to, say, Typed Clojure) is that you really can opt-in one piece at a time, where you need it most.

Hello. Is it possible to specify a complex spec inline (w/o s/defing something)?

@andrewboltachev You mean directly in a s/conform or s/valid? call, for example? Sure, specs are "just" predicates.

@seancorfield To be specific, I want to validate a map

but e.g. (s/keys :req-un [::a]) refers to a symbol defined in (some) ns

For s/keys you must s/def the keys if you want them validated.

i.e. it takes both name and value from it

If you read the spec rationale, it explains why you can't specify both key names and value "types" together.

got it, thanks! looks anyway slightly opinionated solution for me 😉

At least it's the correct opinion 🙂