Fork me on GitHub

@domkm I think you can use your own predicate in root graph spec (s/and asyclic? (s/keys ...)), which internally uses s/*recursion-limit*, but that would traverse graph twice


@U051HUZLD Thanks for the suggestion. I think I wasn't clear. The graph is cyclic. I just only want to validate to N levels and stop validating after that point.


well, I got nothing then, except writing your own validator, and forfeiting all the for-free generators, etc.


or, use 1 spec to generate, and custom walking predicate to validate


predicate can somewhat reuse specs for nodes and edges, for keys, but it is still not pretty


Has anyone found a nice workaround for this: [spec] (s/def ::a ::b) throws unable to resolve error if ::b is not defined


We hack-fixed it by defining this macro:

(defmacro def*
  [name spec]
  `(do (when (and (qualified-keyword? ~spec)
                  (not (#'s/maybe-spec ~spec)))
         (s/def ~spec
           #(throw (ex-info "Spec is declared, but not defined!"
                            {:spec-name  ~name
                             :spec-alias ~spec
                             :args       %&}))))
       (s/def ~name ~spec)))

(def* ::foo ::NaS)            ;;=> :my.spec/abc
(s/valid? ::foo "something")  ;;=> clojure.lang.ExceptionInfo: Spec is declared, but not defined!
(def* ::NaS string?)          ;;=> :my.spec/NaS
(s/valid? ::foo "something")  ;;=> true
And doing something similar with s/fdef, but this broke down when we started using orchestra for instrumentation.


If you can't rearrange the order of your s/defs, perhaps you can (s/def ::b any?) first and then redef it later?


edit: just some more context Using a similar macro as the one above we specced a function’s :ret using a spec that hadn’t yet been defined, but we knew that it would be defined after all the namespaces had been loaded. However, when we started instrumenting the function (using orchestra) the spec was still throwing our «declared, but not defined» exception. It seems that the spec wasn’t overwritten, but describing the specs in the REPL displayed the expected results.


Your suggestion gets rid of the exceptions, but the spec is still not redefed :thinking_face: The function can now return anything because the any? spec still stands


This seems like a silly question, but is there a standard way of checking specs in tests? I suspect it uses (-> (stest/enumerate-namespace 'user) stest/check) and looks something like this:

(deftest specs
    (-> (stest/enumerate-namespace 'user) stest/check)))


@bmaddy Generative tests can be long-running so having them in "unit tests" that you run all the time is a bit of an anti-pattern.


How do you guys do unit testing? Just what percentage of tests are manually written example based ones?


We mostly do classic unit tests. We have one file where we use clojure.test.check.clojure-test/defspec, but I think that was before spec came out. We're about 97% unit tests.

👍 4

Hmm, so do people just remember to run a stest/check function that's in a comment after the function every time they update it? That's what I've been doing so far, but I figured there was a better way.


I think check can be useful when writing the function/spec, but for tests, I tend to use test.chuck and write my properties in the test


then I can use chuck/times to control the number of generative tests (low number during development, higher number on CI)


but I don’t tend to write a lot of functions that have super interesting :fn specs, I just check :args and :ret specs in tests


My tests will instrument my functions, then for any input I can just use s/gen to get the generator, use that with test.chuck


I thought test.chuck was a typo. 😂


heh, no, it’s a really useful lib for generative testing 🙂


And, yeah, big +1 for test.chuck -- we love the regex string generator in it!


@bmaddy I think a good option is to have some scripts that run stest/check and summarize/assert the results are good, and then run those scripts directly as part of either periodic manual or full-suite testing (rather than automated/unit test level stuff).

👍 4

At work we have a few generative tests that run in our "mainline" automated/unit tests, and then we have some more extensive generative/property tests that are in (comment ...) forms in various files that we currently run manually, from time-to-time.


("Rich Comment Forms" -- per Stu's Strange Loop talk)


Thanks @seancorfield. I'd be interested to hear how others do this as well if they feel like sharing.


am I right that the stc namespace for options can only be aliased like:

#?(:clj (alias 'stc 'clojure.spec.test.check))
in cljc code for clojure? Putting it in the ns form didn’t compile.


I think because it doesn’t exist as a file, but is created when loading clojure.spec.test.alpha


that sounds plausible


I didn't know it did that


it’s a bit awkward. trying to think about this:


I think I had a nice solution, but now my ns looks like:

(ns spec-keys-test
   [clojure.test :as t :refer [deftest testing is]]
   [clojure.spec.alpha :as s]
   [clojure.spec.test.alpha :as stest]
   #?(:cljs [clojure.spec.test.check :as stc])

#?(:clj (alias 'stc 'clojure.spec.test.check))


maybe changing the clojure version of the alias stc to clojure.test.check makes more sense than the in-ns thing in clojure.spec.test.alpha


Ah I see why that isn’t the case, because clojure.test.check is lazily loaded (optional dep).


Would it be OK if clojure.spec.alpha created an empty ns in a file, instead of the in-ns? Then both requires for cljs and clj could look the same. This would be nice in .cljc.


Posted an issue about this at JIRA