This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-05-23
Channels
- # announcements (12)
- # beginners (225)
- # calva (7)
- # cider (45)
- # clj-kondo (1)
- # cljdoc (1)
- # cljsrn (3)
- # clojure (112)
- # clojure-dev (45)
- # clojure-europe (6)
- # clojure-finland (2)
- # clojure-india (1)
- # clojure-nl (27)
- # clojure-spec (37)
- # clojure-uk (171)
- # clojurescript (39)
- # core-async (9)
- # cursive (22)
- # datascript (8)
- # datomic (50)
- # emacs (12)
- # figwheel-main (17)
- # fulcro (42)
- # garden (2)
- # hoplon (27)
- # jobs (4)
- # kaocha (8)
- # klipse (2)
- # luminus (2)
- # off-topic (9)
- # perun (33)
- # planck (2)
- # re-frame (9)
- # reagent (48)
- # reitit (5)
- # remote-jobs (1)
- # rum (2)
- # shadow-cljs (23)
- # slack-help (3)
- # spacemacs (18)
- # sql (7)
- # tools-deps (24)
- # unrepl (9)
- # vim (30)
Sorry if this has been mentioned already, but when using tools.deps, using --watch
from the command line just hangs forever. If I set :watch? true
in tests.edn, it works.
Oh, @bbrinck, I see you've found this tool as well. After our conversation on #expound the other day, I decided to try tools.deps after all, and I think I like it. kaocha seems to be the best option for a test runner with tools.deps, but I'm struggling to get instrumentation failures to print properly with my expound printer. It looks like kaocha and orchestra both re-bind s/*explain-out
at different levels, and I'm pretty confused as to what's happening exactly. The closest I can get is the following (some stolen and modified from your test_utils, some taken from https://gist.github.com/kennyjwilli/8bf30478b8a2762d2d09baabc17e2f10):
(ns vertiv-common.demo-test
(:require [clojure.spec.alpha :as s]
[clojure.spec.test.alpha :as stest]
[clojure.string :as str]
[clojure.test :as test]
[expound.alpha :as expound]
[orchestra.core :refer [defn-spec]]
[orchestra.spec.test :as orchestra]))
(alias 'stc 'clojure.spec.test.check)
(defn report-results [check-results]
(let [checks-passed? (->> check-results (map :failure) (every? nil?))]
(if checks-passed?
(test/do-report {:type :pass
:message (str "Generative tests pass for "
(str/join ", " (map :sym check-results)))})
(doseq [failed-check (filter :failure check-results)]
(let [r (stest/abbrev-result failed-check)
failure (:failure r)]
(test/do-report
{:type :fail
:message (binding [s/*explain-out* (expound/custom-printer
{:show-valid-values? true
:theme :figwheel-theme})]
(expound/explain-results-str check-results))
:expected (->> r :spec rest (apply hash-map) :ret)
:actual (if (instance? #?(:clj Throwable :cljs js/Error) failure)
failure
(::stest/val failure))}))))
checks-passed?))
(def default-opts {::stc/opts {:num-tests 50}})
#?(:clj
(defmacro defspec-test
([name] `(defspec-test ~name (stest/checkable-syms) nil))
([name sym-or-syms] `(defspec-test ~name ~sym-or-syms default-opts))
([name sym-or-syms opts]
(when test/*load-tests*
`(defn ~(vary-meta name assoc :test
`(fn [] (report-results (stest/check ~sym-or-syms ~opts))))
[] (test/test-var (var ~name)))))))
(defn check-spec-assertions [test-fn]
(binding [s/*explain-out* (expound/custom-printer {:show-valid-values? true
:theme :figwheel-theme})]
(s/check-asserts true)
(test-fn)
(s/check-asserts false)))
(defn instrument-all [test-fn]
(binding [s/*explain-out* (expound/custom-printer {:show-valid-values? true
:theme :figwheel-theme})]
(orchestra/instrument)
(test-fn)
(orchestra/unstrument)))
(test/use-fixtures :once
check-spec-assertions
instrument-all)
(defn-spec do-it-wrong boolean?
[a int?] 1)
(defn-spec test-a-thing boolean?
[thing int?]
(do-it-wrong false)
true)
(defspec-test gen_test-a-thing `test-a-thing)
But that gives me:
FAIL in vertiv-common.demo-test/gen_test-a-thing (demo_test.cljc:21)
== Checked vertiv-common.demo-test/test-a-thing
(vertiv-common.demo-test/test-a-thing -1)
threw error
#error {
:cause "Call to vertiv-common.demo-test/do-it-wrong did not conform to spec:\ndemo_test.cljc:68\n\n-- Spec failed --------------------\n\nFunction arguments\n\n (false)\n ^^^^^\n\nshould satisfy\n\n int?\n\n-------------------------\nDetected 1 error\n"
:data {:clojure.spec.alpha/problems [{:path [:args :a], :pred clojure.core/int?, :val false, :via [], :in [0]}], :clojure.spec.alpha/spec #object[clojure.spec.alpha$regex_spec_impl$reify__2509 0x2d8a45a "clojure.spec.alpha$regex_spec_impl$reify__2509@2d8a45a"], :clojure.spec.alpha/value (false), :clojure.spec.alpha/args (false), :clojure.spec.alpha/failure :instrument, :orchestra.spec.test/caller {:file "demo_test.cljc", :line 68, :var-scope vertiv-common.demo-test/test-a-thing}, :orchestra.spec/var #'vertiv-common.demo-test/do-it-wrong}
:via
[{:type clojure.lang.ExceptionInfo
:message "Call to vertiv-common.demo-test/do-it-wrong did not conform to spec:\ndemo_test.cljc:68\n\n-- Spec failed --------------------\n\nFunction arguments\n\n (false)\n ^^^^^\n\nshould satisfy\n\n int?\n\n-------------------------\nDetected 1 error\n"
:data {:clojure.spec.alpha/problems [{:path [:args :a], :pred clojure.core/int?, :val false, :via [], :in [0]}], :clojure.spec.alpha/spec #object[clojure.spec.alpha$regex_spec_impl$reify__2509 0x2d8a45a "clojure.spec.alpha$regex_spec_impl$reify__2509@2d8a45a"], :clojure.spec.alpha/value (false), :clojure.spec.alpha/args (false), :clojure.spec.alpha/failure :instrument, :orchestra.spec.test/caller {:file "demo_test.cljc", :line 68, :var-scope vertiv-common.demo-test/test-a-thing}, :orchestra.spec/var #'vertiv-common.demo-test/do-it-wrong}
:at [orchestra.spec.test$spec_checking_fn$conform_BANG___4644 invoke "test.cljc" 115]}]
:trace
[[orchestra.spec.test$spec_checking_fn$conform_BANG___4644 invoke "test.cljc" 115]
[orchestra.spec.test$spec_checking_fn$fn__4650 doInvoke "test.cljc" 126]
[clojure.lang.RestFn invoke "RestFn.java" 408]
[vertiv_common.demo_test$test_a_thing invokeStatic "demo_test.cljc" 68]
[vertiv_common.demo_test$test_a_thing invoke "demo_test.cljc" 66]
...
expected: boolean?
actual: nil
1 tests, 1 assertions, 1 failures.
It's like the output from the printer is getting captured and wrapped in another exception
I feel like https://github.com/lambdaisland/kaocha/issues/45 is really the answer...
@alex.joseph.whitt I haven’t dug into it too deeply, but my guess is that the issue is w/ expound, not Kaocha. I would guess we could find a repro that didn’t include koacha, but only requires that in the process of running stest/check
, we run into a case where we run into an instrumentation failure