Clojurians Log v2

quick request - in kaocha.api/run , there's this bit of code:

99 ...
            (let [config (resolve-reporter config)]
              (let [test-plan (test-plan config)]
                (when-not (some #(or (hierarchy/leaf? %)
                                     (::testable/load-error %))
                                (testable/test-seq test-plan))
                  (output/warn (str "No tests were found, make sure :test-paths and "
                                    ":ns-patterns are configured correctly in tests.edn."))
                  (throw+ {:kaocha/early-exit 0 }))
108 ...

How does this eat load errors exactly? This code is meant to explicitly not eat load errors.

We store load errors in the test-plan so they can be printed out by the reporter. If that's broken in some scenarios then please open an issue and we'll give it priority. A repro repo would be much appreciated!

https://github.com/JJ-Atkinson/minimal-kaocha-bug-repro

Load a repl, and run the comment block in user.clj. Requiring main-test goes well - no dependency errors there. However, afaict test-plan makes a test plan for all nses, including the faulty one random-error-test. All I receive on the repl is

=> (kaocha.repl/run (ns-name 'dev.freeformsoftware.main-test))
WARNING: No tests were found, make sure :test-paths and :ns-patterns are configured correctly in tests.edn.
Execution error (ExceptionInfo) at slingshot.support/stack-trace (support.clj:201).
throw+: #:kaocha{:early-exit 0}

is this only an issue when run from the REPL?

yep - it would not run from the command line because of the file error. the clojure compiler would yell at me 🙂

Here, the compiler still yells. Just under the ::testable/load-error key, which isn't printed

Can you start by upgrading to latest Kaocha?

Caused by: java.lang.RuntimeException: Unable to resolve symbol: x in this context
 at clojure.lang.Util.runtimeException (Util.java:221)
    ...
    kaocha.ns$required_ns.invokeStatic (ns.clj:13)
    kaocha.ns$required_ns.invoke (ns.clj:11)
    kaocha.type.ns$eval3181$fn__3182.invoke (ns.clj:29)
    ...
    kaocha.testable$load.invokeStatic (testable.clj:94)
    kaocha.testable$load.invoke (testable.clj:75)
    kaocha.testable$load_testables.invokeStatic (testable.clj:152)
    kaocha.testable$load_testables.invoke (testable.clj:144)
    kaocha.load$load_test_namespaces.invokeStatic (load.clj:49)
    kaocha.load$load_test_namespaces.doInvoke (load.clj:37)
    ...
    kaocha.type.clojure.test$eval4603$fn__4604.invoke (test.clj:17)
    ...
    kaocha.testable$load.invokeStatic (testable.clj:94)
    kaocha.testable$load.invoke (testable.clj:75)
    kaocha.testable$load_testables.invokeStatic (testable.clj:152)
    kaocha.testable$load_testables.invoke (testable.clj:144)
    kaocha.api$test_plan.invokeStatic (api.clj:56)
    kaocha.api$test_plan.invoke (api.clj:49)
    kaocha.api$run$fn__2854.invoke (api.clj:101)
    ...
    kaocha.api$run.invokeStatic (api.clj:99)
    kaocha.api$run.invoke (api.clj:86)
    kaocha.repl$run.invokeStatic (repl.clj:159)
    kaocha.repl$run.doInvoke (repl.clj:127)
    ...
    user$eval2970.invokeStatic (NO_SOURCE_FILE:1)
    user$eval2970.invoke (NO_SOURCE_FILE:1)
    ...
1 tests, 1 assertions, 1 errors, 0 failures.
#:kaocha.result{:count 1, :pass 0, :error 1, :fail 0, :pending 0}

I get the same results

seems to work here, on Kaocha 1.64.1010

*correction - this might be related to cursive in some way

$ clj
Downloading: lambdaisland/kaocha/1.64.1010/kaocha-1.64.1010.pom from clojars
Downloading: lambdaisland/kaocha/1.64.1010/kaocha-1.64.1010.jar from clojars
Clojure 1.10.3
user=> (require 'kaocha.repl)
nil
user=> (require 'dev.freeformsoftware.main-test)
nil
user=> (kaocha.repl/run (ns-name 'dev.freeformsoftware.main-test))
[E]

ERROR in all (dev/freeformsoftware/random_error_test.clj:1)
Failed loading tests:
Exception: clojure.lang.Compiler$CompilerException: Syntax error compiling at (dev/freeformsoftware/random_error_test.clj:1:1).
#:clojure.error{:phase :compile-syntax-check, :line 1, :column 1, :source "dev/freeformsoftware/random_error_test.clj"}
 at clojure.lang.Compiler.analyze (Compiler.java:6812)
    ...
    kaocha.ns$required_ns.invokeStatic (ns.clj:13)
    kaocha.ns$required_ns.invoke (ns.clj:11)
    kaocha.type.ns$eval3179$fn__3180.invoke (ns.clj:29)
    ...
    kaocha.testable$load.invokeStatic (testable.clj:94)
    kaocha.testable$load.invoke (testable.clj:75)
    kaocha.testable$load_testables.invokeStatic (testable.clj:152)
    kaocha.testable$load_testables.invoke (testable.clj:144)
    kaocha.load$load_test_namespaces.invokeStatic (load.clj:49)
    kaocha.load$load_test_namespaces.doInvoke (load.clj:37)
    ...
    kaocha.type.clojure.test$eval4601$fn__4602.invoke (test.clj:17)
    ...
    kaocha.testable$load.invokeStatic (testable.clj:94)
    kaocha.testable$load.invoke (testable.clj:75)
    kaocha.testable$load_testables.invokeStatic (testable.clj:152)
    kaocha.testable$load_testables.invoke (testable.clj:144)
    kaocha.api$test_plan.invokeStatic (api.clj:56)
    kaocha.api$test_plan.invoke (api.clj:49)
    kaocha.api$run$fn__2854.invoke (api.clj:101)
    ...
    kaocha.api$run.invokeStatic (api.clj:99)
    kaocha.api$run.invoke (api.clj:86)
    kaocha.repl$run.invokeStatic (repl.clj:159)
    kaocha.repl$run.doInvoke (repl.clj:127)
    ...
    user$eval2968.invokeStatic (NO_SOURCE_FILE:1)
    user$eval2968.invoke (NO_SOURCE_FILE:1)
    ...
Caused by: java.lang.RuntimeException: Unable to resolve symbol: x in this context
 at clojure.lang.Util.runtimeException (Util.java:221)
    ...
    kaocha.ns$required_ns.invokeStatic (ns.clj:13)
    kaocha.ns$required_ns.invoke (ns.clj:11)
    kaocha.type.ns$eval3179$fn__3180.invoke (ns.clj:29)
    ...
    kaocha.testable$load.invokeStatic (testable.clj:94)
    kaocha.testable$load.invoke (testable.clj:75)
    kaocha.testable$load_testables.invokeStatic (testable.clj:152)
    kaocha.testable$load_testables.invoke (testable.clj:144)
    kaocha.load$load_test_namespaces.invokeStatic (load.clj:49)
    kaocha.load$load_test_namespaces.doInvoke (load.clj:37)
    ...
    kaocha.type.clojure.test$eval4601$fn__4602.invoke (test.clj:17)
    ...
    kaocha.testable$load.invokeStatic (testable.clj:94)
    kaocha.testable$load.invoke (testable.clj:75)
    kaocha.testable$load_testables.invokeStatic (testable.clj:152)
    kaocha.testable$load_testables.invoke (testable.clj:144)
    kaocha.api$test_plan.invokeStatic (api.clj:56)
    kaocha.api$test_plan.invoke (api.clj:49)
    kaocha.api$run$fn__2854.invoke (api.clj:101)
    ...
    kaocha.api$run.invokeStatic (api.clj:99)
    kaocha.api$run.invoke (api.clj:86)
    kaocha.repl$run.invokeStatic (repl.clj:159)
    kaocha.repl$run.doInvoke (repl.clj:127)
    ...
    user$eval2968.invokeStatic (NO_SOURCE_FILE:1)
    user$eval2968.invoke (NO_SOURCE_FILE:1)
    ...
1 tests, 1 assertions, 1 errors, 0 failures.
#:kaocha.result{:count 1, :pass 0, :error 1, :fail 0, :pending 0}
user=> 

I see the errors when running a raw clj command

sounds like a #cursive issue in that case

I do see this on the version you have in your deps.edn

➜ clj     
Clojure 1.10.3
   (require 'kaocha.repl)
nil
   (require 'dev.freeformsoftware.main-test)
nil
   (kaocha.repl/run (ns-name 'dev.freeformsoftware.main-test)))
WARNING: No tests were found, make sure :test-paths and :ns-patterns are configured correctly in tests.edn.
Execution error (ExceptionInfo) at slingshot.support/stack-trace (support.clj:201).
throw+: #:kaocha{:early-exit 0}
Syntax error reading source at (REPL:3:64).

Maybe. I'm looking to see what's different about the cursive repl

so maybe make sure Cursive is also picking up the latest version, I'm pretty sure we put out a fix for exactly this not too long ago.

Yeah, confirmed it's something to do with the cursive repl. Running the cursive clj repl command, I get the same results as when I run in cursive itself. There's some mystery dep in it's classpath that's messing with reporting.

this is how I invoke kaocha from cursive:

(do
  (require 'clojure.spec.alpha 'expound.alpha 'kaocha.repl)
  (binding [clojure.spec.alpha/*explain-out* expound.alpha/printer]
    (kaocha.repl/run-all)))

I save that as a repl command. Not sure whether it solves your issue though.