Fork me on GitHub
#kaocha
<
2022-04-05
>
JAtkins11:04:15

quick request - in kaocha.api/run , there's this bit of code:

99 ...
            (let [config (resolve-reporter config)]
              (let [test-plan (test-plan config)]
                (when-not (some #(or (hierarchy/leaf? %)
                                     (::testable/load-error %))
                                (testable/test-seq test-plan))
                  (output/warn (str "No tests were found, make sure :test-paths and "
                                    ":ns-patterns are configured correctly in tests.edn."))
                  (throw+ {:kaocha/early-exit 0 }))
108 ...
this eats load errors, which in my case was a compilation error. Can those errors be printed to console? I'm happy to make a pr if desired, but I'm tied up right now with other things at work.

plexus13:04:18

How does this eat load errors exactly? This code is meant to explicitly not eat load errors.

plexus13:04:49

We store load errors in the test-plan so they can be printed out by the reporter. If that's broken in some scenarios then please open an issue and we'll give it priority. A repro repo would be much appreciated!

JAtkins13:04:29

Load a repl, and run the comment block in user.clj. Requiring main-test goes well - no dependency errors there. However, afaict test-plan makes a test plan for all nses, including the faulty one random-error-test. All I receive on the repl is

=> (kaocha.repl/run (ns-name 'dev.freeformsoftware.main-test))
WARNING: No tests were found, make sure :test-paths and :ns-patterns are configured correctly in tests.edn.
Execution error (ExceptionInfo) at slingshot.support/stack-trace (support.clj:201).
throw+: #:kaocha{:early-exit 0}

plexus13:04:30

is this only an issue when run from the REPL?

JAtkins13:04:03

yep - it would not run from the command line because of the file error. the clojure compiler would yell at me 🙂

JAtkins13:04:52

Here, the compiler still yells. Just under the ::testable/load-error key, which isn't printed

plexus13:04:27

Can you start by upgrading to latest Kaocha?

plexus13:04:42

Caused by: java.lang.RuntimeException: Unable to resolve symbol: x in this context
 at clojure.lang.Util.runtimeException (Util.java:221)
    ...
    kaocha.ns$required_ns.invokeStatic (ns.clj:13)
    kaocha.ns$required_ns.invoke (ns.clj:11)
    kaocha.type.ns$eval3181$fn__3182.invoke (ns.clj:29)
    ...
    kaocha.testable$load.invokeStatic (testable.clj:94)
    kaocha.testable$load.invoke (testable.clj:75)
    kaocha.testable$load_testables.invokeStatic (testable.clj:152)
    kaocha.testable$load_testables.invoke (testable.clj:144)
    kaocha.load$load_test_namespaces.invokeStatic (load.clj:49)
    kaocha.load$load_test_namespaces.doInvoke (load.clj:37)
    ...
    kaocha.type.clojure.test$eval4603$fn__4604.invoke (test.clj:17)
    ...
    kaocha.testable$load.invokeStatic (testable.clj:94)
    kaocha.testable$load.invoke (testable.clj:75)
    kaocha.testable$load_testables.invokeStatic (testable.clj:152)
    kaocha.testable$load_testables.invoke (testable.clj:144)
    kaocha.api$test_plan.invokeStatic (api.clj:56)
    kaocha.api$test_plan.invoke (api.clj:49)
    kaocha.api$run$fn__2854.invoke (api.clj:101)
    ...
    kaocha.api$run.invokeStatic (api.clj:99)
    kaocha.api$run.invoke (api.clj:86)
    kaocha.repl$run.invokeStatic (repl.clj:159)
    kaocha.repl$run.doInvoke (repl.clj:127)
    ...
    user$eval2970.invokeStatic (NO_SOURCE_FILE:1)
    user$eval2970.invoke (NO_SOURCE_FILE:1)
    ...
1 tests, 1 assertions, 1 errors, 0 failures.
#:kaocha.result{:count 1, :pass 0, :error 1, :fail 0, :pending 0}

JAtkins13:04:57

I get the same results

plexus13:04:59

seems to work here, on Kaocha 1.64.1010

JAtkins13:04:06

*correction - this might be related to cursive in some way

JAtkins13:04:29

$ clj
Downloading: lambdaisland/kaocha/1.64.1010/kaocha-1.64.1010.pom from clojars
Downloading: lambdaisland/kaocha/1.64.1010/kaocha-1.64.1010.jar from clojars
Clojure 1.10.3
user=> (require 'kaocha.repl)
nil
user=> (require 'dev.freeformsoftware.main-test)
nil
user=> (kaocha.repl/run (ns-name 'dev.freeformsoftware.main-test))
[E]

ERROR in all (dev/freeformsoftware/random_error_test.clj:1)
Failed loading tests:
Exception: clojure.lang.Compiler$CompilerException: Syntax error compiling at (dev/freeformsoftware/random_error_test.clj:1:1).
#:clojure.error{:phase :compile-syntax-check, :line 1, :column 1, :source "dev/freeformsoftware/random_error_test.clj"}
 at clojure.lang.Compiler.analyze (Compiler.java:6812)
    ...
    kaocha.ns$required_ns.invokeStatic (ns.clj:13)
    kaocha.ns$required_ns.invoke (ns.clj:11)
    kaocha.type.ns$eval3179$fn__3180.invoke (ns.clj:29)
    ...
    kaocha.testable$load.invokeStatic (testable.clj:94)
    kaocha.testable$load.invoke (testable.clj:75)
    kaocha.testable$load_testables.invokeStatic (testable.clj:152)
    kaocha.testable$load_testables.invoke (testable.clj:144)
    kaocha.load$load_test_namespaces.invokeStatic (load.clj:49)
    kaocha.load$load_test_namespaces.doInvoke (load.clj:37)
    ...
    kaocha.type.clojure.test$eval4601$fn__4602.invoke (test.clj:17)
    ...
    kaocha.testable$load.invokeStatic (testable.clj:94)
    kaocha.testable$load.invoke (testable.clj:75)
    kaocha.testable$load_testables.invokeStatic (testable.clj:152)
    kaocha.testable$load_testables.invoke (testable.clj:144)
    kaocha.api$test_plan.invokeStatic (api.clj:56)
    kaocha.api$test_plan.invoke (api.clj:49)
    kaocha.api$run$fn__2854.invoke (api.clj:101)
    ...
    kaocha.api$run.invokeStatic (api.clj:99)
    kaocha.api$run.invoke (api.clj:86)
    kaocha.repl$run.invokeStatic (repl.clj:159)
    kaocha.repl$run.doInvoke (repl.clj:127)
    ...
    user$eval2968.invokeStatic (NO_SOURCE_FILE:1)
    user$eval2968.invoke (NO_SOURCE_FILE:1)
    ...
Caused by: java.lang.RuntimeException: Unable to resolve symbol: x in this context
 at clojure.lang.Util.runtimeException (Util.java:221)
    ...
    kaocha.ns$required_ns.invokeStatic (ns.clj:13)
    kaocha.ns$required_ns.invoke (ns.clj:11)
    kaocha.type.ns$eval3179$fn__3180.invoke (ns.clj:29)
    ...
    kaocha.testable$load.invokeStatic (testable.clj:94)
    kaocha.testable$load.invoke (testable.clj:75)
    kaocha.testable$load_testables.invokeStatic (testable.clj:152)
    kaocha.testable$load_testables.invoke (testable.clj:144)
    kaocha.load$load_test_namespaces.invokeStatic (load.clj:49)
    kaocha.load$load_test_namespaces.doInvoke (load.clj:37)
    ...
    kaocha.type.clojure.test$eval4601$fn__4602.invoke (test.clj:17)
    ...
    kaocha.testable$load.invokeStatic (testable.clj:94)
    kaocha.testable$load.invoke (testable.clj:75)
    kaocha.testable$load_testables.invokeStatic (testable.clj:152)
    kaocha.testable$load_testables.invoke (testable.clj:144)
    kaocha.api$test_plan.invokeStatic (api.clj:56)
    kaocha.api$test_plan.invoke (api.clj:49)
    kaocha.api$run$fn__2854.invoke (api.clj:101)
    ...
    kaocha.api$run.invokeStatic (api.clj:99)
    kaocha.api$run.invoke (api.clj:86)
    kaocha.repl$run.invokeStatic (repl.clj:159)
    kaocha.repl$run.doInvoke (repl.clj:127)
    ...
    user$eval2968.invokeStatic (NO_SOURCE_FILE:1)
    user$eval2968.invoke (NO_SOURCE_FILE:1)
    ...
1 tests, 1 assertions, 1 errors, 0 failures.
#:kaocha.result{:count 1, :pass 0, :error 1, :fail 0, :pending 0}
user=> 

JAtkins13:04:07

I see the errors when running a raw clj command

plexus13:04:29

sounds like a #cursive issue in that case

plexus13:04:56

I do see this on the version you have in your deps.edn

➜ clj     
Clojure 1.10.3
   (require 'kaocha.repl)
nil
   (require 'dev.freeformsoftware.main-test)
nil
   (kaocha.repl/run (ns-name 'dev.freeformsoftware.main-test)))
WARNING: No tests were found, make sure :test-paths and :ns-patterns are configured correctly in tests.edn.
Execution error (ExceptionInfo) at slingshot.support/stack-trace (support.clj:201).
throw+: #:kaocha{:early-exit 0}
Syntax error reading source at (REPL:3:64).

JAtkins13:04:38

I think the same happens on latest w/ cursive repl - just tried

plexus13:04:31

maybe inspect (kaocha.repl/test-plan) in that case

JAtkins13:04:47

can do - it's how I found out about the loading error. I'm now much more confused about how I'm not seeing the error in cursive though.

JAtkins14:04:11

You are right - new version of kaocha fixed it. TIL that cursive makes it's own classpath by default and ignores deps.edn.

JAtkins14:04:21

Sorry for all the fuss!

plexus14:04:22

No worries! Happy to hear it's working!

JAtkins13:04:05

Maybe. I'm looking to see what's different about the cursive repl

plexus13:04:24

so maybe make sure Cursive is also picking up the latest version, I'm pretty sure we put out a fix for exactly this not too long ago.

JAtkins14:04:20

Yeah, confirmed it's something to do with the cursive repl. Running the cursive clj repl command, I get the same results as when I run in cursive itself. There's some mystery dep in it's classpath that's messing with reporting.

imre14:04:25

this is how I invoke kaocha from cursive:

imre14:04:28

(do
  (require 'clojure.spec.alpha 'expound.alpha 'kaocha.repl)
  (binding [clojure.spec.alpha/*explain-out* expound.alpha/printer]
    (kaocha.repl/run-all)))

JAtkins14:04:04

New version of kaocha

JAtkins14:04:40

Turns out I thought I tested the new kaocha, but I hadn't refreshed the cursive deps project and I was still using the old version 😕

imre14:04:33

I save that as a repl command. Not sure whether it solves your issue though.