Clojurians Log v2

New Kaocha release! This fixes a bug added to gc-profiling in the previous version: https://github.com/lambdaisland/kaocha/releases/tag/v1.68.1059

Perhaps a post in #announcements and/or #releases may be useful to raise awareness too?

Thanks for taking care and fixing both issues in such a short time!

I'm sorry to bother you guys but I keep getting errors, this one is a bit tricky to reproduce so I'm not opening an issue on Github until I figure out what's causing it

The issue is that "sometimes" (non deterministic so it's probably tied to my particular test cases) kaocha just exit mid way returning exit code 1. The problem is that nothing is printed in the console and I don't know how to get to the underlying cause or exception.

I quickly skimmed thru Kaocha code base and it looks like it's all running in a giant try+ catch and I'm afraid that is blocking the actual exception to bubble up. Is there any way to avoid that?

We only catch our own own early-exit exception in there, any "normal" exceptions will bubble, or be reported as errors depending on which stage they happen in.

Is it possible you're hitting some of your own code that does a System/exit?

If the exit code is 1 then the only way that is coming from kaocha is if there's a single failing test.

Have you tried reusing the random seed to reproduce the issue?

What does "exit mid way" mean exactly?

> Is it possible you're hitting some of your own code that does a System/exit? It's extremely unlikely, we don't call System/exit directly anywhere and it's happening randomly at different points. > What does "exit mid way" mean exactly? I'm using the kaocha.report/documentation and I can see that (when it's stopping early) it's in a different spot everytime. > Have you tried reusing the random seed to reproduce the issue? First thing I tried, I'm always running with a set seed but the exit is not tied to any particular test case My first guess was a memory issue (and that's why I tried the gc-reporter btw) but it's happening even after increasing the heap space (via Xmx). Measuring system memory indicates that we are nowhere near the end of addressable memory.

It does sound a lot like something is killing your JVM...

I know, right? I can't figure out what though and I've got no output message to work with 😓

Maybe I should try running the tests from the REPL. Can I access the results/outputs when running it this way?

It's happening also on the REPL when running (k/run-all) it starts running, it prints a bunch of unit tests and then it just stops running without returning the map with the results. The REPL is still running and everything is still up.

hmmm ok that's fascinating

what's the return value of run-all?

Can you try (kaocha.api/run (kaocha.repl/config))?

> what's the return value of run-all? There's no return value as far as I can see, I'm back to the REPL ns waiting for an input. No message, no anything..

Let me try: (kaocha.api/run (kaocha.repl/config))

btw re-running run-all in the same REPL didn't triggered any error and completed fine returning the map with #:kaocha.result . This at least looks consistent with what I was seeing in the CLI.

Same issue using api/run the output looks something like:

user=> (kaocha.api/run (kaocha.repl/config))
--- unit (clojure.test) ---------------------------
a.test
 testing something
 testing something

[.... other tests .....]

z.test
 testing something
user=>

a function has to either return or error... you can't not have a return value

how about (pr-str (kaocha.api/run (kaocha.repl/config)))?

are you using parallelization?

I see what you mean, I'll try to debug it and see if I can find anything useful..

it's very fascinating, I must admit 😄 trying to come up with a hypothesis and failing...

> are you using parallelization? The tested code is highly parallel but I'm not using the parallelization from Kaocha, I'm running on the latest release..

you could try examing the process with jcmd/`jstack`, but without a hypothesis I'm not sure what that would yield

maybe also use the debug reporter

I'll try to dig deeper into the rabbit hole as soon as I have a little bit of time. In the mean time, thank you for your help!!!

something else, make sure to disable output capturing. That's general advice when debugging issues, you don't want some relevant warnings or other output to get swallowed

could also turn on kaocha debug mode

what's that?

oh, that question came from an unexpected source

I meant https://github.com/lambdaisland/kaocha/blob/main/doc/03_configuration.md#kaochareportdebug

And I just see you suggested that like 3 lines above

Apologies for the noise.

No worries :) I was being a little facetious. I was wondering if maybe you were thinking of the debug thing we have in kaocha-cljs2.