hi @caio, this is great! I didn't even know about Greenlight, or it would have been on my list. You're also the first person to contribute a new test type, so thank you and congratulations! I hope the process wasn't too painful.

A few remarks:

Please add a bin/kaocha as described in the documentation. This is a Kaocha convention, when you open a Kaocha project you should always be able to run all tests with bin/kaocha without having to figure out anything else.

I do have some questions w.r.t. your reporter, you're directly forwarding greenlight events to clojure.test, and then extending clojure.test/report. I would suggest not doing that. Kaocha forwards events it doesn't know about to the original clojure.test/report multimethod, but that's really a fallback for people using Kaocha with legacy assertions+reporters.

Instead the recommended thing to do is to emit custom event types, but to use Kaocha's keyword hierarchy to map them to things Kaocha knows about. To get custom formatting of errors/failures you can extend kaocha.report/fail-summary and kaocha.report/print-expr

that said I see you're aiming for very specific output, maybe Kaocha doesn't yet have all the affordances you need. I'll have to play a bit with greenlight to get a better sense for that.

In ns you're catching all load errors and turning them into warnings, I know that's what the clojure.test implementation used to do as well, but I would no longer recommend it. Once this is merged errors can just bubble up, and they'll be reported as a failing test suite. https://github.com/lambdaisland/kaocha/pull/68

Finally I would suggest using a different namespace, since this is a contributed project rather than a core Kaocha project, so maybe name your test types something like :caioaao.kaocha/greenlight , :caioaao.kaocha.greenlight/ns, :caioaao.kaocha.greenlight/var and namespaces respectively. (it would also be a good idea to think about which maven org you would publish this under, and make your namespace names line up)

FWIW, I'm not ecstatic about the bin/kaocha convention, and I don't use it. clojure -A:test is just as good a convention IMO, and my projects all have Makefiles anyway. In any given project I'd expect one of lein test clojure -A:test make test to work

clojure -A:test doesn't work in leiningen or boot projects

make test would work for everyone 🙂

it also gives a project maintainer a place to add setup like setting env vars (perhaps later on). Finally we might conceivable start distributing a more intelligent wrapper script in the future that further smooths out the difference between boot and the rest.

the latter is in my opinion an argument for not having custom binstubs. that wrapper is guaranteed to do other things than the binstubs people create now

less than half of Clojure projects use make, so my muscle memory can't rely on it

I just don't see any value in trying to introduce a new convention that is very specific to just one tool

and even if kaocha did ship a wrapper script that did smart things I would prefer not to rely on anything outside of my project configuration (e.g. global binaries from $PATH)

but these are just my opinions, take them with a grain of salt 🙂

it's a convention that is strongly recommended from the project's side so that everyone is on the same page. All documentation will reference bin/kaocha rather than any specific tool, and people will be taught to expect a bin/kaocha script when they clone a kaocha-based project, so they don't have to look around how exactly kaocha is run in this project. If you don't want to add it then you're absolutely free not to do so. You can also name tests.edn and explain in the README to run clojure -A:dev:test:dev-cljs --config-file . Do what you must.

hehe

I fully support having a common ground to use in docs etc

the clojure CLI tools aliases are a very good argument for a wrapper BTW. projects start out with a :test aliases but before you know it you need to add three or four to make things run, and if you add one that doesn't exist clojure CLI will barf, so you can't just use -A:dev:test to be "safe"

honestly, I think the best (and most realistic) convention is to make it very visible in the Readme how to run tests, and that it can be done on a fresh checkout

going to the Readme to figure out if this is a kaocha-project is just as much work as going to the Readme to discover you can type make test

but bin/kaocha makes a whole lotta sense for docs etc

I'm not going to battle this out anymore, was just an opinion, feel free to disregard

wasn't my intention to be hostile

no worries, no offence taken. Apologies if my response was stronger than called for. I realize this may raise some objections, people don't like to be told how to organize their projects. I still plan to push for it and see if people can get used to it. We'll see if it catches on or not.

I'm all for conventions where practical. I have no need to name my config file http://chewbacca.xxx, tests.edn is just fine 🙂

😉

This is of course the bottom line, if everybody would already do this I would be extremely happy.

👍

hi @roger429 👋

Hi @plexus Nice blog you had on https://lambdaisland.com/blog/2017-05-25-simple-and-happy-is-clojure-dying-and-what-has-ruby-got-to-do-with-it I'll make sure to read some more posts when I can

I'm curious if you have some tips on using Kaocha in/with Cursive?

I'm working through Tim Pratley's https://timothypratley.github.io/enterprise-clojure-training/manual.html

I'm not really a cursive user, but you should be able to make due with Kaocha's repl interface

Tim's manual describes the leiningen plugin lein-test-refresh that he uses in his workflow. That's what you call the 'Watch mode' in Kaocha?

yes

> I just don’t see any value in trying to introduce a new convention that is very specific to just one tool Also without the wish to come across hostile, I kind of agree. A script doesn’t compose with including other aliases from deps.edn. People should be able to make the scripts specifically for their projects without relying on conventions.

As for plugins, I wonder if there’s an equivalent of fixtures in kaocha. Right now it seems that every plugin has to re-invent how to act as a fixture around some level of test (suite, ns, var). It would be nice to see a more general approach to this. Having wrap-test-var, wrap-test-ns etc. hooks would maybe help?

(and I’m finally getting the hang of the name, I don’t write koacha that often anymore :-D))

Another issue: when you write a kaocha plugin, how does it work in CLJS? I want to turn on speculative specs in CLJS, but does that work automatically when I write my plugin in .cljc?

I'm running tests with kaocha and cljs (nodejs), how can I print stuff? (println) seems to go missing, as does js/console.log, and I can't seem to find the node stdout log anywhere

@plexus thanks for the tips! I'll implement them asap. the process wasn't painful at all, but the docs could help a little bit more. I had to do some reverse engineering on the clojure.test implementation to figure some things out. and those naming conventions and reporting stuff could also be documented. other than that, it was waaay easier than I anticipated 🙂 I'm really impressed with the project

I second the fixtures plugin thing. that would be nice but I also don't like having the tests depend on the test runner, so it looks kinda hard without some sort of abstraction leak

I'll also check the existing report events and see if I can do something similar to what the authors of greenlight wanted for the reports