This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
- # aleph (1)
- # announcements (2)
- # aws (3)
- # bangalore-clj (1)
- # beginners (136)
- # boot (3)
- # calva (89)
- # cider (44)
- # cljdoc (1)
- # cljs-dev (31)
- # clojure (101)
- # clojure-europe (3)
- # clojure-italy (52)
- # clojure-nl (7)
- # clojure-spec (12)
- # clojure-uk (34)
- # clojurescript (62)
- # community-development (46)
- # cursive (25)
- # datomic (6)
- # duct (26)
- # events (6)
- # figwheel-main (4)
- # fulcro (25)
- # graphql (2)
- # jackdaw (4)
- # jukebox (3)
- # kaocha (57)
- # leiningen (31)
- # off-topic (3)
- # onyx (4)
- # other-languages (22)
- # pathom (18)
- # re-frame (9)
- # reitit (3)
- # shadow-cljs (60)
- # spacemacs (9)
- # speculative (22)
- # sql (39)
- # tools-deps (45)
- # vim (37)
hi @caio, this is great! I didn't even know about Greenlight, or it would have been on my list. You're also the first person to contribute a new test type, so thank you and congratulations! I hope the process wasn't too painful.
Please add a
bin/kaocha as described in the documentation. This is a Kaocha convention, when you open a Kaocha project you should always be able to run all tests with
bin/kaocha without having to figure out anything else.
I do have some questions w.r.t. your reporter, you're directly forwarding greenlight events to clojure.test, and then extending
clojure.test/report. I would suggest not doing that. Kaocha forwards events it doesn't know about to the original
clojure.test/report multimethod, but that's really a fallback for people using Kaocha with legacy assertions+reporters.
Instead the recommended thing to do is to emit custom event types, but to use Kaocha's keyword hierarchy to map them to things Kaocha knows about. To get custom formatting of errors/failures you can extend
that said I see you're aiming for very specific output, maybe Kaocha doesn't yet have all the affordances you need. I'll have to play a bit with greenlight to get a better sense for that.
ns you're catching all load errors and turning them into warnings, I know that's what the
clojure.test implementation used to do as well, but I would no longer recommend it. Once this is merged errors can just bubble up, and they'll be reported as a failing test suite. https://github.com/lambdaisland/kaocha/pull/68
Finally I would suggest using a different namespace, since this is a contributed project rather than a core Kaocha project, so maybe name your test types something like
:caioaao.kaocha.greenlight/var and namespaces respectively. (it would also be a good idea to think about which maven org you would publish this under, and make your namespace names line up)
FWIW, I'm not ecstatic about the
bin/kaocha convention, and I don't use it.
clojure -A:test is just as good a convention IMO, and my projects all have Makefiles anyway. In any given project I'd expect one of
make test to work
it also gives a project maintainer a place to add setup like setting env vars (perhaps later on). Finally we might conceivable start distributing a more intelligent wrapper script in the future that further smooths out the difference between boot and the rest.
the latter is in my opinion an argument for not having custom binstubs. that wrapper is guaranteed to do other things than the binstubs people create now
I just don't see any value in trying to introduce a new convention that is very specific to just one tool
and even if kaocha did ship a wrapper script that did smart things I would prefer not to rely on anything outside of my project configuration (e.g. global binaries from
it's a convention that is strongly recommended from the project's side so that everyone is on the same page. All documentation will reference
bin/kaocha rather than any specific tool, and people will be taught to expect a
bin/kaocha script when they clone a kaocha-based project, so they don't have to look around how exactly kaocha is run in this project. If you don't want to add it then you're absolutely free not to do so. You can also name
and explain in the README to run
clojure -A:dev:test:dev-cljs --config-file . Do what you must.
the clojure CLI tools aliases are a very good argument for a wrapper BTW. projects start out with a
:test aliases but before you know it you need to add three or four to make things run, and if you add one that doesn't exist clojure CLI will barf, so you can't just use
-A:dev:test to be "safe"
honestly, I think the best (and most realistic) convention is to make it very visible in the Readme how to run tests, and that it can be done on a fresh checkout
going to the Readme to figure out if this is a kaocha-project is just as much work as going to the Readme to discover you can type
I'm not going to battle this out anymore, was just an opinion, feel free to disregard
no worries, no offence taken. Apologies if my response was stronger than called for. I realize this may raise some objections, people don't like to be told how to organize their projects. I still plan to push for it and see if people can get used to it. We'll see if it catches on or not.
I'm all for conventions where practical. I have no need to name my config file http://chewbacca.xxx, tests.edn is just fine 🙂
This is of course the bottom line, if everybody would already do this I would be extremely happy.
Hi @plexus Nice blog you had on https://lambdaisland.com/blog/2017-05-25-simple-and-happy-is-clojure-dying-and-what-has-ruby-got-to-do-with-it I'll make sure to read some more posts when I can
I'm working through Tim Pratley's https://timothypratley.github.io/enterprise-clojure-training/manual.html
I'm not really a cursive user, but you should be able to make due with Kaocha's repl interface
Tim's manual describes the leiningen plugin lein-test-refresh that he uses in his workflow. That's what you call the 'Watch mode' in Kaocha?
> I just don’t see any value in trying to introduce a new convention that is very specific to just one tool Also without the wish to come across hostile, I kind of agree. A script doesn’t compose with including other aliases from deps.edn. People should be able to make the scripts specifically for their projects without relying on conventions.
As for plugins, I wonder if there’s an equivalent of
fixtures in kaocha. Right now it seems that every plugin has to re-invent how to act as a fixture around some level of test (suite, ns, var). It would be nice to see a more general approach to this. Having
wrap-test-ns etc. hooks would maybe help?
ns are implementation details of
:kaocha.type/clojure.test. Other test types will not have these. Most test types have three levels so far (e.g.
scenario), but that too is arbitrary. You can check however if you're dealing with a top-level testable (`:kaocha.testable.type/suite`), with a bottom-level one (`:kaocha.testable.type/test`), or with some intermediate grouping level (`:kaocha.testable.type/group`). So you would check for those in a
post-test hook. With the helpers in
kaocha.hierarchy that should be straightforward, so I don't think adding extra hooks for those is necessary.
Yeah. The thing I was thinking about here was: Like in Ring-handlers or fixtures, you can have some state. If I did this thing in the pre-step, then I have to undo it in the post-step, but otherwise not.
Unless you hack some atoms into your plugin, but that’s kind of a workaround and it won’t scale with running tests in parallel
Not that I need this functionality now urgently, but I can see it being useful for doing stuff in a
binding like construct, temporarily disabling instrumentation before a test and instrument again at the end of a test
Hey @U04V15CAJ, sorry for the very slow response, I had somehow missed this thread earlier.
> Like in Ring-handlers or fixtures, you can have some state. If I did this thing in the pre-step, then I have to undo it in the post-step, but otherwise not. You can store state in the testable, whatever you return from a hook gets passed further down, so you would do something like this
(defplugin foo (pre-test [testable test-plan] (assoc testable ::did-the-thing? true)) (post-test [testable test-plan] (when (::did-the-thing? testable) (undo-the-thing)) testable))
(and I’m finally getting the hang of the name, I don’t write koacha that often anymore :-D))
Another issue: when you write a kaocha plugin, how does it work in CLJS? I want to turn on speculative specs in CLJS, but does that work automatically when I write my plugin in
There are no mechanisms in place yet for hooking into CLJS, plugins run in Clojure. Like you said, it's still early days for kaocha-cljs.
I'm running tests with kaocha and cljs (nodejs), how can I print stuff?
(println) seems to go missing, as does
js/console.log, and I can't seem to find the node stdout log anywhere
--no-capture-output. If it still doesn't show up then a reproducible case and a GH issue would be very appreciated!
@plexus thanks for the tips! I'll implement them asap. the process wasn't painful at all, but the docs could help a little bit more. I had to do some reverse engineering on the
clojure.test implementation to figure some things out. and those naming conventions and reporting stuff could also be documented. other than that, it was waaay easier than I anticipated 🙂 I'm really impressed with the project
Point well taken, how test type work and the affordances provided to them by kaocha's reporters really need more documentation. I'll make some time for that.
I second the fixtures plugin thing. that would be nice but I also don't like having the tests depend on the test runner, so it looks kinda hard without some sort of abstraction leak