This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-03-10
Channels
- # announcements (1)
- # babashka (44)
- # beginners (188)
- # calva (37)
- # chlorine-clover (28)
- # cider (12)
- # clj-kondo (40)
- # clojars (1)
- # clojure (323)
- # clojure-austin (7)
- # clojure-europe (20)
- # clojure-italy (4)
- # clojure-nl (16)
- # clojure-spec (7)
- # clojure-uk (37)
- # clojuredesign-podcast (1)
- # clojurescript (30)
- # cryogen (2)
- # cursive (30)
- # data-science (1)
- # datomic (26)
- # emacs (1)
- # events (1)
- # figwheel-main (13)
- # fulcro (89)
- # garden (1)
- # graalvm (20)
- # graphql (8)
- # jobs (1)
- # jobs-discuss (1)
- # joker (6)
- # kaocha (125)
- # lambdaisland (1)
- # meander (42)
- # off-topic (18)
- # pathom (3)
- # pedestal (6)
- # shadow-cljs (55)
- # spacemacs (21)
- # sql (18)
- # tools-deps (8)
- # uncomplicate (2)
- # vim (1)
- # yada (3)
I have a shadow-cljs
based react-native project and Iβd like to run some tests with kaocha
however Iβm getting this error:
$ clj -A:test --no-capture-output
[TypeError: Cannot read property 'error__GT_str' of undefined
at Socket.<anonymous> ([stdin]:89:38)
at Socket.emit (events.js:321:20)
at Socket.EventEmitter.emit (domain.js:485:12)
at addChunk (_stream_readable.js:297:12)
at readableAddChunk (_stream_readable.js:269:11)
at Socket.Readable.push (_stream_readable.js:214:10)
at TCP.onStreamRead (internal/stream_base_commons.js:186:23)
E]
Any ideas what that might cause? I tested my configuration in a minimal project and it works. After the E]
the process keeps running with no output.
UPDATE: this error only happens when having thheller/shadow-cljs {:mvn/version "2.8.90"}
as a dependency in deps.edn
.
SOLVED: making use of an additional tools.deps
alias solved the problem by not having shadow-cljs
on the test classpath.
Thanks for reading and sorry for the noise πhey @dharrigan thanks for reporting this, I also assumed that defexpect would just work, but they are probably using custom event types which the reporter needs to know about. Not a lot of work and something we can add support for without directly having to have expectations as a dependency
I don't think #ref will be able to work inside of a #kaocha/v1 btw. Aero is unable to figure out the expected ordering.
Not saying I'm gonna do it π got a lot of things competing for my time right now, but if you file an issue I'll pitch in with what needs to happen. Would be helpful if you could run with --reporter kaocha.report/debug
, and look what values the :type
keys have
@flowthing never looked into it, I agree those stacktraces aren't great. Kaocha's stacktraces in general could be much improved. Would be useful to file an issue on kaocha-cljs
to track this.
@dominicm that makes sense. Note that #kaocha/v1
is really just a convenience, you should be able to do kaocha --print-config > tests.edn.new
and go from there (cleaning a few things up, notably the current cli arguments)
it's more verbose, many more namespaced keys, and all defaults need to be filled in explicitly, but it's not a terrible format to work with
Yeah. That's one solution. You might also want to make kaocha/v1 a aero macro, that would give it control over how things are resolved inside of itself. Haven't thought about how you'd write that yet though.
it's already being handled by aero though, right? does it have a separate concept of "macros"?
Aero has reader "functions" (normal ones) and reader macros (like #profile and #ref). The macros can do special things like case, etc
@defa great to hear you got that working! shadow-cljs is not terribly well supported right now. If your code is compatible with standard clojurescript then you're all good, but if you use shadow-specific compiler extensions you may be out of luck
@dominicm I see, I guess it gets expanded at an earlier stage. Are there any downsides/risks to switching? anything that would behave differently? seems like a no-brainer to just go ahead with
You'd have to think a bit harder to have it work. Presumably users want their tags to expand without #kaocha/v1 having expanded. That's probably the easiest version. Essentially as if their map was the config file itself. The complex version involves trying to expand, then trying to add in the expanded keys if possible, then trying to continue.
@plexus no, there is no complier magic going on, just shadow-cljs on the classpath and a minimal test (is false)
. To clarify Iβm not running the tests via shadow-cljs
at all but separate via clj
.
Any ideas why there is no output captured even if I pass --no-capture-output
to the rest runner? My test fails and starts with:
(deftest foo
(println "Hello!")
...)
that's not enough context @defa... what does your tests.edn look like? can you create a minimal reproduction?
Sureβ¦
;; tests.edn
#kaocha/v1
{:tests [{:id :unit-cljs
:type :kaocha.type/cljs
:test-paths ["src/test"]
:ns-patterns ["-test$"]
}]
}
;; deps.edn
{:deps {}
:paths ["src/main" "src/test"]
:aliases
{:test
{:extra-deps {lambdaisland/kaocha {:mvn/version "0.0-590"}
lambdaisland/kaocha-cljs {:mvn/version "0.0-71"}}
:main-opts ["-m" "kaocha.runner"]}}}
(ns foo.foo-test
(:require [clojure.test :refer [deftest testing is are]]))
(deftest fail
(println "hello!")
(testing "will fail"
(is false "failed by intention")))
ok, so this is for clojurescript tests using node?. I would run with the debug options and see if you see any stdout messages coming back from the JS runtime. One possible explanation would be that the process exits before we receive the stdout. Do you see printlns outside the deftest?
(js/console.log "...")
does produce some output, inside and outside of deftest
but not in an ASCII art βboxβ.
β¦ and yes, this is ClojureScript and I suppose node is involved in the background. Iβm testing code that usually gets compiled by shadow-cljs
with a react-native target.
there are debug instructions in the kaocha-cljs README, which should produce some very verbose output.
> (js/console.log "...") does produce some output, inside and outside of deftest but not in an ASCII art βboxβ.
Maybe you just need an (enable-console-print!)
However it does not look like in the docs:
FAIL in sample-test/stdout-fail-test (sample_test.clj:10)
Expected:
:same
Actual:
-:same +:not-same
ββββββ Test output βββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Bu yi le hu?
β°βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
2 tests, 2 assertions, 1 failures.
β¦ but Iβm okay with that.is that with or without output capturing? you'll only get the frame when output capturing is on
I run kaocha like this:
clj -A:test --watch --no-capture-output
β¦ oh shitβ¦ Iβm so stupidβ¦ sorry. I misunderstood the semantics of the command line option πβ¦ but not always. In my real project
(enable-console-print!)
is necessary even without --no-capture-output
> Sorry if I was wasting your time⦠No problem at all! I like to make sure people are able to make the best of Kaocha
@plexus is it reasonable to pull in expectations in the test alias (in deps.edn) for kaocha so that I can test the =?
macro?
β― unzip -l clojure-test-1.2.1.jar
Archive: clojure-test-1.2.1.jar
Length Date Time Name
--------- ---------- ----- ----
0 2019-12-09 09:41 expectations/
0 2019-12-09 09:41 expectations/clojure/
13888 2019-12-09 09:17 expectations/clojure/test.clj
0 2019-12-09 09:41 META-INF/
57 2019-12-09 09:41 META-INF/MANIFEST.MF
0 2019-12-09 09:41 META-INF/maven/
0 2019-12-09 09:41 META-INF/maven/expectations/
0 2019-12-09 09:41 META-INF/maven/expectations/clojure-test/
114 2019-12-09 09:41 META-INF/maven/expectations/clojure-test/pom.properties
1895 2019-12-09 09:41 META-INF/maven/expectations/clojure-test/pom.xml
--------- -------
15954 10 files
Would it make sense for kaocha.report/fail-summary to look at t/testing-contexts if there's none in the message? They're only in the message from :summary
I'm writing a custom reporter, and I want to include details about the failure when it happens, as opposed to during the summary. This report has to be this format due to CI constraints.
right, you basically want to undo what kaocha does, i.e. capture failure events and only print the details at the end.
I would suggest putting :testing-contexts
in the map when you call it, rather than the other way around
That's what I'm doing for now, although it did feel like reaching as implementation details.
building kaocha's reporting on top of clojure.test is one of the few things I regret. It really should be a separate adapter layer that knows about these clojure.test vars and makes the messages self-contained
so think of it as kaocha's reporting API, where the messages are a generalization of clojure.test's messages
Heh, okay :) I guess everywhere else would be using the api wrong by not providing those keys?
Kaocha isn't picking up that some of my tests have meta when using focus-meta. I'd like to debug why. The printed plan is way too big. What else can I look at?
I've run the kaocha.load stuff that kaocha clojure test does, and I can see that kaocha has found the metadata on the tests, so I'm doubly confused :(
All of my test groups have a focus/skip. One of them has skip set to all of the other metadatas
and do you have test suites that have the same test-paths but are differentiated by skip/focus?
one problem with this is that the same test-id will occur multiple times in the test plan, we should really have a warning for that, as that will confuse plugins like the filtering plugin
That would solve my problem of wanting to make the groups explicitly, especially as I want to define an "inverse" (not :external, not :integration, not :devcards...)
yeah, wouldn't be exceptionally hard to implement either... I'm guesing a ~100 line plugin
although the real recommendation would be to split your tests on the filesystem test/unit/
/ test/integration
etc
I quite like the idea of having a mega suite, and just defining selectors on it. It's probably impractical, but has a good hierarchy in my brain.
This does explain why you run all tests if no focus meta matches. I was getting really annoyed with that :)
Hmm, that would still be useful for running a group of tests across suites. Eg run all unit tests, if there's none in cljs then it doesn't matter.
If I put together a PoC would you be able to get a little contribution going on the opencollective?
I did start floating the idea at the client about that. But they're still figuring out how to let people contribute bug fixes to oss.
I know. Sorry, I was trying to point out that they all haven't solved the problem of that. They are totally unprepared for funding.
in that case I would consider adding a plugin inside the project and doing it yourself, you can use the filter plugin for inspiration
it's really just recursively walking the test-plan, and marking testables as :kaocha.testable/skip true
or not
clj / cljs is fine because the ids are different, the cljs testables are prefixed with cljs:
so it's fine to e.g. have a single directory with cljc
files and consider that to be two test suites, one clj one cljs
right, you basically want to undo what kaocha does, i.e. capture failure events and only print the details at the end.
the other option is removing this association from the hierarchy (derive! :kaocha.type.var/zero-assertions :kaocha/fail-type)
(defmethod report/fail-summary :kaocha.type.var/zero-assertions [{:keys [testing-contexts testing-vars] :as m}])
Unrelated, the missing assertion test seems to be happening for a case which very obviously has assertions. My best guess is that the test is clojurescript async, and that isn't being handled. But I'm not certain at all. Anything ring a bell?
Sorry, I have lots of questions today. Our test suite is apparently not in the state kaocha expects.
ah it's a clojurescript test? although I guess same advice applies, as it piggiebacks on the event type and defmethod created by kaocha.type.var
questions very welcome! it's good to know what is or isn't working for people, or what isn't clear or obvious
I hope my questions aren't too burdensome. Fortunately my employer does allow contributions, so if something was major, I would be able to contribute. I'm hoping that my questions help improve kaocha more quickly under funding. It's a bit of a weak hope, so mostly I just hope I'm not too much trouble :). Open source & time is a hard thing, and I appreciate your time building cool things, and answering my questions.
and yeah when bringing on a big legacy test suite you're going to run into some things like this because kaocha is more strict about certain things (because it tries to prevent you from accidentally doing things you didn't intended). We really should have flags to disable these things to make porting to kaocha easier.
yeah not sure what else would be triggering ::zero-assertions
. Run the test with the debugging enabled and see what events you're getting (--focus on just that test). I would expect you are not seeing any assertions over the wire before the test finishes, in which case the question is why the test/runtime behave that way. If you are seeing assertions over the wire then the question is why kaocha.type.cljs
still thinks that (= pass error fail pending 0)
Figured it out. We have a namespace which runs tests as a top level side effect. I'm guessing they were running in parallel