kaocha

rgm 2022-07-08T05:32:00.578679Z

I'm wondering if I've noticed a known behaviour with kaocha-cljs2 / chui / funnel, a browser connection, and bin/kaocha --watch. (It's entirely possible I've set something up wrong). A test re-run is getting correctly triggered by saving in my editor: only that namespace is re-running. But it seems the tests themselves are being cached. Example; changing (is (= "fred" "ginger")) to (is (= "fred" "barney")) will re-run in the chui browser UI and keep showing expected is "fred" and actual is "ginger" (should still be failing, but be "barney" as actual). If I do the first edit, reload the browser, then do the second edit, the browser picks it up and shows expected "fred" actual "barney" (correct). I tried both firefox and edge on a Mac.

rgm 2022-07-08T05:48:57.012559Z

Hm. Just had the thought that I should try opening the browser devtools and disabling cache. I'll try that in the morning.

plexus 2022-07-09T17:47:35.843059Z

You need to tell shadow that our test collection macro is not cacheable. Maybe that's it?

plexus 2022-07-12T14:03:58.379999Z

That's really great to hear... I sometimes assume no one really uses kaocha-cljs2 because it's such a hassle to set up. Truth is that even though it's a building block kit so it can work with any combination of cljs tooling, compiler settings, js runtimes, etc, 90% of usage is shadow+browser. Just never got around to adding the extra layer of convenience.

rgm 2022-07-08T16:43:02.700349Z

Fascinating: nope ... it's not the browser cache. The signal to re-test is getting through but not the new test code.

Alys Brooks 2022-07-08T17:24:45.737489Z

Strange. Is kaocha --watch printing a message about namespaces reloading?

rgm 2022-07-10T23:33:45.582969Z

maybe ... I've tried shadow's ^:dev/always in the test namespace, and :build-options {:cache-level :off} in my shadow test target, and that hasn't worked. I'm not seeing any namespace reloads in the kaocha --watch pane, though maybe I need to set a verbose option. What does work is just reloading the chui browser tab, and then manually triggering the changed test from within the browser. At that point it picks up the changed test without any (apparent) further involvement by shadow or kaocha.

rgm 2022-07-10T23:36:16.523269Z

so based on that, my working hypothesis is that shadow is re-compiling correctly, and kaocha is triggering a test run correctly, but the chui session is not picking up on it for some reason.

rgm 2022-07-10T23:47:33.464049Z

yep, can confirm that shadow is writing a new copy of target/kaocha-test/js/test.js with the new code, and kaocha is sending a bunch of websocket calls into the browser (fetch-test-data, start-run, start-ns, run-test etc), but the browser session (based on looking at the network dev tools) isn't realizing that test.js has changed so all of kaocha's orchestration is getting run against the old test bundle.

rgm 2022-07-10T23:49:28.344699Z

(I ran tests, saved a copy of the test bundle, triggered another run in the editor, then diffed the two js files)

rgm 2022-07-10T23:51:50.854529Z

I don't think I changed anything, but I'm also not sure it ever worked ... this is the first time I'm trying to TDD something cljs only. I usually only TDD things in cljc namespaces and let CI validate cljs in a headless browser, which wouldn't care about this reload issue.

rgm 2022-07-10T23:56:27.810419Z

Anyway I guess the workaround is to just change code, reload browser, trigger test in browser. Not the biggest deal.

plexus 2022-07-11T11:46:14.722289Z

Have you tried :cache-blockers #{lambdaisland.chui.test-data}?

plexus 2022-07-11T11:47:07.339669Z

I've had a lot of issues like this developing this stuff, and put a lot of time into fixing them... the whole point is to have a good hot-reload autotest workflow.

plexus 2022-07-11T11:47:51.993439Z

you're using :runner-ns kaocha.cljs2.shadow-runner?

plexus 2022-07-11T11:49:35.494069Z

oh wait I think this could be a HTTP issue, how are you serving up your compiled assets? your http server should provide the right caching headers.

rgm 2022-07-11T15:57:19.355409Z

I hadn't tried the cache-blockers but have now... nope, still not loading the new tests. The test.js bundle is coming in with Cache-Control: private, no-cache which is shadow serving it out on port 1818 via :dev-http {1818 "target/kaocha-test"}

rgm 2022-07-11T15:57:41.670139Z

I'll try to extract a minimal repro repository.

plexus 2022-07-11T20:08:22.521379Z

Thanks, sorry for the trouble!

rgm 2022-07-11T20:40:24.305359Z

no worries; I know this is tricky and it's at least 65% likely to be me doing something incorrectly

rgm 2022-07-11T20:41:17.626679Z

also: this was a good slamming on of the brakes to make me realize oh-wait-this-should-be-a-pure-data-model-layer-test anyway 😜

rgm 2022-07-11T20:42:20.790639Z

(I ended up here trying to test something using re-frame-test when it's better tested as "model" ie. map/record transforms)

rgm 2022-07-11T21:46:10.911649Z

@plexus this folder in my experiments is a minimal repro ... this should demonstrate my issue https://github.com/rgm/experiments/pull/72/files

rgm 2022-07-11T21:51:33.897429Z

(Incidentally no part of this should be taken as crapping on your hard work. I'm so grateful cljs2 exists … it's been rock-solid in CI and as I do dev work).