Fork me on GitHub
#kaocha
<
2022-07-08
>
rgm05:07:00

I'm wondering if I've noticed a known behaviour with kaocha-cljs2 / chui / funnel, a browser connection, and bin/kaocha --watch. (It's entirely possible I've set something up wrong). A test re-run is getting correctly triggered by saving in my editor: only that namespace is re-running. But it seems the tests themselves are being cached. Example; changing (is (= "fred" "ginger")) to (is (= "fred" "barney")) will re-run in the chui browser UI and keep showing expected is "fred" and actual is "ginger" (should still be failing, but be "barney" as actual). If I do the first edit, reload the browser, then do the second edit, the browser picks it up and shows expected "fred" actual "barney" (correct). I tried both firefox and edge on a Mac.

rgm05:07:57

Hm. Just had the thought that I should try opening the browser devtools and disabling cache. I'll try that in the morning.

rgm16:07:02

Fascinating: nope ... it's not the browser cache. The signal to re-test is getting through but not the new test code.

Alys Brooks17:07:45

Strange. Is kaocha --watch printing a message about namespaces reloading?

plexus17:07:35

You need to tell shadow that our test collection macro is not cacheable. Maybe that's it?

rgm23:07:45

maybe ... I've tried shadow's ^:dev/always in the test namespace, and :build-options {:cache-level :off} in my shadow test target, and that hasn't worked. I'm not seeing any namespace reloads in the kaocha --watch pane, though maybe I need to set a verbose option. What does work is just reloading the chui browser tab, and then manually triggering the changed test from within the browser. At that point it picks up the changed test without any (apparent) further involvement by shadow or kaocha.

rgm23:07:16

so based on that, my working hypothesis is that shadow is re-compiling correctly, and kaocha is triggering a test run correctly, but the chui session is not picking up on it for some reason.

rgm23:07:33

yep, can confirm that shadow is writing a new copy of target/kaocha-test/js/test.js with the new code, and kaocha is sending a bunch of websocket calls into the browser (fetch-test-data, start-run, start-ns, run-test etc), but the browser session (based on looking at the network dev tools) isn't realizing that test.js has changed so all of kaocha's orchestration is getting run against the old test bundle.

rgm23:07:28

(I ran tests, saved a copy of the test bundle, triggered another run in the editor, then diffed the two js files)

rgm23:07:50

I don't think I changed anything, but I'm also not sure it ever worked ... this is the first time I'm trying to TDD something cljs only. I usually only TDD things in cljc namespaces and let CI validate cljs in a headless browser, which wouldn't care about this reload issue.

rgm23:07:27

Anyway I guess the workaround is to just change code, reload browser, trigger test in browser. Not the biggest deal.

plexus11:07:14

Have you tried :cache-blockers #{lambdaisland.chui.test-data}?

plexus11:07:07

I've had a lot of issues like this developing this stuff, and put a lot of time into fixing them... the whole point is to have a good hot-reload autotest workflow.

plexus11:07:51

you're using :runner-ns kaocha.cljs2.shadow-runner?

plexus11:07:35

oh wait I think this could be a HTTP issue, how are you serving up your compiled assets? your http server should provide the right caching headers.

rgm15:07:19

I hadn't tried the cache-blockers but have now... nope, still not loading the new tests. The test.js bundle is coming in with Cache-Control: private, no-cache which is shadow serving it out on port 1818 via :dev-http {1818 "target/kaocha-test"}

rgm15:07:41

I'll try to extract a minimal repro repository.

plexus20:07:22

Thanks, sorry for the trouble!

rgm20:07:24

no worries; I know this is tricky and it's at least 65% likely to be me doing something incorrectly

rgm20:07:17

also: this was a good slamming on of the brakes to make me realize oh-wait-this-should-be-a-pure-data-model-layer-test anyway 😜

rgm20:07:20

(I ended up here trying to test something using re-frame-test when it's better tested as "model" ie. map/record transforms)

rgm21:07:10

@U07FP7QJ0 this folder in my experiments is a minimal repro ... this should demonstrate my issue https://github.com/rgm/experiments/pull/72/files

rgm21:07:33

(Incidentally no part of this should be taken as crapping on your hard work. I'm so grateful cljs2 exists … it's been rock-solid in CI and as I do dev work).

plexus14:07:58

That's really great to hear... I sometimes assume no one really uses kaocha-cljs2 because it's such a hassle to set up. Truth is that even though it's a building block kit so it can work with any combination of cljs tooling, compiler settings, js runtimes, etc, 90% of usage is shadow+browser. Just never got around to adding the extra layer of convenience.

rgm16:07:02

Fascinating: nope ... it's not the browser cache. The signal to re-test is getting through but not the new test code.