Is there prior art watching (function) vars to trigger tests? I'm experimenting with a really nice test flow where changes to a var (using add-watch underneath) trigger related tests. I also have it give me a notification with the number of failures if any, or a message that all tests passed. It feels like the best of any autotest tool I've ever tried. Instant feedback, no need for a filewatcher, just repl evaluation - so no need to safe the file either, and only run the tests relevant to the changes you are currently making.
this is a really cool idea
Yeah it didn't occur to me before. I think because I was never close to the setup I now have. I have a variation on rfc and rich-comment-tests, both are great and led me here, but I was still missing something. So now I have roughly the following:
(defn foo [& _] )
(testing foo
(=> (foo 1) 2)
)
• When I evaluate the testing block I get the return value :pass or :fail, it reports the test output like with normal tests to the repl so I can inspect any failure. And I get a system notification with a summary
• When I evaluate foo almost the same happens, the testing block runs, but none blocking via a future (to keep immediate feedback)I'm checking if we are in a *repl* context for the above, otherwise I wrap the block in a deftest
The code I have is much less complicated than rfc and rich-comment-tests because I don't rewrite the code in the testing block, everything just evaluates as is
I need to test it a bit more, but I think I can release this as a simple library
If I understand correctly, the general idea is similar to Midje's "autotest" feature which watches changes in namespaces, reloads changed namespaces, and then runs tests that load those namespaces https://github.com/marick/Midje/wiki/Autotest I tried my hand at implementing this for Kaocha as well in https://github.com/lambdaisland/kaocha/pull/284 , given that Midje isn't recommended for new projects anymore, but I wasn't able to push over the finish line
ah okay, you're trying something that doesn't require filewatchers
Yeah exactly, it is really basic, but it works nicely for me so far
sounds interesting; I'd would be curious to see how the workflow is in practice
So normally i develop from the repl and actually deftest is a bit annoying as it doesn't run the test in repl mode (AFAIK). So I already created an adapted deftest macro to "fix" this.
But then the other problem is where do you have your tests. So if they are two different namespaces the overhead is already annoying (switch files editor focus etc).
So I started using rich-comment-test to stay in the context. But this has limited repl features. Then i considered rfc but they also do magic stuff to support the :- syntax.
So now i ended up with my own testing macro that checks if there is a symbol as a first argument. And then starts watching this var via add-watch. If there is no symbol things still work but no watching. I guess you can make this as complex as you want with multiple vars etc. But I was developing one function and then this flow was quite nice.
Probably it's a bit hard to follow just from text alone. I will try to publish something by next week or so
So this is the most interesting part of my macro
(if *repl*
(do
;; TODO need to handle load failures
(let [f# (fn [] (with-repl-test-reporter (fn [] ~@body)))]
~(when var-under-test
`(when-let [v# (var ~var-under-test)]
(let [f2# (fn [] (with-repl-test-reporter (fn [] ~@body)))]
(add-watch v# :autotest (fn [a# b# c# d#]
;; Wrap in future so we return right away
(future (f2#)))))))
(f#)))
(test/deftest ~name ~@body))
When a whole file is loaded/evaluated I have to be a bit smarter about running the tests etc. When there are multiple test blocks result reporting has to wait a bit. Otherwise you get many notifications at once and it is easy to miss one. So that is something I need to work on