@pez is there a way to use the backseat driver to run the tests using calvas test-runner (e.g. command run-all-test) or are there plans / would it even be possible to add this to it?
Add the test procedure to your copilot instructions. The models will use the REPL to run the necessary test on their own via run-tests. No need to use the calva commands here, IMHO.
From my instructions of QClojure (copied from @pez, i think):
Use the REPL for testing:
1. Load test namespace: (require 'namespace-test :reload)
2. Run tests: (run-tests 'namespace-test)
3. Test individual functions interactively
thanks, that also works nicely! 🙏 I still see some small benefits if it could somehow use the calva test functions, e.g. • when running calvas "run all tests" it shows me icons in the ide which tests are fine (green checkmark) and which failed (red cross) • also if I ran the tests manually before, I could ask ai to do "run failing tests again" and then fix them step by step. -> basically it would make it more seemless to switch between manually running the tests and let ai do some work but probably not a priority feature ^^
If you install Joyride you can tell the agent to use it for calling the Calva commands.
I am not sure, but it may work to use VS Code’s built in test runner. For which Copilot has built-in tools.
All that said, it may be that Backseat Driver should have a test tool. The reason being that then the agent doesn’t need to spend tokens/focus on getting the test runner code right.