It is pretty wild to be to tell VS Code, via voice, to make a change to your source files, and then tell it to run the tests and fix any problems 🙂
OK, truly wild...
Then I had it refactor functions -- and it automatically updated the tests to reflect the changes, even tho' it had to make a couple of passes to get everything right... then it explained to me exactly what changes it made.
Sometimes it changes the tests to make them pass. 😃
Well, I was asking it to make changes that I knew would break tests. What I'll try this week, is asking it to change tests to the new behavior I want, so tests fail, and then asking it to change only the source code to make them pass. But Kent Beck has some cautionary tales about attempting TDD with AI, so I don't know how successful that will be 😄
(took me several attempts to get Copilot / Claude 3.7 to understand how to do that -- initially it wanted me to run tests via the terminal -- but now I have it running via the REPL, it's pretty slick!)
You can give more instruction on tool use in copilot-instructions.md I told him to set the namespace to the relevant file we are working on when using the reply tool. Otherwise, by default, it's doing everything from user ns
Keep us posted about how you prompt it and such. There is probably a lot of room for improvements in the tool descriptions too: https://github.com/BetterThanTomorrow/calva-backseat-driver/blob/master/package.json#L91
I was kind of flailing around a bit, and getting frustrated (at least the AI doesn't mind if you yell at it), so I wasn't keeping track of what worked and what didn't. As I try it out this week at work, I'll take more notes...