There’s now at least one function in the codebase that I’m looking at that has a 1.5 pages long docstring. It contains a detailed log of every change made to that function, and a thorough explanation of the function itself plus a lot of the code around the function. The function body itself is a few lines of code.
It's the AI fingerprint!
I used to ask people to comment the “why”, not the “how” of code; tell me why it is out of the ordinary because I can read the code. LLMs suffer from much the same sometimes
Depends on the model, I think. Claude is much more happy to write large comments than Codex seems to be
My experience with Astra is that it completely slaughters comments written by Claude. The result is much more readable in my opinion.
I've seen recommendations to direct your model not to write the docs at all. Ostensibly, even the code becomes better that way.
Writing them in STE with a max of 2 lines and a clear why seems to be prett decent
Is AI aware of version control and its use cases?
It seems very belt-and-suspenders minded at times, at least Claude.
Its just very good at writing text, and isn’t fully aware yet when its a good idea to run with those scissors
Very long documentation is also symptomatic for waterfallish development processes..
That seems correlated, at least. From what I could see, it’s been iterated over using massive “openspec”s.
I think Kent Beck just argued that AI spec driven development is heavily a waterfall-like process, so process wise we taking a step back. I think the presentation was linked on this channel.
I’m not personally comfortable with that mode of development. I tend to discuss a problem for a good while (humans and LLMs), and then write a fairly dense overview (usually with the help of an LLM, they frigging love their own writing, so it helps). Then I tend to implement it piece by piece, usually both LLM and I as writers depending on what I’m touching. Opening Claude Code and writing “yolo” seems to be the path to disappointment, at least for the people around you.
Oh, I didn't mean that you do it like that, sorry! The documentation horrors just reminded me if waterfall, which i have endured too much in the early 2000's 🙂
And I see the similarity in spec-driven AI and waterfall (and Beck had convincing speech about)
No worries, I didn’t read it as if you did. I’ve been on the “receiving end” of it though.
Yes, doc strings do end up with cruft. I use something like this fairly regularly ...
Sweep docstrings and comments in for historical narration: text that describes how the code used to work or how it changed, rather than what it does now. Signals: "previously", "formerly", "used to", "no longer", "was
changed to", "new approach", "migrated from", "replaces the old", "as of v0.x", "NEW:", "Phase 2".
For each one, rewrite it to state the current behaviour in present tense, or delete it if nothing current remains. Keep genuine rationale (why it works this way, what pitfall it avoids) but phrase it as a present-tense fact, not a story about the change. I’ve learned that sometimes it’s hard to fight LLMs with rules etc., and it’s better to redirect their innate tendencies. So I’ve introduced a changelog where the LLMs can channel their need for narration.
Separate to an actual changelog that would be of interest to a human.
“Please write all explanations to the specially dedicated file located at /dev/null ”
I find I have a set of standard "sweeps" which remove some of the cruft that accumulates. One is the dehistoricalise one above.
Then there are ones which sweep unittests trimming (LLMs seems to love adding low value unittests)
I've ended up with a google doc of standard "sweeping" prompts
@mikethompson would you be able to share more of those?