Clojurians Log v2

The only TDD that has actually worked for me is top down and coarse. Start at the boundary: the API or user action. Write a test that says "given X, returns Y and does Z side effects." Then implement to make it pass. Trying to TDD every module or function usually does not work. You do not know the right shape yet, so you end up locking yourself into the wrong abstractions. Sometimes you can do it at a module boundary if the public API is very clear upfront. But in practice, that is rare. Exploration tends to change what those functions should even be. So really it is more like integ test driven design :) The other place this works really well is bugs. Write the failing test that captures the bug, then fix it. Now you have a regression test for free.

Also, in some languages TDD is basically a poor man REPL. Take Java. You are building a class and thinking through its methods and structure, but you cannot really try it as you go. You are working blind. If you write the "tests" first, what you are really doing is creating a harness to call the class. Now you can actually use it and iterate on the design. The test becomes your REPL. It lets you run and poke at the class while you build it. Some people call this TDD, but it is not really about defining all final test cases upfront. It is about setting up a way to iterate REPL style in a language that does not have one.

Uncle Bob's (Robert C. Martin) videos are good

FWIW I don't think I have ever seen proper TDD anywhere except for blogs and vlogs, conferences, etc. At best, IRL in my experience people used it to mean "we try to get as close to 100% test coverage as feasible, and all new code is required to have tests". Personally, I'm a big anti-fan of TDD. :)

https://www.youtube.com/watch?v=4SIpyrko-x4

(someone has apparently chunked up a longer video of his into smaller bits)

Funny thing is that I just saw a 2020 Advent of Code-video of Uncle Bob live-coding, and then a video of Zach Tellman refuting that video. If I am not mistaken I heard Adam Tornhill of CodeScene recently say that he thinks TDD is the way to go (it was in a recent interview with Christoffer Ekeroth, again, if I'm not mistaken). I don't see the value of TDD myself (although I do not yet consider myself in a position to opine just yet) but my question is more broad regarding testing in general. How about unit tests in general? How about non-unit tests?

For what it's worth, I know The Primeagen is a big fan of unit tests but hates TDD. So that says something, even if he might not even be on the same track as us regarding FP at all.

I don't remember seeing any motions towards or talks about any kind of a survey that would delve specifically into testing. There's of course the Clojure Survey, but it doesn't have questions about approaches to testing. With that, I'm pretty sure that the only thing you'd be able to get out of a thread like this is not an answer to your actual question about the prevalent attitude but rather personal anecdotes and links to works of some public figures. And I'm not sure how useful it could be. At the end of the day: • Your project probably should have tests, unless it's trivial and/or a throw-away • The tests should cover at least the critical functionality and all the stuff that's perniciously hard to debug (although one should probably strive towards simplifying such stuff in the first place) • The tests should not rely on hard-coded data where it's possible • The tests should actually be checking for something meaningful (with fine-grained unit tests it's very common to see stuff that basically tests that 1 + 1 = 2, or some inane stuff like "set some field of this object, then test that the value of that field is the one that we set.) I could be missing something there, but anything else I can think of right now is not too far from just fluff. Like microgreens on top of a dish. Kinda help with representation, even have their own taste and texture, but at the end of the day it's the dish itself that matters.

@ilnar I can only interpret what Uncle Bob is saying there in one way, and that is that TDD is like a set of support wheels for programmers who aren't disciplined enough to criticize their own code and care more about the fact that it works than whether it is well-written or not. I agree that could be valuable in a dev team writing Java or C#. I think Clojurists value simplicity and the hammock highly enough to not benefit from this set of support wheels.

A relevant rhickey quote: > We say, “I can make a change because I have tests.” Who does that? Who drives their car around banging into the guard rails!?

@p-himik Would you disagree with the idea that test coverage helps with refactoring in a dynamically typed language? I struggle to envision how that would be done without tests (or something similar, like spec).

I agree with that idea, with the specific way you phrased it. I don't agree that testing is always required, that it's always worth it, that any kind of change (refactoring or otherwise) could be made "safely" if there are enough of tests. > I struggle to envision how that would be done without tests By the sheer power of one's mind and will. :D Refactoring is just moving code around, changing its shape. If nothing in your code relies on its shape (and it shouldn't!), you're golden - just do it. What could possibly break if you rely on your IDE and rename the foo.bar/baz var and all its references to my.proj/stuff? Nothing, if nothing tries to do something like dynamically constructing a symbol that a refactoring tool can't possibly detect.

Renaming a key in a map that's passed around is precisely what I had in mind. I suppose tools could be leveraged to do that (like an IDE with a renaming capability despite the fact that the language is dynamic). But maybe I have designed my code in too brittle a fashion to begin with if I have reached the point where I am afraid to rename a key in a simple map? You just said that the code shouldn't rely on the shape of the data; what does that mean, exactly? How can a function not be required to know the shape of its parameter?

I didn't say that. I said that code shouldn't rely on the shape of the code. Refactoring deals only with code. Changing data is not just refactoring anymore, it's a different kind of change. (Well, technically even renaming a var isn't refactoring, but I'm relying on a common meaning of the word "refactoring", not trying to rely on its root meaning.) Refactoring changes the code without changing its behavior. Changing how the data looks is changing behavior. And yes, it's harder than just renaming a var. Tests/specs/schemas help out here significantly more than during refactoring. But even in relatively large code bases, map key renames can be done without any tests or anything else. And I'd say that's how you should approach it anyway - after all, a green test does not mean that all is good. If you use simple keywords as keys (i.e. without namespaces), the process of renaming is quite tedious - search for all keyword usages, remove all search results that have nothing to do with the data in question (might require some deep digging here), go through the rest and rename, while double-checking. If you use fully-qualified keywords, and if you're also impeccably diligent about "same key => same meaning", then it's also just a single shortcut away - all :foo/bar can be renamed to :foo/baz if all instances of :foo/bar mean exactly the same thing. But that "impeccable diligence" is toxic in the sense that if :foo/bar means something in your project, it must mean exactly the same thing in every single place that touches your project, directly or via data. So being impeccably diligent is not always possible. So you might end up in a situation where e.g. at the ingress endpoint you must keep the keyword the same since the existing data is still out there with that key, and at the presentation level you must change the keyword to something else, for whatever reason (and yeah, not all reasons are valid). The same approach as for simple keywords would be required here.

Surely code is data? Unless you're referring to the data that the user sees, but that's not what I meant. Rewriting code is basically what I spend 95% of my time doing. I aim for the highest possible quality of my code, and that is really difficult to get right the first time (I would even say impossible). Rewriting includes renaming symbols and keywords, reshaping data, as well as refactoring in the traditional sense. I'm not sure how namespaced keywords can truly mitigate this problem. Most keywords are simple things like :highlighted? as the argument to a function that generates hiccup. It is very often that I decide that :highlighted? is a sub-optimal name and want to rename it to something else (e.g. :selected?). This keyword might be something that is not created "near" the location where the function is, and I'm starting to think that maybe that is the problem. Maybe I should be more diligent in reducing the number of references to a symbol from places "far away". Proximity (e.g. same file) certainly makes rewriting easier. I also feel like using a tool to rename symbols is somewhat of a hack and not something I want to rely on if the "proper" solution is writing less brittle code.

> Surely code is data? For all practical purpose, code is data only when you treat it as such. When you only run it, without programmatically writing or reading, it can be considered as non-data. > Unless you're referring to the data that the user sees Yes, with a very generic meaning of "user" - anything that uses the data that your code produces. Could be other code. Could be your own code. > Rewriting code is basically what I spend 95% of my time doing. TBH, this seems very, very skewed. But maybe that's just because we're in #beginners, I dunno. > Proximity (e.g. same file) certainly makes rewriting easier. There's an approach where all usages of particular keywords are hidden behind getter and setter functions. Sometimes it's useful, but usually I'd say it isn't. It's most definitely not something that should be used only to facilitate keyword renaming.

Replying to the initial question, there are lots of opinions on TDD (and use of REPL) and the context important. TDD approach is a design technique to help you design code effectively, the unit tests are just one output of this approach. Some people built there design experience following a strict TDD approach - write a failing test (thing about what success is), write code to make the test pass (consider design decisions) and then refactor (is this the write design choice with the information I currently know). Running the unit tests is one way to provide feedback whilst you design the code, but less important than the thinking process. The right level of unit tests can be very useful for picking up regressions in a code base, especially as that code base grows. Write too many tests and they will take too long to run (people wont run them as often or at all). And tests that run too deep into an implementation will be brittle, either preventing change or breaking with changes. Testing the public API of namespaces in Clojure is a common approach with keeps the test coverage light but effective. With Clojure you have a REPL that also provides feedback, arguably faster and more specific than running a test (as you can evaluate a single form through to the whole project). Using a REPL can be more of an ad-hoc approach to design as there is less structure, so relies on the engineer to have sufficient focus (this is good for trying lots of things, not so good if distracted in trying lots of things). For new projects or a new domain, I will use the Clojure REPL to experiment with design choices which may change quite often or want to compare several different design approaches. e.g. I use the REPL heavily when doing a spike (exploring options for a new feature). Output from my REPL experiments go into the unit tests and source code (from the design journal or rich comment form created as I experiment). I tend to take a test-first approach when I am adding to established code base that has multiple design decisions I should build upon, or any change where there is far less need to experiment. I will still used the REPL to test out assumptions quickly though (or test a function / expression I am not sure about). There is value in mixing TDD and REPL as they are both ways to enhance design and increase feedback. For myself, the test suite is an output and not the focus. When either of these techniques are not valued or fully understood though, they wont provide value. There are also people using Clojure who choose to not to use either of these techniques (and unfortunately its quite easy to tell). One thing that seems to be quite widespread (at least in all the commercial project I've worked on) is the use of Specifications (Clojure.Spec or Malli) and using Generative test data inside unit test code.

Thanks for all the thoughtful responses. I suppose I have some deep research to do.

It's also worth noting that with Clojure's REPL, there's usually some hot key for your editor that will run some specific subset of tests, so you don't even need to switch context to "run tests" (running the full test suite at the command-line is still an option, of course). Before I came to Clojure I was pretty strongly pro-TDD. What I found with Clojure was that the REPL supplanted a lot of the design/thinking/exploring that I'd previously done with TDD -- I was doing a lot of that same level of design/etc interactively via (comment ...) forms in my code, and eval'ing as I go (Rich Comment Forms -- RCFs). What I had to be disciplined about was remembering to extract some of that code into actual tests once it solidified, so that I had the benefit of catching regressions, which is the other thing you get from TDD. I still am pro-TDD -- overall, and especially for non-Clojure code -- but I would say my workflow is more RDD now (REPL-Driven Development), even tho' I still use some of the principles from TDD. And my code has a lot of RCFs 🙂

Stu Halloway coined the term RCF -- based on Rich Hickey working inside (comment ...) forms and often leaving them at the end of a namespace, with a bunch of code that shows how to use the functions in that ns. And in that way you get another benefit of having a lot of tests: they act as documentation -- well, so do RCFs.

I also think of REPL-driven development as a form of TDD in spirit, like you evaluate expressions, give them input, see what comes out, get a feel for it, establish what you know about the problem, and once it seems ripe enough, start extracting it into version-controlled code. I don't think it matters whether the tests or production code comes first in practice, as long as the tests convey to others that the code works, you've understood the problem, how your solution interacts with the rest of the program, and ideally serve as documentation for later reference

How much coverage to you aim for (if that's even measurable)? Do you test every function? I'm basically only interested in catching regressions (at least until someone shows me what other values tests bring). TDD vs RDD would be equivalent to me in this regard if they result in the same tests.

I think test coverage is... a questionable goal (unless you're writing safety-critical code, perhaps).

Test coverage is one of those things where Goodhardt's Law applies IMO. It's measuring "output" when you'd perhaps be more interested about "outcome"

I write tests for "public APIs" and "interesting private functions" but exactly what I write tests for is very subjective, based on my instincts as someone who has been writing code for nearly 50 years (45 professionally). Too many tests and you probably have brittleness, too few tests and you won't catch regressions (or maybe some logic bugs).

When you are just beginning, I'd say write tests for public functions, or any complex function that you are struggling to write.

I feel like even simple functions can break easily when you're rewriting stuff all the time. Maybe that's where the REPL comes into play? I run them right away after my changes to see if they still work rather than waiting for the tests to tell me something broke?

+1 to test.check.generators , Malli, Prismatic Schema, Spec etc. They're super nice for reducing boilerplate in tests and production code alike. Piggybacking off the keyword example earlier, you might not need an explicit test to verify a keyword's presence when you've got a spec for it, it will throw if the validation fails. Depends on the context if finding out about the missing key through an exception is the right thing though

I like https://github.com/nubank/matcher-combinators/ for writing tests that check for shape and structure, when comparing literal data structures isn't necessary (because that has the downside of making the tests brittle)

Re: times before, I also have found TDD much more beneficial when writing other languages such as JavaScript, especially data-heavy applications that need a ton of context and define state left and right. For me, not having a REPL to interact with the application as it's running, the next best thing is writing a unit test for your new function, or running the relevant test scaffolding to arrive at the application state your task at hand is concerned with

Thanks for the insights.

> when you're rewriting stuff all the time I'm curious why you're finding yourself doing so much rewriting? Are you exploring the solution space via the REPL before committing to actual code? (e.g., using RCFs and eval'ing code from your editor -- not typing into the REPL)

I don't rewrite stuff because they don't work, but because the code isn't as clean as it could be. Sometimes I go back and forth because I can't decide (e.g. indirection vs no indirection). Would RCFs assist here somehow? I don't see how it would be possible to envision the future of the code base accurately to write everything correctly the first time.

You could explore multiple design approaches to some extent in RCFs, yes. But if you're doing "cleanup" to improve code while keeping its behavior the same, I guess I would consider that "refactoring" rather than "rewriting", and if you were doing TDD, that would be part of the workflow anyway: Red-Green-Refactor.

Yeah with tests it would be easy. I'm wondering if I'm missing something when it comes to not testing pervasively. It seems like low test coverage would cause regression bugs very often when refactoring a code base in a dynamically typed language. I might have arrived at the answer already during this conversation, but I am uncertain.

Probably moreso when you are a beginner, but over time, as your REPL-based workflow improves and you get a better sense of what sort of bugs you tend to introduce, that will get less and less so, IME. Idiomatic Clojure means bugs due to mutable state are rare and usually localized, and with a separation of pure functions from stateful functions, and a focus on small, composable functions, several other classes of bugs become less frequent too -- and your code is easier to test (and therefore easier to develop with the REPL).

At work, we have a mature, evolving codebase, parts of which date back to 2011, and it breaks down like this:

Clojure build/config 25 files 396 total loc,
    218 deps.edn files, 3550 loc.
Clojure source 632 files 122302 total loc,
    5214 fns, 1259 of which are private,
    761 vars, 45 macros, 107 atoms,
    85 agents, 19 protocols, 54 records,
    855 specs, 10 function specs.
Clojure tests 174 files 26101 total loc,
    168 deftests, 476 defexpects, 21 defdescribes,
    5 specs, 1 function specs.

Clojure total 806 files 148403 total loc

Polylith:
   24 bases 294 files 76266 total loc,
   170 components 506 files 71682 total loc,
   24 projects 6 files 455 total loc,
  total 806 files 148403 total loc.

Babashka scripts 6 files 287 total loc.

I figured it's probably important to draw strict boundaries between files. Inside a file it's easy to refactor, but not if it is referenced from a bunch of other files. Testing the "public API" as you phrased it makes sense. I suppose the challenge is to minimize the contact area between modules. Right now I'm doing UI development and it's really difficult to predict what data shape I'm going to be happy with. It's a real pain. But maybe UI development is one of the harder challenges with all the proliferation of different shapes you want to render and thus a case where high test coverage is warranted?

A big bag of :fully-qualified/keywords is probably a reasonable shape to start with, in terms of data, rather than trying to work with complex, nested data -- if you can. Obvs other than Hiccup which will mirror your UI of course (but the core app data doesn't need to be that shape).

Yes. It's the hiccup data that's the giant headache, haha.

Well, that's "just" your actual UI so it'll always be as complex as your UI is 🙂 But you can make sure that the code generating Hiccup does nothing else, so you can "refactor the UI" as much as you want, independently of the application data itself and the "business logic". And if that code is written as a set of reusable, composable "component" functions, you should find the "UI refactoring" to be more localized.

Yeah, that's another challenge. How much indirection and component proliferation do I want? Maybe I need two hammocks!

Rich Hickey commented, "Clojure is for optimists", and back in version 1.0, there was no clojure.test. But Clojure Spec includes a really thought-provoking integration with generative testing! At another Conj, Tim Ewald made the analogy of woodworking with hand tools. Back in 2008, Clojure was an answer to Java, a language in which anything was possible, but only with the exercise of superhuman discipline. From immutability to STM to macros (to generative testing), there is a strong theme in Clojure to unyoking superhuman tasks from humans, or cutting big problems down to human scale. I conveniently take this as a clue that there is a healthy ambivalence about testing among experienced Clojurians, but the ambivalence does not extend to TDD. TDD is a superhuman discipline imposed specifically to steamroll the creative spark and it is just plain off the table. :-)

> I don't think I have ever seen proper TDD anywhere except for blogs and vlogs, conferences, etc I agree a lot with this. And the few messages 👎 the goal of 100% coverage. From this perspective I don't find Clojure folks to be much different from people using other languages. You'll find the same mix of: • people caring a lot or less about testing • people giving different definitions for the same label (more on things like functional test or integration test than TDD). Worth mentioning that • In the talk "Design, Composition, and Performance" Rich Hickey says explicitly (whilst touching very briefly on this) that good design enables good testing. I think this, implicitly, throws out of the window the notion of tests driving design and I personally agree with this. • I remember both Rich Hickey and Stu Halloway praising property-based testing and you can see this is reflected on some of the tools available (generative testing with Spec). • I'm not sure if it's just me but I stumbled many times on people misinterpreting the emphasis on design and property-test as being anti-TDD or against writing tests. The first might be true (I don't know personally, I'd leave the word to anyone in the Clojure core team), the second is definitely bonkers. At the end of the day this doesn't mean you must: • use property-based testing for everything; • agree on everything with everyone from the Clojure core team (but strongly recommended listening to them as they consistently deliver high quality talks, etc). People should use their judgment on what to test and why as they should on everything.

> Rich Hickey says explicitly (whilst touching very briefly on this) that good design enables good testing. I think this, implicitly, throws out of the window the notion of tests driving design and I personally agree with this. The TDD thinking is that if you start with tests, then you focus on design choices that make testing "good". Rich is right that "good design enables good testing" but that doesn't preclude writing (good) tests as part of the design process, i.e., his statement doesn't preclude TDD. And, yes, you can definitely write PBT as part of TDD, because at that stage you're focusing on the properties / invariants that are part of the design. Yes, you can also write PBT after writing the implementation to verify properties of the implementation itself. PBT is orthogonal to TDD.

> The TDD thinking is that if you start with tests, then you focus on design choices that make testing "good". I'm a fan of TDD and in particular Kent Beck book was really important for my personal development, but I never digested the "test before design" part. Maybe it works for others, great for them. What really struck with me with TDD is the emphasis on testing-first, which could be seen also as an excuse to think-more-and-better first (yep, direct line to hammock-driven here). > you can definitely write PBT as part of TDD Sure, absolutely. I don't know why sometimes people get stuck in options as they were necessarily opposite (sort-of if you do REPL-driven then you're against TDD to name an obvious classic, and this sort of stuff) when you can steal all those good ideas out there and combine them when there's a suitable situation 😁

> I never digested the "test before design" part Not sure what you mean here. Coming up with tests before writing (production) code is deeply intertwined with the design process. The tests are the reification of your design thinking, essentially. You don't write tests then go off and do your design and then come back and write code to make the test(s) pass. Am I misunderstanding you?

> you don't write tests then go off and do your design and then come back and write code to make the test(s) pass What I hear (from the "tests drives design" way of thinking) is that if you write/define your tests first then design will follow. Precisely because this doesn't make sense and doesn't work for me I'm not qualified to articulate how this is supposed to work in practice.

I think you are in agreement. Mario is thinking in three steps: • design • test • writing code And he's saying that he doesn't buy that tests should come before designing, but after it (together with the code-writing).