unit testing considered harmful?

architecture-unit-testing.md

in searching more in depth for topics in software architecture, i have come accross Simon Brown's Software Architecture vs Code lecture, for GOTO 2014.

the section that most caught my attention, was his reference to Why Most Unit Testing is Waste ¹, by James O Coplien.

in the paper, a specific quote from Coplien caught my attention:

(...) notions such as: “Every line of code has been reached,” which, from the perspective of theory of computation, is pure nonsense (...)

which is, undeniably true and, as i tought more about it, also comically unfortunate that lines of code reached is used as a measurement of test coverage.

the lecture and paper also reminded me of Dijkstra's Go To Statement Considered Harmful. mostly because the former was given at GOTO, but also due to how the latter would seemingly fit into the classic considered harmful phrasal template.

Footnotes

1. here, i have included a reference to a reproduction in PythonTest because the original source seems to be a dead link ↩

couple more quotes from the paper:

It was no longer possible to reason about the execution context of a line of code in terms of the lines that precede and follow it in execution, since those lines of code are no longer adjacent to the one you are concerned about. That sequence transition now took place across a polymorphic function call — a hyper-galactic GOTO.

Remember, though, that automated crap is still crap. And those of you who have a coroporate Lean program might note that the foundations of the Toyota Production System, which were the foundations of Scrum, were very much against the automation of intellectual tasks (http://www.computer.org/portal/web/buildyourcareer/Agile-Careers/-/blogs/autonomation). It’s more powerful to keep the human being in the loop, as is more obvious in exploratory testing.

The psychological perspective is instructive here, and it’s important because that — rather than any computational property — most drives developer behaviour.

It’s much easier to avoid putting bugs in than to take them out.

Don’t underestimate the intelligence of your people, but don’t underestimate the collective stupidity of many people working together in a complex domain.

Some people tell me that this doesn’t apply to them since they take more care in writing tests than in writing the original code. First, that’s just poppycock. (The ones that really make me laugh are the ones who tell me they are able to forget the assumptions they made while coding and bring a fresh, independent set to their testing effort. Both of them have to be schizophrenic to do that.) Watch what your developers do when running a test suite: they’re doing, not thinking (like most of the Agile Manifesto, by the way).

I tend to find this more in Eastern European countries, where the lack of widely available computing equipment forced people to think. There simply weren’t enough computers to go around. When I first visited Serbia back in 2004, the students at FON (the faculty where one learned computing) could get to a computer to access the Internet once a week. And the penalty for failure is high: if your code run doesn’t work, you have to wait another week to try again.