I Love Test Coverage

Cards on the table, I love test coverage, it’s one of the few measures you can get in software development that has some tangible meaning. I particularly enjoy opening up the HTML generated coverage reports and seeing little conditional blocks that got missed, or even just finding out that one part of the code is run hundreds of times.

It’s a wealth of information, insight, and a chance for me to find the mistakes I have made clearly pointed out.

Why you Hate it

So, why do so many people hate test coverage? Why do they scoff at the idea of 100% test coverage, and then debate whether it should be 70% or 80% instead?

Quite simply, the answer is Goodhart’s law. Now if you know what that is, hopefully you can see where I’m going with this. However I won’t make you google it, Goodhart’s law states that "When a measure becomes a target, it ceases to be a good measure”

That’s the rub, teams destroy a valuable measuring stick by turning it into a target, or a stick no longer to measure but to beat either a drum or developers who are a little less savvy with writing tests.

And yes for the pedants out there Goodhart is talking about statistics and government policy, but I still think it gets at something human that erodes away the value of what few measurements we have in this world.

Learning to Love Test Coverage Again

You can drop a rock in your measuring jug to make the last of your milk touch the line. But it doesn’t mean your cake will taste better.

Coverage is a measurement, not a target, stop trying to drop rocks or lowering the line. Your tests should build confidence in your software, and enable developers to refactor it.

Let go of those ideas, the only target you should have for test coverage is for it to NOT be a target. This is easily done by ensuring coverage is 100% and spending your time on other aspects of testing.

“But what about X file that can’t be unit tested”, add it to the coverage ignore list. Make an active decision that it doesn’t add value to be tested. Or face the fact, that you should learn how to test it.

Let go of the worldly percentages that are tying you down, reach 100% and transcend into loving test coverage.