Testing Appendix
Who tests the tests?
That’s a good point. Why write your program twice?
Consider your cars dashboard. When you start up your car, you see a bunch of little lights light up and fade away on your dashboard. That’s the automated test suite which tells you if your car is safe to drive.
Those lights don’t make your car run. Your cars computer is just constantly probing (testing) critical components of your car to see if everything is alright. These test systems are small and simple compared to your cars engine.
Of course, you wouldn’t try to get a car on a discount and argue that you don’t need those dashboard lights.
And when a light goes on when driving, what do you do? You stop and check that system. You perform a manual test. And if the system is faulty, you repair it. But what if the system which your car reported as defective is actually working fine? Then you’ll check the sensors.
When you see that your production code is fine, but your tests are failing, you know that it’s the tests which you have to fix.
Notes on coverage
Does a software project have to have 100% coverage? Most of the time, this simply isn’t practical or possible.
However, the level of trust you have in your tests, will directly impact your speed of development. If you trust your tests completely, you can push to production constantly - multiple times a day. Your team can even work in a single branch. Those are topics for another blog post.
If you don’t trust your tests, you’ll be supplementing them with manual testing which brings you back to square one. It is here where writing tests becomes a waste of time.
Therefore, teams which are not after achieving 100% trust in their test suite, are wasting time writing it.
Coverage is not the only metric which tells you how reliable your tests are. A test which runs a piece of code but doesn’t check if it is doing what it should, will cover that piece, but this coverage is useless. That’s just to say that assertions are what give coverage meaning.
How do I know if my assertions are correct? Doing mutation testing can help you increase the quality of your assertions.