Good. Evil. Bratwurst.

Why don’t they test?

Posted on by arlen

I’ve heard from some more developers on the subject of testing, who claim they aren’t allowed to do unit testing because there isn’t time. I’m not blaming the developers here, but their management for being just plain stupid. An elementary analysis of the question proves just how wrongheaded they are.

The first question to ask is do they test the software at all before they supply it to the customer. It’s possible they don’t, in which case they have a point that testing will take more time, and my only advice, to both the developers there and to their customers, is “Run Away!” It’s irresponsible and unprofessional to make people pay for untested software.

Let’s assume they actually do test the software before releasing it to the customer. So, now we have the question of “When?” Most likely, they’re waiting until the end of the development process. Which means that if a bug is detected at that point, not only is there a lot of code to sift through to find it, there is also the very real risk that other code has been built upon the original buggy code and it may have to be reworked as well as just the buggy code itself. Making the cost in time required to both find and fix the bug much higher than it would have been if the bug were caught earlier.

Obviously, then, the earlier in the process the bug is detected, the easier and cheaper it will be to fix it. One way to detect it earlier is to move testing farther up in the process by breaking the testing up into stages, and testing what’s been developed in that particular stage. Then you catch the bugs earlier, making it cheaper to find and fix them. But if you do that, you’ll still need to retest everything from the previous stage, to be sure you haven’t broken something along the way.

It’s cheap-er but not cheap-est. The logical extreme of this approach will be to test all code immediately after writing it. But there’s still a cheaper way to do it.

Let’s take a step back from the word “testing” and consider the entire development process. Before writing any piece of code, the developer must first know what the code is expected to do. (The idea of starting to write and just going where the code takes you, like fiction writers do, just doesn’t scale.) This idea is captured in the documentation somewhere (the precise where and format is procedure-dependent) and then the developer starts to write the code. And, at some later point, tests are devised to make sure the code does what it is supposed to do.

This means documentation of what the code will do is written, then it and/or the code itself is later studied to derive tests to be performed on it. What if we combined those steps into one? We’ll think about what this particular piece of code to be developed will be doing, and instead of writing it out in english or pseudo-code or whatever, we document it by writing a test that says when this input is received, this output should be produced. It will take less time, because the same person that writes the spec will write the test, so no time is spent transferring the knowledge from one to the other.

But we can find a better way even than that to speed the process: if the developer who will write the code writes the tests, it will help fix in the developer’s mind a clear image of exactly what the code should be expected to do, making the coding itself take less time. (A good developer will consider the problem and the edge cases anyway, all we’re doing is writing them down in the form of the test.) And, while doing the actual coding, if there are determined to be better ways to handle the task than the one originally chosen, adding new tests or editing existing ones will perform the same clarifying function for this new direction. What’s more, if the tests are used as the documentation of the expected function of the code, the documentation will always be current, instead of having to be updated at a later point (or worse, left to “rot”).

Tests only take a few seconds to run. (One major project I’ve been involved with has over 2000 tests written so far, and the entire suite takes a little over a minute to run, testing several thousand lines of code. Any one segment of the testing is done in a mere handful of seconds.) If the tests are written first, and done right, the developer can have confidence that the task is complete, because the tests say so.

There’s a little startup cost, in terms of time, when you first begin this approach, because it will take a little time to get up to speed on writing tests, but it won’t take long. Good developers see “edge cases” for their code rapidly, so test cases will not be difficult to come up with at the start, and will take less time with every cycle. Developing the ability to find these cases quickly and easily through this practice will only make for a better developer. There really is no downside to this practice.

We’ve seen writing tests before coding leads to finding bugs faster, fixing them more cheaply, developing finished code faster, and I’m not done, yet.

How many times have developers “fixed” a bit of code, only to break the code somewhere else? (Come on, be honest, we’ve all done that a few times.) Or added a new feature that breaks an older one? Rerunning the entire test suite on a regular basis (called “regression testing”) will catch those events quickly, again when the bugs are most easily found and fixed. This means the code resulting from this practice will be better code, as well. Setting up the entire suite to be automatically run on every check-in will precisely locate the files where the bug was introduced. No more hunting through thousands of lines in hundreds of files. Again, a speed gain, not a loss.

When a bug escapes detection and is found “in the wild,” the first thing to do is write a test case for that bug. Then code the fix, rerun the tests and see that not only has the bug been fixed, but no new bugs were introduced. Keeping the bug test cases around will prevent the bug from resurfacing after some later changes. Once again, higher quality code is achieved.

Management Summary: Far from costing you time, not only does the practice of writing tests up front ultimately speed up your development, it will also raise the overall quality of the code and make maintenance easier. The only possible way it won’t is if your developers are perfect, and never make mistakes. Didn’t think so.

Don’t have the time to write tests? Son, I’m too busy not to write tests.

Leave a Reply

Your email address will not be published. Required fields are marked *

January 2010
« Sep   Feb »