Recently in testing Category

Sometimes (when you aren't really doing true TDD), you make a change that breaks a lot of tests. When this happens, nose's output can be long and gross, especially if your tests use print statements to help debugging.

In this case, you really want to just fix the errors one at a time. Its handy to use the --stop (or -x) option of nosetests, which makes the run stop at the first sign of trouble.

This makes it easier to read the output and helps morale, since you can tackle one failure at a time.

The Google Testing Blog has a great post about how to write code that is impossible to test. The list is a handy scorecard to compare your work to. I was happy to see my sentiments about evil constructors echoed in one of the items:

Heavy Duty Constructors - Make constructors that do lots of work in them. The more work you do in the constructor, the hard it is to create your object in a test fixture. And if your constructor can construct other things that are hard themselves to construct, that's even better! You want the transitive dependencies of every constructor to be enormous. Enormous is hard to get under test.

I collect mocking frameworks like some people collect figurines. So I'm very interested to see if Google's Mox is any good.

At first glance, it seems to be very mocker-ish, but I'm sure I'll find some noteworthy differences. If you beat me to the punch, please post a comment and lemme know what you think of Mox.

[via AgileTesting].

One of the key benefits of testing and TDD is delaying the Eff-it moment.

The Eff-it moment happens when you're diligently slinging code: writing tests and building modules to satisfy them. One day you do some quick math and realize you're not going to hit the deadline at your current pace. You quietly declare, "Eff-it, I'm just gonna get it to work".

You stop testing and start slamming crap in to make your deadline.

This has happened to me. I'll argue that it happens to just about everyone who attempts any degree of test-driven development.

I think it's important to see TDD as a way to stave off the Eff-it moment, not it's executioner.

If you ARE human, and you don't make it all the way through before you hit Eff-It, don't be sad. You're still OK. The tests you manage to write help you and will help others.

This kind of discipline takes nerves of steel. It's like base jumping (righteous video, btw). You need to trust your tests and stay calm in the face of a rapidly approaching deadline/canyon floor.

Just as trust in equipment is built over time, so is trust in testing. It won't happen the first time out of the gate. You'll chicken out eventually, but just keep testing as long as you can.

Each time you'll learn more, and gain more confidence and trust. Eventually you'll be doing whole projects with the calm nerves of a seasoned extreme athlete.

Can you write a test that does more harm than good?

In the intro to his upcoming book The Art of Unit Testing, Roy Osherove describes a case where poorly-designed tests actually hurt his team's progress:

Worse yet, some tests became unusable because the people who wrote them had left the project and no one knew how to maintain the tests, or what they were testing. The names we gave our unit test methods were not clear enough. We had tests relying on other tests. We ended up throwing away most of the tests less than 6 months into the project.

I'm on the fence. Anything that exercises your code in a automatic, repeatable way has got to be useful. On the other hand, poorly written tests cost time and effort to understand, refactor, and fix.

Are the drawbacks to bad tests worse than having no coverage at all?

I think  the answer is that in the short term, even bad tests are useful. Trying to squeeze a extra life out of them beyond that, however, pays diminishing returns.

Just like other software, your tests should be built for maintenance, but in a crunch, you can punch something in that works. It's better to have bad tests than to have untested code.

(photo courtesy of Still Burning, some rights reserved)

A while back Michael Feathers wrote a short post about optimizing unit tests. In it he gives a list traits that prevent something from being a true unit test.

When trying to convert non-testers, I often use a reverse version of Michael's list to help explain what is a unit test. So here's the same list with a positive spin:

A test is a unit test if :

• It doesn't talk to the database
• It doesn't communicates across the network
• It doesn't touch the file system
• It can be run at the same time as any of your other unit tests
• You don't have to do special things to your environment (such as editing config files) to run it.

Of course there are more qualities. Think of this as a list of traits as necessary but not sufficient for something to be a unit test.

I know I'm probably late to the game, but I just found a great breakdown of kinds of test doubles.

There are a lot of terms floating around when it comes to automated testing:

• Mock Objects
• Fakes
  
• Dummies
  
• etc
  

This isn't a big deal when you first write tests. As your testing becomes more advanced, things get complicated. Eventually confusion sets in when someone goes to use your mock object and you have to tell them it only works for a specific test, or vice versa.

Martin Fowler mentions testing vocabulary in one of his essays, and refers to terminology from Gerard Meszaros. I did a little digging, and found that Gerard authored (among other things) a neat chart on the aforementioned page that shows how different kinds of test doubles compare in their behavior, along with some examples.

Don't wait until the end of a project to write tests. It doesn't work.

Writing tests is never urgent, but it is very important. Devotees of Stephen Covey will tell you this puts it smack in the middle of quadrant two: things that are important but not urgent.

Quadrant one, urgent and important things, prevent any other work from being done. Unfortunately, the final stages of any project are chock full of urgent and important things, like fixing bugs, building deployment packages, scheduling rollouts, and other fun. All these are important and urgent. If you haven't written automated tests at this point, don't start.

No manager will ever tell you, "Sure, we'll delay the launch by a week so you can write some code that won't ever be executed in production and has no bearing on the project making it out of QA".

So, if automated testing lives in quadrant two (it does), and that no real quadrant two work is possible at the end of a project (it isn't), you need to write your tests early, when quadrant two is in full effect, at the beginning of the project.

I'm not saying you need to do test driven development. That's hard. All I'm saying is that it's important that tests keep pace with code. Get in the habit of running coverage tools when you finish modules, and write tests before moving on.

If you do find yourself at the end of the project with no tests, don't panic. You're not a bad person. You're not a bad programmer. You just didn't write tests. Don't try to write them all at the end. You'll only get frustrated and stressed out.

Finish the project, fix any last-minute bugs, and deploy. After that you'll be back into quadrant two, and you can write all the tests you missed the first time around.