Agile Testing with Lisa Crispin

From my previous post 12/24, Mykhailo Poliarush asked about testing task cards. I’ve seen several different approaches to planning testing tasks during iteration planning and managing them throughout the iteration.

One approach is to use stickers on each development task cards. For example, a yellow sticker could denote the task is ready for testing, a green sticker could denote that it has been verified. Or one could simply write testing activities needed for each development task on the back of the card (or in its description, if you’re using an electronic story board such as Mingle).

My team has used this approach successfully for the past five years: When we write task cards for a story, we write testing cards as well as the cards for coding, design meetings, database tasks, infrastructure tasks and so on. When we had a physical story board, we used green index cards for testing cards, and white for other types of tasks. That way the testing cards stood out easily and we could tell if testing activities were getting squeezed towards the end of the sprint. We estimate each testing task card in hours, just as we do the development task cards.

Now that we use Mingle for our story board, we don’t have a way to color code them, so we write “Test” in front of the description of each task, for example:

Test – Write high level test cases for adding a validation for non-active employees

The details about the testing activities go on our team wiki, not in the task card itself. The task card has the estimated hours and the status. Each task starts in the “to do” column. When someone picks up a testing task card, it goes to the “WIP” column and is assigned to that person. When finished, it goes into the “done” column.

I like having separate, easily identifiable testing task cards, because we can tell really quickly if we’re getting into a bind with testing. If a lot of stories have their development cards in “done” but there are still lots of test task cards in “To Do”, and there are only a couple of days left in the iteration, that’s trouble. When we see this trend, the developers stop working on new functionality, and start taking on testing task cards along with the testers.

I’m often asked “How do you know how much time to estimate for a given testing task?” This definitely depends on the application being tested, on the infrastructure available for it (for example, do you have a test schema with useful test data already in it?), the testing expertise of the team, and other factors. You’ll start to notice trends as you go. Testing task estimates for any given story of ours usually adds up to about 50% of the total development task estimates. So if the testing estimate is way above or below 50% of development estimate, we look to see if maybe our development estimates are off, or if it’s just a situation where a story is either very easy to test or highly test-intensive.

Never get too hung up on estimates – they’re just guesses. Even when you’ve been working on the same application for five years, any story can surprise you!

The main points here are:

• Make your testing tasks visible
• Include your testing task estimates in your story estimates
• Everyone on the team should pick up testing tasks when necessary
  

This sprint has been an excellent example of why the whole team must commit to quality for the team to succeed.

In the past few sprints, we started to slide into the bad practice of having leftover testing task cards that got carried over into the next sprint. Some of these were cards to write new FitNesse fixtures to automate tests for a particular new piece of functionality. Some were end to end testing cards that simply take a long time to complete. There were also development cards left over. A couple of stories had dragged on more than two sprints. This was getting awful! It felt like we were dragging through waist-deep mud of technical debt.

In last sprint’s retrospective, we got serious about solving these problems. Here are our action items:

• Finish leftover cards before starting new stories in new sprint. This meant taking on less new work for the sprint.
  
• Ask remote team member to work on existing cards before bringing in new stuff
• Have main developer for a story put his name on the story card and take responsibility for making sure all tasks are done
• Write “end to end” test cards for each story (these are cards that remind the developer assigned to that story to test that all the pieces work together)
• Developer responsible for story should think about all task cards ahead of time, make sure all necessary cards are there
• Ensure everyone needed is copied on emails
• Stop checking in untested code

We focused on finishing all the “leftovers” first. For the new FitNesse fixtures, the developers asked that I first write example tests with my idea of how the test should work. The remote developer then wrote the fixtures and updated the tests as needed so that they pass. These fixtures were difficult to do as they had to use a lot of legacy code, but the ROI will be big because they test critical areas where we’ve had expensive production problems in the past.

There was only one incident of untested code being checked in, and we talked about it right away and discussed ways to ensure this stops happening.

6 days into the 2 week sprint, we still had lots of testing cards and not many development ones. The developers, of their own volition, decided during the Scrum to attack the test cards. By the end of the day, most of the test cards were finished! Then they went on to the stories that were new that sprint.

Since a glance at the online storyboard now tells us who the main developer is on each story, communication has improved. When more than one developer works on tasks for a story, they’re communicating better. Lots of pairing is happening too. Communication is better all around, including with the remote team member.

8 days into the sprint, a developer was writing up the information on the functionality he had changed for a batch processing story, and as he wrote, he realized that he missed a use case. He fixed it, but was concerned there wasn’t enough time left in the sprint to adequately test the new functionality. We decided to postpone that story to the next sprint. Consciously deciding to put off testing to the next sprint isn’t a bad thing, when there’s a good reason for it.

Although we are still very busy, it’s a great feeling to see our storyboard for the sprint with almost all the cards moved to the ‘done’ column. Since it’s holiday time, people are taking time off, so time is short. A task to automate testing for a new UI feature implemented with Ajax has been rolled over to the next sprint, because the test script is blowing up (anyone out there have experience with the mouseover command in WebTest?) Again, this was a conscious decision, not a case of “Oops, today is the last day of the sprint and we didn’t get this finished”.

Many teams struggle with trying to finish all the testing for all the stories by the end of each iteration. If this happens to you, the first action to take is to limit the amount of new work brought into the next sprint, so you can focus on finishing the leftovers. As you catch up and are able to reduce your technical debt by finishing testing and test automation tasks, you’ll be able to increase your velocity later, without sacrificing quality. This has to be a team effort.

Please post comments and questions and I’ll follow up on those. This is such a central factor to team and agile testing success, I’d like to talk about it more.

I’m transitioning over to this new website. Please visit often and see what is new. You can find links to articles and so on at my old site.

I’ll be blogging here too. Stay tuned! And Happy Holidays.