Team Commitment to Quality

There are so many test frameworks, drivers and other tools out there I want to try. For example, Selenesse has been at the top of my list of things to try for months (and I’ll get to it sometime soon!) However, I couldn’t resist the offer from Pekka Klarck, one of the Robot Framework contributors, to give me a live one-on-one demo of Robot Framework a couple of weeks ago. This weekend, I finally had time to try to write my first test (really, to finish up a test that Pekka had started for me).

My first test is below. I always struggle mightily when I learn some new tool or language, and this was no exception. (Apparently my more-geeky friends also struggle with new things, but they don’t whine publicly about it like I do). The user doc for RF leaves a bit to be desired from a non-programmer perspective. However, they are actively improving it. With much help from Pekka (and suggestions from many in the Twitterverse who also use RF), I have gotten over this first beginner hump, I can see lots of possibilities.

You can write RF tests in various formats, including HTML tables. I like the plain text format, because to me, simple is good. It makes the tests more amenable to version control, because you can easily diff versions to see what has changed. If you try that with html table tests you’ll have lots of noise.

I love the flexibility of this tool. I am trying it out with the RF Selenium libraries, but you can use it with a variety of tools, including tools to test non-Web apps such as Java Swing. There’s a lot of potential there.

I like the fact that you can define whatever “keywords” you want, so you can create your own domain-specific language and the test can document itself and the code that it is testing. The inline-documentation aspect is also one of my favorite features of FitNesse.

My team is pretty happy with our current tools: FitNesse for testing behind the GUI, Canoo WebTest for GUI regression tests, and Ruby/Watir scripts to help with exploratory testing. We’ve long wanted to try Selenium, though, and RF would give us an easy way to do that. We’re always experimenting, and an experiment with RF will tell us whether there’s value in adopting it.

For myself, I think RF will be useful in my public presentations and classes. I’m always looking for ways to teach good test automation design, and I think I can do this fairly easily with RF. I’ve seen Dale Emery and Elisabeth Hendrickson do good demos with RF. I’ve also seen Antony Marcano, Andy Palmer and Gokjo Adzic do good demos with FitNesse, so I’m not saying one is better for this purpose than the other!

Here’s my test. I also copied it in below, though the spacing might be wacky there. I put comments in for myself that might help you, too. Download the RF-Selenium library demo and try it for yourself. If you have any issues I am glad to try to help. It would help me learn, too!

—-

=”color: #333399;”>Developer responsible for story should think about all task cards ahead of time, make sure all necessary cards are there

We focused on finishing all the “leftovers” first. For the new FitNesse fixtures, the developers asked that I first write example tests with my idea of how the test should work. The remote developer then wrote the fixtures and updated the tests as needed so that they pass. These fixtures were difficult to do as they had to use a lot of legacy code, but the ROI will be big because they test critical areas where we’ve had expensive production problems in the past.

There was only one incident of untested code being checked in, and we talked about it right away and discussed ways to ensure this stops happening.

6 days into the 2 week sprint, we still had lots of testing cards and not many development ones. The developers, of their own volition, decided during the Scrum to attack the test cards. By the end of the day, most of the test cards were finished! Then they went on to the stories that were new that sprint.

Since a glance at the online storyboard now tells us who the main developer is on each story, communication has improved. When more than one developer works on tasks for a story, they’re communicating better. Lots of pairing is happening too. Communication is better all around, including with the remote team member.

8 days into the sprint, a developer was writing up the information on the functionality he had changed for a batch processing story, and as he wrote, he realized that he missed a use case. He fixed it, but was concerned there wasn’t enough time left in the sprint to adequately test the new functionality. We decided to postpone that story to the next sprint. Consciously deciding to put off testing to the next sprint isn’t a bad thing, when there’s a good reason for it.

Although we are still very busy, it’s a great feeling to see our storyboard for the sprint with almost all the cards moved to the ‘done’ column. Since it’s holiday time, people are taking time off, so time is short. A task to automate testing for a new UI feature implemented with Ajax has been rolled over to the next sprint, because the test script is blowing up (anyone out there have experience with the mouseover command in WebTest?) Again, this was a conscious decision, not a case of “Oops, today is the last day of the sprint and we didn’t get this finished”.

Many teams struggle with trying to finish all the testing for all the stories by the end of each iteration. If this happens to you, the first action to take is to limit the amount of new work brought into the next sprint, so you can focus on finishing the leftovers. As you catch up and are able to reduce your technical debt by finishing testing and test automation tasks, you’ll be able to increase your velocity later, without sacrificing quality. This has to be a team effort.

Please post comments and questions and I’ll follow up on those. This is such a central factor to team and agile testing success, I’d like to talk about it more.