Agile Testing with Lisa Crispin

Are you a tester wondering what the heck you should be doing now that you’re suddenly on an agile team? Are you a manager of a new agile team puzzled at why the QA group refuses to cooperate with you? Are you an agile developer trying to figure out how to deliver the best possible software? If so, you might be interested in the three-day agile testing course that Janet Gregory and I are developing. We’re offering this course in partnership with LeanDog in Cleveland and with Program Utvikling in Norway, and will be offering it in other locales as well – please contact me if you’re interested.

Here’s a course description:

Over three days, we put theory into action through a variety of exercises. This course teaches testers how to fit into agile projects, contribute to the whole team and overcome common cultural and logistical obstacles in transitioning to an agile development process. It explains the values and principles that help testers adopt an agile testing mindset and how to accomplish traditional testing processes, such as defect tracking, metrics, audits, and conforming to quality models. Students will learn how to complete testing activities in short iterations, and how testers contribute on a daily basis during each iteration and release cycle. Through interactive exercises and group discussions, participants will discover good strategies for driving development with both executable and manual tests. The course is filled with real-life examples of the many ways agile testers add value.

I made a difficult decision to move on from my incredibly wonderful agile team at ePlan Services to take on new challenges at the equally wonderfully agile Ultimate Software. This leaves the team at ePlan with four programmers and one tester (plus two system administrators, a DBA and a ScrumMaster).

Even with two testers, the programmers pick up some of the testing tasks – ours is a testing-intensive financial application. The programmers often document stories on the Wiki, write FitNesse tests as well as automating them, do manual testing, and investigate functional test failures in the builds. Everyone on the team is willing to wear multiple hats. The sys admins sometimes do development and testing tasks, the ScrumMaster sometimes helps with manual testing.

Conversely, we testers help with production support, occasionally take on a simple development task such as changing text in a report, fix SQL query bugs if the solution is obvious to us,and the like. It’s one of the things I enjoy most about working on an agile team. Nobody’s stuck in a pigeonhole, and everyone focuses on common goals.

In the past, when we’ve been down a tester, the whole team pitched in to take up the slack. For example, when our Ruby/Watir expert left a few years ago, every team member bought the Pickaxe book, and helped maintain the Watir scripts and even write new ones.

So, I was surprised the other day when I heard a plan to have one of the programmers, who started out here as a tester, revert to the tester role until a new tester could be hired. Yes, he’s a terrific tester, and indeed has continued to do some testing tasks along with coding work. But why should he have to abandon his new role, especially as he’s really getting traction with his new skills, to do testing activities full-time? It makes no sense. Naturally, with his experience in the tester role, he’ll be an important resource, but his job shouldn’t suddenly revert.

I reminded everyone of how we normally handle this situation. It was a bit of a “doh” moment as everyone realized the whole team approach to picking up the slack has worked so well in the past, and there’s no reason not to keep using it. Everyone can pitch in on testing tasks, just like always. Every role on our team is equally valued, so nobody feels that testing is somehow “beneath” them. Everyone is committed to delivering the best software possible. The team velocity will drop while they’re short a member, but quality won’t suffer.

The Whole Team approach to testing has been so effective on my teams, and I expect it will work well on my new team too!

The Agile 2009 program is ready, and it’s chock full of practical content. I know training and travel budgets have been cut, but this program is worth the investment. Of course, the best thing about any conference is the informal networking and learning from other participants. This year’s presenters include not only a lot of “big names”, but lots of in-the-trenches practitioners dealing with the same issues as your team.

I’m particularly excited about our testing stage program. Our track is in a large room, so we have included sessions that are highly interactive, and useful for a wide variety of participants. We have a great range of topics and session types. There are also lots of sessions in the Developer Jam and Tools for Agility stages that will be highly interesting for anyone involved in testing.

I just reviewed a book proposal that raised my ire. The skills and techniques that would be taught in the book were fine. The emphasis on context-driven testing was great. But the book appeared to be framed in a setting where testing is all about dealing with crises, trying to figure what testing you can squeeze in to a very short timeframe, making the testers somehow in charge of quality.

Testing skills are important, of course, and we testers add value with our special experience and viewpoint. But we can’t have much impact on quality by ourselves.

On the plus side, I’m seeing more and more teams around the world figure out that the best way to deliver a high-quality product is to move testing to the front, make testing and coding part of one process, have the whole development team take responsibility for quality, and work incrementally, iteratively and at a sustainable pace.

On the minus side, I see things like this book proposal that ignore the evidence of how this new whole-team approach is working, and that keep pushing a way of working that is 20+ years old and hasn’t resulted in consistent, high-quality software.

I don’t care if you’re using a traditional waterfall process, or agile, or something else. There is a pragmatic, practical approach to software development and testing that works. It’s more than specific testing skills and tricks. Discipline, communication and collaboration are the key skills and characteristics we need. Continuous integration and automated build processes that run good regression tests are as just as essential as automated source code control.

Testers do need to learn new skills and techniques. More importantly, they need to get up from their workstations and (physically or virtually) go talk to other development and customer team members. We need to find out how we can all help each other do our best work and delight our customers.

What do you think is the most important thing testers need to do or know? What book is missing that would send newbie testers in the right direction?

Yesterday evening I had my weekly dressage lesson with my trainer, riding her young horse, Dutchman. I’ve been riding since I was three, and started learning dressage almost 30 years ago. For several years recently, I rode a wonderful upper-level dressage horse. Some people would wonder why I still need lessons.

“Dressage” is a French word for “training”. It’s something I have to continually learn, and I expect that’s true of most dressage riders. I might be slower than most, though. Yesterday, my trainer commented, “You know how to do this; when I remind you, you can do it. Why can’t you remember to do it without my telling you?”

Well, right now I only ride once a week, in this lesson. So it’s hard to remember the many different things I need to be doing at one time. That’s why I need a reminder. My job is that way too. In any given day, I have to engage in several different activities. It’s easy to forget some good practice that I ought to be doing.

Agile development helps us remember the right behaviors by providing us with a framework of values and principles. For example, when we need to solve a problem, we can apply the value of simplicity. Agile practices, such as pairing, help us maintain the discipline that we need.

I find that I also have to continually learn, reading articles, blogs, watching webinars, brainstorming with co-workers. Just as I ride better when my trainer reminds me of things I already know, I am a better tester and development team member when my co-workers remind me of our team goals and good practices.

If I only had one thing to do all day, every day, I could probably become perfect at executing. But since real life and real work involve so much variety, I have to continuously learn. There are new tools  to investigate, new business requirements to learn, new problems to solve.

There are lots of great books, magazines, courses, conferences, blogs and other online offerings to help us learn. See the links on the sidebar to your right. Elisabeth Hendrickson and Dale Emery have a great Agile Testing Series course. Janet Gregory and I are now offering our own course, Agile Testing: A Roadmap to Success. Go learn something new today!

The second issue of Quality Matters is available for download, and features my article, “Helping Testers Transition to Agile: What Testers Can Do for Themselves”. In it I offer advice to testers who find themselves on agile teams and aren’t sure how to adapt. Please comment on it!

Quality Matters is aimed at Eastern Europe, but I’ve found the first two issues quite good, helpful information no matter where in the world you work! I particularly enjoy Alessandro Collino’s articles, this month it’s a comparison of Robot Framework and Fit.

Quality Matters is published by Quality House, which is organizing SEETEST, June 16-17, Sofia Bulgaria. I’ll be there, and I look forward to meeting software professionals from southeastern Europe there!

In other news – there are starting to be a lot of “agile testing” training courses offered. Watch this space, because Janet and I are working on our own three day training course in agile testing.

The world’s a tough place, so take advantage of a chance for some chuckles. Read about the newest test tool from the eminent Alister Scott, WATIF. And if you’re feeling the need to get some kind of certification, check out Scott Ambler’s new master class. (A reminder – and this part sadly isn’t an April Fool joke – Janet Gregory and I do not endorse any certifications for agile testing or anything else, especially any you might see on a website that hasn’t been run thru a spelling or grammar checker, and that had material lifted from our book on it until recently).

My team was lucky in having Brian Marick visit us last week, interviewing some of us for his book “Travels in Software”. He had visited us back in 2004, and I hope it was interesting for him to see that 1) we used a lot of ideas we got from him, such as using examples to drive coding and 2) we’ve changed and grown a lot and are a productive team.

It was fun to be interviewed by Brian, and he asked thought-provoking questions. One question was, what would you go back and tell your 2003 self, when you first started on this team? Or presumably, even my 2000 self on my first agile team. This has started a couple different thoughts in my head.

Here’s one thing I’d be sure to tell myself. “Lisa, you are good at picking up a new domain, and you’ll learn about 401(k) plans, all their government regulations, how all the different users use the application and other knowledge directly related to the application under test pretty fast. However, there’s lots more to know than you think. You’ll still be learning about parts of the app you never saw before in 2009! Also, you’d better learn about the whole business, not just the parts that are automated in the app.”

Since Day 1 of implementing Scrum, our team has been committed to delivering the highest possible quality software, and I think we’ve done a fairly good job. However, one of our biggest problems has been that we sometimes misunderstand customer requirements, or miss requirements entirely, even though we are driving development with tests.

It was only after we made a conscious, concerted effort to sit with the business people, learn all their processes in detail, and document them, that we got a full enough understanding of the business. For example, only recently did I learn about all the different bank accounts through which money moves around, because much of the movement is done manually. It’s critical to know this, however, because it’s very bad when cash balances stop reconciling correctly, or when the clearinghouse or bank says we have a different amount in each account than we think we do.

Maybe it was important that we first master agile practices such as TDD, CI, refactoring, driving coding with examples and tests at all levels, and test automation, before we moved on to getting a broad and deep understanding of the business and of every person’s job. You can’t do everything at once. Still, it would have helped me to at least have a sense of what I didn’t know yet, so I didn’t have a false sense of confidence.

Do you know your business inside and out? Or only the application for which your team writes code?

Chapter 21, Key Success Factors, is available on InfoQ.

Please read and comment! I look forward to your feedback.

Jeff Langr and Tim Ottinger have added a flash card to their “Agile: In a Flash” project (that will become a book) based on our book’s “Ten Principles for Agile Testers”. They did a nice job of summarizing the information in Chapter 2 of Agile Testing: A Practical Guide for Testers and Agile Teams. We are honored to have this information as part of the terrific Agile flash card project!