Agile Testing with Lisa Crispin

I love conferences – I get to meet so many smart practitioners and come home bursting with new ideas. I’m back from Agile Development Practices and I learned a ton. One thing that’s on my mind both from the conference and other reading I’ve done this week is the importance of a culture that enables and even promotes learning.

It’s easy to forget that failure is good – if you can fail fast enough. That’s one reason agile works so well – short iterations and work transparency let us fail fast, and everyone knows when we fail. But failure shouldn’t only be seen as negative. We learn a lot more when we fail.

I recently read a great post titled “Are you building a learning suppression system by Daryl Kulak from his upcoming book. Here’s an excerpt.

What happens in organizations is that people get punished for committing sins of commission but they do not get punished for sins of omission. This shapes the mind of a person into saying “No, we better not try that,” attempting to avoid the nasty results of commission errors while ignoring the problems of omission errors, which don’t seem to cause as much havoc.

We don’t want to suppress innovation. Mistakes should be tolerated, not punished. We want a learning organization so our teams can always adapt to change, and thrive.

More on Learning from Linda Rising

Linda gave an enlightening talk Wednesday at ADP about retrospectives, and she talked a lot about having time for thinking and learning. These are my notes on her talk, so I hope I haven’t misinterpreted anything she said. You can find more material direct from her on her site.

When confronted with practices such as pair programming or retrospectives, some managers have this reaction: “We don’t have time for thinking”.

But we aren’t trying to stop, analyze the root cause of all of our problems, and fix them all. We need to do little experiments to see if they solve some problem we’re having. If the solution works, we don’t even have to know why, we can just use it. Linda compared this to tuning a musical instrument – we can make tiny changes, and these give us lots of value.

Linda quoted Tom DeMarco’s book, Slack. You can’t learn or change if you have no slack. Linda cited research showing that not being able to express feelings and thoughts in the normal course of business weighs us down, slows us down. It sounded a lot like technical debt to me! We fret about things that aren’t going well, and if we don’t get a chance to experiment with ways to improve them, we become less and less productive, more and more unhappy.

Going back to that idea that we must tolerate failure, it’s important that we don’t criticize or blame our teammates. Every team member should feel safe to raise issues and propose ideas. Linda cited Norm Kerth’s prime directive for retrospectives: We must believe that everyone did the best job he or she could, given what was known at the time. Everyone used their skills and abilities, the available resources, to their best advantage, given the situation at hand. When we start from there, we can avoid pointing fingers, and simply try to learn: what’s going well? What’s not going so well? How can we improve? What still puzzles us?

Linda advises us to watch for stereotypes: “These people are just no good at…” She was talking in the context of retrospectives, but this is good advice for everyday work. We even stereotype ourselves. We have to believe, instead, that we can get better. Then we will.

Focus on experiments. Identify the next experiment, and ask the three questions about it at the next retrospective. Share knowledge. Look for patterns.

Time to Learn

As Linda noted, everyone says they want to learn, but few take the time to do so. Take some time to learn something new today. See what you can do to promote a learning culture within your organization. Show your manager the Google 20% rule, and the value that Google has derived from that! Lots of companies, big and small, have something similar: Wiki Wednesdays, Engineering Sprints, one hour per day devoted to professional growth. People on agile teams love learning, and it makes us happy. When we’re happy, and when we are always learning and experimenting, we can do our best work. To me, that’s what agile is all about.

I meet more and more teams, both within my company and externally, who are getting really good at agile testing. They’ve adopted a Whole Team approach, they write tests first, they collaborate and communicate well, they use feedback to improve. Agile teams are functioning much better overall than what I was seeing a couple of years ago.

One perennial issue – and this isn’t new with agile – is that testers and teams still struggle with how to design and write automated tests that have a good return on investment. That is, the tests are reliable, stable, maintainable, and provide good value for the amount of work that goes into them. We also all struggle with how much to automate, what to not automate, what to keep in our regression suite.

Janet and I devoted a section of our book to automation, and we presented principles, techniques and strategies that will help. We also include this in our Agile Testing course, and we both do conference tutorials on how to succeed with automation in agile projects. I think in addition to that, a lot of people need some practical hands-on experience. They need to be guided through an example of automating tests for a story, then they need to try it for themselves.

I’m developing an internal class for people at my company, and we also have ideas for workshops and other ways for teams who have solved some automation problems and developed good practices to share with other teams. I just mind mapped the ideas we have so far. At Adam Goucher’s suggestion, I’m posting my first draft here.

It’s my hope that I could extend this to a more generic class that would be appropriate for a conference tutorial. I’ve talked with some folks about collaborating on something like this and kind of making it open source, with several of us who could teach it, like the Watir tutorial that Bret Pettichord, Charley Baker and others have taught. But, others better than I have tried to teach hands-on courses like this and run into issues. What tools would we use that can easily be installed on all participants’ laptops? What if the participants have different skill levels? So, I don’t know if this is really possible. I’m interested in ideas, or if anyone has done this successfully.

Actually I’m sure this has been done successfully by people like Elfriede Dustin who have been specializing in test automation for years – I need to get her book out and re-read it. I’m not trying to steal anyone’s idea or say that this is a new original idea I thought up. I just see a gap in many teams’ ability to create the ‘right’ automated regression tests.

So, here is the mind map, you’ll have to open it in a new window (I’m sure there’s a better way to do this but I have to catch a plane), feedback welcomed.

Last month I had the great pleasure to participate in Agile Testing Days. I did a tutorial on using the Agile Testing Quadrants for test planning, and a keynote on 7 key success factors for agile testing success. More importantly, I got to attend a lot of great sessions by leading agile testing practitioners.

There were so many good sessions and I’ll only mention a few here. Mark your calendars for next year’s Agile testing Days, Oct. 4 – 6.

My slogan after this conference comes from Dave Evans (and from Mike Scott – not sure which one of them coined the phrase but they both talked about it): Respect the Tests. More about that in a minute.

Elisabeth Hendrickson always inspires me, and her keynote didn’t let me down. I’ve heard her definition of agile before, but it bears repeating: Delivering value in the form of releasable software at frequent, regular intervals at a sustainable pace, while adapting ot the changing needs of business. She quoted Mary Poppendieck as saying, if you want to go very fast, you have to be very disciplined. Sustainable pace means that we keep technical debt down so that the code doesn’t impede us.

Elisabeth presented her four sources of technical risk: Ambiguity, assumptions, dependencies and capacity, and her 7 key testing practices: TDD, ATDD, Exploratory Testing, Automated System Tests, Collective Test Ownership, CI, and one more that I think is automated unit tests (I can’t read my notes!)

Elisabeth also quoted Tobias Mayer, who refers to the Product Owner as the Voice of the What, and the dev team as the Tribe of the How. I like this, but I take issue with it a bit. The Poppendiecks have influenced me to experiment with having the dev team learn the business inside out, so the dev team itself can help the business prioritize and even think of features. This worked well for my team at ePlan, and I have to agree that funneling all the business requirements through the Product Owner has a lot of drawbacks.

What was new to me in this talk was something that several of us had talked about a night or two before, the idea of Collective Test Ownership. This ties in with the Whole Team Approach. Everyone on the team takes responsibility for quality, testing, and test automation.

Another great session, again from someone whose work I’m familiar with, was David Evans‘ talk. One big takeaway for me was this: If a test is worth writing, it’s worth automating, and it must always pass in the future. I often see teams that ignore failures in their CI tests because “We have to get these new stories out, this is the PO’s priority.” So the PO doesn’t care whether what she prioritized in previous sprints still works or not? When the CI breaks, the team should stop the line and get it green.

Dave pointed out that if all tests pass, you don’t need extra metrics. Defects are evidence of missing tests. So don’t call it a defect – call it a failing test, write the failing test and fix it right away. If the customer really doesn’t want it fixed now, write a story for later.

Dave recommend fixing time and quality, but don’t fix functional scope. If time is running out, thin the requirements. It’s not like leaving a wheel off or leaving out the engine. Respect the tests – any test worth writing is worth running all the time.

Dave presented his Agile Quality Framework. I won’t go into all the details, but one of my favorite parts is applying balloon patterns. An empty balloon is still a balloon, but if you don’t have the rubber outside, it’s hard to have the air and then add the rubber later. Do the rubber first – the CI, the test framework.

Mike Scott elaborated on the balloon pattern in his demo of Testify Wizard. In one hour, Mike went from zero to a full CI process, including the project two continuous builds, one running unit tests, and one running FitNesse tests. It even generated code coverage reports! I think he did this with both Java and .Net during the same session. He did this with a couple of pushes of the button,some minor configuration and a wave of his crutch.

This shows how easy it is to respect the tests. It’s not that hard to build a framework to integrate our code and run our automated tests so that we get quick feedback all the time.