Agile Testing with Lisa Crispin

I’ve just been struggling with a title for this post. Some of my ideas were: “Visibility Taken to New Heights”, “Yes, There Are Teams Who Do This Just Like In the Books”, “A Real Commitment to Continual Improvement”, “Inspiration”, but none of them really capture my amazement at what I saw visiting the team at Energized Work in London. Finally it struck me that this team embodies the Whole Team approach to software development in a way that I’ve rarely seen.

I’ve just updated this post with links to photos of the Big Visible Charts in the Energized Work Lab, accompanied by explanations from Simon Baker. Check ‘em out, they will give you ideas for your own team to use.

What a Team!

I’ve always maintained that I work for the coolest team on the planet, and most of what I try to help others learn are things I learned from and with my own awesome teammates. I don’t want to be disloyal, but the team at Energized Work raises the bar for cool. Simon Baker (@energizr on Twitter) invited me to visit while I was in London earlier this month (and he gave me permission to write this post about it). I love to visit development teams wherever I go, and learn what other practitioners do to improve software quality, so I was happy I could make the time. Mohinder Kohsla had breakfast with me that morning, and was kind enough to guide me to the Energized Work lab, as he already knew some of the folks there.

Big Visible Charts

What blew me away right off the bat at Energized Work was the creative ways they use whiteboards. One whiteboard had personas, complete with photos and bios, and assumptions. That’s not too unusual, but it’s still nice to see someone actively using personas in real life.

What intrigued me most was the way they represent their backlog for their current project as a site map on a big whiteboard. They start drawing site map for the web application they’re working on on a big whiteboard, with placeholders for each screen, and arrows showing how the navigation flows. Since this is on the whiteboard, it’s easy for them to evolve the map as the project proceeds. They overlay cards with user goals so they can identify which journeys deliver value. Screen prints of completed pages are stuck to the board over the original placeholder, along with the cost to develop it.

The team starts with minimum functionality delivered to achieve a goal, working in fast, tiny iterations. As Simon explained to me, the customer can decide based on that functionality and what it cost whether to invest more in that user goal, for example, to “enrich the feature with more functionality”. The site map / backlog is part of the team’s mechanism which allows the customer to decide “just in time” whether to “go deep or broad” every week in response to user testing.

The Energized Work team has experimented with different ways to do story/task boards as well. Instead of a traditional Scrum task board or Kanban board, they represent each story with a big square on the whiteboard. In the middle of the square is a story card has some high level acceptance tests written on the back. Surrounding it is plenty of space to draw mock-ups, prototypes, write questions, write down test cases, whatever they need to discuss or do. A team member only has to look at the board to see story requirements.

Instead of moving cards from column to column, eg. from “Work in Progress” to “Verify”, they represent the vertical slicing of the story with different colored dots, representing feedback from different activities including customer review, UX reviews, and manual exploratory testing. Some cards may cycle through coding, testing, reviews, and back to coding several times before they’re done. Gordon Conroy, the tester, keeps an eye on the “big picture” as new functionality is created slice by slice. He was so knowledgeable about all aspects of the project, it’s clear he works constantly together with the rest of the development team. If companies with separate test teams could see this in action, they’d get why testing can’t be a separate phase or done by a separate team!

My own team uses a fairly standard task board with rows for stories and columns for status, and we write high-level requirements and questions on a separate whiteboard. We also work in these fast, tiny iterations with multiple slices of each story, and I am wondering if we can borrow some of these ideas to better represent that visually. We put more details about slices, test cases and specifications on the team wiki, but developers don’t always look at the wiki. Of course we also do specification by example with executable tests, but whiteboard drawings and notes would make a good, quick-to-access supplement during development. I’ve told my team everything I’m writing here, and we’re ruminating on how we might experiment to improve visibility, while keeping our remote teammate involved.

Another Big Visible Chart in the Energized Work lab lists usability heuristics. Clearly, no aspect of software quality is neglected here, and there were so many visual reminders to keep the team on track. Just about every problem my team has experienced, we solved by making it more visible, and seeing how well another team accomplishes this is affirming.

Driving Development with Tests

The Energized Work team are clearly expert practitioners of TDD and specification by example. They showed me the continuous builds in Jenkins for a couple of different projects, it looked a lot like what my own team does. I’ve personally met few teams that have as sophisticated a build job set-up as ours. I found it interesting that they’ve used different test frameworks and tools on different projects, they are clearly committed to finding what works best for each situation, and experimenting with new approaches. Some of the tools they use are new to me, such as Spock for unit testing Groovy and Java, using a BDD-style syntax.

All database changes and data migrations are automated, kept under source code control and managed with Liquibase. This is one area where my own team could improve, so it was helpful to see this in action.

Gordon Conroy showed me some clever solutions they’ve come up with to test having many concurrent users testing realistic situations, and how they can monitor these scenarios as the automated tests run.

Customer Collaboration

They explained how they work with their current customer. He comes in several days a week to collaborate with all team members, including developers and testers, and answer questions. The customer’s goal is to have a website to show potential investors, so a lot of effort goes into the look and feel and showcasing the functionality. The customer gets continual feedback from tests and from the backlog site map, and in turn is able to review the work completed so far and give feedback to the development team on what changes are still needed. I wish I could have met the customer because I’m betting he is truly delighted (as we want all of our customers to be!)

Continual Improvement

Most of all, I admire the Energized Work team’s obvious commitment to always finding better ways to work, even though they are already functioning at such a high level. They had just experimented with using causal loop diagrams for their retrospectives, to help them identify behaviors and invisible work. They told me that they aren’t just trying to improve the process within the system, but better understand the system itself. As a result, they’re able to make real improvements in effectiveness, not only efficiency. Here’s another example of output from a retrospective.

It was exciting to meet a team of people who are passionate about software quality, love what they do, and clearly have a lot of fun. They obviously have time to learn, innovate and experiment, which is reflected in some incredibly creative solutions to some tricky testing problems.

I’ve been telling my own teammates the ideas I brought away from my visit, and we’ll see what experiments we can think of trying. Some of our best ideas were “stolen” from other teams, and I expect in another six months I’ll be able to tell you what has resulted from my inspiring visit to Energized Work.

Everyone Focused on Quality

Janet Gregory and I, and our teams, have been practicing the Whole Team approach to delivering high-quality software for years, and working hard to explain this concept to others. It was affirming to see a team commit to a high standard of quality and then work relentlessly to keep raising the bar.

I highly recommend that you and your team visit other teams, in your own area or when you are traveling. Every team can teach you something, even if it’s “what not to do”. Visiting a team who has found good ways to produce great software shows you what’s possible in real life, and leave you enthused about trying something new to do better work.

My team creates software to manage 401(k) retirement plans. Back in 2006, the U.S. Congress passed legislation allowing Roth contributions in 401(k) plans. Plan participants can elect to pay taxes on their retirement deferrals up front, rather than the usual tax-deferred 401(k) contributions. Then, when they retire and withdraw their savings, they don’t have to pay taxes on their withdrawals. This is called a “qualified” distribution.

There is a catch – if you make a Roth contribution, you have to leave it in your account for at least 5 years. If you withdraw it early, you have to pay taxes on whatever gains you made from interest and dividends on your investments. We tested the difference between qualified and unqualified distributions both with in-memory FitNesse tests that verified the algorithms in the code, and by manual testing by faking out the dates as needed. But there was no chance yet to have a qualified distribution in real life.

We have many tests for Roth withdrawals, from the unit level on up to the GUI level. One FitNesse test verifies the results when an employee withdraws money that includes funds from their Roth account. Since 2006 was the first year anyone could make Roth contributions, there was no possibility to make a “qualified” distribution until this year. The test set up a participant whose first Roth contribution was in 2006, and verified that the system withheld taxes for the gain portion when the participant withdrew the money.

At the start of 2011, this test started to fail in our continuous integration. Surprise – it’s 2011, the participant has now been contributing Roth funds for 5 years, thus now the distribution is qualified and should on longer have any tax withheld! The actual results of the test showed this difference in behavior.

Some people would say we shouldn’t have an end-to-end test like this, that it’s expensive to maintain. But I think it was cool to see this functionality actually work in production. It allowed me to give a heads-up to the CSRs that they would now see Roth distribution checks going out that didn’t have tax withheld. I changed the expected results, because before this, we weren’t able to have any end-to-end test for this case. It also shows how our tests are living documentation – as soon as we have a test failure, we have to figure out the reason, and either change the code (if it’s a bug) or change the test (if it’s correct behavior).

What’s your opinion? Do you like this sort of test, and this sort of surprise?

My teammate Tony Sweets (sys admin, programmer and genius) just posted a terrific writeup on StickyMinds/Techwell of how our continuous integration and build process has evolved over the years. This includes a history of the hardware we’ve used and how the builds have been set up, both in CruiseControl and Hudson, and all the different things the build process does.

One of my favorite excerpts:

This method worked too well. The testers started splitting up their test suites and were creating new build jobs left and right mainly because it was so simple. They didn’t have to deal with hostname issues and whatnot. They simply had to create the new job from an existing Hudson job and change whatever they needed in order to run a different set of tests. When the job ran it did not care what else was running because it was running in its own separate environment.

I love the easy-to-use Hudson UI, and this freedom to configure our own test jobs the way we want. It’s also easy to stop jobs or kick them off when we want.

Whenever I speak to a conference session or user group meeting, I always tell people, “If you aren’t doing continuous integration now, go back to your office and drop everything and get your CI going. It isn’t hard to do, there are a bunch of good tools available to help, even Testify Wizard to help you set it up. A programmer can do it in a matter of days or less. There’s no excuse to not do CI.”

I’m convinced that in 5 years at the most, any team not doing CI will be looked upon the same way we look upon teams that don’t do source code control. It would just be crazy to not do it! Automated tests don’t have much value if they aren’t giving you quick feedback several times a day. Without CI, your technical debt is bound to bury you quickly.

If I had to pick one reason our team has been so successful the past 7 years, our CI process is it. It’s the pulse of our team, and if it stops (as it did a few weeks back – see Tony’s blog post!), we all just about have a heart attack! When it’s ticking along, we feel healthy and happy.