Naturally, I replied with a typically long-winded answer. She must have liked it though, because she suggested I cut and paste my reply into a blog post. Here it is!

The key thing with a newbie agile team is to not say “move testers on all teams into automated testing” but “help all teams learn to take whole-team responsibility for quality and testing, and learn how programmers and testers collaborate to automate tests at all levels”. Automated test code is code, and generally, it’s much quicker and better in the long run for programmers to code the automation fixtures. Testers know what to test, and when programmers do the coding, testers have lots of time to help customers come up with examples of desired behavior to turn into tests. Then they can pair with the programmers to automate regression testing and any other types of tests (performance, load etc). Then they will still have time to do the all-important manual exploratory testing.

If they read our book Agile Testing, they will learn that they have to be patient and invest a lot of time in learning and experimenting with different test frameworks. And they should use Mike Cohn’s test automation pyramid concept to see where they’ll get the best ROI in automation. Generally, they should always start by learning how to do TDD and get traction on unit level tests. Then they can move on to API or service level tests, then GUI. But that’s not a rule, each team has to figure out what’s best for them.

I highly recommend Gojko Adzic’s Specification by Example book, and also his blog, gojko.net. Markus Gaertner has a great ATDD book coming out but it’s not published yet.

There are SO many test frameworks, drivers, tools, it’s overwhelming. Recently we had to find a way to automate GUI regression tests for our new code that uses Dojo. Our existing GUI tools aren’t able to interpret the Dojo JS properly. It was truly a team effort, though not a smooth one. Our sys admin had earlier played around with Selenium, and we thought Webdriver might work. The sys admin did a spike and proved it did work, then he and our senior architect spiked a couple of different frameworks – one homegrown, one using Geb. They demo’ed it and got everyone’s opinions. For now we are going with the Geb framework, but down the road we might decide it’s not quite right and try something else.

Anyway, that’s a long-winded answer. One good place to start getting an idea about functional test tools is the Agile Alliance Functional Test Tools group’s spreadsheet comparing various tools, at http://bit.ly/AgileTestTools. When the teams are ready to think about functional testing, they might take a look and get an idea of all the choices. It’s a good idea to stop and plan a test automation strategy.

It is really, really hard to get started with test automation. There’s a period where it’s just extra work and no reward. But eventually they’ll cross over that Hump of Pain and start reaping the benefits, eating away at their technical debt!

Agile Testing Quadrants

You might want to start with Brian Marick‘s original posts on his agile testing matrix, which we called the Quadrants and (with his permission) made the heart of our Agile Testing book.

The quadrant numbering system does NOT imply any order. You don’t work through the quadrants from 1 to 4, in a waterfall style. It’s just an arbitrary numbering so that, in our book and when we are talking about the quadrants, we can say “Q1″ instead of “technology-facing tests that support the team”. 

Most projects would start with Q2 tests, because those are where you get the examples that turn into specifications and tests that drive coding, along with prototypes and the like. However, I have worked on projects where we started out with performance testing (which is in Q4) on a spike of the architecture, because that was the most important criterion for the feature. If your customers are uncertain about their requirements, you might even do a spike and start with exploratory testing (Q3).

Q3 and Q4 testing pretty much require that some code be written and deployable, but most teams iterate through the quadrants rapidly, working in small increments. Write a test for some small chunk of a feature, write the code, once the test is passing, perhaps automate more tests for it, do exploratory testing on it, do security or load testing on it, whatever, then add the next small chunk and go through the whole process again.

The quadrants are merely a taxonomy to help teams plan their testing and make sure they have all the resources they need to accomplish it. There are no hard and fast rules about what goes in what quadrant. Think through them as you do your release, theme, and iteration planning, so your whole team starts out by thinking about testing first.

Michael Huetterman adds “Outside-in, barrier-free, collaborative” to the middle of the quadrants, see his Agile Record article or his excellent book Agile ALM

Visit my Presentations page for some slide decks that contain more information on the quadrants, or check out our book. I’m always happy to talk about the quadrants, just send me an email!

Software Quality Connection, “How to Improve Communication between QA and Development“

SearchSoftwareQuality.com, “Agile development: The whole-team approach”

Also on SearchSoftwareQuality.com, “The whole-team approach to Agile development: Examples and benefits“

I’ve got a bunch of tips and Ask the Expert Q&A on SearchSoftwareQuality.com, please check them out! I’d appreciate feedback.

The Real Me, with Chester and Ernest at the Brewpub

I’m looking forward to learning about lots of people in this series!

I’ve just updated this post with links to photos of the Big Visible Charts in the Energized Work Lab, accompanied by explanations from Simon Baker. Check ‘em out, they will give you ideas for your own team to use.

What a Team!

I’ve always maintained that I work for the coolest team on the planet, and most of what I try to help others learn are things I learned from and with my own awesome teammates. I don’t want to be disloyal, but the team at Energized Work raises the bar for cool. Simon Baker (@energizr on Twitter) invited me to visit while I was in London earlier this month (and he gave me permission to write this post about it). I love to visit development teams wherever I go, and learn what other practitioners do to improve software quality, so I was happy I could make the time. Mohinder Kohsla had breakfast with me that morning, and was kind enough to guide me to the Energized Work lab, as he already knew some of the folks there.

Big Visible Charts

What blew me away right off the bat at Energized Work was the creative ways they use whiteboards. One whiteboard had personas, complete with photos and bios, and assumptions. That’s not too unusual, but it’s still nice to see someone actively using personas in real life.

What intrigued me most was the way they represent their backlog for their current project as a site map on a big whiteboard. They start drawing site map for the web application they’re working on on a big whiteboard, with placeholders for each screen, and arrows showing how the navigation flows. Since this is on the whiteboard, it’s easy for them to evolve the map as the project proceeds. They overlay cards with user goals so they can identify which journeys deliver value. Screen prints of completed pages are stuck to the board over the original placeholder, along with the cost to develop it.

The team starts with minimum functionality delivered to achieve a goal, working in fast, tiny iterations. As Simon explained to me, the customer can decide based on that functionality and what it cost whether to invest more in that user goal, for example, to “enrich the feature with more functionality”. The site map / backlog is part of the team’s mechanism which allows the customer to decide “just in time” whether to “go deep or broad” every week in response to user testing.

The Energized Work team has experimented with different ways to do story/task boards as well. Instead of a traditional Scrum task board or Kanban board, they represent each story with a big square on the whiteboard. In the middle of the square is a story card has some high level acceptance tests written on the back. Surrounding it is plenty of space to draw mock-ups, prototypes, write questions, write down test cases, whatever they need to discuss or do. A team member only has to look at the board to see story requirements.

Instead of moving cards from column to column, eg. from “Work in Progress” to “Verify”, they represent the vertical slicing of the story with different colored dots, representing feedback from different activities including customer review, UX reviews, and manual exploratory testing. Some cards may cycle through coding, testing, reviews, and back to coding several times before they’re done. Gordon Conroy, the tester, keeps an eye on the “big picture” as new functionality is created slice by slice. He was so knowledgeable about all aspects of the project, it’s clear he works constantly together with the rest of the development team. If companies with separate test teams could see this in action, they’d get why testing can’t be a separate phase or done by a separate team!

My own team uses a fairly standard task board with rows for stories and columns for status, and we write high-level requirements and questions on a separate whiteboard. We also work in these fast, tiny iterations with multiple slices of each story, and I am wondering if we can borrow some of these ideas to better represent that visually. We put more details about slices, test cases and specifications on the team wiki, but developers don’t always look at the wiki. Of course we also do specification by example with executable tests, but whiteboard drawings and notes would make a good, quick-to-access supplement during development. I’ve told my team everything I’m writing here, and we’re ruminating on how we might experiment to improve visibility, while keeping our remote teammate involved.

Another Big Visible Chart in the Energized Work lab lists usability heuristics. Clearly, no aspect of software quality is neglected here, and there were so many visual reminders to keep the team on track. Just about every problem my team has experienced, we solved by making it more visible, and seeing how well another team accomplishes this is affirming.

Driving Development with Tests

The Energized Work team are clearly expert practitioners of TDD and specification by example. They showed me the continuous builds in Jenkins for a couple of different projects, it looked a lot like what my own team does. I’ve personally met few teams that have as sophisticated a build job set-up as ours. I found it interesting that they’ve used different test frameworks and tools on different projects, they are clearly committed to finding what works best for each situation, and experimenting with new approaches. Some of the tools they use are new to me, such as Spock for unit testing Groovy and Java, using a BDD-style syntax.

All database changes and data migrations are automated, kept under source code control and managed with Liquibase. This is one area where my own team could improve, so it was helpful to see this in action.

Gordon Conroy showed me some clever solutions they’ve come up with to test having many concurrent users testing realistic situations, and how they can monitor these scenarios as the automated tests run.

Customer Collaboration

They explained how they work with their current customer. He comes in several days a week to collaborate with all team members, including developers and testers, and answer questions. The customer’s goal is to have a website to show potential investors, so a lot of effort goes into the look and feel and showcasing the functionality. The customer gets continual feedback from tests and from the backlog site map, and in turn is able to review the work completed so far and give feedback to the development team on what changes are still needed. I wish I could have met the customer because I’m betting he is truly delighted (as we want all of our customers to be!)

Continual Improvement

Most of all, I admire the Energized Work team’s obvious commitment to always finding better ways to work, even though they are already functioning at such a high level. They had just experimented with using causal loop diagrams for their retrospectives, to help them identify behaviors and invisible work. They told me that they aren’t just trying to improve the process within the system, but better understand the system itself. As a result, they’re able to make real improvements in effectiveness, not only efficiency. Here’s another example of output from a retrospective.

It was exciting to meet a team of people who are passionate about software quality, love what they do, and clearly have a lot of fun. They obviously have time to learn, innovate and experiment, which is reflected in some incredibly creative solutions to some tricky testing problems.

I’ve been telling my own teammates the ideas I brought away from my visit, and we’ll see what experiments we can think of trying. Some of our best ideas were “stolen” from other teams, and I expect in another six months I’ll be able to tell you what has resulted from my inspiring visit to Energized Work.

Everyone Focused on Quality

Janet Gregory and I, and our teams, have been practicing the Whole Team approach to delivering high-quality software for years, and working hard to explain this concept to others. It was affirming to see a team commit to a high standard of quality and then work relentlessly to keep raising the bar.

I highly recommend that you and your team visit other teams, in your own area or when you are traveling. Every team can teach you something, even if it’s “what not to do”. Visiting a team who has found good ways to produce great software shows you what’s possible in real life, and leave you enthused about trying something new to do better work.

To what are you devoting  your 10,000 hours of practice?