Practice, Practice, Practice!

April 22nd, 2011

The latest issue of Agile Record is out, and includes some terrific articles by Gojko Adzic, Jurgen Appelo, Ellen Gottesdiener, Johanna Rothman, Linda Rising, Catherine Powell and several more – what an incredibly talented and diverse group of contributors! My “Agile Testing in Real Life” column is about practicing our software development (that includes testing) skills. I like to lead by example, so in this article I recount everything I’ve done so far this year to grow my own skills and become a better software tester.

To what are you devoting  your 10,000 hours of practice?

“Our manual testing…”

April 5th, 2011

Yesterday I had a call from a participant in my recent BayAPLN session. I completely screwed up and, when I tried to save the phone message, I deleted it, and somehow the number that my phone saved, I could not call back. So, I’d like to try to answer the question, as best I remember it, here. If you still have questions, please email me, because this is such an important topic.

Since I listened to this question in my car and could not write it down, I don’t have it exactly, but it was along the lines of: “My team has no automated testing. One of our user stories did not have all the testing completed by the end of the sprint, because there was not time to complete the manual tests. Would you please recommend tools to help us?”

I know of a lot of great testing tools, GUI drivers and frameworks. If you want to do GUI testing, you can’t go wrong with Canoo WebTest, Selenium, Watir. Robot Framework, FitNesse and other frameworks can be used for testing at the API level or combined with a driver for testing through the GUI. Those are just a few examples of what’s available. Even better, your team can write its own testing frameworks and harnesses.

But tools aren’t the point. The point is that the best people to solve your testing problems is your own development team. By “development team”, I mean programmers, testers, BAs, DBAs, sys admins, everyone involved in delivering software. If you aren’t delivering user stories in a sprint because testing isn’t finished, you need to look at that problem as a team. By bringing all your team’s different roles, experiences and viewpoints to bear on this problem, you will think of experiments to try, and you will arrive at solutions.

Give yourselves time to experiment. Engage everyone involved in delivering software in solving the issues of getting all testing activities done each sprint or iteration. This diversity of viewpoints means you will eventually succeed. YOUR TEAM can solve this problem. Give it a chance!

Bay Area Testing Practitioners Brainstorm About Learning

April 5th, 2011

On March 24, I facilitated a workshop at Agilistry Studios in Pleasanton, CA, on Learning for Testers. 19 practitioners from the Bay Area got together to consider how, as testers, we can learn ways to contribute more to our teams, as well as raising the bar for our profession. I’d like to share some of the ideas generated by this engaging group!

Why We Should Learn

We started out by considering good reasons that testers should learn. Participants self-organized into groups, brainstormed ideas, and each group chose their top three ideas to share. Some of the items surprised me. One reason to learn was “to reduce stress”. I asked the group to elaborate. They pointed out that it is stressful if you aren’t sure whether you have the appropriate skills to do your job. That is so true, I have felt that stress before!

Another category of reasons that resonated with me was “to have fun” and “not to be bored and have ADD”. Finding joy in our work should be our highest priority IMO!

Rather than take up room on this page, I invite you to see the photos of the actual results on my Picasa site.

What We Should Learn

I love the Bay Area, it is truly a crucible of software creativity. It is no surprise, therefore, that the ideas of what to learn were surprising to me.

“Knowing when to stop” – wow, so true. Any good tester can find things to test forever. When have we achieved the “minimum”? What is “enough”? Given that in real life, we have time and resource limitations, this is a crucial skill.

“How to invent test ideas” – cool!

“Deciding what is most important to test” – that kind of ties in with “Knowing when to stop”, I think. Knowing how to analyze risks, learning customer and user priorities, these are necessary skills for testers.

I was glad to see “thinking skills” were a priority, such as: Communication, Integrity, How to Teach. Integrity is particularly interesting to me. It’s essential, but how do we learn it? How do we teach it? Comments, please!

Another interesting skill to learn was the “Elevator Pitch”. If you can describe what you do in under a minute, you must really understand it. This exercise produced many intriguing skills – please check out the photos of the sticky notes.

How To Learn These Things?

I think the most innovative ideas in this workshop surfaced during the “How To” brainstorming.

Consider this: “Note your assumptions. Later, compare what happens to your assumptions. When you stop, note your reasons for stopping. Mindfulness.” I don’t know about you, but I am going to try this.

Journalism class, to learn how to make elevator pitches – what a great idea! We have to go outside our profession to get skills that will help us do our best work.

Risk taking – set out to fail! This takes a lot of courage, but failure is a great way to learn, so why not?

And finally, my favorite, “Learn by doing”. Isn’t that the best way? But to learn, we need time. If we’re always focused on meeting deadlines, we won’t absorb any learning.

Make some room today for learning. Your team has the best solutions to your problems. You just need to take the time to talk about them and think of small experiments (as Linda Rising says) to address them.

My colleague and co-author Janet Gregory and I have a new article in this month’s Better Software about Learning for Testers, with some additional material on Techwell.com. We’d love to know what you think, please comment!

Newly Available Online

April 1st, 2011

My Software Quality Connection article, Selling Agile to the CFO, has been reformatted by ITBusinessEdge into a nice little slide deck. I think it is really easy to read & digest in this new format.

I have a new article on TechTarget’s SearchSoftwareQuality.com site, “Getting on the Same Page: How Testers Can Help Clarify Requirements”. I’d love to hear of your own experiences in this area.

Shortening the Feedback Loop

March 20th, 2011

I’ve been driving donkey-drawn carts for five years or more. The donkeys and I are pretty good at it. Ernest was obstacle-driving champion five years in a row at the Castle Rock Mule and Donkey Show. Ernest and Chester have won several Feed Team races (Mini class).

Recently we upgraded to a wagon. This is a miniature buckboard with a fifth wheel. The wheelbase is about 8 feet, and the tongue another 8 feet. As the driver, I perch WAY up on a seat, far from the donkey team. Look how far away I am.

Wagon Team


The first time I took encouraged the donkeys to some speed pulling this wagon, they started to lope, and then I could tell they were not exactly under my control anymore. I flashed back to those runaway stagecoach scenes in old Westerns. Not a good feeling.

Luckily, we have an awesome donkey driving trainer, Tom Mowery. Tom suggested that I sit WAY forward on that seat, leaning over the front of the wagon, taking a short hold on the lines (what would be reins, if we were riding instead of driving). This way, I could react quickly if they got out of control. OH – a short feedback loop!

I practiced getting the donkeys to lope, then bringing them back to a trot – all while in this aggressive position hanging over the front of the cart with short lines. We practiced getting into a lope, and coming right back to a trot. With immediate feedback, the donkeys (they are SO smart) figured out that the wagon wasn’t a scary thing chasing them, that they were actually controlling its speed, and they could slow it down on command.The short feedback loop let them fix the problems right away without any panic.

Short feedback loops work the same way with software teams. If we have a continuous integration process that runs our regression tests on each new version of the code, we know within a few minutes or hours whether new or updated code has broken something. When we know right away, it’s easy to fix. Problems don’t worry us, because we know we can fix them in a timely manner and move on.

Short feedback loops give us confidence. Confidence leads to enjoyment. I love driving my donkeys, and I love working on a software development team that is guided by continual quick feedback! Look how much fun this is!

An Agile Team!

Agile Documentation

March 14th, 2011

If you’ve ever wondered how much documentation is enough in an agile project, or you’ve heard you don’t even need documentation with agile, please check out my article on StickyMinds about how my teams have approached agile documentation.

Agile in a Flash!

March 10th, 2011

I was an early adopter of the terrific Agile in a Flash cards by Jeff Langr and Tim Ottinger, but left in the dust in the new fashion of having one’s photo taken with the card deck! I have remedied that situation with help from my friend Anna Blake, who was my photo shoot Art Director and photographer. We got up bright and early today on a cold Colorado morning, put on our Carrharts and Elmer Fudd hats, and showed the cards to the donkeys and goats.

Jo, Edgar and Chester are keen to get up to speed on Agile

Ernest is the cautious late adopter, but he is moving in for more

Belgium Testing Days: A Look at a European Conference

March 3rd, 2011

I wrote up my experiences at the excellent Belgium Testing Days conference which took place in February. In particular, I thought about what is different for me at conferences outside the U.S. compared with “domestic” conferences. They’re all great learning and networking opportunities, but culture makes for some nice differences. Overall it was a wonderful time and I’m still thinking how to apply everything I learned. Read about it on SearchSoftwareQuality.

CodeRetreat Boulder: A Tester’s Experience

February 28th, 2011

This past Saturday I was fortunate to be one of the 60 people enrolled for the largest Code Retreat ever, Code Retreat Boulder, facilitated by Corey Haines and Chad Fowler, organized by Prakash Murthy. This was my first Code Retreat, though I’ve worked on teams practicing TDD, pair programming, and good design practices for more than ten years.

I was the only tester attending, and as I had feared, I was somewhat out of my depth. Nevertheless, I write automated tests, and I need to apply the same design basics to those as we do to production code. The past couple of years, I’ve tried to think of ways to help testers learn those principles, and written articles such as “An Intro to Test Design” in the Testing Planet. One goal of Code Retreat is stretching yourself, making ugly messes and not worrying about it even though our goal is perfection. I was definitely stretched way beyond my comfort zone!

Corey explained the basics we should practice:

  1. Tests pass – J.B. Rainsberger says this is “like breathing”.
  2. Don’t Repeat Yourself, keep code DRY
  3. Tests and code reveal intent – name things well
  4. Small – work in tiny increments.

The day was divided into 45-minute pairing sessions; we switched pairs each time. We wore name tags that indicated our programming language preferences. At the end of each session, we had to delete our code, stand up, and debrief for a few minutes about what we learned and what to try next. We also had a long lunch break (with delicious food!) The coding problem to solve was Conway’s Game of Life. As I dislike both math and puzzles, this scared me.

Mike Clark generously offered to pair with me in the first session. I’ve known Mike for ten years and he’s a TDD guru. We used RSpec and TextMate, both new to me (at work, we use test::unit with Ruby, and I use IntelliJ Idea). It was so natural for Mike to start writing a tiny test, then a tiny bit of code, which he put in the same file to start off – quite efficient. He had brought graph paper, which was handy for writing down examples of what a Game of Life grid would look like. We immediately set about creating and testing a board of cells. Corey came around to look at what we’d done, and commented that we should try to avoid numbers and X/Y coordinates. Hmmm. He mentioned encapsulation, which I understood to mean we don’t need to know about cells, only the board.

Apparently most everyone had started out creating a grid and using X/Y coordinates. For the second session, I paired with Erika Bricey, who works at Rally Software. We decided to follow Corey’s advice to start with the rules, to focus on the names for the methods, and whether the methods are doing too much. Corey mentioned that a class with plural name is a collection of other things. Erika and I had some good ideas on how to approach the problem, but for sure I couldn’t turn those into code, and Erika struggled too. Erika also used RSpec, but she worked in Eclipse.

Seeing our frustration, Corey and Chad suggested we less skilled persons find highly skilled pairs. So I waylaid Marty Haught, a test-obsessed programmer I’ve been wanting to talk to. He went along with my desire to focus on rules and on the names of things. For example, we started out with a module called “alive”, for cells that are alive. Later we needed to have dead cells, so we renamed that to “remain_alive” and the other module was called “dead_cell”. We wrote tests and code for most of the rules. We tried to follow Corey’s advice that “anytime you need to DO something, ASK another object”. For example, to find out if a cell is alive, ask “How many neighbors do I have?” and that will invoke an object that says “Here’s my number of neighbors, am I alive?”

Corey told us that if every time you have to make a decision, you ask someone(thing) else, you end up with no “if” statements. He challenged us to see how far we could go before we had to have a number. He noted that there are two parts to the rule: the hard-coded rule such as “Any live cell with fewer than two live neighbours dies” also states the spirit of the rule: “as if caused by under-population”. Aha! We should write our code in terms of under- and over-population instead of hard-coded boundaries.

My first pair after lunch, Anthony, used MacVim. I’m an old fan of Vim, but hadn’t seen MacVim. We talked about the fact that the grid is infinite, and decided to see if we could get information about the neighbors of any one cell, working with relative numbers. Anthony was pretty sharp at figuring out how to do this, and we made a lot of progress in the 45 minutes. Other pairs tried starting with the outside abstraction and working in.

I was getting tired and didn’t take as many notes in the afternoon – it was all I could do to keep up with what my pair was doing. I had low expectations with regard to my Ruby coding knowledge, but I learned how much I don’t know! For the last session, I was the kid who didn’t get picked for the softball team, I didn’t have a pair, so I observed Rob Park and his pair working in Java. I work on a Java team. It was interesting to see the immediate contrast of working in Ruby vs. Java for the same coding problems. Java looked much slower and more cumbersome to me.

We finished with a closing circle where each person stated what we learned, what surprised us, and what we’ll do differently back at work on Monday. Many of the participants were new to pairing and TDD. I’m not, but it reinforced the value I’ve gotten by learning through pairing with a programmer. I learn more about code design in an hour of pairing than from days of reading books. I plan to be more diligent in applying good design principles to automated test code – for example, pay more attention to using good names. I won’t just take the first solution that occurs to me, I’ll experiment with different approaches. Let’s see if I achieve that goal!

Using Mind Maps for Test Planning

February 28th, 2011

Recently I wrote a post about how our team used a mind map to help us with theme planning. We also sometimes use mind maps to help us in planning our testing. Some people asked if I’d post an example. Here it is:

Theme Testing Mind Map

Theme Testing Mind Map

We’re a financial services company, and this theme was a rewrite of one of the oldest and most critical parts of our system: sweeping a tiny percentage of the assets in each participant’s account as pay for our services.

My fellow tester Lori Thayer (who came up with the idea to mind map test cases) and I wrote the initial mind map, then discussed it with the rest of the development team, the product owner, the controller and other stakeholders. Our central node is “Run”, because this is a batch process that needs a way to be kicked off, make sure prerequisites are in place, provide a way to monitor progress, and be restartable in case of problems. Not only is it important to test these parts of the system, but they’ll be invaluable to help us execute and monitor tests.

Without boring you with all the details of this them, you can see that the map includes some primary nodes such as “collection” for sweeping shares out of the account, “sale” for selling the shares once they’re in our company account (definitely in our best interest!), and “instructions”, each representing one participant’s account. We drew dotted lines between related nodes, such as “collection” and “sale”, “sale” and “fast401k account”. The map includes details such as the internal account number where the funds should end up. On the right side, we wrote questions that nobody was able to answer yet. These got answered as we learned more through talking to stakeholders and doing exploratory testing on a design spike.

We referred to this map many times over the next several iterations as we implemented this theme. We checked off the areas that were tested, added more questions and/or answers, even added more nodes as we thought of more areas to test. We didn’t get down into level of test case detail, but the map shows the relationships we needed to test, in addition to the individual components. (Unfortunately, I neglected to take an “after” photo).

We got into more details on the team wiki, using color-coding to show questions and answers. Our most senior programmer lives in India, and the wiki is a good way for him to participate in discussions like this.

Wiki Page Example

Wiki Page Example


Here’s a snippet of some test scenarios we wrote. When we’re satisfied with a test, we put our initials next to it to indicate it’s done.


Example Test Scenarios

Example Test Scenarios

It took us five iterations (ten weeks total) to finish this theme. At the end, we felt confident that we’d thought of and discussed all angles of the process, its potential impacts on other parts of the system,  talked to all stakeholders about their individual needs and concerns, and done adequate testing not only at the functional level, but also performance and usability.