Agile Testing with Lisa Crispin

Today I was lucky to hear part of the Thomas Jefferson Hour as I drove down to the ranch to work the donkeys. Thomas Jefferson, channeled by the actor and scholar Clay Jenkinson, quoted from his first inaugural address: “Every difference of opinion is not a difference of principle”. Jefferson believed that our elected officials should respond to moments of tension (and as he says, there was just as much rancor in political debates back in his day as now) with politeness, good humor and respectfulness. He recommended that our current elected officials, if they can’t find common ground, should adopt “artificial good humor” and assume the best about their opponents.

TJ and I might both have our head in the clouds, but I’m a big believer in civility. I’m not recommending that we all walk around on eggshells and worry about offending people. But I do think we need to all work hard to make sure that everyone on our software team feels safe to express opinions, and conversely, is willing to accept the team consensus or majority rule, and work to make whatever approach is taken successful.

This is something I want to take to heart for myself. Here’s an example. I’m trying to explain an issue to the programmer who is on production support this sprint. I’ve analyzed the problem and think I know how it should be fixed. He seems not to understand, gets impatient with me, throws up his hands and tells me I am mistaken. I could feel mad and hurt by this, or I can remember that being the production support monkey is frustrating and stressful. I’ve given him the information I wanted to convey, and I can leave him to process it. He’s smart and motivated to take care of our customers, so I know he will find a good solution. He shouldn’t be rude to me, but if I can avoid escalating the tension, we will have a good outcome.

Indeed, later the programmer comes over and apologizes, says now he understands what I was trying to say, and appreciates the research I did into the issue. We talk about the fix and how to test it.

I’m lucky to work on a team that, though there is lots of joshing and joking, is composed of people with a deep commitment to providing a quality product, and open minds willing to consider anyone’s ideas. I’ve been on teams that didn’t provide this atmosphere of respect and civility, and I must confess, I hightailed it away from those toxic environments.

The “agile” movement was built on principles and values. “Every difference of opinion is not a difference in principle”. Give your team time and space to agree on principles and values. Going forward from there, each team member will feel free to float their ideas and opinions, and have healthy discussions. That’s what makes a project successful.

News!

Janet Gregory and I have an article, and are interviewed in, this month’s Software Test and Performance Magazine, devoted to “Women of Influence in Software Testing”. Please check it out, it’s a great issue with lots of meaty articles.

I have a sidebar in Dawn Cannan’s terrific “Be the Worst” article in the inaugural issue of Agile Record.

Matt Davey wrote a wonderful thumbnail summary of our book in his review of it, we are grateful.

I work for a mid-sized software company with around 27 development teams, mostly Scrum but also a few doing Lean and Kanban. We have a large and diverse product of HR-related software, I think there are around 6 million LOC, on three different platforms, including an ASP product and a .NET product. I’m fairly new, I started at the beginning of June, so I’m still learning about how everything gets done. We release four times a year and the software has won many prestigious awards. I’ve been impressed with this large-scale implementation of agile development. Just the speed of the daily Scrum of Scrums is amazing.

Many of the teams do a great job of automating their tests, and a few teams have their own continuous integration (CI) that runs multiple times a day. Other teams have automated tests, but must wait for the company-wide build to run, which is usually only once per day and sometimes not that often. They don’t get feedback fast enough, and that slows them down – a lot.

I’ve been asking why some teams struggle to automate tests, and why some teams don’t have their own CI. I’ve also asked my own team why most of our functional tests are automated through the GUI, rather than behind the GUI. We use FitNesse, but mainly to drive GUI tests with SWAT.

The answer I get a lot is “It’s the framework”. We have three architecture teams, and there is a framework (maybe there is more than one, I don’t fully understand it yet) that apparently makes it hard to test at the page or object layer. In addition, some teams’ code has tentacles into legacy code written in Delphi whose tests are written in WinRunner. Also, the current framework appears to not be fully supported with automated unit tests, it was developed a few years back. Programmers new to the framework seem to have a hard time understanding how to work with it. The framework clearly causes a lot of frustration. A new framework is under development, but it won’t be available for some time.

In order to understand the existing framework and find out whether the new one will facilitate testing behind the GUI, I was fortunate to get to meet with the leader of the architecture side of the development group. He gave me an overview of the new framework’s current design (which may change), aimed at making our product easily configurable by customers, using web services. It looks like this design allows testing under the GUI level.

Additionally, I learned that it is possible to test under the GUI in the current framework, and I met someone on another team who has automated many FitNesse tests at the object level. I plan to get her to show me how these tests work, and see if what she did would work for our code.

I haven’t worked at a company this large since the early 90s, and I’ve never worked in a company, agile or otherwise, that had “architecture teams” and “frameworks”. I get it that our product needs to look and work consistently across the board, so it does make some sense to centralize the design of the UI and perhaps other parts of the system. However, if the architecture is done by one team, what does that mean for programmers on other teams? Is the fun part of their job taken away? Do they get a chance to think for themselves and improve the product?

It also seems to me that there needs to be a lot of communication amongst the entire development community of our company, that all programmers should have some input to the architecture process. I’ve found that communication between teams in general is a bit lacking. Each team is so focused on delivering its part of the product, they don’t always feel they have time to go talk to other teams. So, for example, perhaps nobody on my team has talked to the person who did a lot of FitNesse functional tests.

Does your company have an architecture team, and/or a framework that everyone works with? Please comment and share your experiences. As a tester, I’d like to know what I can do to help my team deliver the highest possible quality when we have to work with a framework designed by someone else.

This month’s issue Dressage Today had some articles about the two-time Olympian dressage rider and trainer, Steffen Peters. He talked about the role of discipline in turning dreams into reality. For him, this means keeping himself mentally and physically fit through a thoughtfully planned exercise and diet routine, as well as understanding the capabilities of each horse and how it can learn best.

My own dressage trainer tells me all the time, “It takes a million repetitions to change a bad habit.” That’s only a slight exaggeration in my case. I remind myself (and she reminds me) about 50 times a ride to correct each bad habit, for example, to put my hands closer together and keep them centered over the horse’s withers. Some of my bad habits are 50 years old, so they’re hard to fix, but I’m slowly getting better.

Discipline is a critical component of delivering high-quality software, too. We need to thoughtfully plan to use the right practices for us. We need to remind ourselves every day, via big visible charts, information radiators, retrospectives, any means that works to learn new good habits and reinforce the things we do well.

During a recent estimating meeting, our product owner (who normally knows better) asked, “About 6 weeks ago you estimated this XYZ story at 5 points. Is there any cheap, hacky way to do it for one point?” One of the developers responded, “Possibly, but we are not going to do a cheap, hacky solution”. It’s tempting to cut corners to meet a business need, but hacking in a quick fix that increases our technical debt down the road isn’t the way to do it.

Another example where discipline and good habits came into play occurred in this morning’s Scrum. A couple of the customers brought up a significant piece functionality that they had accidentally omitted from a story we’re already working on. It was tempting to say “OK, we’ll add that in too.” But it was too big a change to put into the story at this late date. We discussed possible workarounds, and one of the customers came up with one that would do the job. They will write a new story, and if we can, we’ll pull it in this sprint. If not, they’ll get it two weeks later.

Sometimes we do make compromises about quality, and implement a solution that is less than ideal. There are always business trade-offs to consider. But we don’t make these lightly. And even after more than five years of agile development, we spend time on a serious retrospective every iteration, and work hard to keep improving the way we work and the quality of our code. Some of our bad habits have been hard to fix, but we keep working at it. Our business continues to grow, which to me is the best feedback about our disciplined approach to keeping up good habits and eliminating bad habits.