Leading quality in software organizations

Leading quality in software organizations – what is the best way? For more than two decades now, software organizations have wondered: when we transform to agile or a DevOps culture, what do we do with the testers / quality engineers? The leading practice is obviously to embed them in teams. Should they report to the same manager as the other development team members? Should they report to a separate testing / quality manager? What’s the best way?

As a tester on agile teams in companies ranging from small startups to 40+ engineering teams,  I’ve worked in a variety of organizational structures. There are benefits and downsides of each. I’ll go through those, and then lay out what I think works well for most organizations.

Testers report to same manager as other development team members

The small startups where I’ve worked had only one or two development teams. These were truly cross-functional, including coders, testers, database experts, system administrators, designers, architects, and individuals performing two or more of those jobs. One agile motto is “do the simplest thing that could possibly work”. That means everyone to reports to the same manager. This worked out super well for me. My teams had good managers who understood the whole team approach to quality. They valued testers as highly asa everyone else on the team. Often, we had more testers than programmers on our team, because our application carried a lot of risk and was testing-intensive.

We were truly agile teams. We set our goals together, and collaborated to achieve them. While we had frequent one-on-ones with our manager, we didn’t endure traditional individual performance evaluations. Our self-organizing teams were supported by the business and we achieved great success.

All that said – in my experience, most executives and even IT managers don’t really understand quality. They don’t know the value of making the big investment in letting teams have time and resources to learn how to build quality into the product. I often see programmers on newbie agile teams continue to throw their code over the wall to the testers who are on their same development team. Sigh.

Testers are embedded on teams, and report to a quality / testing manager

This is another organizational structure that can work really well with the right people. (Maybe it all comes down to having the right people, no matter what the structure…) I’ve had a couple of jobs where I was embedded on one or sometimes two cross-functional delivery teams, but I reported to a “director of quality engineering” or someone with a similar title. Again, I was lucky that all the managers, whether their direct reports were testers, coders or others, embraced the whole team approach to quality. They valued testers as highly as everyone else.

One advantage to this structure is that the quality / testing director was the same “rank” as the development director. Occasionally, the development managers might want to go in a direction that would have a negative impact on quality. The quality director an intervene and influence those decisions in favor of quality and of us testers. For example, if there is manual regression testing to do, the quality director can make sure that the whole team feels the pain of doing it – together.

Another benefit I saw with having a “quality team”, though we were all embedded in delivery teams, was we could coordinate quality initiatives for the whole engineering organization. When we were a dozen quality engineers and there were 40 delivery teams – obviously, we couldn’t embed testers into every team. We came up with models and techniques that we could give each delivery team to help them along their quality journey. We organized ensemble exploratory testing sessions to help programmers on all teams gain those skills. It wasn’t easy but it was effective.

I’ve also been in an organization where we testers were on cross-functional teams, happily reporting to the same development managers as everyone else. And suddenly, a new CTO was brought in and made changes. We were told out of the blue that a new “QA Manager” had been hired and we would now report to that person. We were still embedded on teams, but our performance evaluations (ugh) would be done by this new manager. The new manager had no understanding of agile development. He was not interested in joining our ensemble programming sessions to see how the whole team worked to build quality in. Nor was he interested in us doing any quality initiatives together across the engineering organization. Until a new CTO came in and fired him, he really didn’t do a thing!

My preferred organizational structure

Based on my own experience and what I’ve seen in organizations I’ve worked with, I think I know what would work well in many contexts. I haven’t worked with this structure myself, but I know people who have. Embed testers / quality engineers in cross-functional teams. And – have one or more “quality practice leads” at the director level (and manager level, for large organizations).

The testers / QEs report to the same manager as the rest of the cross-functional delivery team. And, the quality practice lead provides extra support for them. Consider an organization newly transforming to agile or to a DevOps culture. A quality practice lead can organize appropriate training for teams to learn how to implement a whole team approach to quality. If testers / QEs are having any struggles on their delivery teams, the quality practice lead can collaborate with the development directors and managers to help the team get on track together. They can lead a quality community of practice for the organization – open to everyone, regardless of their specialty or role.

This approach provides the benefits of testing and quality specialists being equally valued delivery team members, with the whole delivery team collaborating to build quality in. The testing and quality specialists help their fellow delivery team members learn testing skills. They have someone to go to when they need help that they can’t get from someone on the delivery team.

Having a quality practice lead at the same organization level as the highest development organization leaders gives visibility to quality and testing. That lead can work with the other engineering leaders to enable all teams to keep improving their quality practices. They can lead initiatives such as quality practices assessments to help teams improve. For me, it’s the best of both worlds – being a fully engaged delivery team member, and, having the support of testing and quality specialist who’s in a leadership position.

I hope I’ve explained this in a way that makes sense. What do you think is the most effective way to lead quality in modern software organizations? I’d love to hear your experiences.

The title of this post is inspired by the book Leading Quality by Ronald Cummings-John and Owais Peer. I highly recommend this book to help your managers and execs understand how an investment in quality pays off.