My Agile Testing Days USA 2023 experiences, Part 1!

Agile Testing Days USA 2023 brought me lots of new learnings, new friends, new energy! I plan to share my experiences here over the next several days, in multiple posts. Delivering small chunks, sustainably!

The Holistic Testing Model in practice

I’m most excited about the interest in a holistic approach to quality and testing. Especially heartening was seeing so much desire to learn about continuous delivery, DevOps, and observability. The first sign of this was a great turnout for the all-day Holistic Testing tutorial that Janet Gregory and I facilitated on the Monday. Most of the tutorial consists of hands-on practice in small groups. By afternoon, our groups were collaborating with so much trust and engagement.

We got positive feedback on many techniques we introduced. One of the most popular was the pipeline exercise to understand testing in the workflow to production. Another was Ashley Hunsberger’s Test Suite Canvas, which guides conversations around care, maintenance, and value of automated test suites.

Our tutorial content is drawn from our two training courses, “Holistic Testing: Strategies for Agile Teams”, and a brand new course, “Holistic Testing for Continuous Delivery”. Our participants’ enthusiasm is good affirmation for how we have designed the courses, especially the new one.

Testing, quality and deployment pipelines

Everyone in the testing community should get involved in DevOps, deployment pipelines, monitoring, observability, and testing in production. I consider these as testing activities, though some people think they “belong” to operations specialists, site reliability and platform engineers. See my page here on my site with a long list of resources that help people learn these important parts of software development.

It’s great to see more and more sessions on these topics at “testing” conferences. A big crowd of us attended David Dang‘s talk, “Optimizing Quality Stages in CI/CD Pipelines”. I loved his approach to making “key quality stages” in deployment pipelines highly visible.He highlighted static code analysis, “smart” code coverage, unit tests, unit integration tests automated smoke tests, notifications, and dashboards. These are important for fast feedback on a wide range of quality attributes, including security vulnerabilities. He recommends automatically rolling back changes if any of these test stages fail, using clear pass/fail criteria. His advice to making pipelines transparent is spot on.

Testing beyond what some people call “testing”

João Proença‘s talk “A Little Less Testing, A Little More Quality” delved into more areas of quality that might not come to mind when you hear the word “testing”. He focused on ways we can manage risk in the changes we release to production. Diverse activities can prevent, accept, and transfer risk. And, when those three approaches aren’t enough, we can observe and react. João shared a personal story to illustrate ways he managed risk. He led an effort to carve a high-risk part of an application out of the monolith to allow frequent releases and fast feedback. He and his team succeeded with controlled rollouts, short feedback loops, and trunk-based development.

João emphasized that “good quality” isn’t enough. We have to give customers products that are useful. Everything we do to deliver software that is useful, good and correct can be considered testing. We need facilitation skills, ability to get team and business buy-in, and work with other specialists such as architecture. The term “quality engineering” may capture this better.

Human connections, and more to come!

My favorite part of any event is getting to see old friends and meet new ones. ATD USA 2023 provided many opportunities. There was a fun and friendly vibe. We had awesome discussions at Lean Coffee each morning.

I’ll share more experiences here in the next days! I also recommend reading Toni Lauderbaugh’s thoughts on the conference, and Lisi Hocke’s comprehensive post with her awesome sketch notes.