AGILE TESTING

Observability, Continuous Delivery, DevOps & Related Resources

We all need to be involved in testing activities and building quality in, all the way around that continuous DevOps loop! It’s what Holistic Testing is all about. Here are some awesome books, blogs, videos, courses and more to help you get engaged with the right-hand side of the DevOps loop. Send me any you have that aren’t on here already!

Join the DORA Community of Practice to learn a variety of ways to enable continuous improvement. I joined it last year, and am now a DORA Guide! I learn so much from the monthly short webinars and Lean Coffee discussions. Most discussions are fundamentally about testing and quality.

Check out Abby Bangser’s blog for so much great insight on observability, site reliability engineering and more. For example, here is a great list of resources and insights from Abby to help testers learn how to get started with observability. Also, this podcast interview with Abby includes many resources in the show notes.

Janet Gregory and I have developed a new remote live course, “Holistic Testing for Continuous Delivery: For a quality DevOps Culture”. Learn more about it on our Agile Testing Fellowship website, it’s available through our training providers around the globe. We also have a new mini e-book, Holistic Testing: Weave quality into your product, available for free download on the same website!

Shifting left and right in the continuous DevOps loop — Let’s get involved all the way around the loop! Graphic from Janet Gregory, inspired by Dan Ashby’s continuous testing loop.

ARTICLES, EXTERNAL BLOG POSTS, FORUMS AND MORE

General Testing, DevOps, CD, Observability…:

“Observability, what to instrument?” Chris Armstrong
“Holistic Testing: What it means for agile teams”, Janet Gregory and Lisa Crispin (check the blog for more posts related to the Holistic Testing model)
“Why platform teams are useful”, Colin Humphries
Google’s SRE site (with books)
SRE vs DevOps Chronosphere (warning, annoying sales stuff there, but the article is helpful)
DevOps Roadmap
“They aren’t pillars, they’re lenses“, Danyel Fisher
“Beep, beep! Pager Duty caller, Pager Duty caller!”, Jessica Greene
“How to get into SRE“, Alice Goldfuss
“Continuous Compliance“, Dave Farley (and see all his other blog posts too!)
“Understanding Observability”, Sanjeev Sharma
“Defining Platform Engineering vs. SRE from a testers vocabulary”, Abby Bangser
“Learning from production use”, Lisa Crispin
“Power Hour: Operability“, Rob Meaney, Ministry of Testing Club
“Use Testing to Develop Better Software Faster“, Marit van Dijk
“Developing a Culture of Observability“, Honeycomb.io (requires name and email info)
“Baby steps into tracing“, Abby Bangser
“Why you should be testing in production“, Kristin Jackvony
“Testing in Production: the hard parts“, Cindy Sridharan
“I test in production“, Charity Majors, increment.com
“Accelerate: 2019 State of DevOps Report“, Dr. Nicole Forsgren
“Continuous testing and continuous delivery in a DevOps world“, Ashley Hunsberger
“Tracing our pipelines“, Abby Bangser
“Building resilient APIs with chaos engineering“, Joyce Lin
“Power Hour: Stuck or Need Guidance on DevOps or Observability“, Ministry of Testing forum Q&A with Abby Bangser
“Quality at the Speed of DevOps“, Part 1 of a series by Brendon Connolly
“Logging, Monitoring and Alerting“, by Kristin Jackvony
“3D&d=www.mabl.com&c=2&c=3&c=6&rp=3&ab=true&corr=&hs-expires=1582920883&hs-version=1&hs-signature=APUk-v5NYc_rwfAe6x6RV9Ujshy49AnmlA”>DevTestOps according to the experts” by Lisa Crispin
“Testing in production, the safe way” by Cindy Sridharan – has a lot of the terminology that will help!
“Monitoring and Observability” by Cindy Sridharan
“No Fear: How to test software in production“, Amber Race
“Continuous Testing” (includes links to more good resources) by Jez Humble
“How Continuous Delivery Impacts Testing“, interview with Jeff Morgan
“A Tester’s Story of Adapting to the New World of Pipelines“, by Parveen Khan
“Does the DevOps tester really exist? Maybe, with some work“, by Justin Rorhman
“The Road to DevOps: How enterprises can meet the challenge“, by Christian Paulus
“Why So Much Talk Around DevOps Culture?“, Federico Toledo
“Q&A: David Farley on Everything Test Teams Need to Know“, LogiGear Magazine
“The world of DevOps can make a newbie’s head spin“, InformationWeek, interview with Peter Varhol and others
“DevOps/QA Era has Started!” by SP Sankar
“From QA to Engineering Productivity“, Ari Shamash
DevOps Topologies website
“DevOps = Dev + Error Budget + Ops“, Jack Shirazi
“DevOps-QA era has been started“, QOrigin’s Blog
“Continuous Testing Using Shift Left Testing Approach“, Homanshu Sheth
The Human Value in Automation-Driven DevOps Testing“, Komal Lopez
The NewStack, has podcasts, ebooks, events, newsletter
“Explaining DevOps at different levels”, Atlassian

ML apps and testing in DevOps / CD

“Continuous Delivery for Machine Learning“, Danilo Sato et al

Test automation:

“Serverless Framework – AWS Lambda Guide – Testing” , Serverless.com
“How to automate your tests with Travis CI“, Stephen Smith
“How do you know test automation isn’t working?” Dennis Martinez – check out more publications on his DevTester website.

Exploratory and other manual testing:

“The Place of Manual Testing in DevOps“, by Chelsea Frischknecht
“Session-based Testing in an Agile/DevOps Environment“, by Lalitkumar Bhamare
“Exploratory Testing: It’s All About Discovery“, Garima Tiwari
“Exploratory Testing: Expanding Testing Across the Delivery Cycle“, Ingo Philipp

BOOKS

A Practical Guide to Testing in DevOps, Katrina Clokie
DevOps for Dummies, Emily Freeman
Grokking Continuous Delivery, Christie Wilson
Team Guide to Software Operability, Matthew Skelton, Alex Moore and Rob Thatcher
Continuous Delivery, Jez Humble and David Farley
Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations, Nicole Forsgren PhD, Jez Humble, Gene Kim
Measuring Continuous Delivery, Steve Smith
DevOps for Developers, Michael Hüttermann
The DevOps Handbook, Kim, Humble, Debois, Willis
Developer Testing: Building Quality into Software, Alexander Tarlinder
Site Reliability Engineering, Google

Webinars, videos, online courses…

Online courses:

DeliveryConf 2020 sessions with slides, audio and group discussions – a gold mine from the leading practitioners in DevOps and CD!
“Test Automation in DevOps“, Test Automation University course from Lisa Crispin (all TAU courses are free! Check them out!)
“Test Automation Essentials“, another free online course from Lisa Crispin

Podcasts and audio (including conference sessions):

Ali Hill on quality Coaching Roadshow, “security controls on an AWS infrastructure“
20QA%20Therapy&utm_content=222412807&utm_medium=social&utm_source=twitter&hss_channel=tw-775332218004799489″>”How to fit testing into DevOps”, QA Therapy Podcast with Lisa Crispin, Sergio Freire and Christian Cunha
2316,20and20Engineers20Abby20of%20MOO”>Observability and Test Engineers – The O11yCast podcast, with Abby Bangser, Charity Majors and Liz Fong-Jones
DeliveryConf 2020 sessions – a gold mine of information!
Maintainability Podcast, including this great episode with Abby Bangser
O11yCast observability podcast
Arrested DevOps podcast
Ministry of Testing podcast with Theresa Neate on testing and testers and devOps
“QA-Driven DevOps, an interview with Tanya Kravtsov“, Skytap podcast
“Advanced Agile with DevOps“, The Testing Show from Qualitest with Michael Larsen, Matt Heusser, and guests Lisa Crispin and Jessica Ingrasselino
Chaos Engineering podcast with Tammy Butow, Joe Colantonio’s TestTalks podcast
“Testimonias del Squad” Podcast, topics include testing, DevOps, agile and more.

Videos, webinars:

“Let’s call platforms what they really are, ways to provide something-as-a-service”, Abby Bangser
“Pact Contract Testing for Quarkus” Holly Cummings
“20 minutes with Rob Meaney on Operability”, Rob Meaney
Dave Farley’s Continuous Delivery YouTube channel
“Build it, test it, run it together”, with Abby Bangser, Maryam Umar, Marit van Dijk
“How to shift to continuous testing in DevOps”, with Andrew Morton, Maryam Umar, Angela Riggs, Isabel Evans and Lisa Crispin
Fast Flow Conf YouTube channel – from a Team Topologies conference
“Devs should love your tests“, Carlos Kidman, SeConf London 2019
Sarah Wells on FT’s transition to DevOps, InfoQ Engineering Culture Podcast with Shane Hastie
“Test Automation in DevOps” course by Lisa Crispin, Test Automation University
Panel discussion with Simon Prior, Joe Colantonio, Parveen Khan and Lisa Crispin, “Why is DevOps breaking testing?”
“Exploring a DevOps Transformation like a Tester“, Parveen Khan
“Testing Ask Me Anything: CI/CD and Delivery Pipelines” with Abby Bangser and Vernon Richards, Ministry of Testing
“The Joy of Testing in Production” with Amber Race, March 2019
“Testing for DevOps with Ashley Hunsberger, Marit van Dijk and Amber Race“, March 2019
“Testing in DevOps: A Testing Panel with Theresa Neate and Hilary Weaver-Rob“, March 2019
“CI and CD Delivery Pipelines” AMA with Abby Bangser video
“Changing Roles of Testers Transitioning to Agile & DevOps“, Lisa Crispin and Joel Montelvisky
“The road to enlightenment: How we learned to stop worrying and start treating infrastructure like any other feature” – Abby Bangser and James Spargo video
“Thunderstruck by Serverless: A Tale of Learning and Testing“, a TestBash Manchester talk by Conall Bennett
“Quality for Cloud Natives“, Sarah J. Wells, European Testing Conference 2019
“DevOps Challenges Facing QA“, Qualititest
“Why We Should Test Programmable Infrastructure“, Matt Long, TestBash Manchester 99 second talk
“Does DevOps Need Continuous Testing?” Gerie Owen, DevOps Days Des Moines 2018
“DevOps and Groupthink: An Oxymoron?” Gerie Owen, DevOps Days Kiev 2018

General DevOps, not testing-related:

“Building and Deploying Your Code with Azure Pipelines“, Abel Wang

Observability, Continuous Delivery, DevOps & Related Resources

ARTICLES, EXTERNAL BLOG POSTS, FORUMS AND MORE

BOOKS

Webinars, videos, online courses…

Some takeaways from the Software Teaming 2024 conference

Celebrating 10 years of friendship, courage, and joy

A look back at release engineering

What DevOps culture means to me – and how you can help it grow

More learnings from Agile Testing Days 2023 – part 3

©2024 Agile Testing with Lisa Crispin. All rights reserved.