Test Automation Design and Strategy Training

I meet more and more teams, both within my company and externally, who are getting really good at agile testing. They’ve adopted a Whole Team approach, they write tests first, they collaborate and communicate well, they use feedback to improve. Agile teams are functioning much better overall than what I was seeing a couple of years ago.

One perennial issue – and this isn’t new with agile – is that testers and teams still struggle with how to design and write automated tests that have a good return on investment. That is, the tests are reliable, stable, maintainable, and provide good value for the amount of work that goes into them. We also all struggle with how much to automate, what to not automate, what to keep in our regression suite.

Janet and I devoted a section of our book to automation, and we presented principles, techniques and strategies that will help. We also include this in our Agile Testing course, and we both do conference tutorials on how to succeed with automation in agile projects. I think in addition to that, a lot of people need some practical hands-on experience. They need to be guided through an example of automating tests for a story, then they need to try it for themselves.

I’m developing an internal class for people at my company, and we also have ideas for workshops and other ways for teams who have solved some automation problems and developed good practices to share with other teams. I just mind mapped the ideas we have so far. At Adam Goucher’s suggestion, I’m posting my first draft here.

It’s my hope that I could extend this to a more generic class that would be appropriate for a conference tutorial. I’ve talked with some folks about collaborating on something like this and kind of making it open source, with several of us who could teach it, like the Watir tutorial that Bret Pettichord, Charley Baker and others have taught. But, others better than I have tried to teach hands-on courses like this and run into issues. What tools would we use that can easily be installed on all participants’ laptops? What if the participants have different skill levels? So, I don’t know if this is really possible. I’m interested in ideas, or if anyone has done this successfully.

Actually I’m sure this has been done successfully by people like Elfriede Dustin who have been specializing in test automation for years – I need to get her book out and re-read it. I’m not trying to steal anyone’s idea or say that this is a new original idea I thought up. I just see a gap in many teams’ ability to create the ‘right’ automated regression tests.

So, here is the mind map, you’ll have to open it in a new window (I’m sure there’s a better way to do this but I have to catch a plane), feedback welcomed.

3 comments on “Test Automation Design and Strategy Training

  1. Lisa, I think I need to read this section in your book to find the answers, but I’m curious about your view on automation strategy, specifically the criteria for [when/when not] to automate and automating enough.

    There seems to be confusion at my company about what automate – some think it’s more about automating appearance, location, than verify a button exists.

  2. Of course it depends on the application, and what’s important to the business. It’s all about ROI and risk, right? But I think it can be hard to automate whether something LOOKS right, and I don’t mind doing a quick visual check. If it’s a test that has to be done every build, it has to be automated. If it’s a test that has to be done every iteration, and it’s really tedious and nobody really has time, it has to be automated. If it’s a one-off or can be easily done before every release and someone will remember to do it, don’t automate it.

    I like to automate a minimum set of test cases for a particular piece of functionality in the regression suite. I might automate lots of other test cases during development to make testing go faster, but I won’t keep every single one of them in the regression suite, unless risk tells me I should.

  3. >>I just see a gap in many teams’ ability to create
    >> the ‘right’ automated regression tests.

    Hi Lisa,

    ‘Right’ automation solution should start with ‘right’ requirements but do those teams really know what they want (and what they can get using particular tool or approach)?

    In the other hand, requirements could be clarified during implementation process. That’s why I use Test Automation Requirements Matrices while discussing new project with a Client.

    Here’s how I documented them:


    Once requirements breakdown has been performed, we can match them against available tools, resources, timeframe – and refine in the second iteration, if required.
    For example, the roots of your statement “I like to automate a minimum set of test cases for a particular piece of functionality in the regression suite” are in the creation and maintenance cost. But what if you can invest in development of more advanced framework that reduces creation and maintenance cost by 2-3 times?

    Thank you,
    Albert Gareev

Leave a Reply

Your email address will not be published. Required fields are marked *


Recent Posts: