Agile Testing with Lisa Crispin

We like small stories. If we keep stories to a size we can finish in two or three days, there is less likelihood of a story “blowing up” because we vastly under-estimated it. Doing lots of small stories every iteration helps us get into a steady rhythm. We can manage our work in process more efficiently and keep cranking stories through.

Some pieces of functionality are too big to fit into a small story, so we have to break them down. This is a hard skill to learn, but trust me, everything can be broken down into smaller chunks, somehow. This can get to be somewhat artificial and go against the principle that a story ought to deliver something of value to the business that we can release. However, we think the tradeoff is worthwhile. When we work on a big new piece of functionality, we simply hide the new stuff by means of properties until it’s ready for prime time. I guess we could branch the code too, but we prefer to work on the main trunk.

If you have some bit of functionality that doesn’t go end to end, how do you test it above the unit level? How can you show it to customers? This requires a bit of extra effort, but again, we’ve found it worthwhile. (My team has been chopping stories ever-smaller for six years now).

Here’s an example. Our application manages 401(k) plans (employer-sponsored retirement plans, to which employees and employers may contribute). Right now we are rewriting the functionality that allows a plan participant to “roll in” money from a 401(k) plan at a previous employer. We also needed to add functionality to allow users to roll in money that is not tax-deferred; this is called “Roth” money. As part of this rewrite we are rewriting the back-end processing.

When this is all finished, users will go to the UI, enter the information about the money they’re rolling in, and when they submit, instructions will be created that will allow us to receive the money and buy appropriate shares in mutual funds on behalf of the participant. One story read something like this: “As a participant, I want the system to be able to create instructions for Roth roll-ins”. Originally the story was to process the instructions, but we realized that was too big a story and cut it down.

There’s no UI yet, and we aren’t even processing the instructions, so how do we test this? The developer created a FitNesse fixture to accept information that users would enter in the UI, send these inputs to the production code, and the code in turn would create the correct instructions. Then we could use a “data fixture” (or we could use DbFit) to verify the data in the database.

I started out by just writing a FitNesse test to execute the fixture with my inputs and verifying the database with my own SQL queries. Then I made an automated regression test by setting up data about the plan and user in the database, operating the fixture to create the instructions, verifying the instructions, and then cleaning up the data. Here is the “meat” of the test. The basic data is set up earlier in the test. The “do” fixture specifies the roll-in data just as the user would in the UI, then sends it to the code which creates instructions in the database. The test then checks the data in the database. We can confirm this code works, even without the UI or ability to process the resulting instructions. And we can try lots of different test cases.

FitNesse Test for Partial Functionality

I’ve submitted three proposals to Agile 2010 and would love your feedback. Please review them – you will need to create a free account on the submission site if you don’t already have one.

Workshop: Who Pays for All the Fun? Selling the Value of Testing in Agile Projects

Tutorial: The Whole Team Approach to Testing

Talk: How Low Can You Go: Doing the Defect Management Limbo

I have a thick skin, so don’t hold back. I know there will be lots of great sessions, there’s a high standard, and I want to make a useful contribution.

Chris McMahon has blogged about our Writing About Testing group. I’m so grateful to Chris for starting this group. We have a wonderful bunch of testers/writers. I really enjoy helping members who are new to writing get published for the first time. I also really appreciate the great help and advice I get about my own writing.

Now, if I can just find the time to write! Check out my News page to see why I’m so busy. (I’ve also been on vacation for nine days! Back to work soon, hopefully with lots of new energy and ideas after this nice rest!)

Last sprint we had a half-point story, which in our world should be about a day of coding and testing, to do some “simple” text content changes on a wizard in our web application. It’s been a year since we had updated this part of the application (it mainly gets used from late January to mid-March). While we were planning our sprint, we didn’t think too hard about how the wizard worked, and how many permutations it involves over the span of time that our end users are using it.

First Try

Our product owner asked me for screenshots of some of the pages, and had his mockups of the changes ready to go in a Word document before the start of the sprint. We wrote test cases on our wiki accordingly.

Second Try

One of the programmers started working on the story and realized the code had a lot of different cases that depended on dates and statuses of various items. He printed out some of the code and went over it with the product owner to get the appropriate text for each of the cases. The original mockups we had were no longer useful.

Third Try

When I started to test the updates, I kept getting confused. There were subtle differences between the different permutations, and several different target and deadline dates involved. I showed the new pages to the product owner, and he got confused himself, and kept making more changes to the text and even to the logic behind what text is displayed when. The programmer patiently updated the code over and over again, but I could tell he was losing the will to live.

I finally cut and pasted paper mockups myself for each possible condition with the text I thought was correct, went over it with the product owner who made his changes, and the programmer worked from the paper mockups. This did the trick, but by now, the half-point story had taken eight days instead of one.

What We Learned

In hindsight, we think that we should have started by researching all the different possible permutations before we started on the story, giving the PO a screen shot for each one, and having him give us paper mockups for each condition. This would certainly helped me test, as I tend to be more visual, rather than being able to mentally interpret what the code should produce.

When we showed this to more of the stakeholders, they asked if anyone besides the PO and us had reviewed the new text. We had been depending on the PO to do this, so we didn’t show it to anyone else until the coding was finished. Next time, we’ll ask the PO if other stakeholders have reviewed his changes.

Sometimes stuff just happens, but if we always remember to get a mockup for each change needed and each case, it might save us time in the future.

News

There’s a lot going on – I have new articles published and lots of tutorials, workshops and other events coming up. Please check out my News page for details.

It’s that time of year for resolutions. One important agile principle (to me, at least) is the idea of continuosly improving. We are always looking for experiments to try, ways to work better. (There’s discussion on Twitter right now about the label ‘agile’ – maybe it’s time to drop it and just let it be the way we develop software! But that’s another post!)

Recently I wrote a Sticky Minds blog post about being nice. On New Year’s day, I read one of those newspaper articles you see every year that gives advice on sticking to resolutions. It suggested that rather than resolve to “being nicer”, set specific goals, such as, “I will compliment one of my co-workers each day on one of their accomplishments”. That seems smart.

So, instead of resolving to act nicer, I will resolve to notice and appreciate each day at least one of my teammates’ contributions. I also want to pair for some period of time with a coworker at least a couple of times a week. It would help me improve my skills, and I think it helps the team improve.

I have lots of other ways I’d like myself and my team to improve, watch this space. Meanwhile, I’m more interested in what other people are doing to improve how they work or behave. Please comment and share!

Agile Coaching, in my opinion, isn’t only for people who coach agile teams. If you’re in the software business, you’ll learn something valuable from this book. Start with the last chapter – “Growing You”. The only way to succeed in our industry is to continually learn and improve, and this chapter gives great suggestions how to do that.

This book is a model of concise, clear writing. It’s packed with information in the form of narrative, bullet points, graphics, photos, stories of real teams and projects, exercises, tips and examples. The authors have been walking their talk for a long time, so you can feel confident about following their advice. Their flair for language makes the book fun to read. It could be a quick read, except you’ll find yourself stopping to reflect on your own experiences, and thinking about how you could apply the technique you just read about on your own team. Each chapter includes hurdles you may face, and a checklist summarizing action items for you, the reader.

I particularly enjoyed the little vignettes with dialog from a typical team illustrating topics such as “Not Quite Done Yet” and “Getting Ready to Demo”. The authors have insight into all aspects of coding, testing, and managing software teams. You’ll find advice you might not expect in a software-related book, such as “Be kind to yourself”. The focus on people and teams will make you a better person and team member.

I wrote a blog post on StickyMinds about my views on the importance of being nice. I particularly like some of the stories that commenters have shared about how they were able to get a good working relationship with difficult coworkers. Please share your stories!

It’s that time of year where we may have a chance for extra socializing and pleasant distractions from the short, dark (at least in this hemisphere) days. Here’s a bit of holiday cheer from me, my husband Bob Downing, our dogs Tango and Bruno, and our donkeys Ernest and Chester.

Happy Holidays!

Beautiful Testing, with a wide variety of fascinating chapters contributed by leading software professionals, is available. Not only will it inspire you and your team to improve your testing and your product, but all royalties go to a good cause. I’m honored to have written a chapter. Please see the page on my site for more details and a free copy of my chapter – to whet your appetite for the rest of the book!

I meet more and more teams, both within my company and externally, who are getting really good at agile testing. They’ve adopted a Whole Team approach, they write tests first, they collaborate and communicate well, they use feedback to improve. Agile teams are functioning much better overall than what I was seeing a couple of years ago.

One perennial issue – and this isn’t new with agile – is that testers and teams still struggle with how to design and write automated tests that have a good return on investment. That is, the tests are reliable, stable, maintainable, and provide good value for the amount of work that goes into them. We also all struggle with how much to automate, what to not automate, what to keep in our regression suite.

Janet and I devoted a section of our book to automation, and we presented principles, techniques and strategies that will help. We also include this in our Agile Testing course, and we both do conference tutorials on how to succeed with automation in agile projects. I think in addition to that, a lot of people need some practical hands-on experience. They need to be guided through an example of automating tests for a story, then they need to try it for themselves.

I’m developing an internal class for people at my company, and we also have ideas for workshops and other ways for teams who have solved some automation problems and developed good practices to share with other teams. I just mind mapped the ideas we have so far. At Adam Goucher’s suggestion, I’m posting my first draft here.

It’s my hope that I could extend this to a more generic class that would be appropriate for a conference tutorial. I’ve talked with some folks about collaborating on something like this and kind of making it open source, with several of us who could teach it, like the Watir tutorial that Bret Pettichord, Charley Baker and others have taught. But, others better than I have tried to teach hands-on courses like this and run into issues. What tools would we use that can easily be installed on all participants’ laptops? What if the participants have different skill levels? So, I don’t know if this is really possible. I’m interested in ideas, or if anyone has done this successfully.

Actually I’m sure this has been done successfully by people like Elfriede Dustin who have been specializing in test automation for years – I need to get her book out and re-read it. I’m not trying to steal anyone’s idea or say that this is a new original idea I thought up. I just see a gap in many teams’ ability to create the ‘right’ automated regression tests.

So, here is the mind map, you’ll have to open it in a new window (I’m sure there’s a better way to do this but I have to catch a plane), feedback welcomed.

I work for a mid-sized software company with around 27 development teams, mostly Scrum but also a few doing Lean and Kanban. We have a large and diverse product of HR-related software, I think there are around 6 million LOC, on three different platforms, including an ASP product and a .NET product. I’m fairly new, I started at the beginning of June, so I’m still learning about how everything gets done. We release four times a year and the software has won many prestigious awards. I’ve been impressed with this large-scale implementation of agile development. Just the speed of the daily Scrum of Scrums is amazing.

Many of the teams do a great job of automating their tests, and a few teams have their own continuous integration (CI) that runs multiple times a day. Other teams have automated tests, but must wait for the company-wide build to run, which is usually only once per day and sometimes not that often. They don’t get feedback fast enough, and that slows them down – a lot.

I’ve been asking why some teams struggle to automate tests, and why some teams don’t have their own CI. I’ve also asked my own team why most of our functional tests are automated through the GUI, rather than behind the GUI. We use FitNesse, but mainly to drive GUI tests with SWAT.

The answer I get a lot is “It’s the framework”. We have three architecture teams, and there is a framework (maybe there is more than one, I don’t fully understand it yet) that apparently makes it hard to test at the page or object layer. In addition, some teams’ code has tentacles into legacy code written in Delphi whose tests are written in WinRunner. Also, the current framework appears to not be fully supported with automated unit tests, it was developed a few years back. Programmers new to the framework seem to have a hard time understanding how to work with it. The framework clearly causes a lot of frustration. A new framework is under development, but it won’t be available for some time.

In order to understand the existing framework and find out whether the new one will facilitate testing behind the GUI, I was fortunate to get to meet with the leader of the architecture side of the development group. He gave me an overview of the new framework’s current design (which may change), aimed at making our product easily configurable by customers, using web services. It looks like this design allows testing under the GUI level.

Additionally, I learned that it is possible to test under the GUI in the current framework, and I met someone on another team who has automated many FitNesse tests at the object level. I plan to get her to show me how these tests work, and see if what she did would work for our code.

I haven’t worked at a company this large since the early 90s, and I’ve never worked in a company, agile or otherwise, that had “architecture teams” and “frameworks”. I get it that our product needs to look and work consistently across the board, so it does make some sense to centralize the design of the UI and perhaps other parts of the system. However, if the architecture is done by one team, what does that mean for programmers on other teams? Is the fun part of their job taken away? Do they get a chance to think for themselves and improve the product?

It also seems to me that there needs to be a lot of communication amongst the entire development community of our company, that all programmers should have some input to the architecture process. I’ve found that communication between teams in general is a bit lacking. Each team is so focused on delivering its part of the product, they don’t always feel they have time to go talk to other teams. So, for example, perhaps nobody on my team has talked to the person who did a lot of FitNesse functional tests.

Does your company have an architecture team, and/or a framework that everyone works with? Please comment and share your experiences. As a tester, I’d like to know what I can do to help my team deliver the highest possible quality when we have to work with a framework designed by someone else.