Recently I wrote a post about how our team used a mind map to help us with theme planning. We also sometimes use mind maps to help us in planning our testing. Some people asked if I’d post an example. Here it is:
We’re a financial services company, and this theme was a rewrite of one of the oldest and most critical parts of our system: sweeping a tiny percentage of the assets in each participant’s account as pay for our services.
My fellow tester Lori Thayer (who came up with the idea to mind map test cases) and I wrote the initial mind map, then discussed it with the rest of the development team, the product owner, the controller and other stakeholders. Our central node is “Run”, because this is a batch process that needs a way to be kicked off, make sure prerequisites are in place, provide a way to monitor progress, and be restartable in case of problems. Not only is it important to test these parts of the system, but they’ll be invaluable to help us execute and monitor tests.
Without boring you with all the details of this them, you can see that the map includes some primary nodes such as “collection” for sweeping shares out of the account, “sale” for selling the shares once they’re in our company account (definitely in our best interest!), and “instructions”, each representing one participant’s account. We drew dotted lines between related nodes, such as “collection” and “sale”, “sale” and “fast401k account”. The map includes details such as the internal account number where the funds should end up. On the right side, we wrote questions that nobody was able to answer yet. These got answered as we learned more through talking to stakeholders and doing exploratory testing on a design spike.
We referred to this map many times over the next several iterations as we implemented this theme. We checked off the areas that were tested, added more questions and/or answers, even added more nodes as we thought of more areas to test. We didn’t get down into level of test case detail, but the map shows the relationships we needed to test, in addition to the individual components. (Unfortunately, I neglected to take an “after” photo).
We got into more details on the team wiki, using color-coding to show questions and answers. Our most senior programmer lives in India, and the wiki is a good way for him to participate in discussions like this.
Here’s a snippet of some test scenarios we wrote. When we’re satisfied with a test, we put our initials next to it to indicate it’s done.
It took us five iterations (ten weeks total) to finish this theme. At the end, we felt confident that we’d thought of and discussed all angles of the process, its potential impacts on other parts of the system, talked to all stakeholders about their individual needs and concerns, and done adequate testing not only at the functional level, but also performance and usability.
12 comments on “Using Mind Maps for Test Planning”
Linda,
Great post, as usual. I love documenting test ideas using mind maps too. One of the reasons I first started to use them was because you used them so well at the beginning of your chapters in Agile Testing.
I’m curious about how you selected tests once the ideas had been laid out. How did you handle the combinatorial explosion that emerged from these ideas? E.g., did you include a test for (a) value of shares = “not enough to pay the fee” together with retirement plan type = “Fast 401 (k)”? Did you include a test for (a) fee to come from “cash” and (b) type of fee = “X”? One very efficient and effective approach to select a manageable set of tests to execute from the millions of possible tests is to ensure that your tests include, at a minimum, every single possible pair of test conditions using a pairwise test case generator. Alternatively, more thorough coverage can be achieved through the use of combinatorial test case generators that will find a manageable set of tests that combine every possible combination of 3 test inputs and test each of them at least once.
Did you use a combinatorial test design approach? If not, with the millions of possible test cases you had to chose from, how could you be “confident that [you had] thought of and discussed all angles of the process, [and their] potential impacts on other parts of the system”?
Justin
Justin, that’s a great point. Because we work (both coding and testing) in such small increments and slices, we don’t usually have that many combinations to test. For this theme, we only implemented one fee type, custodial fees. We work incrementally.
The theme started with a spike to make sure the design solution would scale. The programmers were only concerned with performance at this time, not functionality. Once everyone was happy with the performance, we started doing user stories.
The programmers designed the code so that we could run the job against a single 401(k) plan (that had been done for the spike performance testing, too). We picked a plan and started designing code to work just for the circumstances of that plan. We began with a happy path of an active plan, in a product to which the fee applies. We could test all the cases that applied to that particular plan. It’s surprising how much you can cover that way. We could verify active participants with balances, inactive participants with balances, participants with no balance, participant with a balance in ecash, a participant whose fund was exchanged and on down our list of test cases. We could also automate tests as we went along.
Once we were happy that the code worked in all the circumstances we could produce with one plan, we increased it to 10 plans, but all within the same product, and spot-checked results. Then we moved on to plans in another product and other scenarios. It wasn’t until close to the end of the theme that I started running the job against all the plans in the test system. That was more of a load and performance test.
I have no doubt that if I would ever take the time to learn combinatorial testing it would help us, I guess I haven’t felt enough pain of feeling we had so many combinations to get through to motivate me to learn. When we work in such small increments, we never have millions of possible test cases. But of course we are concerned with the impacts on other parts of the system, so I know combinatorial testing would help with that.
QuickLinks for March 2011…
Here a quick list of articles I´ve read during the last month: Checking Invisible Elements UI Test Automation Tools are Snake Oil Jenga Driven Development Silverlight/WPF: Don’t name elements unless you have to: why x:Naming everything is bad WPF…
Aren’t tools like the mindmap or FIT tables the right way to write test cases ? I am mentioning this because you have also written some text. The problem in many offshore establishments is this text format. They don’t use any diagrams or tables. All their test cases are just text and text is mostly misleading because each one writes in a different way.
I think all our test cases – acceptance or otherwise – should be either code or a type of table or diagram. Am I right ?
There should be other tools or diagrams ?
Hi Mohan,
You make an excelelnt point. I do think describing a test with text has a lot of downsides. Concrete examples, preferably executable ones such as we can do with tool such as Fit/Fitnesse. Diagrams are powerful and help clarify specifications.
Our team has found through trial and error that what works best for us is to present some high level narrative – just quick bullet points – along with a happy path executable test to start with, then adding more test cases as coding proceeds. However, the narrative is helpful for retaining our ‘corporate memory’ of why we made certain decisions. Often we have to go back to a story we did a year or more ago. We have the tests to tell us what the code does, but if we have nothing else, we don’t know why we made certain design decisions. We also want to document anything we decided not to test, or decided to not automate a test for it. So the text is more for historical purposes. Does that make sense?
Lisa,
Thanks for sharing. I noticed that you used a digital photo of the board instead of creating a mind map using an electronic tool. Do you still use digital photos of the board for collaborating with remote teams, or does somebody capture it in a tool and share it that way?
Have you found that a mind map created in a tool is more difficult for people to remember and refer back to than one created on a whiteboard?
Hi Dan,
We have tried various online mind map tools, such as XMind and Freemind, but these weren’t easy to read when stored on the wiki, and the team didn’t all want to use the online tool. We turned to mind mapping on the whiteboard when using the online tool didn’t work, and the whiteboard method worked best for everyone, in spite of our remote guy not being able to draw on it (at least he can watch us draw it, and see the pictures later). Maybe at a future date, we’ll try the online option again.
— Lisa
Hi Lisa,
I am preparing to introduce Mind Maps in my own company (SMART) and we will most likely settle on Xmind. Exporting to their web format and embedding into our wiki seems like the best way to go, but I haven’t take the time to test that feature out just yet. Thanks for the great example on how you work. Have you ever considered a digital whiteboard?
It would allow your remote participants the ability to participate on the whiteboard and allow you to store all the notes. I am not trying to push my product, just trying to get a better understanding of what barriers you would face, especially considering your use case is one we want to try and support.
You can always take this offline: oliviermireault@smarttech.com
Hi Olivier,
I’ve always wanted to try a digital whiteboard. I’ve heard there is a Google whiteboard and people have said they were successful with that. It seems to me that ought to work well.
There are new mind map tools all the time – shows how useful mind maps are! I need to try out http://www.mindmup.com/, I tried an early alpha of it but they’re improving it all the time.
I’d love to hear about your experiences, will you be blogging about that by any chance?
thanks
Yes, once it’s all said and done and the sprint is completed, I should have a minute to go through my experience. My blog own blog is relatively sparse at the moment, I can never seem to get into the habit of writting. I will send you the post once I have it up.
[…] https://lisacrispin.mystagingwebsite.com/2011/02/28/using-mind-maps-for-test-planning/ […]
[…] onde são informadas as fases do projeto, atividades e colaboradores relacionados. O uso dos mapas mentais também tem se mostrado uma valiosa técnica para a organização e entendimento das atividades de […]