When Brian Marick came up with his ideas about using examples to drive development (I think that was back in 2003), my team latched on to the concept. We are fortunate in having an expert product owner who is talented at coming up with good examples. People often ask me how this works. Our current and previous sprints contain a good illustration of how we use examples to drive testing and coding.
We’re working on a theme to display internal rate of return on account statements. This involves a complex algorithm. Here’s just one part of the formula. Our development team hashed out this formula together with the product owner in the course of a couple of meetings to discuss it. Because our programmers and testers have a lot of domain knowledge, we were able to question the PO’s initial formula, and come up with one that was simpler to code and still provided a correct value.
The reason we were doing this theme is that our previous IRR formula got way off with edge cases, for example, if a participant started out a quarter with a zero balance, and got a big asset rollover in the middle of the quarter. The product owner researched nine different edge cases, pulled their data out of the production database, and created spreadsheets showing all the cash flows and the resulting IRR value. Here’s a small snippet of one of the spreadsheets, showing some of the cash flows for the first quarter of 2008, and the IRR value.
A developer and tester worked together to first write a simple FitNesse test (here’s a snippet of the FitNesse test) using data from one of the product owner’s examples. The IRRs came out different than the product owner’s. The PO did some research and found some problems with his own calculations. This process iterated a few times until both the example in the spreadsheet and the one in the FitNesse test produced the correct values.
From there, the tester has been creating more FitNesse tests using the other examples from the spreadsheet. All the known edge cases are in the spreadsheet, so this is a great start on covering all the possible test cases. We’ll do more tests, both through FitNesse, and through the UI, until we’re satisfied that the correct value displays on the account statements.
It’s taking us two sprints to complete the different stories in this theme. I think it would take much longer if we didn’t have the examples. The PO has saved us lots of time by researching the examples from production data up front, and putting those in a spreadsheet so we can see how he reached his IRR values. He worked closely with programmers and testers as the tests and code were developed in several tiny iterations until everyone was satisfied that the code was performing correctly, at least initially. Now we have his edge cases as examples to make sure we cover all possibilities, along with all the test cases we think of ourselves.
We have good confidence that we can deliver the right business value. As we have a firm deadline, this is a good thing. Some of the FitNesse tests will be put into our regression test suite that runs in a continuous build process. There are so many benefits to using examples to drive development.
Do you use examples to drive testing and coding? Please share your stories.
6 comments on “Using Examples to Drive Development”
Great to read a post with real world examples – even if trying to understand that formula gave me a headache !!
In our “not-Agile” environment I had a similar experience with examples. By eastern last year we decided to get rid of our fragile chained tests we kept in a shell-script based test framework. We decided to come up with new Tests based on FIT driven by the ideas from Brian as tests as examples.
We managed to make good progress with it. Previously the testers were associated with “long feedback”, “irrelevant corner-case issues” etc. During the last summer I realized that this picture changed. Our development lead came to my office asking for input on the current behaviour of our system under test. We have a huge service dealing with the distribution of money around some accounts, that seemed to be broken and he wanted to write an email to our customer on how the system should be have. He had prepared seven examples, he wanted us to exercise. With our previous scripts this would have taken about 2 weeks or so to prepare and execute. With our FIT tests I was able to give him the seven answers in 10 minutes, where the most time I had to take in order to write down the actual behaviour of the system. While doing so I also included the tests into our regression suite, since I found them worthwhile to have when we got an answer from the customer.
I described the same story on Matt Heusser’s blog for becoming a First class citizien as a tester: http://xndev.blogspot.com/2009/03/how-to-be-first-class-citizen-as-tester.html
Examples are a great way to communicate business requirements, and FitNesse is an excellent tool for exercising them. It is important to augment this with diligent refactoring and attention to code quality.
What I have observed on a unhealthy project is that each additional example required code changes (for test code and production code). Each additional story took longer to implement. The warning signs were there, but the business team stressed how important it was to make the date so developers decided to ‘refactor it later’. Now, the project is drowning in technical debt because the code that makes the examples pass was not simplified. Some developers are now saying “if we would have done more upfront design we wouldn’t have these problems”. This upsets me because they see it as a failure of agile, but the real reason for the failure is because they did not keep the code clean enough to make it easy to change.
Phil, that formula gave me a headache too! (And it’s not even the whole thing).
Markus, I love your example. Thanks for the link to your post. It’s great how you can provide such quick feedback even if your environment is “not-Agile”. That’s what agile testing is all about!
Tim, you make an excellent point. Examples on their own aren’t enough, that’s for sure. There is a lot of other discipline involved, and good principles that need to be followed.
[…] ThinkingTester – Shrini Kulkarni Tom DeMarcos confession Agile Testing – Lisa Crispin Using examples to drive development […]