Posts Tagged ‘fitnesse’

Oh, Look, That Does Work!

Friday, January 7th, 2011

My team creates software to manage 401(k) retirement plans. Back in 2006, the U.S. Congress passed legislation allowing Roth contributions in 401(k) plans. Plan participants can elect to pay taxes on their retirement deferrals up front, rather than the usual tax-deferred 401(k) contributions. Then, when they retire and withdraw their savings, they don’t have to pay taxes on their withdrawals. This is called a “qualified” distribution.

There is a catch – if you make a Roth contribution, you have to leave it in your account for at least 5 years. If you withdraw it early, you have to pay taxes on whatever gains you made from interest and dividends on your investments. We tested the difference between qualified and unqualified distributions both with in-memory FitNesse tests that verified the algorithms in the code, and by manual testing by faking out the dates as needed. But there was no chance yet to have a qualified distribution in real life.

We have many tests for Roth withdrawals, from the unit level on up to the GUI level. One FitNesse test verifies the results when an employee withdraws money that includes funds from their Roth account. Since 2006 was the first year anyone could make Roth contributions, there was no possibility to make a “qualified” distribution until this year. The test set up a participant whose first Roth contribution was in 2006, and verified that the system withheld taxes for the gain portion when the participant withdrew the money.

At the start of 2011, this test started to fail in our continuous integration. Surprise – it’s 2011, the participant has now been contributing Roth funds for 5 years, thus now the distribution is qualified and should on longer have any tax withheld! The actual results of the test showed this difference in behavior.

Some people would say we shouldn’t have an end-to-end test like this, that it’s expensive to maintain. But I think it was cool to see this functionality actually work in production. It allowed me to give a heads-up to the CSRs that they would now see Roth distribution checks going out that didn’t have tax withheld. I changed the expected results, because before this, we weren’t able to have any end-to-end test for this case. It also shows how our tests are living documentation – as soon as we have a test failure, we have to figure out the reason, and either change the code (if it’s a bug) or change the test (if it’s correct behavior).

What’s your opinion? Do you like this sort of test, and this sort of surprise?

A Different Approach to FitNesse “Macros”

Tuesday, July 28th, 2009

My teammates Maykel Suarez, J.P. Erkelens and  Eddy Lara ought to be the ones blogging about this as they come up with these cool ideas, but I have the time and the inclination so I will. I hope they’ll read this and correct any errors I make.

I’ve used FitNesse since 2003 and obviously have not learned enough up to now!

We want our FitNesse tests to be DRY (Don’t repeat yourself), easy to maintain and easy to read and understand. Maykel and I were pairing on some FitNesse/SWAT tests this morning and found we needed the same test steps in more than one place. Maykel created this page under our “HelperMacro” area. Each FitNesse variable contains a SWAT test table.

!define NavigateToRatingScale (
!|SWAT |
|StimulateElement|Expression|

innerHtml:Performance Management;onclick:USGPM|onclick|A|
|StimulateElement|Expression|
innerHtml:Rating Scale |onclick|A|
)

!define NavigateToAddChangeRatingScale (${NavigateToRatingScale}
!|SWAT|
|StimulateElement|Expression|

innerHTML:Overall Review|onclick|A|
)

!define InputNumberOfLevels (
!|SWAT|
|SetElementAttribute|

Expression|Id:txtLevelNums|value |${NumberOfLevels}|INPUT|
|StimulateElement   |Expression|Id:txtLevelNums|
onchange|INPUT|
)

These are ‘macros’ defined as variables. Cool, huh? Notice that the second one, NavigateToAddChangeRatingScale, uses the first one, NavigateToRatingScale.

Here are examples of how we used these in test pages. First we include the page at the top of the test page;

!*> includes
!include -c <SuiteBookworms.HelperMacro.

RatingScales
*!

Then we can use whichever of the macros within RatingScales that we need at any time.

Navigate to add/change rating scale page
${

NavigateToAddChangeRatingScale}

When you view the test page in the browser, you see:
Browser rendering of navigate steps

It’s easy to understand what the test is doing, but we only have to maintain those steps on one page.

Here’s an example of setting the number of levels in a rating scale:

!define NumberOfLevels {5}
${InputNumberOfLevels}

In the browser view, this looks like:

Browser view of set rating scale steps

We have the kind of macros I’ve always been used to as well for example, a login macro:

{HomeUr}!*> login to $ with ${UserName} and ${Password}
!|SWAT|
|NavigateBrowser|${HomeUr\l}|
|SetElementAttribute|Id|ctl00_

Content_Login1_UserName|value|${UserName}|INPUT|
|SetElementAttribute|Id|ctl00_
Content_Login1_Password|value|${\Password}|INPUT|
|StimulateElement|Id|ctl00_
Content_Login1_LoginButton|onclick|INPUT|
*!

which are included in a test in the way to which I am accustomed:

!define UserName {hobbest}
!include <SuiteBookworms.HelperMacro.LoginMacro

I can’t articulate why I like the first, different approach, using variables to hold the macros, better than the way I’ve done it for years. It seems more streamlined to me, while retaining the ease of maintenance and understandability. Say sometimes you want to navigate to Page A, and other times you need to get to Page 2 which requires first going to Page A. If you’re including a bunch of one-line macros – one to navigate to Page A, one to navigate to Page 2 – your test gets real clunky looking.

Something Really Slick

Maykel, J.P. and Eddy used a similar technique so that a test could iterate through the same steps with different inputs each time. A variable is defined with a SWAT FitNesse table as its contents, including both a variable for the input value and a variable for the assert. Then it’s quick and easy to have multiple test cases using the same table. It took me a little while wo work out what they’re doing, but I love it.

!define AssertValidInputNumberOfLevels (
${InputNumberOfLevels}
!|SWAT|
|@AssertElement\${Exists}|

Expression|innerHtml:Invalid numeric value. Value should be a number from 1 to 99.;id:ctl00_errMsg|div |
)

This test expects an error message to come up, since 5.3 is an invalid value.

!define NumberOfLevels {5.3}
!define Exists {Exists}
${
AssertValidInputNumberOfLevels}

The next test expects that no error message will appear, as 42 is a valid value.

!define NumberOfLevels {42}
!define Exists {DoesNotExist}
${ AssertValidInputNumberOfLevel}

(there are several more of these with different values, some which expect an error message, some which expect no error message)

In the browser these look like this, so they’re perfectly easy for POs or anyone to read.

Slick way to use variable to iterate different inputs, expected results

Is this something everyone does and I just didn’t run across it before?