Weekend before last I had the great pleasure of participating in the Austin Workshop for Test Automation. Test tool developers and users exchanged experiences and ideas. You know that automated testing has come a long way when you see the approaches organizations such as Opera and Expedia use to overcome massive testing challenges.
One discussion that helped a penny drop in my brain was the concept of test libraries vs. test frameworks. A library is a reusable component. Watir is a family of Ruby libraries that drive the browser. A framework is a set of libraries that gives you everything you need to test. Cucumber is a test framework that supports behavior-driven development using plain text. Taza is a test framework that uses the Watir libraries and generates RSpec test assertions. At least – I think so! Please correct me if I’m wrong.
A good analogy someone came up with at AWTA is that Kevin O’Connor’s (host of This Old House) tool shop is a framework – it has everything needed for any project, whereas Marek J‘s garage is just a bunch of tools. This is not to cast aspersion on Marek J’s own cool test automation framework, Watirloo.
What I learned is that my team has our own test automation framework using Watir, which consists of our Ruby scripts and test/unit. The BDD frameworks such as RSpec and Cucumber seem to be the cooler thing to use than test/unit, but I don’t know how we would take on the huge task of refactoring all our Ruby/Watir scripts to use a different test framework.
That doesn’t mean we shouldn’t look at some of these newer frameworks to see if they have something to offer that our current automation solutions don’t include. My fellow tester Lori Bergan has been looking into Cucumber. I’ve been intrigued by BDD since last year’s CITCON Denver. I like tools that seem natural.
What tools have you been trying lately? Are you happy with the ones you currently use?
2 comments on “Test Libraries and Frameworks”
After taking a look on WikiPedia on the definition of both words, I came up with an entry on my blog about it: http://blog.shino.de/index.php?/archives/26-Testing-Libraries-vs.-Frameworks.html
The essence seems to be, that frameworks can be built of many libraries. Libraries are just some collection of some useful functions/classes which belong together. Frameworks make you able to proceed in your work faster without the need to adapt to the underlying library interfaces.
[…] Lisa Crispin reported last week on the difference between Test Libraries and Frameworks. When reading her blog entry I felt the urge to comment to it on the approach we used at work for our two-step based testing functions. In the end I figured that I quiet had not understood the point on where the difference between test libraries and frameworks might be. Here is the comment I made to the blog-entry: Since I’m working in the specialized business of mobile phone rating and billing, we introduced our own framework for testing. During the last year we switched our legacy test cases, that were fragile and suffering from high maintenance costs, to a FIT based approach. We came up with two steps for this. […]