Archive for the ‘agile testing’ Category

The Team’s Pulse: CI/Build Process

Monday, August 23rd, 2010

My teammate Tony Sweets (sys admin, programmer and genius) just posted a terrific writeup of how our continuous integration and build process has evolved over the years. This includes a history of the hardware we’ve used and how the builds have been set up, both in CruiseControl and Hudson, and all the different things the build process does.

One of my favorite excerpts:

This method worked too well. The testers started splitting up their test suites and were creating new build jobs left and right mainly because it was so simple. They didn’t have to deal with hostname issues and whatnot. They simply had to create the new job from an existing Hudson job and change whatever they needed in order to run a different set of tests. When the job ran it did not care what else was running because it was running in its own separate environment.

I love the easy-to-use Hudson UI, and this freedom to configure our own test jobs the way we want. It’s also easy to stop jobs or kick them off when we want.

Whenever I speak to a conference session or user group meeting, I always tell people, “If you aren’t doing continuous integration now, go back to your office and drop everything and get your CI going. It isn’t hard to do, there are a bunch of good tools available to help, even Testify Wizard to help you set it up. A programmer can do it in a matter of days or less. There’s no excuse to not do CI.”

I’m convinced that in 5 years at the most, any team not doing CI will be looked upon the same way we look upon teams that don’t do source code control. It would just be crazy to not do it! Automated tests don’t have much value if they aren’t giving you quick feedback several times a day. Without CI, your technical debt is bound to bury you quickly.

If I had to pick one reason our team has been so successful the past 7 years, our CI process is it. It’s the pulse of our team, and if it stops (as it did a few weeks back – see Tony’s blog post!), we all just about have a heart attack! When it’s ticking along, we feel healthy and happy.

Intro to Test Automation Design, in The Testing Planet (& more)

Monday, July 26th, 2010

The Testing Planet from  the Software Testing Club has indeed landed! It is chock full of excellent articles about testing, both technical and cultural. Just a sample of the titles: “Yes, It’s Personal”, “Context-driven Jumping”, “Testing & Creativity”, “Weekend Testing Fun”. I am extremely proud that my article, an introduction to test automation design, is on the front page!

Though The Testing Planet is available digitally, I urge you to subscribe to the print version. 1) it is way more fun and convenient (at least for us folks who prefer an actual newspaper with our morning tea), 2) it supports the magazine, and we need to support it and keep it going! The Software Testing Club rocks, it’s a great way to meet and exchange ideas with testers all over the planet. Give a little back to your testing community by supporting The Testing Planet.

Methods and Tools

I wrote a tool review of FitNesse for the Summer issue of Methods and Tools magazine. Please let me know if you have any questions about it.

Podcasts

Wednesday, June 30th, 2010

Podcasts are a nice way to learn about new topics, pick up new ideas and hear the experiences of other testers and agile practitioners. I highly recommend the Testing Podcast site for informative interviews with a wide range of testing practitioners.

This interview with Michael Kircher of Software Engineering Radio was posted in June 2010 and can be found on testingpodcast.com as well, along with some other podcasts I’ve done in the past. Michael and I cover several topics ranging from the role of the tester in agile teams, over test automation strategy and regression testing, to continuous integration. I really enjoyed it because we got to talk for over an hour (though the podcast is not that long.)

Published on InfoQ: A Tester’s Learning Journey

Friday, June 25th, 2010

InfoQ just published an article I wrote to try to inspire more testers to grow their skills. It begins:

The software industry is changing fast. More and more teams put testing up front and center; they use tests to drive development. New and improved automated test frameworks and drivers burst onto the scene every month. Teams with more automated regression suites need testers with sharp exploratory testing skills. But most people do not learn the needed skills in university: where will these testers come from? Read more

Making Lemonade

Tuesday, June 8th, 2010

If you’re at Better Software/ Agile Development Practices West this week, make sure you read the issue of Better Software that came in your conference bag. I’m especially pleased to have an article in this awesome issue with extraordinary fellow testers such as Jennitta Andrea, Markus Gaertner, Pradeep Soundararajan and Parimala Shankaraiah.

We’re All In the Same Boat

Monday, May 31st, 2010

Please see my StickyMinds Blog for a new post inspired by Jeff Patton’s recent presentation to our Agile Denver user group. The Whole Team Approach isn’t just for developing the software! We need to really think Whole Company. Comments welcome!

SQuAD Workshop: Learning for Testers

Sunday, May 16th, 2010

The Software Quality Association of Denver held a workshop May 7 to share ideas about why testers need to learn, what they need to learn, and how they can learn all of that. I facilitated an enthusiastic crowd of about 80 people. The room wasn’t set up in a suitable way, but participants found creative ways to work. And creative they were. I’d like to summarize some of the outcomes.

Why We Learn

We started by dividing into small groups to brainstorm the reasons why testers need to learn. Beyond the obvious “stay employed” and “do your job better”, some fascinating reasons came out: “To stay interested and sharp”, “self-esteem”, “curiousity”, “freedom” – I’ve always thought that job security is in your mind. If you believe you have valuable skills, you’ll know you can get a job no matter what happens. Now, that’s freedom!

Groups Brainstorming

Group Brainstorming Activity

Other reasons that I might not have thought of included: Help others learn,  keep up with changing environments and be more flexible and adaptable, spawn new ideas. “Challenge yourself”, “Be a better contributor” – this is the attitude I want in my peers! There were several comments on the theme, “avoid reinventing the wheel” – find out what works and what doesn’t, build on previous learning and trials, make work easier and more efficient.

Another theme was “prevent stagnation”: keep your brain young, become more well-rounded, balance working hard with getting help when needed, be ready for the next opportunity, job satisfaction, and one of my favorites: “It’s fun to learn”. Well, yeah!

What We Need to Learn

Our second group activity was to think of all the things we need to learn. I was gratified to see a lot of what some people call “soft skills”, what Diana Larson and Jim Shore call “essential skills”: negotiation skills, listening skills, stress management, leadership skills, management skills, team mentality, how to motivate people, conflict resolution, creative problem solving, how to work with developers, communicate without jargon, simplify concepts, how to think like a customer, politics, team dynamics. An especially intriguing item was “Who tells the truth?” It’s critical to know that, right?

Creative Use of Body and Chair for Brainstorming Surface

Creative Use of Body and Chair for Brainstorming Surface

Flexibility and versatility were a theme here. Design patterns, programming/scripting languages, platforms, levels of abstraction, being able to speak the developers’ language were important outcomes. “Keep your brain in good learning shape” is something essential that hadn’t occurred to me before. One group noted the importance of learning estimating and planning skills. Long lists of test tools, processes, methodologies and other technical skills were also popular items on the “What to learn” lists.

How We Can Learn

This all sounds wonderful, but how do we learn all those diverse skills? Where do we find time to gain that experience? In our third group activity, the SQuAD groups came up with lots of innovative ways to learn.

One theme was trial and error. Learn from mistakes – what worked, what didn’t. Retrospectives are one way to do this. Experiment, do pilot projects, and importantly, build slack into the system so you have time to do this. A learning culture where mistakes are tolerated is a prerequisite for innovation and creativity.

Mentoring was another common theme, but a twist on this was not only to get a mentor, but to be a mentor. Mentor the tester next to you! Learn by teaching.

Collaborating within and across teams is another learning method that several groups came up with, but again with different ideas. Creating a shared space and being able to sit with people that have different skills is a great way to learn. Using wikis for collaboration is another. Pairing with other testers and with developers was suggested. Getting feedback from coworkers is another simple but effective idea. Our peers are a great resource.

If the sticky notes won't stick, just use the floor!

If the sticky notes won't stick, brainstorm on the floor!

Participants had a lot of interest in learning about their business, learning the domain. To do this, they suggested brown bag lunches with Subject Matter Experts, using the product like a customer would, talk to experts, observe. One of my favorite ideas was “Write a manual!”

Some outcomes built on our existing skills that already help us be good testers. We’re good at asking questions, so ask! Ask followup questions. Learn about the product with exploratory testing. Do compoarison testing. Evaluate tools to learn more about them.

There are obvious online sources of information, such as webinars, podcasts, blogs and white papers. I was surprised how many groups listed “Google” and online searching as a way to learn (though it shouldn’t be surprising – we all do that all the time!) Social networking also came up a lot as a good way to get information and benefit from others’ experiences. In addition to mailing lists, we have many online communities at our disposal, such as Software Testing Club and Weekend Testing.

One group suggested applying principles from non-related areas. Our reading and learning shouldn’t be limited to testing or even software development. Inspire yourself by attending (in person or virtually) conferences such as TED: Ideas Worth Spreading.

I was gratified to see several groups suggest that we learn by doing. “Practice!” The best way to learn is just go ahead and try something.

Outcomes on Door

Outcomes on Door

More Photos

You can see the groups at work, and their ideas, on my Picasa site. Please take a look and think about what you will do today to learn something new.

StarEast 2010 – Self-Organized CWAC

Monday, May 3rd, 2010

I have LOTS to write about StarEast, but I’d like to start with the conference-within-a-conference that a group of us planned on our own. Well, I didn’t do much planning, but Matt Heusser took the lead and lots of other folks including Justin Hunter planned a couple of fun and enlightening events.

We were self-titled the “Rebel Alliance”. I don’t think we were rebelling against anything, there are just a lot of Star Wars fans in the group. (Though there was a good lightning talk suggesting good things we do rebel against). I was going to be a Bantha until someone reminded me of a gruesome scene involving one.

But I digress. We had dinner one evening, at which we even got t-shirts with a star fighter on it. So much fun to get to know people such as Yvette Francino, Shmuel Gershon, Lanette Creamer, Selena Delisie to name just a few (and how great to have lots of women!) I’ve known Adam Goucher awhile and met him at Agile 2009 (if not before), but it was great to see him and his fellow editor of Beautiful Testing, Tim Riley (who did a great keynote at the conference).

Shmuel Gershon's lightning talk

The next evening our group had our own lightning talks, which were as intriguing and creative (maybe more) than anything at the conference. Adam talked about pirates, and Lanette talked about herding cats. This photo of Shmuel presenting a new session testing tool he wrote shows how much energy the speakers had.

Jon Bach had surprising and thought-provoking exercises, as always. These are just a few examples.

At the Rebel Alliance Lightning Talks

Selena and Dan took lots of videos, I hope those will show up someplace. David Gilbert provided his own tasty home brew – what a treat! He did a good talk too, on what we’re rebelling against, and for – learning vs. pass/fail, programmer’s best pal vs. quality police.

Conferences always get my brain buzzing, just being around such smart and creative testing professionals. But being a part of this terrific group was amazing. From old friends like Janet Gregory to more recent friends such as Sean Stolberg and Dawn Cannan, to people I only knew on Twitter up to now, such as Alex Kell, to people I didn’t know at all before, like Mark Vasco -  it was almost overwhelming! I wanted to talk to everyone at once!

After this experience, I suggest you consider this at your next conference: get a group of like-minded people together and plan an evening ahead of time. You’ll look forward to the conference that much more, and take exponentially more ideas home with you. And you’ll be part of a small community that hopefully will live on past the conference.

First RobotFramework Test!

Monday, March 29th, 2010

There are so many test frameworks, drivers and other tools out there I want to try. For example, Selenesse has been at the top of my list of things to try for months (and I’ll get to it sometime soon!) However, I couldn’t resist the offer from Pekka Klarck, one of the Robot Framework contributors, to give me a live one-on-one demo of Robot Framework a couple of weeks ago. This weekend, I finally had time to try to write my first test (really, to finish up a test that Pekka had started for me).

My first test is below. I always struggle mightily when I learn some new tool or language, and this was no exception. (Apparently my more-geeky friends also struggle with new things, but they don’t whine publicly about it like I do). The user doc for RF leaves a bit to be desired from a non-programmer perspective. However, they are actively improving it. With much help from Pekka (and suggestions from many in the Twitterverse who also use RF), I have gotten over this first beginner hump, I can see lots of possibilities.

You can write RF tests in various formats, including HTML tables. I like the plain text format, because to me, simple is good. It makes the tests more amenable to version control, because you can easily diff versions to see what has changed. If you try that with html table tests you’ll have lots of noise.

I love the flexibility of this tool. I am trying it out with the RF Selenium libraries, but you can use it with a variety of tools, including tools to test non-Web apps such as Java Swing. There’s a lot of potential there.

I like the fact that you can define whatever “keywords” you want, so you can create your own domain-specific language and the test can document itself and the code that it is testing. The inline-documentation aspect is also one of my favorite features of FitNesse.

My team is pretty happy with our current tools: FitNesse for testing behind the GUI, Canoo WebTest for GUI regression tests, and Ruby/Watir scripts to help with exploratory testing. We’ve long wanted to try Selenium, though, and RF would give us an easy way to do that. We’re always experimenting, and an experiment with RF will tell us whether there’s value in adopting it.

For myself, I think RF will be useful in my public presentations and classes. I’m always looking for ways to teach good test automation design, and I think I can do this fairly easily with RF. I’ve seen Dale Emery and Elisabeth Hendrickson do good demos with RF. I’ve also seen Antony Marcano, Andy Palmer and Gokjo Adzic do good demos with FitNesse, so I’m not saying one is better for this purpose than the other!

Here’s my test. I also copied it in below, though the spacing might be wacky there. I put comments in for myself that might help you, too. Download the RF-Selenium library demo and try it for yourself. If you have any issues I am glad to try to help. It would help me learn, too!

—-

lisas_first_test.txt

# new line and 2 spaces are cell delimiters.
# the asterisks have to start in the first col
# Look at the selenium keywords in http://code.google.com/p/robotframework-seleniumlibrary/wiki/LibraryDocumentation
# Look at the quick start guide for more http://code.google.com/p/robotframework/wiki/QuickStartGuide
Actually you don’t even need the # if you type up above the first table. Tables can be in any order. ‘*** Test Cases***’ is an example of a table.
Run it with ./rundemo.py lisas_first_test.txt
To start the server for the app under test manually, python httpyserver.py start

*** Test Cases ***

invalid account
navigate to the login page
type in username  invalid
type in password  xxx
click submit
verify the invalid account error message

*** Keywords ***

navigate to the login page
log  hello, world!
open browser  ${LOGIN PAGE}  ${BROWSER}
title should be  Login Page
${title} =  Get Title
should start with  ${title}  Login

type in username  [Arguments]  ${username}
input text  username_field  ${username}

type in password  [Arguments]  ${password}
input text  password_field  ${password}

click submit
Click Button  login_button

verify the invalid account error message
Title should be  Error Page
Page should contain  Invalid user name and/or password

*** Settings ***
Library  SeleniumLibrary
Test Teardown     close browser

*** Variable ***
${LOGIN PAGE}   http://localhost:7272
${BROWSER}      firefox

The One Right Thing

Wednesday, March 10th, 2010

During a flurry of tweets and blog posts about whether or not ATDD is a good thing, I posted a blog on Stickyminds with my opinion. I’ve gotten some great comments, please add yours!