Fun experiments

I joined the Pivotal Tracker team as a tester on August 13. I felt so lucky to get on board there, it’s a wonderful team, and Pivotal Labs has an amazing culture. But in spite of having amazing helpful teammates, I struggled to get to a point where I felt like I was contributing.

Every new job I’d started in the past, I’d brought something to the party. Either the team needed help with testing, or they needed help transitioning to agile, so even before I learned the software or the business domain, I could help. But Pivotal’s been doing agile since way before we called it that. And their long-time focus on quality means they are masters of testing and test automation at all levels.

After a lot of pairing and about seven weeks, I finally felt like a useful team member. Like all teams, we have problems to solve, and there are areas, such as automation at the API level and ATDD, where I’ve got valuable experience to share. Those are both areas where we need to improve, so I’ve been participating in some experiments to find a good approach.

Starting our experiments

In our first experiment, the team decided that a pair should try creating a reasonable framework for ATDD (Acceptance Test-Driven Development, aka Specification by Example). I was fortunate to get to pair with Glen Ivey.  We used an RSpec-based approach, but made our own DSL. The team already had RSpec controller tests at the functional levels, but our new scripts tested the API through the HTTP layer. When we demo’d them to the team, though, there was a consensus that there was too much duplication between our framework and the existing tests. I disagreed, but in the Whole-Team approach, the group decides.

Next steps – building on what we learned

After more team discussions, we decided that we would specify test cases for each story just in advance of coding, so the developers could use those to help make sure they meet all the requirements. We hoped to reduce the number of stories that get rejected one or more times. We thought about using FitNesse or a similar spreadsheet-style approach. Marlena Compton helped me think of alternatives, and encouraged me to go with what seemed best to me – the DSL that Glen and I had created, because it worked well for me personally.

I wrote test cases in a pseudo-code fashion using this DSL for an API endpoint that was already in progress. It certainly helped me test the endpoint, and I showed the tests to the rest of the team. I asked that a programmer pair with me on some more tests, so we could get better feedback on whether this approach was helping. My teammate Jeff volunteered, and we wrote test cases for a different endpoint whose stories were in our backlog. Jeff had great ideas for refactoring our pseudocode to make it DRYer, and we came up with some good test cases. On my own, I wrote tests for yet another endpoint. The developers used these tests as they worked on the stories, found them helpful, and made suggestions for improving them, as well as additional test cases.

But they should be automated, too

The next step was to actually try automating these tests. I paired with my teammate Nick, and then again with Jeff, and we automated the functional tests with RSpec using the library already used for other functional tests. There were some test cases we couldn’t automate because these tests can’t access the HTTP layer. The team is concerned that might be too expensive. Nevertheless, in the process of automating the tests, we found three bugs, and now have solid regression tests for that endpoint.

Enjoying work!

I’m having so much fun and learning so much pairing with my teammates who are expert programmers. Some of our experiments have been more successful than others, but they’re all a success if we judge by what they taught us. It’s hard to juggle these efforts with other top business priorities, but we work to achieve a balance of getting new features released and building a more solid base of driving development with higher-level tests.

Come Join Me in Belgium and England!

I’ll share these experiments and more in Belgium and England in the next couple of months. I’ll be at Belgium Testing Days the last weekend in February. I’ll be doing two workshops. One is on “avoiding the testing squeeze“.  I talk about my workshop on Advanced Topics in Agile Testing in this video – the sound quality isn’t the best but hopefully you can get a good idea what we’ll do.

I’ll do a similar workshop on advanced topics in agile testing in Cambridge, UK on March 19. Later that same day, I’ll be one of four speakers at the Software East meetup at RedGate software. Please join us for fun discussions and networking!

And maybe the most fun: TestBash 2.0 in Brighton, March 22! Join me and a bunch of really cool speakers and participants. Please also consider supporting TestBash as a micro-sponsor.