I had the opportunity to talk with Gil Zilberfeld, product owner with Typemock, a software company whose products are designed to help developers with their unit testing. I don’t often get to talk to product owners, and I was interested in his views on testing and quality. (Disclaimer: I have no experience with using Typemock’s products, and this is not a commentary on those products!)
An Agile Approach
Gil works with a team of three developers, who also provide technical support to customers. They practice agile development, working in small increments and short iterations, releasing new versions of their products a few times per year. They use agile practices such as pair programming and continuous integration.
Gil’s team doesn’t have any testers, but they clearly do plenty of testing, both unit and integration testing . Gil explained to me that his team practices “dog fooding”: they use their own product to help with their own unit testing. They have thousands of unit tests running in their continuous integration, providing quick feedback after every check-in. I thought it was interesting that, although their team is co-located, they use an online board product, Trello. Gil said the sticky notes kept falling down.
Developing What Customers Want
I asked Gil how his team comes up with requirements for things such as user interfaces and APIs. He said that rather than compose formal user stories, they “throw ideas in the air”, sketch out what they want on whiteboards, break features down and code them. They don’t make changes to requirements during the iteration, but they can continue to tweak the features in subsequent iterations. They try to innovate and come up with new features that can help both advanced and new users. Once they release a new feature, they use feedback from customers to enhance it. They listen to requests and fix problems quickly.
Using their own product allows the developers to evaluate the user experience and performance aspects, rather than doing formal usability or performance testing. Since the developers rotate the technical support duties, they each get time working directly with end users who are having issues with the software. This sounds to me like a great way to get a feel for quality from the customers’ perspective. I wish my own team had this kind of direct contact with users.
Value to Customers
Gil noted that when Typemock’s products were new, the early adopters were more flexible – not everything had to be perfect. Now, they’re producing enterprise software, the customers have changed. Developers with different experience levels have different needs. Inexperienced programmers may value an easy learning curve over sophisticated features. Even though they use their own product, Gil’s team doesn’t always know what the customers will like. They use beta testing as needed, choosing the types of customers that can provide the most useful feedback.
Gil told me a great story about his previous experience working for a company that produced large medical devices. His team didn’t have direct contact with customers. His first visit to a real customer was eye-opening. The customer had many large devices, and very little room. Gil could see that requirements he had thought weren’t very important were actually critical because of the space limitations. For example, computers needed to be inside the machine, with touch screens, to save space. This is a good lesson in why we need to understand value from the customer’s perspective.
Since Gil’s team doesn’t have testers and does their own testing, I asked what their main pain points were with respect to testing and quality. He said they want to make some improvements to their continuous integration process, and reduce some technical debt they have incurred there. Next, they would like to address velocity issues. They work at a good velocity, but management always wants more output. Gil said this is a question of matching expectations. They have to balance developing new features with meeting support standards and fixing bugs.
Since I work on a small team that works on a now-mature product, I was interested to hear about experiences of a similar team. Though they’re in a different domain, it sounds to me like they have a similar commitment to delivering the best possible quality software product that they can. My experience is that there are more and more teams like Gil’s that think about many different aspects of quality. They go beyond functional testing to think about different quality aspects the customers value, and try innovative approaches to delivering those.
4 comments on “A Product Owner’s Perspective”
I do not believe that developers can plan for beginner user experience of the product they build. When developers build a product they tend to do happy path testing forgetting about unexpected behavior and more creative ways of using the software. That’s why you need testers on project. Numbers of Test Cases and line coverage don’t say anything about quality o the tests.
Krystian, thanks for your comment. The developers on Gil’s team are also customers of the product, but you make a good point that they are now expert users and may not be able to understand the beginner user experience. I hope Gil will comment on this.
First, let’s not generalize. Different developers have different experience levels, and so may give the needed feedback for what we consider either entry or expert level.
Most developers do take the happy path. The thing is, when using test first, like our team does, you find that you cover a lot of unhappy ones upfront. Can’t say we don’t have bugs (I wish), but as long as everyone is in a quality-first state of mind, I feel we’re doing fine.
Finally, since we have the support rotation, the developers are actually exposed and communicate with customers. I found that if we the customers are hurting, they will tell us straight in the face. And there’s something “magical” about interaction with customers – it’s an eye opener. So I have cases where the developers try to convince me (sometimes succesfully 🙂 to fix something or build a feature for the customers that actually requested it.
Overall, I feel that working incremently, listening to customers, and responding quickly, the story moves away from product quality to customer experience. I think the latter is more important and the best one for us.
One could also argue the same for testers, the more they test the product, the more familiar they become with the product.
Therefore, they could miss the more obvious things as time goes by. That’s why when someone new joins a testing team, they go ‘Is this an error/defect’ and you go, “Yes, it is..wonder how I missed that’ – Not saying it happens all the time, but I think the same dangers can exist.