Skip to content

Month: September 2016

Gathering Meaningful Feedback

One of the great challenges as a Product Manager is gathering meaningful feedback.  While I recently wrote to encourage users to Please Speak Up… about their experiences with a product, I want to take a look at the flip side of that coin and talk about approaches Product Managers can take to help gather feedback that can be put to use.

Most of these are from my direct experiences in working with B2B software, most recently with eBillingHub, but many apply to products across the board.

User Surveys

User surveys are great tools for gathering high level information across a large group of users.  You’re not going to get a whole lot of details in this format, but it doesn’t mean you can’t get value.  While questions need to be easy-to-answer and you need to try to limit them to get response rates, a well-formed survey is a powerful tool.

You’re not going to learn why that a designed workflow doesn’t line up with how they expect to use your product, you can learn what sections of your system may need more focus.  Or you can use a tool like Net Promoter Score (NPS) to gather the general heartbeat of your product in the marketplace.  While that won’t turn directly into something you can fix or change, it does help to start shedding light on where your focus should be.

User Interviews

Scripted or unscripted, user interviews are a great opportunity to both get to know your customers better and identify their pain points.  When I sit down with customers, I always start by asking them to tell me anything about our product and the experiences with it: good, bad, or awful.  And then I emphasize that any feedback will help me to make sure we craft a better product for them as we move forward.

It’s the opportunity for them to provide their unfiltered feedback to someone who can help do something about it, which you, as Product Manager, may not have received through sales or support.  In a face-to-face, small group setting, it’s even more powerful.  One person doesn’t feel put on the spot, and thoughts bounce off one another to help even more information come to light.

But few things will lead to a good User Interview more than empathy. The User Interview isn’t about you, or your product…it’s about your customers.  There’s no need to get defensive, but be honest.  Admit when there are areas you know need improvement, and take the time to understand how a seemingly minor bug may have ruined their day on multiple occasions.  It may be the light you need to realize that your roadmap priorities could use some refinement.

User Experience (UX) Interviews

I classify UX Interviews a little differently than the User Interviews, mostly because different information comes to light in them.  While my User Interviews don’t involve actually using the product, the discussions revolve around what the product can and can’t do for them.  However, UX Interviews have the product front-and-center and are more focused on whether or not the product’s design makes it easier to do what they need to do, since you’re actually going through the product (or a prototype) with them.

For example, you may be following along with a customer going through their expected workflow when they stop and unexpectedly say “and then I go over to System X to do Task Y”.  It’s the perfect opportunity to either step in and learn more about what they need to do for Task Y, or to identify and address gaps in your product you may have never encountered otherwise.  You might even be able to tap a whole new market!

Product Planning Boards / Customer Focus Groups

Product Planning Boards are collections of customers that have been asked or have volunteered to be more active in their feedback around a given product.  This is the name we’ve given to these groups in the Legal division at Thomson Reuters, and they’re very similar to customer focus groups, except that they are ongoing groups rather than short-term.  The goal is to have an active, representative subset of our customers that can be referenced for business questions, new feature demos and other feedback.

I run my Product Planning Boards through a private messaging board/forum where the group can communicate with me and each other.  I’ve worked hard to cultivate an environment where they can be honest about the usage of the application without any sales pressure, counter-arguments or rushing to upper management (in their or my organization).  I also ask questions about more than just our product, but about how they do their jobs and what operational headaches they may be dealing with. And they can see each other’s responses, so we can get agreement, disagreement and discussion among the group, since everyone operates a little differently.  I’m there to listen, probe further and understand more about their needs and day-to-day work so that we can make a product that better fits their work and needs.

While I’ve gotten valuable feedback from all of these interactions with our customers, none have been quite as directly impactful for me as the work with the Product Planning Boards.  They’ve truly helped to reshape our product roadmap in a way that will help provide them swifter improvements that are needed as eBillingHub moves forward.  However, using these tools in concert is critical to ensuring that I have a “full world” view of my users and my product.