Skip to content

Category: Product Management

Gathering Meaningful Feedback

One of the great challenges as a Product Manager is gathering meaningful feedback.  While I recently wrote to encourage users to Please Speak Up… about their experiences with a product, I want to take a look at the flip side of that coin and talk about approaches Product Managers can take to help gather feedback that can be put to use.

Most of these are from my direct experiences in working with B2B software, most recently with eBillingHub, but many apply to products across the board.

User Surveys

User surveys are great tools for gathering high level information across a large group of users.  You’re not going to get a whole lot of details in this format, but it doesn’t mean you can’t get value.  While questions need to be easy-to-answer and you need to try to limit them to get response rates, a well-formed survey is a powerful tool.

You’re not going to learn why that a designed workflow doesn’t line up with how they expect to use your product, you can learn what sections of your system may need more focus.  Or you can use a tool like Net Promoter Score (NPS) to gather the general heartbeat of your product in the marketplace.  While that won’t turn directly into something you can fix or change, it does help to start shedding light on where your focus should be.

User Interviews

Scripted or unscripted, user interviews are a great opportunity to both get to know your customers better and identify their pain points.  When I sit down with customers, I always start by asking them to tell me anything about our product and the experiences with it: good, bad, or awful.  And then I emphasize that any feedback will help me to make sure we craft a better product for them as we move forward.

It’s the opportunity for them to provide their unfiltered feedback to someone who can help do something about it, which you, as Product Manager, may not have received through sales or support.  In a face-to-face, small group setting, it’s even more powerful.  One person doesn’t feel put on the spot, and thoughts bounce off one another to help even more information come to light.

But few things will lead to a good User Interview more than empathy. The User Interview isn’t about you, or your product…it’s about your customers.  There’s no need to get defensive, but be honest.  Admit when there are areas you know need improvement, and take the time to understand how a seemingly minor bug may have ruined their day on multiple occasions.  It may be the light you need to realize that your roadmap priorities could use some refinement.

User Experience (UX) Interviews

I classify UX Interviews a little differently than the User Interviews, mostly because different information comes to light in them.  While my User Interviews don’t involve actually using the product, the discussions revolve around what the product can and can’t do for them.  However, UX Interviews have the product front-and-center and are more focused on whether or not the product’s design makes it easier to do what they need to do, since you’re actually going through the product (or a prototype) with them.

For example, you may be following along with a customer going through their expected workflow when they stop and unexpectedly say “and then I go over to System X to do Task Y”.  It’s the perfect opportunity to either step in and learn more about what they need to do for Task Y, or to identify and address gaps in your product you may have never encountered otherwise.  You might even be able to tap a whole new market!

Product Planning Boards / Customer Focus Groups

Product Planning Boards are collections of customers that have been asked or have volunteered to be more active in their feedback around a given product.  This is the name we’ve given to these groups in the Legal division at Thomson Reuters, and they’re very similar to customer focus groups, except that they are ongoing groups rather than short-term.  The goal is to have an active, representative subset of our customers that can be referenced for business questions, new feature demos and other feedback.

I run my Product Planning Boards through a private messaging board/forum where the group can communicate with me and each other.  I’ve worked hard to cultivate an environment where they can be honest about the usage of the application without any sales pressure, counter-arguments or rushing to upper management (in their or my organization).  I also ask questions about more than just our product, but about how they do their jobs and what operational headaches they may be dealing with. And they can see each other’s responses, so we can get agreement, disagreement and discussion among the group, since everyone operates a little differently.  I’m there to listen, probe further and understand more about their needs and day-to-day work so that we can make a product that better fits their work and needs.

While I’ve gotten valuable feedback from all of these interactions with our customers, none have been quite as directly impactful for me as the work with the Product Planning Boards.  They’ve truly helped to reshape our product roadmap in a way that will help provide them swifter improvements that are needed as eBillingHub moves forward.  However, using these tools in concert is critical to ensuring that I have a “full world” view of my users and my product.

Please Speak Up…

Let’s face it, providing feedback can be a tricky wicket.  While we all have opinions on just about everything in the world around us, we tend to put a voice to very few of them.  And, ironically, we tend to express most loudly the opinions that impact us the least.

Think about it:

  • When was the last time you vocally questioned a sports coach’s decision?  Or criticized the inability of your favorite artist to live up to their quintessential albums?
  • When was the last time you gave your employer feedback about the quality of their healthcare offerings?  Or let them know if the tools provided were sufficient to most effectively perform your job?

As a Product Manager, I often consider myself one of the “Great Consolidators of Information”; it’s my job to filter through all the data, business cases, and feedback to help determine the best path forward for the product.  Often there are a lot of factors at play, but bringing additional benefit to our customers is the ultimate goal.

However, one of the limitations is that if I never get feedback about a feature or process, I’ll never know that it’s something that needs to be addressed.

So, I’ll ask of you: Please speak up…

Your feedback does matter, and it does play into the decisions being made.  Some feedback will have visible results, while others won’t.  However, that doesn’t mean that the latter was any less meaningful or valuable.

You might think that this is limited to customers, but interestingly, it isn’t.

In most organizations, there are also a number of internal “customers” of a software product; we Product Managers need to make sure support has the tools needed to assist with customer problems and that we are in tune with the market that our sales teams our encountering.  However, many people in these groups are often used to the status quo, or see that a vision or initiative is in place, and go along with it.  So instead of speaking up with insight they may have garnered in years on the job, or from previous roles, they fall silent.  Yet, again, we Product Managers are working to do the best with the information we have at hand…and additional insight helps provide a clearer picture of what that looks like.

I understand that providing feedback isn’t always fun, fulfilling, or fruitful.  However, know that in every organization, somebody is listening.  And your feedback might just give them the insight they need to deliver you a better solution.

So, I’ll ask you one more time: Please speak up…

How Is Secondary

evPmdi1471311822Having been a software developer for the first 7 years of my career, the natural approach to work was to receive a task (a new feature, bug or performance issue) and asking “How do I make this happen?”.  And, honestly, how was the fun part.  It was applying all the algorithms and best practices I had learned throughout getting my Bachelor’s and Master’s in Computer Science, mixing it with the capabilities of the servers I had access to, and making something new that helped solve a problem.

However, as I grew more senior in my profession, had more exposure to the business problems at hand, and began interacting with the people that had to use my solutions, that approach evolved.  Sure, I had made applications that were usable, but not always very intuitive.  As I grew into a Developer/Quasi-Product Manager role, I did start interacting with a handful of key contacts with our customers, and the seed of thoughts around “What is the user trying to accomplish?” had been planted.

I then attempted a transition into Project Management for the next step of my career.  I quickly transitioned from coding with occasional customer interaction to a role that was fully customer-facing.  It was a bit of a shock to the system for a natural introvert like myself, but even more of a shock was no longer having accountability for the answers.  Focused on software implementations, life as a project manager was about “How do we get new customers using our system?”.  And when customers asked for functionality that our application didn’t have, my job wasn’t about addressing that, but figuring out “How can we work around that problem for now?”.

Admittedly, it was difficult, mostly because my job was more about solving our company’s problems than it was about solving our customer’s problems.  That “What?” seed had been planted and continued to grow, and it was anguish to not have any control over addressing those problems.  I knew that I needed to get back to Product Management.

Now back in a Product Management role, I’m back to that “What?”-first mindset, and continually refining it to identify the many levels of “What?” within our customers organization:

  • What are users trying to accomplish?
  • What do users need to do their jobs more effectively?
  • What does management need from our software to see the big picture?
  • What are the key values that purchasers see in our software?
  • What can we do to improve the ROI for our customers?

After we can paint a clear picture of what our customers need, only then can we effectively begin asking “How do we help our customers?”, start putting those plans into action, and then completing UX testing to make sure we haven’t missed the mark.

And it’s not just a one-time exploration.  We must continually repeat this process as our markets change, customers evolve, roles change, and new needs and capabilities arise.  But even in that iteration, the order remains the same: We must first understand what our customers problems are before we can provide effective answers on how to address them.