Skip to content

Greg Nilsen Posts

Gathering Meaningful Feedback

One of the great challenges as a Product Manager is gathering meaningful feedback.  While I recently wrote to encourage users to Please Speak Up… about their experiences with a product, I want to take a look at the flip side of that coin and talk about approaches Product Managers can take to help gather feedback that can be put to use.

Most of these are from my direct experiences in working with B2B software, most recently with eBillingHub, but many apply to products across the board.

User Surveys

User surveys are great tools for gathering high level information across a large group of users.  You’re not going to get a whole lot of details in this format, but it doesn’t mean you can’t get value.  While questions need to be easy-to-answer and you need to try to limit them to get response rates, a well-formed survey is a powerful tool.

You’re not going to learn why that a designed workflow doesn’t line up with how they expect to use your product, you can learn what sections of your system may need more focus.  Or you can use a tool like Net Promoter Score (NPS) to gather the general heartbeat of your product in the marketplace.  While that won’t turn directly into something you can fix or change, it does help to start shedding light on where your focus should be.

User Interviews

Scripted or unscripted, user interviews are a great opportunity to both get to know your customers better and identify their pain points.  When I sit down with customers, I always start by asking them to tell me anything about our product and the experiences with it: good, bad, or awful.  And then I emphasize that any feedback will help me to make sure we craft a better product for them as we move forward.

It’s the opportunity for them to provide their unfiltered feedback to someone who can help do something about it, which you, as Product Manager, may not have received through sales or support.  In a face-to-face, small group setting, it’s even more powerful.  One person doesn’t feel put on the spot, and thoughts bounce off one another to help even more information come to light.

But few things will lead to a good User Interview more than empathy. The User Interview isn’t about you, or your product…it’s about your customers.  There’s no need to get defensive, but be honest.  Admit when there are areas you know need improvement, and take the time to understand how a seemingly minor bug may have ruined their day on multiple occasions.  It may be the light you need to realize that your roadmap priorities could use some refinement.

User Experience (UX) Interviews

I classify UX Interviews a little differently than the User Interviews, mostly because different information comes to light in them.  While my User Interviews don’t involve actually using the product, the discussions revolve around what the product can and can’t do for them.  However, UX Interviews have the product front-and-center and are more focused on whether or not the product’s design makes it easier to do what they need to do, since you’re actually going through the product (or a prototype) with them.

For example, you may be following along with a customer going through their expected workflow when they stop and unexpectedly say “and then I go over to System X to do Task Y”.  It’s the perfect opportunity to either step in and learn more about what they need to do for Task Y, or to identify and address gaps in your product you may have never encountered otherwise.  You might even be able to tap a whole new market!

Product Planning Boards / Customer Focus Groups

Product Planning Boards are collections of customers that have been asked or have volunteered to be more active in their feedback around a given product.  This is the name we’ve given to these groups in the Legal division at Thomson Reuters, and they’re very similar to customer focus groups, except that they are ongoing groups rather than short-term.  The goal is to have an active, representative subset of our customers that can be referenced for business questions, new feature demos and other feedback.

I run my Product Planning Boards through a private messaging board/forum where the group can communicate with me and each other.  I’ve worked hard to cultivate an environment where they can be honest about the usage of the application without any sales pressure, counter-arguments or rushing to upper management (in their or my organization).  I also ask questions about more than just our product, but about how they do their jobs and what operational headaches they may be dealing with. And they can see each other’s responses, so we can get agreement, disagreement and discussion among the group, since everyone operates a little differently.  I’m there to listen, probe further and understand more about their needs and day-to-day work so that we can make a product that better fits their work and needs.

While I’ve gotten valuable feedback from all of these interactions with our customers, none have been quite as directly impactful for me as the work with the Product Planning Boards.  They’ve truly helped to reshape our product roadmap in a way that will help provide them swifter improvements that are needed as eBillingHub moves forward.  However, using these tools in concert is critical to ensuring that I have a “full world” view of my users and my product.

Please Speak Up…

Let’s face it, providing feedback can be a tricky wicket.  While we all have opinions on just about everything in the world around us, we tend to put a voice to very few of them.  And, ironically, we tend to express most loudly the opinions that impact us the least.

Think about it:

  • When was the last time you vocally questioned a sports coach’s decision?  Or criticized the inability of your favorite artist to live up to their quintessential albums?
  • When was the last time you gave your employer feedback about the quality of their healthcare offerings?  Or let them know if the tools provided were sufficient to most effectively perform your job?

As a Product Manager, I often consider myself one of the “Great Consolidators of Information”; it’s my job to filter through all the data, business cases, and feedback to help determine the best path forward for the product.  Often there are a lot of factors at play, but bringing additional benefit to our customers is the ultimate goal.

However, one of the limitations is that if I never get feedback about a feature or process, I’ll never know that it’s something that needs to be addressed.

So, I’ll ask of you: Please speak up…

Your feedback does matter, and it does play into the decisions being made.  Some feedback will have visible results, while others won’t.  However, that doesn’t mean that the latter was any less meaningful or valuable.

You might think that this is limited to customers, but interestingly, it isn’t.

In most organizations, there are also a number of internal “customers” of a software product; we Product Managers need to make sure support has the tools needed to assist with customer problems and that we are in tune with the market that our sales teams our encountering.  However, many people in these groups are often used to the status quo, or see that a vision or initiative is in place, and go along with it.  So instead of speaking up with insight they may have garnered in years on the job, or from previous roles, they fall silent.  Yet, again, we Product Managers are working to do the best with the information we have at hand…and additional insight helps provide a clearer picture of what that looks like.

I understand that providing feedback isn’t always fun, fulfilling, or fruitful.  However, know that in every organization, somebody is listening.  And your feedback might just give them the insight they need to deliver you a better solution.

So, I’ll ask you one more time: Please speak up…

How Is Secondary

evPmdi1471311822Having been a software developer for the first 7 years of my career, the natural approach to work was to receive a task (a new feature, bug or performance issue) and asking “How do I make this happen?”.  And, honestly, how was the fun part.  It was applying all the algorithms and best practices I had learned throughout getting my Bachelor’s and Master’s in Computer Science, mixing it with the capabilities of the servers I had access to, and making something new that helped solve a problem.

However, as I grew more senior in my profession, had more exposure to the business problems at hand, and began interacting with the people that had to use my solutions, that approach evolved.  Sure, I had made applications that were usable, but not always very intuitive.  As I grew into a Developer/Quasi-Product Manager role, I did start interacting with a handful of key contacts with our customers, and the seed of thoughts around “What is the user trying to accomplish?” had been planted.

I then attempted a transition into Project Management for the next step of my career.  I quickly transitioned from coding with occasional customer interaction to a role that was fully customer-facing.  It was a bit of a shock to the system for a natural introvert like myself, but even more of a shock was no longer having accountability for the answers.  Focused on software implementations, life as a project manager was about “How do we get new customers using our system?”.  And when customers asked for functionality that our application didn’t have, my job wasn’t about addressing that, but figuring out “How can we work around that problem for now?”.

Admittedly, it was difficult, mostly because my job was more about solving our company’s problems than it was about solving our customer’s problems.  That “What?” seed had been planted and continued to grow, and it was anguish to not have any control over addressing those problems.  I knew that I needed to get back to Product Management.

Now back in a Product Management role, I’m back to that “What?”-first mindset, and continually refining it to identify the many levels of “What?” within our customers organization:

  • What are users trying to accomplish?
  • What do users need to do their jobs more effectively?
  • What does management need from our software to see the big picture?
  • What are the key values that purchasers see in our software?
  • What can we do to improve the ROI for our customers?

After we can paint a clear picture of what our customers need, only then can we effectively begin asking “How do we help our customers?”, start putting those plans into action, and then completing UX testing to make sure we haven’t missed the mark.

And it’s not just a one-time exploration.  We must continually repeat this process as our markets change, customers evolve, roles change, and new needs and capabilities arise.  But even in that iteration, the order remains the same: We must first understand what our customers problems are before we can provide effective answers on how to address them.

Baseball Fan Score Survey

Trying out a bit of a social experiment here.  While we’ve seen maps of what baseball teams are the most popular in each area across the country, I was curious about the average baseball fans opinion was of every team.  I’m going to try and base it on the Net Promoter Score (NPS) metric to try and determine how fans across the globe view each team.

If you could complete the survey below (or at this link), I’d appreciate your input.  All responses are anonymous, and if you’d like to share the survey on social media, that would also help me to get a better picture of the current state of this ever-changing landscape.  Thanks!

Whose Data Is It Anyway?

In a world coming to terms with the ideas of “Big Data”, one of the really interesting discussions in the area is around data possession.

Most people think that the data collected about them intrinsically belongs to them.  This is especially the case when it comes to health and biometrics data, like those collected by Fitbit, Garmin, Withings, or any number of step/heart rate/weight/activity trackers.  After all, it’s data about you and your body, right?  So we tend to believe that they are like our medical records, and we should be able to take them with us from one provider to another.

However, we don’t necessarily feel this way about other data gathering and tracking practices.  I mean, if you wanted to review if you’ve been eating better, you probably wouldn’t ask your local grocer to pull you a report of the shopping habits associated with your loyalty card.  That’s information they’ve tracked about you to bolster their offerings and they wouldn’t want it shared with another grocery chain.  Yet, we’d probably expect Fitbit to give us our data so we could transfer it over to Garmin if we changed devices.

You might think I’m comparing apples and oranges, so lets go with a more direct example involving biometrics.  Let’s say you got screened for TSA Pre-Check recently and had your fingerprints scanned.  Then you volunteered to chaperone an upcoming field trip at your kid’s elementary school, but need to complete a background check.  If you called up the TSA, would you expect them send your fingerprints over for the background?  Likely not, and fingerprints are one of the most unique personal identifiers we know of today.

It’s a fine line that many of these businesses and agencies end up walking, but it’s important to realize that much of the data that is tracked about us that we consider “ours” probably isn’t.  Much of that data likely wouldn’t be captured without the devices we use, and we voluntarily provide it to them, so we shouldn’t presume ownership.

And as devices start collecting more and more data about us, the water is going to get much murkier, and it will be interesting to see how the law progresses alongside those changes.

Taking Stock

When I started this blog back in 2006 (I just now realized this site has been around for more than a decade…wow), it was as an outlet for me to comment on the things that interested me.  Over time, it’s evolved and devolved, changed platforms, changed focus, and lost focus, but mostly just existed.

Almost 2 years ago, I introduced the Midweek Music Break series that, in all honesty, I started as a motivation to blog more regularly.  It was supposed to be the intermission between more regular postings on my random topics.  However, that portion never materialized, and the Midweek Music Break took over as the central focus of my blogging, with me setting myself a weekly reminder to stay on top of it.

While I always enjoy sharing my broad musical interests, that’s not what I want to focus of my blog to be.  It should be a sidenote, but there wasn’t a main article.  In fact, I hadn’t much considered what I wanted the main focus of this blog to be, and thus, it never had a focus.  Sure, there were technical discussions, I was able to share some of my baking and cooking interests, and I was able to talk about my development as a runner, but most of the time I just never felt I had expertise worth sharing…or at least not interesting enough for others to read.

Over the last year, I’ve come back around to realize that I do have a story to tell, and I believe it’s a matter of putting pen to paper (or at least “fingers to keys”, but that sounds weird).  So, I’ll probably still have posts about running, baking and other current events.  However, I also want to make more of an effort to share insight from my career in Product Management.  I’m in a very exciting position at Thomson Reuters, and want to help develop the profession through discussion.

So, I’ll be working to find the central story to this blog, and I hope you’ll continue to join me on this journey.  Hopefully we’ll all learn something along the way as well!

Midweek Music Break: Sugar Ray – Every Morning

Sugar Ray is one of those funny bands from the 90s.  They started out as a punk metal band, and then manage to turn out a series of alternative pop hits into the early 2000s.  While “Fly” is their iconic hit, and “Falls Apart” is probably my favorite from the band, “Every Morning” falls squarely in the middle, combining catchy with more complex lyrics.

As they temperature rises, this is quintessential summer music for me.  Relaxed, fun and generally upbeat.  When it comes on, I just want to cruise along in the car with the windows down and take in a beautiful sunny day.

And sometimes, just those snapshots of nostalgia are worth the listen…

Midweek Music Break: Gorillaz – Rhinestone Eyes

I just realized that the Midweek Music Break, took 80 posts to feature one of my favorite bands: Gorillaz.  Beyond their great music, there’s this whole intricate alternate world with Gorillaz, and each of their songs and albums tells a part of a larger story.  “Rhinestone Eyes” is a song/story that didn’t get the full animation treatment, but is a great song nonetheless…

With their trademark catchy and unique beats, combined with intricate lyrics that are delivered very plainly, it’s basically the antithesis of a pop song.  Yet, it’s beautiful and mesmerizing.  This is what Gorillaz is all about.  Yes, it’s a great song that stands on it’s own, in their style but distinct from the collection, but is part of something bigger.

When I first started listening to Gorillaz, I thought the whole animated personas was a bit of a gimmick, but fun and unique.  However, as the albums progressed, the world grew, the stories evolved, and it was clear that the characters were a storytelling platform.  It lets the band grow, change and evolve over time.  And while it looked like the journey had come to an end several years ago, it looks like there may yet be another chapter


Midweek Music Break: Seeed – Augenbling

Apparently, every once in a while the YouTube suggested videos are actually useful, since that’s how I came across this 2012 song from German band Seeed.

Now, I’ve been known to enjoy a number of foreign-language songs (at least to this native English-speaker), even though I don’t always know what the song is about.  In fact, there’s often a freedom in enjoying the music and the vocals without the need to know what it’s about.

In this case, it’s a smooth beat combined with catchy vocals (and an entertaining video) that make for a great listen where I don’t even car what the song is about (but I will admit I looked them up, just to make sure it wasn’t objectionable).  And that’s a great thing to have happen when you end up stumbling across a song.