User feedback examples, and what each one is actually evidence of

User feedback is anything a person who uses your product tells you about using it, and the examples below are sorted by what each one can be used to decide. The useful question is never what a piece of feedback looks like. It's what decision the thing in front of you is allowed to carry.

Nine shapes turn up again and again. They're not equally good evidence, and two of them are routinely treated as proof when they're not. Every example below is written for this post, not taken from anybody's board, because publishing what identifiable customers wrote is their decision rather than ours.

Most of these arrive somewhere other than a board, which is the whole problem. A public feedback board is one place where the shape and the person who sent it stay attached to each other,.

What counts as user feedback?

Anything a user tells you about the product, including the things they tell you by doing rather than by writing. A message is feedback. So is a cancellation. So is a support ticket that arrives for the third time about the same screen.

What doesn't count, and is worth naming because it gets filed as feedback constantly, is somebody inside your own company saying what they think users want. That's a hypothesis.

It may be a good one. It's not evidence, and the moment it enters the same queue as the real thing it starts outvoting it, because internal people are always the most available writers.

There's a board carrying all nine of the shapes below if you want to look at one, and the demo is open without an account.

The nine kinds, and what each one can decide

The column that matters is the last one. Almost every bad prioritisation call is a piece of evidence used for the decision in the fourth column, not the third.

A vote on somebody else's request

Bulk CSV export · 84 votes

A vote is one bit of information, that this person recognised the problem when somebody else described it. That's worth having, and it's the cheapest thing a user can give you, which is exactly why it's over read.

A vote tells you nothing about how badly. It can't separate the person who would leave over it from the person who clicked because the sentence sounded agreeable. Two requests with 84 and 61 votes are ordered. Two with 84 and 81 aren't ordered by anything but noise.

A written request with the situation attached

We run onboarding for new agencies in batches of twenty. Right now each one is added by hand and there is no way to check afterwards which of them actually got set up. If I could paste a list and see what happened, that morning goes away.

This is the best shape there is, and it's rare. It carries the situation, the current cost and the shape of the answer, and it lets you build something for the problem rather than for the sentence.

Note what it doesn't carry. One person's morning isn't a business case. The next question is whether anybody else runs onboarding in batches, and the way you find out is to put the request somewhere the other agencies can see it.

A bug report

The changelog page is blank on my phone. Fine on my laptop.

A bug report is evidence that the product did something other than what it promised, which is a different category from a request. It goes in a different queue and it competes for time against other bugs, not against features.

It's also the most under weighted thing in most products, because somebody reporting a bug has already decided the product is worth the trouble of telling you. Most people just leave the page.

A cancellation reason

Too expensive for what we ended up using it for.

A cancellation reason is the only feedback you get from somebody with nothing left to lose, which makes it the most honest and the least actionable at once.

Read it as a statement about value, not about price. This sentence doesn't say the price is too high. It says the value that landed was below the price, and those have different fixes.

Building the feature they name is usually the wrong response, because if they had valued the product they would have asked for it while they were paying.

The same support ticket for the third time

Where do I change who gets the weekly email?

One of these is a person. Three of these is a product problem, and the fix is usually a label, not a feature.

The reason this shape gets missed is that support queues are organised by ticket rather than by subject, so the third one looks like a new conversation instead of a recurrence. Anything that lets you count tickets by subject turns your support queue into the cheapest research you own.

A rating with no sentence attached

7 out of 10

A number with nothing attached is evidence that the number moved. It can support watching a trend. It can't support any claim about cause, and it's treated as though it can more often than any other item on this list.

The usable version asks one narrow follow up question in the same breath, because the sentence somebody writes in the ten seconds after choosing a number is worth more than the number.

A request relayed by a colleague

Sales says the enterprise accounts all want single sign on.

This is evidence that somebody in your company heard something. It may be an accurate summary of four conversations. It may be one conversation, six weeks ago, repeated until it sounded like a pattern.

The relay always loses the part you need, which is the situation. Treat it as a lead rather than as a request, and go back to the customer for their own words. If that can't be done, the ledger on the decision should say the evidence was second hand.

A drop in how often something gets used

Behaviour is feedback, and it has the advantage of including the people who never write to you. It has the matching disadvantage of never telling you why.

A drop is a place to go and look. It's not a finding. The finding comes when you match it to something written, which is why usage numbers and a feedback queue are worth more together than either is alone.

A comment thread on an existing request

A thread is the shape that most often changes what gets built, because the disagreement in it's the scope. Three people asking for an export and arguing about whether it needs filters have told you that the export is the request and the filters are a second one.

What a thread can't carry on its own is priority. Ten comments from ten people who pay you nothing and one comment from an account that pays you a third of your revenue read identically in a list.

The two that look like evidence and are not

A vote count with no idea who voted. The number is real. What it stands for isn't, because a board that anybody can vote on measures whoever showed up. Until each vote has a person behind it with something known about them, the count is a popularity score rather than a measure of demand.

A relayed summary with the original words gone. By the time a request has been through a colleague, a meeting and a document, the words that reach the backlog are a description of a solution. The situation that made it necessary is missing, and it's the only part that would have told you whether the solution was the right one.

How do you turn these into something you can rank?

Rewrite each one as a sentence with a subject, a situation and a cost, then put it somewhere the other users can see it.

The rewrite is the work. "Bulk export" isn't rankable. "Agencies onboarding in batches can't check afterwards which accounts were set up" is, because two people can now agree or disagree with it.

Putting it where others can see it's what converts one message into a count. That's the only mechanism that turns the first shape on this list into the second, and it costs nothing but a public page.

What does a tool charge you for, and what is a tracked user?

Worth knowing before you pick anywhere to put this, because it changes what you're willing to collect.

A tracked user is a pricing unit used across this category. Every end user who has interacted with your board in the billing period counts toward a number, and the bill moves with it. The effect is that collecting more feedback costs more money, which is a strange incentive for a feedback product.

There's a longer version of that on what a tracked user costs.

How do you do this without a tool?

A spreadsheet with four columns does most of this, holding the words as they arrived, who sent them, the date, and the rewritten sentence. The rewrite column is the one that matters and it's the one people leave out.

What a spreadsheet can't do is let the other users vote, which means every count in it's your own guess at how many people care. It stops working at the point where you've more requests than you can hold in your head, which is sooner than most people expect.

How does this product do it?

Requests arrive on a public board. Nothing a visitor writes appears publicly until somebody approves it, so a board doesn't become a wall of noise the first week it's open.

Each request carries its own comment thread, and each voter carries whatever you know about them, including the revenue behind the account, so a count can be read by weight rather than by headcount.

The export is a CSV with twelve columns, including the weighted vote count beside the plain one, so the count you argue about in a meeting is the count you can check afterwards.

The types of user feedback sorts every one of these by what decision it can carry.

14 days, a card at signup, then $9 or $39 a month.