How (not) to get feedback from users

October 29, 2014

What do you think about xyz feature? What can we improve so that the product fits your needs? What would you change on this screen?

Asking users for an opinion about our design might be the quickest way to get feedback, but it is arguably the worst way to do it. Furthermore, taking opinions or requests as facts and translating them directly into a design can be terribly misleading.

In reality, people are often not very good at articulating their needs and wants, let alone their latent needs or factors that influence their behaviour. Their abilities to understand why they are doing something are poor and their abilities to predict their own behaviour in different circumstances are even worse. Most of their decisions are made unconsciously so this should not come as a surprise.

There is another problem with people. They sometimes make things up. They might criticise our design heavily because they feel they are helping us this way, even though it is obvious they complete tasks with ease. Or they might feel uncomfortable telling us that they don’t like our product, so they lie. Or they just don’t know, so they make up the answer to fit the current context.

Here is an example. During a research for one of our projects we interviewed a participant, seasoned sports coach, who was talking about his skills, students and busy daily schedule. But when we observed him later, we discovered that his perception didn’t entirely matched the reality. And this wasn’t an isolated case. Should we have only listened to what these people were saying, we would have come to wrong conclusions. In another example, our client was asking potential customers whether they would need xyz functionality and most of them were confident they needed it. The functionality was implemented. But when my team came onboard we realised that the functionality has not been used at all. Not only our client wasted resources to build the functionality, they also needed to invest more into fixing the problem this functionality has made.

Takeaway here is that opinions are completely unreliable. Instead of basing our decisions on what others say, we should focus our efforts on discovering authentic behaviours and stories through indirect research. Usability testing, for instance, is a relatively cheap method used to discover problems with existing designs. It helps us understand what users actually do when interacting with our design, rather than what they say they do. Ethnography studies helps us discover deepest insights about people: how they live and work, what motivates them, what are their needs, problems, intentions, beliefs and values. Through observation, contextual inquiry or diary studies (to name a few) we can inform a design process with valuable information that cannot be discovered with other methods.

This is nothing new, but sadly we need to keep saying it, again and again. Consequences of relying on opinions, attitudes or customer requests are hazardous: useless products, loss of money, dissatisfied users. Investing into a proper research, even lightweight one, can pave the way for creating products that perform better for us and our users.

Let's discuss this on twitter.