• Alessio Ferracuti

Most common mistakes in User research & product development

Updated: Mar 9, 2020

Product Tank in Edinburgh

Last week I attended Product Tank's latest meetup in Edinburgh, along with other 150 professionals from all over Scotland, and had the pleasure to gain insights on the most common mistakes in User Research and Product Development (shared with you in this article).

Speakers at the event included user researchers from The Scottish National Investment Bank, Education Scotland, XDesign, and product owners and lead designers from Heineken and Sustainably.

The event was kindly sponsored by Deliveroo, Lloyds Banking group, and xDesign. You can also check out xDesign's article where they talk about key learning points and insights from Product Tank's User Research's event

The most common mistakes in User Experience Design (UX)

The First 2 speakers were senior User Researchers who have been in the design industry for over a decade; Bobby King is a Senior User Researcher at Scottish National Investment Bank, where he is planning a User Research Programme for the new public-sector investment bank, while Mike Jefferson is a freelance User Researcher currently working with Education Scotland; Mike is also the organizer of the UX Edinburgh meetup in Scotland.

Bobby and Mike shared with us some valuable advice on how to avoid the most common pitfalls in user research.

If you are a User Researcher, talk to users!

The first mistake they talked about is not talking to users. This is a common issue for user researchers that are in a new environment and have new job positions. If you are a user experience designer/user researcher and you represent users of an organization, going out and talking to users is a must.

How can you design an application without talking to the people who will be using it? Get used to talking to users and get away from the mindset that you need a finished product before putting it in front of the users.

Prototyping before researching users

Bobby King talked about the second common mistake in product development, which is going straight to prototyping before talking to the users. Bobby’s advice is to begin a project with researching human beings first, finding out their preferences, habits and philosophies…. and we do that by sitting down with them and asking them questions.

It’s not hard to develop a Prototype that will only capture assumptions about what we think our customers want.

Going solo as a User Researcher

Mike found himself in the position to be the only user researcher in an organization, and was asked many times to go “solo” on research and interviews.

In your Job position as user researcher, you will probably be asked to go out there and research users by yourself, just like Mike. Especially if you are working for a new organization, it seems like a very good idea to go out there on your own, and do things your way.

However, there is still many things that you don’t know in the organization, and it would be very beneficial to the research process to take someone with you that is more familiar with the domain of the company. Moreover, involving others in the research, helps you get distance between you and the users… so you can watch and observe better.

If possible, get some of the stakeholders involved with the interviews, and let them watch the users next room. Interviews are a very powerful research method and stakeholders will stop being skeptical after being involved in it.

How to choose the right User Research method

Be more collaborative with others, choose research methods together and if you are outnumbered or not respected, then you are not working with the right team, and you should change organization.

How do you know which type of research is most suitable to your project?

All research methods have their place and purpose, and you need to find out what it is that you are trying to understand. Perhaps you need find out why people are not visiting a certain page, or what they cannot find on a page, and so on. Think also outside the box and use different methods in conjunction. For example, Surveys used in conjunction with other research practices could be very powerful, and you could be finding out very useful things. Another example is focus groups… focus groups are not liked by researchers as there is always a predominant person in groups that will mislead others, but if the session is being directed the right way, it gets excellent results.

Getting consents and protecting privacy in User Research

Bobby is very cautious when doing research, and always makes sure that the people he is interviewing are consenting what he is doing, and that means bringing the right copies, consent forms, and protecting their privacy.

Some of his other advice included; not talking about confidential research on the train, while going back home. You never know, someone might know the organization you are speaking about.

Confidentiality and privacy is just as important as organizing the research. Do not underestimate the time frame for your research, because planning interviews takes weeks, research takes hours and analysis of the research takes days! Also keep in mind for your interview scripts to not to include too many details when asking the questions…just observe and listen to the users, and you will eventually find out what you are looking for.

Alessio Ferracuti UX designer
That's me networking at Product Tank's event on User Research in Edinburgh

Heineken and their Product development strategy

Alanna Innes & Duncan Forbes came to talk about how Heineken employs User Research to ensure commercial & product success. They described their experience conducting user testing sessions to challenge assumptions and how an evidence based approach to product definition can complement Agile feature development. Alanna is a Product Owner with xDesign, one of Scotland’s leading digital product development agencies, ensuring product success for innovative companies, and Duncan is a Product Manager at Heineken, one of the world’s largest brewers & pub landlords.

They worked together on a project where they were trying to figure out how they could use technology to improve the costumers experience (club owners, stadiums, etc) and prospects. They held beers, ideas, and volunteers sessions and came up with something they thought that their clients would love and that their company needed at this time. They had ambition, and the confidence to proceed, so they wanted to make a prototype out of their assumptions and hire a UX agency.

Short after, Heineken contracted Xdesign to make a prototype, even though Heineken did not know the scope of the project and understand what problem they were trying to fix. Xdesign made a beautiful prototype, and invited users to test it. They set up a room with cameras, microphones, invited a few Heineken employees and stakeholders to participate, and of course the users to be recorded.

While watching live users using the prototype, Alanna and Duncan realized that users did not like the cell phone onboarding at all! Users thought it was a joke for them to fill out all this information online, and that they would prefer speaking to someone over the phone. Heineken stakeholders could not believe the user testing's outcomes, and became defensive about their product, without accepting the fact that their prototype was not suitable for the users.

Their costumers were not ready for cellphone boarding, and Duncan and Alanna were very worried about how the project was going. Eventually, Adam and Alanna adopted a different prospective, listened to their costumers, discovered their problems, and in the end, managed to complete the project in a 14 weeks design sprint. Their project turned out to be a success and will be made public in April 2020.

What did Duncan and Alanna learn from this experience?

Startups are not easy, and they can result in failure. Thanks to usability testings, they were able to identify their costumers’needs and proceed with the development of a successful product. 

Alanna said: "Usability is king, and once you have that truth, it is hard to dispute by stakeholders or managers"

Duncan says that collaboration can be hard sometimes, but as long as there is trust, and stakeholders believe in what you are doing, you will find consents for more research and new approaches.

Usability Testing is the cheapest and most effective research method

Jenny Bjorkman, UX Lead at Sustainably, talked about small start up budgets, and how Sustainably still keep the user at the heart of everything by carrying out user testing.

Before you start a project, defining the right target market is crucial. If you are designing a product that will be worth a lot of money, convince the stakeholders to invest in user interviews and testing!

Jenny also shared with us how she conducts her interviews on a low budget; You should definitely watch her video if you are looking for step by step instructions on how to make a usability testing. Gathering targeted users can be very cheap (around £50 each user for 1 hour of their time), while the cost for the tools and programs for testing is only around £300 (if you use mac), which includes microphone, face time, mirror screening, and so on.

I hope this article on User Research was useful for the ones that were not able to attend the Product Tank event in Edinburgh.