User research or usability testing?
The type of research you'll want to conduct will vary depending on where the project you're working on is at.
- User research: we're figuring out what to build. Can be messy and time-consuming.
- Usability testing: we have a thing and we want to improve it.
User research
We don't know the problem or have a solution yet. We need to observe users' behaviour to understand their desires and context.
- More common in the early stages
- Broad research questions
- Usually less structured
- Are we solving the right problem?
- Uncovers users habits, preferences and mental models
Usability testing
We already have a solution and we want to test it. We want to see if it's solving the problem.
- Iterative. We do it continuously.
- More specific research questions
- Usually shorter and more structured
- Is our solution working? How can we improve it?
- Test our structure and labels
Define your research objective
Each round of user research should have clear objectives.
Choose the relevant testing method:
- Which problems are we trying to solve?
- Which assumptions on our product or users do we want to challenge?
- What do we need to know to make informed decision about what to do next?
- Are we testing something or exploring a new area?
- Remote or in person?
- Where does it stem from? Metrics (e.g. we saw a drop-off in signups) or business requirements?
Be clear on your research question
- A workshop can be a good way to capture research questions, opinions and assumptions. It's a quick way to prioritise for what we're trying to solve and, by involving stakeholders, making sure that everyone is clear on what we aim to achieve.
- Operationalization is a complex word for something fairly simple: it basically means to define the subject or object of study so that the method is replicable and transparent
- Bad example: What do our young users eat?
- Good example: What do our users aged between 16 - 30 eat at lunchtime?
- Avoid "think", "feel", "believe", "perceive" or other similar verbs. We want to uncover what people are doing, their behaviour.
- Prefer concrete over abstract nouns.
Be clear on what you're going to test
- Pick what you're going to test: a new idea, a competitor's solution to a problem we're exploring, an improvement to an existing flow.
- Set up a clear goal for participants to achieve.
When not do to research
- To evaluate designs. Do you like x? Liking is subjective and superficial.
- To collect requirements (which features would you like?).
- To ask about hypothetical scenarios.
Finding humans
How many you'll need will vary on the type of research.
- User research: quite broad; might require multiple rounds of 10 participants before we can make a bet
- Usability testing: as a rule of thumb 5 participants are enough to start hearing similar feedback. It's usually better to do multiple smaller round of research than larger and less frequent rounds.
The first step is having good screening questions. Without good screening questions we get irrelevant participants, and as a result misleading research.
- Make sure you select participants to whom the task or problem is relevant.
- Demographic usually matters less than behaviour. Their existing behaviour is going to determine whether the participant is relevant to our research or not.
- What level of knowledge do they need? Are they developers or specialised individuals?
- Because our customers tend to have a certain profile, it's important to screen for certain attributes. Make sure to look at their job so that you don't end up talking exclusively to engineers (unless this is relevant to your research!)
- If needed ask them extra questions when contacting them. We usually include questions about career (are you a developer, PM or PD?) and industry (do you work in financial services?).
Do your admin
The easy but boring part:
- Data protection
- Incentives
- Logistics
- Getting permissions
Write your interview guide
It's important to be guided by clear goals when doing a round of user research.
What are we interested in?
- User needs
- What problems do our users have?
- What barriers are they encountering?
- Values
- What's important to them?
- What are their habits?
- Context
- What's their environment?
- What tools are they already using?
- What subjects (people or companies) interact with them?
- Desires and motivations
- What's their goal? What is the context of the task?
On the day
During the interview
Always make sure you have at least a 15 minute buffer before any research session in case the participant arrives early.
Conducting the interview
- Create a welcoming environment
- Start each interview with a general description of the goal
- Open by reassuring the participants: explain the purpose of the research, how long it will take, the confidentiality and ask for consent for any recording or note taking activity.
Asking questions
- Listen more than you speak
- Aim for open-ended questions. Avoid yes/no questions.
- If they reply with yes/no ask to elaborate (follow-up questions).
- Summarise and paraphrase what the participant has said if you're unsure on the meaning or need them to elaborate.
- Avoid: judgmental language, leading questions and compound questions.
Taking notes
- Always try to assign a note taker to each session. This is a great opportunity to involve some developers in a couple of research sessions.
Closing the interview
- Do you have anything else you'd like to add about X?
- Thank the participants and remember how the incentive will be delivered to them
- Ask whether they'd be willing to participate to further rounds of research
The pitfalls of recording
I once worked in a place where they wanted every single session to be recorded and stored (audio/video) in a folder. Of course — no one ever went back watching those sessions, but the recordings reassured us. We had them, just in case.
But — "just in case" isn't a good privacy stance for your participants, neither is a good use of time getting all those permissions in place.
My advice is to think twice before recording anything. You should aim to take good notes the first time around, and only if that is impossible (perhaps you are both the interviewer and the note taker) record the session to integrate your notes as soon as possible.
The privacy of your users and participants is paramount. Once you have analysed and transcribed your notes, you should delete the recordings. The findings from your research will be invaluable and will guide you in your decisions, but the raw data will loose its value quickly.
Analyse and share findings
Review notes
- Write each observation on a sticky note and group similar notes together
- Pay attention to the vocabulary the user is using
Identify common patterns and derive insights:
- Add your observations to a research database
- Update our goal, problem or solution
- Update our personas
- Communicate outcomes to everyone, with associated actions