THE UX INSIGHTS COMPANY

User
Experience
Research

101
Introduction 3

Who should read this guide? 5

Planning UX research 6
Start now, whatever the status of your product or service 7
Get to know your users 8
Four stages in the early UX research lifecycle 9
Making a UX research plan 10
Using time wisely 11
How many users do you need to recruit? 12
What Sample Size do I need? 13
Criteria for selecting users 13
Using personas 14
Choosing a UX research method 14
Quantitative 14
Qualitative 15
Unmoderated vs. moderated 15
Unmoderated testing 15
Moderated testing 16
Common UX research methods 17

Doing UX research 21
How to ask questions 22
Asking users about their expectations 23
Making questions relevant and actionable 24
Three stages of questioning 25
Removing bias 26
How to observe 27
Understanding users and their mental models 27
Contextual enquiry 28
Checklist for effective interviewing 29
Pitfalls to avoid when interviewing 31
Examples of UX research questions 32

The results are in! 33

Collating and analyzing data for action 34
Pros and cons of spreadsheets 35
UX data: ‘one truth’ 35
UX data analytics: the basics 36

Acting on UX research results 37

How to prioritize issues 38
How to sell your recommendations 39
Know what must be done 39
Talk to the right audience 40
Lay the groundwork 41
Tell a story that grabs them 42
Bring the data! 42
Story structure 43

Conclusion 45
Introduction
Let’s start with a simple question:

Why is UX research important?

It stops you from making a product that your customers

don’t want.

Or to be more accurate…

It stops you from making a product that your customers might want at first,
but after using it a couple of times and getting so frustrated with it, they try to
set it on fire.

Then they tell everyone they know that your product is responsible for
burning off their eyebrows.

I know. The stakes are THAT high.

3
For any product or service to succeed,
you need to know what your users
want as early as possible in your
design process. This is where UX
research comes in.

Appearance, functionality, and price are

traditionally what businesses focused on
when competing with other products in
the market. But what differentiates a
product now is how good it feels for
someone to use your product, and
how this experience meets and hopefully
exceeds their expectations.

This is what we talk about when we talk

about the ‘user experience’.

And that’s why it’s important to find out

– from actual, real-life, honest users who
aren’t just mannequins you’ve dressed
up to fool your CEO – what they want
and expect from your product.

That way you can produce a truly user-

focused experience that they love
and won’t try to kill with fire (or complain
about on Twitter).

4
Who should
read this guide?
User experience is important enough to justify everybody in your
organization taking 30 minutes out of the day to find out how user-focused
design can benefit the business. Seriously.

If there’s someone in your team that hasn’t come across a feature of a

product (whether it’s an app, a website, a kitchen utensil, the door they
entered the office through) that hasn’t frustrated them to the point of
swearing never to use that product again, you can assume a body snatcher
has replaced them.

UX isn’t just the concern of the design team. Your sales team makes
connections every day with real-life humans, so they’ll be particularly
receptive to your customers’ frustrations. Your marketing team should
know that your customer’s experience is ultimately what will make or break
your product upon launch. Your dev team will momentarily glance your way
without removing their headphones and carry on keeping your product from
falling off a cliff – but they know how important usability is.

Naturally, people looking to specialize in UX will be among the first to

benefit from this guide, however it’s also written for the busy CEO or business
manager, in order to deliver the key points of UX research with the minimum
of waffle, so they can go back to reading their yacht brochures in peace.

5
 THE UX INSIGHTS COMPANY

Planning UX
research

Part One
 planning UX research

Simply guessing what people want or merely copying your competitors isn’t
designing for your customers.

It’s the same with just adding features because they look cool.

If your friend gave themselves an Iron Maiden tattoo, would you do the same?
No, you don’t even like Iron Maiden – you like Slayer. But can you guarantee
you’d still like Slayer in 10 years time?

Even if your product finds an audience - and there are plenty of badly
designed products out there used by some very masochistic people - you’ll
be cursed to round after round of expensive product fixes as you play
catch-up with your users’ expectations and their many complaints.

However, there is some good news. Yes you will still like Slayer in 10 years
time. Phew. But more importantly UX research can be straightforward,
effective, time-saving and cost efficient.

Especially if you follow a few simple rules…

Start now, whatever the status of

your product or service

If you’re reading this and your product is already out there – then keep
reading. Honestly, it’s not too late!

Yes, it’s true that the earlier you find mistakes, the easier and cheaper they
are to correct (sadly, the same can’t be said for that Iron Maiden tattoo), but it
is never too late to benefit from UX research.

Whether you apply the insights from your research to improve a product
you’re selling today or a product you want to build tomorrow, you can always
increase value by making it more user-focused.
part one

7
 planning UX research

The great thing about our digital-first, service-led economy is that fixes and
improvements can be delivered immediately to your customers. It’s
not like when your 1980s’ He-Man action figure didn’t fit through the Castle
Grayskull entrance because he was too wide. That was just tough luck and
nothing would ever change it.

Get to know your users

It’s not as terrifying as it sounds.

The whole point of UX research is to understand who your

customers are, what is important to them, and how they
actually use your products.

Sure, your intention may have been to develop a ‘Massively Multiplayer Online
Role Playing Game’, but maybe your true destiny is to become a photo-
sharing network (see the pivot of Flickr).

The most useful UX research is facilitated with specific tools and methods to
help you acquire the maximum amount of understanding. Your research also
needs to be well-planned and organized, so you can have confidence that
insights are derived fairly and accurately.

You’ll also be pleased to learn that UX research is founded on

straightforward logic and common sense, rather than some kind of
complex data science.

Gaining real observations from the real people using your product is
one of the best methods of testing, rather than just indirectly analyzing data
about them.

Data may tell you what people do with your product and where they stumble,
but it takes the people themselves to tell you why.
part one

8
 planning UX research

Four stages in the early

UX research lifecycle

While the principles behind UX research remain the same, every UX

research project is different.

You’ll ask different questions and use different approaches depending on the
users, your product and the stage of your product lifecycle.

At the beginning, you’ll probably find yourself in one of these phases:

#1 Discovery
You don’t yet know what people need, so this is where you’ll find out.
Understanding a user’s expectations could mean the difference between
continuing to develop a project or changing course entirely.

#2 Exploration
User expectations and your product can work together, but now you need to
understand the scope of your design and how it can really help people.

#3 Testing
Here you’ll check your product designs, either during development or after your
launch, to ensure that it works for users while noting aspects to fix or improve.

#4 Listening
Monitor comments to see if there are bugs or issues that have cropped up,
but also listen out for patterns or trends which may inform new designs and
help evolve future projects.
part one

9
 planning UX research

Making a UX research plan

The very first thing you need is a plan. Without a plan, your team will be
unfocused, you’ll forget what you’re doing and the whole thing will descend
into chaos – which is exactly how camels were first designed.

And the most important thing your research plan needs is

a goal.

The goal will likely be a problem that your users have experienced that your
product will heroically solve.

Working backwards from this point, you can work out:

• How you’re going to solve that problem (documenting everything along

the way)

• By what specific date (just writing ‘ASAP’ won’t fly)

• The resources you’ll need to achieve it (people, time, budget, pizza)

This plan will be a constant reference for you and anybody else contributing
to your UX research.

Remember that plans can, and often do, change for a variety of reasons.
If your plan needs to change remember to give your colleagues and
stakeholders enough warning to adapt.
part one

10
 planning UX research

Using time wisely

Time isn’t on your side, The Rolling Stones were lying. Time is constantly
working against you, plotting your downfall with every passing second. Which
is more of a Leonard Cohen theme.

However, you can mitigate some of this panic and frustration by factoring-in
the following time-sucks in your initial plan:

#1 Stakeholder time
The results of your research should be available to stakeholders as early as
possible. Never underestimate the power of quick-wins to buy yourself some
bonus breathing room. Find out what your stakeholders need first, then work
backwards to figure out how everything else fits into your timeline.

#2 Recruiting time
You’ll need to find suitable users AND you have to make sure there are
enough of them. A UX research platform can make recruitment easier by
offering access to larger communities of testers.

#3 User time
The availability of testers can be a challenge. Some research approaches
allow users to try products and give feedback at their own convenience, over
a period of time, without you being present. Others, like interviews, must be
scheduled for specific dates and times.
part one

11
 planning UX research

How many users do you need to recruit?

What’s the magic number of people you need per user group to provide valid,
useful insights?

At some point in your UX career you will likely hear UX guru Jakob Nielsen’s
advice that, ‘Testing with FIVE users is enough!’

You may also hear the following contradictory information from the other UX
guru Jared Spool stating, ‘EIGHT is NOT enough!’

Meanwhile, market researchers doing multivariate, A/B testing and analytics

say sample sizes of 1,000 or even 50,000 aren’t enough.

So, which is it?

As Kuldeep Kelkar points out in his article on sample size, if you read Jakob
Nielsen’s article closely it’s clear that sample size of five is enough to identify
85% of the problems. What’s not always clear is that this 85% only applies to
high probability problems that have a 31% chance or higher of occurring
with at least one participant.

If your objective is just to identify low hanging fruits, a sample size of five is
enough. But if you want to go beyond that, and there is a business value
to fixing and optimizing several workflows, you’ll want to go with larger
sample sizes.

The higher the sample size the more issues you will find.

In many cases, especially where design has matured and lots of optimization
is needed (ecommerce, B2C websites, etc) its recommended to test with
larger sample sizes of 20, 30 or even more per segment.

Depending on your intended outcomes, here are our recommendations

about sample size. The bold and blue rows are the most common set
of numbers we see across hundreds and hundreds of studies here at
UserZoom:
part one

12
 planning UX research

What Sample Size do I need?

problem/insight
occurrence
40% 30% 20% 10% 5%
Identify
Usability Issues
sample size needed 4 5 9 18 37

margin of error (+/-) 24% 15% 10% 8% 5% 3% 2%

Estimating
Parameters KPI sample size needed 90%
10 28 65 103 268 749 1,689
confidence

difference to detect (90%

50% 30% 12% 10% 5% 3% 2%
confidence)

Comparing sample size within

17 29 93 115 246 421 1,297
Options subjects

sample size between

22 64 426 614 2,468 6,866 61,822
subjects

All of this is, of course, budget and resource dependent. If you find
recruitment difficult, remember that testing with only a handful of people
is better than no testing at all.

Criteria for selecting users

When deciding which users will potentially give you the ‘best’ data, you’ll
probably want to find people who:

• Use (or will likely use) your product reasonably often

• Use (or will likely use) your product to be more effective or efficient at work

The term ‘best’ here means relevant, useful and objective. Users should tell
you honestly what they think about using your product.

Sometimes this can be tough to hear, so prepare yourself. If you’re doing

UX testing as an exercise in fishing for compliments, you’re doing UX
testing wrong.

But that doesn’t mean you don’t deserve a nice cup-of-tea/stiff-drink in a quiet
room after you’re done.
part one

13
 planning UX research

Using personas
Personas can be useful in identifying users to recruit.

They are a way to help organizations understand their potential and

existing audience in a more personal way.

Personas are detailed profiles of a particular audience member, who

represents a distinct group of people - in that they share similar behaviors,
attitudes, personalities and preferences of your product - but are the
‘figurehead’ for a larger demographic.

The two criteria mentioned in the previous section (they use the product
often, they use it to do more efficient work) already define two specific
personas. Either a potential user matches one or both of these criteria or
they don’t, in which case they shouldn’t be recruited.

Choosing a UX research method

The next step in planning UX research is deciding which research methods you
will use. There are two main categories of research methods to choose from:

Quantitative
Quantitative refers to any kind of research where the results can be
presented in numbers, i.e. ‘how many, how often or how much’.

For a complex example of quantitative data, just take a look inside the
analytics of your website – pageviews, sessions, bounce-rate, frequency of
visits over time – are all quantitative data.
part one

14
 planning UX research

Qualitative
Qualitative refers to any kind of research where the results take the form
of observations, comments, thoughts and feelings, instead of cold, hard
numerical data.

This can be further subdivided into behavioral and attitudinal methods. For
instance, a behavioral method examines how people use your product. An
attitudinal method would then investigate what people think of your product.

Specific research techniques can belong to either the qualitative or

quantitative category, depending on how they are used. You may also find
that a blend of quantitative and qualitative research techniques will
help you cover all possible bases.

Unmoderated vs. moderated

Another thing to consider when choosing from the variety of UX testing
methods at your disposal, is whether you’re going to run the tests ‘moderated’
or ‘unmoderated’. So, whether yourself or a researcher (a moderator) is going
to be present during the test, or whether the test participant is left to carry
out the task without anybody else in the room.

Here’s a little more detail on both options:

Unmoderated testing

Unmoderated tests are how we describe unobserved tests,

where a participant is left alone to complete tasks without the
presence of a moderator.

These sessions can be recorded for later viewing as part of a qualitative study,
or the data is collected and analysed as part of quantitative research.
part one

15
 planning UX research

In an unmoderated test, a participant will interact with a digital product, while being
prompted with questions or tasks. A UX researcher will then look at the results of
the project and take note of where the participant is experiencing problems.

You will need a specific online platform or service to help you accomplish
unmoderated tests. The service will also help you build the study, find
participants and analyse the data. This can be a lot easier, cheaper and
quicker than running moderated sessions.

However as you will be reviewing the results ‘after the fact’, you’ll be missing
the chance to ask more in-depth questions. So if something unexpected
crops up, you’ll have to launch another round of tests to investigate.

Moderated testing
Here the participants are observed by a researcher, in-person or remotely.
Moderated tests tend to be used when testing more complex websites or
incomplete prototypes, as you can probe a little more and ask users more ‘in-
depth’ questions.

Many of the following methods on the next page can be run as either moderated
or unmoderated...
part one

16
 planning UX research

Common UX research methods

Here are examples of some of the most common techniques

used to carry out UX research.

A/B Testing: A/B testing requires you to use a third-party piece of software
that helps you set up two different web pages, where one page has an
element that’s slightly altered from the other. For instance, if you can’t
decide on the text for a ‘buy’ button, you could use an A/B test to present
one version of the button that says ‘add to cart’ to half your traffic, and the
other version that says ’buy now’ to the other half of your traffic. Then you
can then see whether or not changing the text of this button has made any
difference to the number of people clicking on it.

Card sorting: In a card sort, participants are presented with a list of items
(for example, all the products featured in an online supermarket) and
asked to group them in a way that makes the most logical sense to them.
Depending on the type of card sort, participants can also choose names
for the groups they’ve put together, forming the potential categories and
subcategories of a website.

Clickstream Analysis: Clickstreams allow you to view and analyze the

paths participants took while performing their tasks, which percentages
followed that path, and their final status (complete, error, abandon or
timeout). Certain software even allows you to analyze the heatmaps (the
aggregated areas where users clicked on the page) on each page for
further behavioral analysis.

Concept/Prototype Testing: Concepts or prototypes are inexpensive

versions of a design for engineers to test how the real thing will function.
UXers can test prototypes with users to get early feedback before sinking
resources into a bad design. In the long run, it will save you time, money,
and headaches rather than doing all of your UX testing when a product is
nearly complete.
part one

17
 planning UX research

Customer Feedback (or Voice of the Customer Study): A Customer

Feedback or VoC study is aimed at collecting ‘true’ data on participants who
visit a site. You can obtain valuable statistics about who your users are and
what they want from your website, by asking them questions via a survey
triggered by feedback link, button, form or email.

Desirability Studies: Desirability studies help you identify and define some
quality of your product or brand. You will show your participants your
product (whether it’s a prototype, live website or even some marketing
copy or images), you then ask them to describe what they see using a list of
pre-selected words.

Diary/Camera Studies: Diary studies gather information about a user

experience over an extended period of time. Participants write about their
experiences with a particular product or service in a diary. They may also
take photos or perform other activities to record their experiences. Once
the study period is over, the researcher analyses the findings.

Ethnographic Field Studies: Ethnographic studies involve talking with

people and observing them perform their tasks in their own natural
context. Its aim is not just to gather information on how people behave
and interact, but also how their location, environment and other contexts
affects their day-to-day lives. UX designers take this ethnographic research
and use it to solve a problem through a product or technology.

Eyetracking: Eyetracking lets UX researchers see precisely where

participants look on a screen when performing tasks. It requires a special
piece of equipment that tracks the user’s eyes as they look around the
screen, and this information generates a heatmap of where on the page
the user concentrates the most. This information can help you determine
where your most valuable content or CTAs should reside.

Focus Groups: A group of participants from your target market are

gathered together in one room with a moderator, where they discuss
topics around your potential product or service. These thoughts and
feelings are collated and used to inform the direction of your product.
part one

18
 planning UX research

Intercept Surveys: By adding a few lines of JavaScript code to your website

or integrating your mobile app with an SDK, you are able to intercept site
visitors on your site or app, with an automatically triggered survey.

Interviews: Participants meet with a researcher one-on-one to discuss

topics around your potential product or service. These thoughts and
feelings are collated and used to inform the direction of your product.

Lab Study: In a traditional lab-based study, between 6-10 (varying

according to needs and perspective) are brought into a ‘lab’ environment
to run through a series of tasks. Participants work on a pre-configured
computer or mobile device while being observed in a separate room either
via monitor or through a one-way mirror. During the study, participants are
given tasks and asked to perform them with a researcher sitting next to
them or in the other room.

Remote Moderated Usability Testing: You are live online with

participants but from your own location, connected to them with screen
share technology and an audio bridge. The moderator is there to ask
participants questions, respond to their questions and feedback, and guide
them through the tasks.

Remote Unmoderated Usability Testing: This method requires having

participants, chosen from a panel of testers who fit into your particular
demographic, interact with your prototype, website, app or any other
digital product in order to capture their feedback, actions, behaviors and/or
spoken-aloud thoughts.

Tree testing: Tree testing is typically used to validate the results of a card
sort. In tree testing, the main categories and subcategories for a website
are already established. Your participants are asked to explore these
categories in order to find a particular item or piece of content. They click
through the various links until they find the category where they expect the
item to reside.

True-Intent Studies: True Intent studies enable you to intercept organic

visitors on your website, mobile site or mobile app and ask them what their
intentions are for visiting as well as getting actionable information about
their experience.
part one

19
 planning UX research

Usability Benchmarking: UX benchmarks allow you to measure your

digital products baseline performance and measure how changes are
affecting the UX needle over time. Typical benchmark studies are either
longitudinal, in which you continually measure your own products over
time, or competitive in which you measure your products performance
against others.

For more detailed information, check out our Comprehensive Guide to 23

Quantitative and Qualitative Research Methods.
part one

20
 THE UX INSIGHTS COMPANY

Doing UX
research

Part Two
 doing UX research

If you’re planning on going down the unmoderated route, you’ll want to look
for a flexible UX testing platform that can help you manage usability studies,
schedules and information. It can also offer additional UX research tools like
remote testing, benchmarking, card sorting, tree testing and other survey and
ranking methods.

Ease-of-use for such a platform is as important to you as it is to a UX

researcher, just like the ease-of-use of your product is to your customers. So
pick the platform that’s right for you.

You can then save time and effort in automatically collecting data from different
UX research techniques, so that you can rapidly and accurately assess the
overall satisfaction of the user experience.

However if you’re going down the moderated route, you’ll need to think about
how you’re going to interview and observe your chosen test participants.

Much of the next section can be summed up as – don’t terrify your users; do
placate them with sweet things.

How to ask questions

Half the challenge in carrying out ‘effective’ UX research is asking the

right questions.

It’s a skill that takes time, practice and experience. It can also mean the
difference between a useful test and a complete waste of time and money.

If you’re a UX research newbie though, there are helpful rules to get started in
writing useful questions.

Mainly it’s about asking things that encourage users to share their point
of view, without your influence or bias.
part two

22
 doing UX research

Your job as a UX researcher is to uncover truth and honesty. Your job is to

gather user feedback that isn’t colored by your own hopes or expectations.
Your job is to listen, and to be deliberate with your words and actions.

To ask good questions – the ones that get to the heart of the user experience
– consider the aim and scope of your UX research.

The aim is the problem or issue you want to solve, or the information you
want to obtain.

The scope is the extent of your research, including the number of users
involved and the different research methods applied.

With these in mind, you can plan questions that will help you discover
what your user’s goal is when using a product and how they go about
achieving it.

With this information in place, you can begin to explore what works well, what
works not so well and what doesn’t work at all for users.

Remember to explore all three of these areas as fully as possible. The point is
to discover problems that currently exist, but it’s also good to highlight what
is working well. Don’t dwell on them too long, but it’s good to report positives
back to stakeholders, as these can help ‘cushion the blow’.

Your time with the user will be limited, so you’ll want to stick to the task at
hand, but always keep an ear out for behavior or comments you weren’t
expecting – as ultimately these can all help improve the overall experience of
your product.

Asking users about their expectations

At any point while using a product, there is a natural next step that your users
will want to take (i.e. when you add a product to a basket, you might expect
to be offered the choice to view the basket or continue shopping). A key part
of your questioning will be to ask users what they expect those next steps
to be.
part two

23
 doing UX research

Knowing what your user’s expectations are will help you build
workflows and customer journey maps to match them.

And the only way to find out what those expectations are is to ask
users directly.

Other questions may deal with their initial impressions of a product,

their feedback about common examples of use of the product, and their
comments on any changes you may have in mind for an existing product
or service.

Making questions relevant and actionable

Before you ask users your questions, try them out on a test audience. This
audience can be people you know, who you can rely on to give sincere and
constructive opinions on whether your questions make sense or not. Again,
you’ll want to prepare that cup-of-tea/stiff-drink and the quiet room.
After you’ve finished the test, the answers will be collated and analyzed. It
helps to consider this stage when coming up with your questions beforehand.

Your users’ responses should be actionable, either directly or

after analysis (i.e. offers practical guidance on how you can
improve the product).

If you cannot see how you will practically use the responses to improve user
experience, then think of a different question.

Analysis of quantitative data may be easy (count the number of yes and no
replies), but this is more complicated to action. Information from qualitative
questions may be harder to analyze but can give you a clearer picture of how
to improve your product.
part two

24
 doing UX research

Three stages of questioning

The questions themselves can be organized into three stages.

You start with a broad introduction, then a focus on the specifics, before
ending with a more general perspective.

These three different phases can look like the following:

#1 Opening section
These are broad questions, where you find out about your users’ work, use of
the product and any overall feedback they may have. “Tell me about a typical
day for you at work?” and “What are your biggest daily challenges at work?” are
typical icebreakers.

#2 Middle section
This is more focused. Ask for opinions on specific features of interest that are
tied to your research goals, why they hold these opinions and what they expect
to see or achieve.

#3 Closing section
When announcing the end of the session, you can open opportunities to gain
extra information and insights. Users often feel more able to express themselves
freely at the end of a session as though the pressure is off, so don’t ignore the
value of the closing section. Instead, use it to ask pivotal questions like “If you
could change one thing, what would it be?” or to rate their experience, or ask
how useful they think a specific function or feature will be for them.
part two

25
 doing UX research

Removing bias
Often without meaning to, we can influence people’s answers by the way
we phrase a question.

Here’s an example of two very similar questions that could lead to wildly
different answers:

1. What do you like about this product feature?

2. What do you think about this product feature?

In the first question, we’re restricting the answer to only the good things
they’ve discovered about your product.

In the second question, the field is open: the user can reply with good things,
bad things, or both, depending on the feature.

We might not like the answer to question 2 as much as question 1, but it’s
the answer to question 2 that’s the one we need to hear and will make
a difference.

Also keep in mind that what users say and what users do may be two
completely different things. Remote observation of how someone uses a
product or service may uncover important truths that simply talking about it
may not have revealed.
part two

26
 doing UX research

How to observe

Besides properly observing how a user interacts with a product or service, it is

also important to notice any signs that might indicate the interviewee is
uncertain or feels under pressure.

Uncertainty and stress can skew what you observe, so this should be reduced
or eliminated where possible. Likewise, be alert to any thoughts, opinions or
preconceptions that the interviewee has, as these should be probed further.

You should always include in your preamble before the session that
you are testing the website and NOT the user!

Reassure them that there is no ‘right’ or ‘wrong’ and that all feedback is
appreciated and will only help to improve the website.

Recording the session (with the interviewee’s prior permission) is one way
to capture as many signals as possible. Objective observation, attention to
detail and taking notes are good practices while interviewing, whether you’re
recording the session or not.

Remember that practice makes perfect – you can hone your skills in any of
these areas by practicing in simulations with colleagues, before doing real
interviews with users.

Understanding users and their mental models

A mental model is a concept that expounds the importance of how every
person has their own individual way of seeing the world.
part two

27
 doing UX research

Designers typically create products based on their own perceptions, assuming

users will understand the way it works in the same way they do. A designer’s
challenge is to ensure that developers build a product with the users’
mental model in mind.

It takes more than an hour in an interview room or a tabletop session to

understand all the facets of a user experience. This is especially true of
products or solutions that are used in non-office settings, such as factories,
hospitals and outdoor sites.

In these cases, you may have to ‘walk a mile in their shoes’ to understand
how users relate to your product in their own environment and how that
affects their user experience.

This is where contextual enquiry can help.

Contextual enquiry
Contextual enquiry is a method that allows you to gain deeper understanding
of users’ wants and needs.

There are four main phases:

Context - Go to their place of work. Even if their office is very similar to yours,
being in their own environment can make a significant difference. Bring
doughnuts.

Participation - With your users’ permission, do the things they do, like
handling service calls or managing inventory. Experience their pain points for
yourself - you’ll find out far more about the usability required.

Interpretation - Check that your observations tally with those of the

participant, that any assumptions you make are justified and that your
conclusions are correct.

Focus - It’s good to keep an open mind, but it’s also good to plan the areas,
tasks, and behaviors you need to observe to maximize the benefits of the study.
part two

28
 doing UX research

Checklist for effective

interviewing

Think of the following list as the basics you always need to cover, no matter
what specific area or product you’re working on.

Attitude - Make a point of being appropriately positive, approachable,

open, curious and focused.

Appropriate location - Ensure that the location lends itself to realistic

usability testing in a normal, stress-free (for you and your interviewee)
environment.

Build rapport - Trust, empathy and freedom of choice are critical. Be sure
that your interviewee knows that the usability testing is voluntary and not
imposed.

Consent forms - These inform participants of the purpose of the interview

and record their agreement, while protecting everyone legally.

Supporting devices - At a practical level, remember to organize easy

access to the product or solution they’re testing. Ensure your recording
devices are ready to note or track user behavior. And make sure all
necessary batteries are charged before you start.

Focus on outcomes - How does the participant feel when using or after
having used your solution? What useful feedback can you get to help
develop or modify product design accordingly?

Be patient with silences - Your interviewee may be remembering a

relevant point or thinking how best to express feedback.
part two

29
 doing UX research

Check understanding - It is essential to correctly understand what your

interviewees say and what motivates them to use your product as they do.
Rephrase or summarize what you have heard or noticed out loud to your
interviewees. Ask them if you have accurately captured all their comments
and feelings.

Balance note taking with engagement - You want to record as much

of your interviewee’s input as possible, but make sure you stay engaged.
Using shorthand and making an audio or video can let you better focus
on making the most of your time with your participant. Even better would
be having someone else from your team taking notes while you do the
questioning. It’s worth remembering that you can make the session
appear very formal if you’re taking notes and this can lead to extra
pressure. People generally feel more inclined to speak when it feels like
natural conversation.

Thank your interviewee when the interview is over - May be obvious,

but politeness can be an afterthought if your day is packed with running
multiple tests.

Remuneration - People are sacrificing a significant part of their day to help

you, so they will expect to be paid for their time. Obviously the amount you
pay will depend on your budget, but the expected rate can be anywhere
between $25 - $60 per hour. The more niche your product or market, the
more expensive it will be to recruit the users you’re looking for.

WHY? - When all else fails during the interview (if the user has a question, if
the user is stuck, or the user makes a comment about something that’s not
working) always ask WHY? Asking “Why?” is a researcher’s best friend -
“Why did you do that?” “Why did you have that expectation?” etc.
part two

30
 doing UX research

Pitfalls to avoid when

interviewing

There are four pitfalls that commonly undermine effective interviewing and
UX research testing.

Avoid the following:

Taking over the user’s role - Your point of view will never be (exactly) the
same as somebody else’s, and you are not the user. Listen actively to your
interviewees, but do not usurp their role.

Diverging into training - If a user has difficulty in using a product or

service, note down all the relevant information. But resist the temptation
to resolve the difficulty there and then. In real life, you won’t be there to
help them.

Being too general - User feedback during UX research sessions must be

precise and targeted, if it’s to help you build better products and solutions.
Probe accordingly.

Unnatural behavior - Also known as the Hawthorne effect, people often

behave differently when they know they are being observed. Counter this
effect by building rapport, trust and putting your interviewees at ease, so
that they behave naturally and give you valid information.
part two

31
 doing UX research

Examples of UX research
questions

The important thing here is to build rapport with your interviewees, while
also helping to set the scene and position your main objective.

There is no standard set of questions that can be used for any given test,
because every product, user group and test objective is different.

Your questions should be related to what your testing. They should begin
by warming up your user, before transitioning to more specific enquiries
about the product or service.

As an example of a line of questioning for a specific product, here are some

questions you might ask if testing a piece of team management software:

• How long have you been in your current role?

• How would you describe your typical workday?

• What are the biggest challenges for you in your daily work?

• If there was something you could change about your job, what would it be?

These questions would however be useless if you’re testing a travel website.

In this instance it would make more sense to ask:

• Where was your last holiday?

• What kind of trips do you have planned for the future?

• What was your favourite holiday destination and why?

part two

32
 THE UX INSIGHTS COMPANY

The results
are in!
Part Three
 the results are in

Congratulations, you’ve now successfully planned, organized and ran a UX

research project. But before you push the big red button on the confetti
cannon, I have some important news to deliver. You’re only halfway through.

Also your research plan only allowed budget for one confetti cannon, so
you’d better save it till after you’ve collated, analyzed and delivered your
recommendations.

Collating and analyzing data

for action

Now you have this massive bundle of notes and records from your UX
interviews and tests, you’ll need to find a way to extract all the useful and
relevant information.

Your first step might be to go back over your notes and refresh your
memory – this will help put them into context. Remember that your main
goal is to get actionable insights.

Don’t stop analyzing until you have enough practical recommendations

of what to do next with the design.
part three

34
 the results are in

Pros and cons of spreadsheets

A simple way to organize your data is to use spreadsheet software, like
Microsoft Excel.

Spreadsheets allow you to calculate the averages of various scores and

rankings, find out the variance of the results and color-code user responses
to help you see the consensus at a glance.

While spreadsheets are well understood in most organizations, they have

a few drawbacks, especially if they’re the sole way of communicating your
insights.

First, they can easily be altered. This can lead to several different versions
of your spreadsheet in circulation at the same time. Second, they don’t
always integrate directly with other tools that are useful for UX research.
Third, user errors are all too frequent. This leads to distorted results and
erroneous conclusions.

Finally, they’re really boring to look at.

UX data: ‘one truth’

By comparison, a purpose-built UX research platform can avoid all the

above issues.

The immediate advantage is that it offers ‘one truth’ for all, as well as
protecting you from unfortunate Excel-formula errors.
part three

35
 the results are in

A suitable platform can also offer several more advantages:

• It links easily to other UX testing tools you already have.

• It provides consistent benchmarking. Using the same baselines and

measurements for UX, you get ‘apples to apples’ comparisons that show
you the real issues in your company’s products and services.

• Useful data analytics are part of the package. No need to be a data scientist
– common sense and curiosity are enough. By combining UX metrics with
analytics, you get better insights into what is or isn’t working, and why.

UX data analytics: the basics

User experience data on its own is lifeless; it can’t tell you anything other than
somebody really likes data-entry.

Whether the data is in rows of numbers, sheets of scores or lines of text

containing the answers to your open questions, it often takes some extra
tools or resources to get the data to reveal its secrets.

Data analytics is a way of giving data a voice. The simplest analytic is

probably the average: you just sum up all the scores or rankings in the
answers to a question, then divide that total by the number of questions.

More sophisticated analytics can pick out trends, as well as degrees of

commonality and differences of interest. Besides working with numbers, they
can also pick out words, phrases and even ideas and feelings that appear
commonly in user feedback.

Good user experience is an important part of a data analytics package too.

A platform and its data analytics should take you from user data input to
useful insight without needing to get your hands dirty.
part three

36
 THE UX INSIGHTS COMPANY

Acting on
UX research
results

Part Four
 acting on UX research results

This is the final stage of your UX research and the reason why you started it in
the first place.

This is where you can turn surveys, interviews, tests, trends and insights into
product or service improvements and competitive advantages.

This is also where your abilities as a master show-person are put to the test as
well as your research. And yes you should definitely wear a top hat and tuxedo.

How to prioritize issues

One of the hardest things once we’ve finished all the interviews and data
gathering is deciding how to prioritize issues.

Its will help if you begin by summarizing all the issues found and deciding
which ones are a priority.

If you’ve forgotten, take a look back at why your research was performed
in the first place. What were your research questions? If there’s something
specific you wanted to explore then only look at issues related to that issue.

Often we notice many other issues when undergoing user testing, and we
want to put those all findings into the report, but then we need to remind
ourselves that we’re going beyond the scope of the project and need to
stay focused.

Prioritize issues into:

• High severity (show stoppers, things that prevent users from completing
a task)

• Medium severity

• Low severity
part four

38
 acting on UX research results

Also keep track of how many users are being affected by the issue. You
could have one issue that’s low severity but if it’s affecting every single user
then it may be something you want to take a look at.

How to sell your

recommendations

To get your recommendations for UX improvements approved, you need to

get your message across to the people who can make those improvements.

Often, the best way is to tell them a story.

One that makes them sit up and pay attention, one that gets them going
“ooh” and “ah,” one that leads them directly to the truths you’ve
uncovered, one that makes them say “this is the greatest story I’ve ever seen,
and I’ve seen Jaws and Terminator 2!”

Too much? Yeah probably too much. Nothing’s better than Terminator 2. But
you get the idea.

Know what must be done

The right platform and tools can go a long way to revealing the strengths
and weaknesses of a product’s UX, as well as showing how user expectations
are evolving. However, the final step of recommending what to change in a
product will come from you.
part four

39
 acting on UX research results

You need to be clear about what to do and the why to do it.

It’s also crucial to understand that, at this final stage, you are responsible
for a providing a great user experience for the people who are listening to
your report.

If you want to improve the UX of a product, then make sure the UX of your
presentation is crystal clear.

Talk to the right audience

Whether you’re selling an idea, a product or a service, anyone ‘buying’ should
have three characteristics:

1. They must need what you’re offering

2. They must have the authority to say, “Yes, we’ll take it and
anything else you have on offer!”

3. They must have the right resources (whether it’s budget, time,
tools, or probably all three)

The identity of these people will depend on your organization.

It may be the Head of Marketing or the Product Manager, keen for an

edge to help sell more, justify higher margins, build brand equity or increase
customer satisfaction and loyalty.

It may be the R&D or Production Manager who wants better quality ratings
from users or fewer product returns because of user dissatisfaction. It may
be the CEO of your company. It may be a combination of any of these people.
part four

40
 acting on UX research results

Whatever the job description, these people have something in

common: a stake in the future of the product concerned.

Your company may already have a process in place for reviewing and deciding
on product changes, with the stakeholders already identified. If it doesn’t,
then look for the people who need to know about product improvements,
who have the authority to say yes, and have control of the resources to make
it happen.

Lay the groundwork

It doesn’t matter if people know how your UX research story ends before you tell it.

It’s often better they know upfront, instead of surprising them with an
unexpected conclusion and provoking a reaction you hadn’t expected. You
are not M. Night Shyalaman. Well I suppose there’s a chance you could be.
Ooooooh TWIST!

Before you bring people together to tell them your UX research story,
find out if the people in your audience are likely to be receptive or
unreceptive to your conclusion. Then construct your story to reassure
them as needed.
part four

41
 acting on UX research results

Tell a story that grabs them

The most effective story is one that addresses the motivations of your audience.

Theatricals aren’t necessary if you know which buttons to push (woah, keep
your hand away from that confetti cannon – this is just a metaphor). You need
to find out what these are for your own organization.

Here are some common motivations:

Make more money - Selling more of a product or service at higher margins

can do this.

Save money - Cut costs through better products that need less support.

Increase customer loyalty - Better UX keeps competitors at bay and

increases repeat sales.

Increase usage - For pay-per-use services, you’d like to retain more

customers.

Decrease usage - Products that help customers faster can also help justify
higher prices.

Brand building - Better UX for one key product can boost the image of the
entire brand.

Bring the data!

It’s essential to have your data on display as you tell your story.

Sometimes, one significant metric or statistic is all it takes to get

everybody in agreement about a point or a recommendation.

Or perhaps it’s a video clip in which a user gives sincere, relevant and useful
feedback. Decide which data does the job best and use it.
part four

42
 acting on UX research results

Don’t swamp your audience with unnecessary data or insist on dragging

them through all the analysis that led to a specific statistic.

But do be prepared to drill down into data to answer any pointed questions, or
to satisfy their need to see that your UX research has been done methodically.

Story structure

While your aim should be to keep your audience’s attention throughout your
presentation, you need to start strong.

You want to quickly capture their interest and make the promise of a
helpful, logical solution to a pressing problem or business opportunity.

Your opening statement might look something like the following:

“Over the last 12 months, customer calls for support for our flagship product
have increased by 25%. Yet repeat sales have dipped by the same amount.
This means costs have gone up and profits have gone down. In the UX team,
our mission is to find out what users think of our products as they use them
and to help optimize that user experience for business benefit and customer
satisfaction. To see why support has risen while repeat sales have declined,
we interviewed and observed a group of end-users. We gathered input and
feedback to see how to improve and cut support costs, while encouraging more
repeat sales. Here is what we did and what we found.”
part four

43
 acting on UX research results

But perhaps with more shark attacks/time-travelling robots.

The main sections of your presentation then might look like this:

• Briefly introduce yourself, then describe the user group and how you
collected the users’ observations. Mention any overall trends that are
relevant for setting the scene.

• Go through the challenges. Use strategically placed commentary from

the users, like a telling response in a survey response, or a short video clip
that illustrates a specific point.

• Offer a solution. An effective user comment will put the message directly
across to your audience, and the stronger your case will be for moving
towards a solution.

• Summarize what you have observed and describe an agreeable

resolution defined by the recommendations from your findings.

Remember that you may need to present more than once. Your audience
may not be together at the same time, or it may change, bringing in new
stakeholders to whom you have to present.

For each audience, adapt the presentation to address the motivations of

the people listening to you, while always bringing the data in each case.

The more you give your presentation, the better you will know your audience
and how to play to their emotions.
part four

44
Conclusion
UX research can be described as plan, do, check and act.

First, you plan your UX research – decide what you want to find out, who you
will ask and how.

Next, do the data and insight collection – whether it’s via interviews or UX tests.

After that, check what you have collected to see what insights you have on
where to improve the UX.

Finally, act to make recommendations based on your insights and present them to
the people in a position to transform the product or service accordingly.

Then start the cycle again!

The by-product of this user-focused way of doing things is that you end up
making better stuff – stuff that people really want to use; stuff that makes
people happy.

And that’s reason enough to FIRE THE CONFETTI CANNON!

Oh you already pressed the button on page 34. Well that wasn’t very
good planning.

45
 acting on UX research results

Want to learn
more about
UserZoom?

Contact us today to learn how we can support

your UX research goals

Schedule a call

San Jose Denver London Manchester Madrid Barcelona

part one

www.userzoom.com marketing@userzoom.com 866-599-1550

THE UX INSIGHTS COMPANY HQ: 10 Almaden Blvd #250 San Jose, CA 95113 USA 46