Hello, I'm David Travis and in this episode
of the UX Tea Break I answer the question:
"What does good look like when it comes to
Discovery research?"
So this question comes from Brett Parker.
Brett asked this: "I'm a product owner and
I don't know a great deal about user research.
I have a user researcher on the team who's
doing discovery research for us. He seems
competent but how do I know if his work is
any good?"
I thought this was an interesting question
because often most of us have got some kind
of background in quantitative research: enough
that we can have what I like to call "quantitative
literacy". So for example if I showed you
the results of a survey you would ask me questions
such as, well, you know, "How were the people
sampled for this survey? Were they representative?"
You'd also know, if I gave you the data in
Excel, how to create an average and a standard
deviation and so on. You kind of… most people
know about the basics of quantitative research
because we're taught it at school.
But when it comes to qualitative research
that's something that often people find a
lot harder to judge. And I thought I would
give you four criteria that I would use to
decide if the piece of Discovery research
that you're looking at is any good.
First of all, I should define some terms.
By 'Discovery' research I mean research that's
carried out early on in design in order to
understand the user needs for a particular
product or system. Secondly, I'm talking about
digital products and services. I'm not talking
about things like, I don't know, toothpaste.
But if you're doing research in that kind
of area, then… well actually some of these
things might be useful but it's not really
about that. And I'm also not talking about
qualitative research generally. I want to
emphasize I'm focusing on Discovery.
So I think there are four criteria you could
use, Brett, to decide whether or not the research
is any good. So the first criteria is to look
at the research and ask, "Did it use small
samples?" And by small samples I mean somewhere
between five and… I don't know… 20 users.
In contrast to say a research study that might
use hundreds of users. And the reason we want
small samples in qualitative research is because
we want to go in depth. We really want to
understand the range of experiences that people
have so that we can fully understand the space
that we're designing for. We're not interested
in creating a kind of an average like you
would be if you were doing quantitative research.
So that's the first one: it uses small samples.
Second and kind of related to that is that
the research focuses on behavior. What I mean
is what we're interested in is what people
do rather than what they say. So we're not
so interested in opinions. It's not that we're
not interested at all in people's opinions
and what they say it's just that it's very
risky to design digital products or services
based on people's opinions. Instead we want
to see what they actually do. So that means
in the research that you're trying to evaluate
look at how long the researchers spent with
the participant where the participant was
actually doing something — the meaningful
activity I like to call it. If the research
session was about an hour in length I'd like
to think that the participant was doing stuff
— demonstrating the way they go about doing
the meaningful activity — for about half
an hour or so. So about half the time, as
a kind of a rough rule of thumb.
Related to that is the third principle which
is that the best kind of discovery research
is done one-to-one rather than in groups.
The reason you want to run one-to-one research
is you want to get this in-depth understanding
of people and if you run people as groups
it's much harder to get that understanding.
Because often what people will do is they'll
coalesce around some norm or description of
ways they behave. And you won't understand
the differences as much as you would do if
you went one-to-one.
Then the fourth criteria is that the research
that you carry out is carried out in context.
So by 'in context' I mean it's carried out
in people's homes or places of work, as opposed
to say, remote research mediated over the
internet, or pop-up research in a cafe, or
lab-based research in a studio. So if your
users do this meaningful activity in their
homes, that's where the Discovery research
should take place. If it happens at work,
that's where the Discovery research takes
place. If it happens in their car, that's
where the Discovery research should take place.
So the purpose of that is to make sure that
you're in the user's context. And the reason
that matters is because first of all people
forget things that they remember when they're
in context. They forget that they do things
in a certain way. They forget about other
devices they may use or other applications
they may use to achieve their goals when you
see them out of context. But in context, when
you can ask them to do stuff, those things
act as triggers to remind them of exactly
what it is that they do in that particular
situation.
So those four criteria: first of all, small
samples. Secondly, it should be behavior-based
research rather than opinion-based research.
Thirdly, it should be one-to-one. Fourthly,
it should be in context. If you can tick all
of those boxes it's going to be very hard
to do poor quality Discovery research. All
of those four are really good indicators that
the research has been done well.
Finally, I should say that I'm not saying
that every piece of Discovery research needs
to hit all four of those criteria. It could
well be the team have a very specific question
that you could go out and answer by speaking
to some people in a cafe or somewhere local.
By all means go ahead and do that. It's just
that that shouldn't comprise the majority
of the Discovery research. The majority of
the Discovery research should hit those criteria
that I've listed.
Well, I'm sure somebody disagrees with that
so let me know in the comments. What resonated
with you? What things did you disagree on?
What other criteria would you use to decide
that Discovery research is any good? And of
course, as always, if you liked the video
please give it a thumbs up. That way, YouTube
recommend it to other people. And subscribe
to the channel.
