Introduction to quantitative and qualitative user research

Who can help you define or verify a customer's real need? Who can tell you if your interface works? Don't look too far: it's obviously the users themselves!

To improve your product and its UX - whether it's to increase your conversion rate, generate better reviews or avoid answering the same question over and over again to support - you need to understand your users and how they use it. To do this, you have a whole arsenal of different techniques that can be grouped into two main categories: qualitative and quantitative research. Here is a brief overview of these methods:

Quantitative research, a story of numbers

Quantitative research consists of collecting statistical data from a large number of users in order to describe a phenomenon, to test or confirm a theory.

A quantitative study quantifies and obtains numerical results. It answers the questions "What?" and "How much?" thanks to metrics that are evaluated. The results are generally presented in the form of graphs and percentages.

To do a good quantitative study, you need to ask closed-ended questions, which invite specific answers, where the choices of answers are predefined or limited. For example: How many purchases have you made this year on the Internet?

In your range of methods for your quantitative studies, you can use, among others:

• Surveys: questionnaire with closed questions sent to a large number of people.
• A/B testing: this consists of presenting two different versions of the same functionality to establish which of the two gives the best results with users.
• First Clicks test: test where we examine which element the participant clicks on first when completing a given task.
• Web analytics: study of the paths of Internet users on a site via analytics tools.

The choice of one or the other technique depends on the stage of the project. I won't tell you more, you'll know everything in a future article, I promise 😃 !

The size of responses to collect depends a lot on the study in question: you have to take into account the size of the target population, the margin of error or even the level of confidence. This article from SurveyMonkey helps you determine the right size for your own quantitative study or you can look at the very detailed method described by Testapic. However, a commonly accepted number is 200 responses.

Qualitative research plays on words

Qualitative research consists in observing or talking directly with a limited number of users to gather impressions, opinions, motivations, behaviors from respondents.

It is used to understand a concept or thoughts, to describe a problem or a need. It answers the questions "How?" and especially "Why?".

In qualitative research, the questions asked are open-ended - participants can express their thoughts freely - because unlike quantitative research, qualitative research does not seek to measure a phenomenon, with numerical data, but to describe it using words, expressions, emotions. In qualitative research, a problem is analyzed in depth.

Depending on the circumstances of your project, you can use qualitative methods such as:

• Usability testing: consists of having users test their product/prototype to ensure that the path is clear and to detect any possible blockages.
• User interview: consists of a dialogue between the designer of a product or service and a potential user.focus group: group interview (usually with 8 to 10 participants).
• Co-creation workshop: a workshop in which users actively participate in the design of a product.

But that's all another story, to be told in another episode 😉

To determine the number of testers to meet during a usability test, it is common to apply the concept stated by Jakob Nielsen that "5 users are enough to find 85% of the problems". This article explains Nielsen's theory by giving the percentage of usability problems encountered as a function of the number of testers, for the same persona (i.e. people using the application or product in the same way):

Two very different methods...

As you can see, quantitative and qualitative research are two very different types of research.

Here's a table to summarize the differences (it's classic, but effective! 😉 ):

Unquestionably, each method has its advantages and disadvantages. It is easier to collect quantitative data, but the data is indirect: you have to work on the answers obtained to bring out the important information. On the other hand, it is difficult to obtain qualitative data, but it is direct: the analysis is done without reworking.

To present the analysis of your study to your product team, for example, quantitative results appeal to the rational side of the audience, whereas words and verbatims appeal to the emotion and sensitivity of the interlocutors.

... but complementary

Driven by the same desire to study the user and his needs, these two types of research are used at different stages of a project, one calling for the other.

For example, with analytics you can see that 60% of your visitors abandon the checkout process before the end. The problem is that you don't know why! Qualitative research then comes into play to provide you with an explanation.

Another case: during an interview with a participant, you become aware of a problem you didn't know you had and identify a need for a new product to solve it. Is this one shared by other users? A survey will allow you to quantify it.

Qualitative and quantitative work together to form a research spiral, like the iterative design cycle illustrated by Nielsen Norman Group:

‍

It is tempting to focus on one or the other of these methods - the one you are most comfortable with - and to put aside the second. Of course, each direction has its drawbacks!

Doing only qualitative research? Without numerical data, it becomes difficult to prioritize one issue over another.

Doing only quantitative research? Without an in-depth understanding of the user's problem, you run the risk of developing a solution that is not adapted to the real need.

Qualitative and quantitative research are not mutually exclusive, nor do they completely overlap. They are complementary. Used in parallel, they cover all the questions that arise in order to understand users completely:

Quantitative or qualitative method? Sometimes difficult to say!

This qualitative versus quantitative description is very dualistic. The reality is more nuanced. There is a wide variety of research methods that form a continuous spectrum between 100% qualitative and 100% quantitative, all at a slightly different level. This means that some are quantitative but with a small qualitative component, and vice versa.

In particular, there are techniques that are almost at the limit between the two. This is the case of guerrilla testing, for example. Guerrilla testing is the practice of testing a feature with passers-by in the street, in cafes or in train stations, in just a few minutes. A large number of tests are carried out in a row and it is still possible to explore a problem with the participant!

Rohrer's illustration: Evaluating the techniques against each other

It is also possible to distinguish between research methods according to their interest in behavior or attitude, because there is a difference between "what the person says" and "what the person does".

Every person forms an image in society that may be different from his or her actual behavior. The most famous example is that of the yellow Sony Walkman. When they were about to launch a new yellow Walkman (the ancestor of the iPod for those who might be too young 😝 ), the company had a lot of people interviewed, all of whom answered that it was great and revolutionary. But when they were offered a free Walkman from their range, no one selected the yellow Walkman.

Each search method will study the user according to one or the other of these views. As with the qualitative or quantitative characteristic, these methods can be classified along a spectrum from 100% behavioral to 100% attitudinal.

To summarize, it is possible to evaluate, for each method :

its level of quantitative or qualitativeits attractiveness for the behavior or for the attitudeThis gives the following graph:

For more details, I invite you to read the article explaining the graph in question written by Christian Rorher.

‍

‍

Today, quantitative is more popular! It is true that in this world of data, quantitative is terribly sexy. But beware, it does not answer all the questions and sometimes create new ones. Qualitative research will always be necessary when you really want to understand your users' behavior, their problems and needs - in short, when you ask yourself "But why? And qualitative research is exciting! Don't hesitate any longer, get started and recruit your first users in a few clicks on our platform 🤩