Quantitative Research

Quantitative research is the methodology which researchers use to test theories about people’s attitudes and behaviors based on numerical and statistical evidence. Researchers sample a large number of users (e.g., through surveys) to indirectly obtain measurable, bias-free data about users in relevant situations.

“Quantification clarifies issues which qualitative analysis leaves fuzzy. It is more readily contestable and likely to be contested. It sharpens scholarly discussion, sparks off rival hypotheses, and contributes to the dynamics of the research process.”

— Angus Maddison, Notable scholar of quantitative macro-economic history

See how quantitative research helps reveal cold, hard facts about users which you can interpret and use to improve your designs.

Use Quantitative Research to Find Mathematical Facts about Users

Quantitative research is a subset of user experience (UX) research. Unlike its softer, more individual-oriented “counterpart”, qualitative research, quantitative research means you collect statistical/numerical data to draw generalized conclusions about users’ attitudes and behaviors. Compare and contrast quantitative with qualitative research, below:

Quantitative research is often best done from early on in projects since it helps teams to optimally direct product development and avoid costly design mistakes later. As you typically get user data from a distance—i.e., without close physical contact with users—also applying qualitative research will help you investigate why users think and feel the ways they do. Indeed, in an iterative design process quantitative research helps you test the assumptions you and your design team develop from your qualitative research. Regardless of the method you use, with proper care you can gather objective and unbiased data – information which you can complement with qualitative approaches to build a fuller understanding of your target users. From there, you can work towards firmer conclusions and drive your design process towards a more realistic picture of how target users will ultimately receive your product.

Author / Copyright holder: Teo Yu Siang and the Interaction Design Foundation. Copyright terms and license: CC BY-NC-SA 3.0

Quantitative analysis helps you test your assumptions and establish clearer views of your users in their various contexts.

Quantitative Research Methods You Can Use to Guide Optimal Designs

There are many quantitative research methods, and they help uncover different types of information on users. Some methods, such as A/B testing, are typically done on finished products, while others such as surveys could be done throughout a project’s design process. Here are some of the most helpful methods:

• A/B testing – You test two or more versions of your design on users to find the most effective. Each variation differs by just one feature and may or may not affect how users respond. A/B testing is especially valuable for testing assumptions you’ve drawn from qualitative research. The only potential concerns here are scale—in that you’ll typically need to conduct it on thousands of users—and arguably more complexity in terms of considering the statistical significance involved.
• Analytics –With tools such as Google Analytics, you measure metrics (e.g., page views, click-through rates) to build a picture (e.g., “How many users take how long to complete a task?”).
• Desirability Studies –You measure an aspect of your product (e.g., aesthetic appeal) by typically showing it to participants and asking them to select from a menu of descriptive words. Their responses can reveal powerful insights (e.g., 78% associate the product/brand with “fashionable”).
• Surveys and Questionnaires – When you ask for many users’ opinions, you will gain massive amounts of information. Keep in mind that you’ll have data about what users say they do, as opposed to insights into what they do. You can get more reliable results if you incentivize your participants well and use the right format.
• Tree Testing –You remove the user interface so users must navigate the site and complete tasks using links alone. This helps you see if an issue is related to the user interface or information architecture.

Another powerful benefit of conducting quantitative research is that you can keep your stakeholders’ support with hard facts and statistics about your design’s performance—which can show what works well and what needs improvement—and prove a good return on investment. You can also produce reports to check statistics against different versions of your product and your competitors’ products.

Most quantitative research methods are relatively cheap. Since no single research method can help you answer all your questions, it’s vital to judge which method suits your project at the time/stage. Remember, it’s best to spend appropriately on a combination of quantitative and qualitative research from early on in development. Design improvements can be costly, and so you can estimate the value of implementing changes when you get the statistics to suggest that these changes will improve usability. Overall, you want to gather measurements objectively, where your personality, presence and theories won’t create bias.

Learn More about Quantitative Research

Take our User Research course to see how to get the most from quantitative research: https://www.interaction-design.org/courses/user-research-methods-and-best-practices

See how quantitative research methods fit into your design research landscape: https://www.smashingmagazine.com/2018/01/comprehensive-guide-ux-research/

This insightful piece shows the value of pairing quantitative with qualitative research: https://usabilitygeek.com/ux-designers-quantitative-data/

Find helpful tips on combining quantitative research methods in mixed methods research: https://blog.optimalworkshop.com/what-is-mixed-methods-research/