If you aren’t sure about when to use quantitative vs qualitative research, you aren’t alone. It’s tough to discern between the two and select the best approach for your business needs.
But in the same way your team performs best when each individual’s strengths are considered, quantitative and qualitative research methods perform best when leveraged according to their benefits.
Quantitative research examines hard data and is measured in numbers. You would likely use it for testing a hypothesis or measuring the effectiveness of a strategy. Qualitative research examines behavioral data and is measured non-numerically. You might implement it for the creation of a hypothesis or to inform the direction of an optimization strategy.
In this article, we’ll answer the following questions:
- What is the difference between quantitative and qualitative research in CRO?
- When do I use qualitative research? When do I use quantitative research?
- What are examples of quantitative and qualitative data?
- What are the best quantitative and qualitative research methods?
Let’s dive in.
What’s the difference between quantitative and qualitative research?
Quantitative and qualitative user research describe two different types of data that can be used to improve website usability and conversion rates.
Here are practical definitions of the two:
1. Quantitative research is primarily concerned with the numbers.
It uses metrics like completion rates, bounce rates, and conversion rate to measure the effectiveness of your sales funnel.Quantitative research answers the question: 'How much of what is happening, and where?' Click To Tweet
Quantitative research answers this question: “How much of what is happening … and where?” This data is typically used to form and test a theory that it will ultimately support or reject. In conversion optimization, that’s usually best accomplished with A/B testing.
2. Qualitative research is primarily concerned with your website users.
It leverages special tools like heatmaps, clickmaps, and surveys to find out “Who is doing what … and why are they doing that?”Qualitative research answers the question: 'Who is doing what, and why are they doing that?' Click To Tweet
This data tells you what design features are difficult or easy to complete. Research following a qualitative approach is exploratory and seeks to explain ‘how’ and ‘why’ a particular phenomenon or behavior operates as it does in a particular context.
The screenshot above is a heatmap. It shows you where visitor attention is most centered. You can use that information to optimize page layouts and guide visitors along your sales path.
Whereas quantitative data tells us that a specific number of visitors behaved in a particular way at a certain point along your pathway to sales, qualitative data provides clues about why they did so.
Both types of research are essential to a data-driven design cycle. Much of the time, though, companies collect quantitative data only.
That’s a mistake.
Qualitative data allows deeper insight and can help you quickly determine where your user experience is sagging. The more user-friendly you can make your path to sales, the more customers you’ll have walking it to the end over and over again.
Here’s a summary of some of the key differences:
|Quantitative Research||Qualitative Research|
|Representative sample size||large and representative||small and non-representative|
|Analysis||statistical relationships||trends in behavior and sentiment|
|Drawback||lacks psychological context||lacks empirical evidence|
How to apply quantitative and qualitative research: a step-by-step process
Digital marketing managers and conversion rate optimization specialists sometimes disagree over which type of research should be conducted first: quantitative or qualitative.
While the best strategy is always the one that fits your particular needs, the conversion rate optimization team at The Good recommend the following sequence in most cases:
- Check to ensure appropriate data collection tools are in place and set up correctly
Do an audit of your tool stack to make sure their respective data streams are flowing properly. In terms of website behavior, this means making sure Google Analytics is set up correctly and feeding real-time visitor data.
Sign in to Google Analytics, and then go into incognito mode on another browser to visit your website. If everything is fully operational, you should see yourself pop up on the real-time map.
- Begin by observing quantitative data to identify stuck points in your sales funnel
Look at the metrics within Google Analytics that reflect the different stages of your sales funnel. Analyze things like bounce rate and cart abandonment rate to search for evidence of bottlenecks in your funnel.
- Use qualitative tools and observations to find out why the bottlenecks exist
Tools like heatmaps and click maps allow you to gauge the effectiveness of your page content. Strategic user testing campaigns allow you to see sentiment trends regarding which content is driving action, and which content may actually be preventing action.
Enjoying this article?
Subscribe to our newsletter to get more just like it, sent straight to your inbox every week.
- Propose solutions to the problems, activate those solutions (through A/B and multivariate testing), then observe results
Combine the insights from your quantitative and qualitative research to create a solution for any problem you have identified. Put those solutions in action and then compare their effectiveness against each other through A/B testing.
- Continue observing, constructing theories (hypotheses), and iterating on the testing to monitor and improve conversion rates across the board
Keep the iterate, test, analyze loop going until you reach your target conversion rate. It is rare that your first round of A/B testing will yield the ultimate increase in conversions. Once this process is implemented the first time, it should be perpetually running as a part of your overall growth strategy.
Tool selection can be tricky. Even if you’re the digital marketing manager of an enterprise-level business with in-house conversion optimization capabilities, you’ll probably save money by contracting with a conversion optimization partner to get started.
At The Good, we help businesses work through the development or fine-tuning of their conversion optimization strategy. This includes identifying and recommending necessary tools and helping secure the best price for those tools.
Since conversion rate optimization is our business, we stay up to date on the latest developments and can help keep teams from getting mired down with the wrong tools and counterproductive methods.
Quantitative vs Qualitative Research Methods in CRO
It is nearly impossible for a CRO strategy to succeed if it is built using purely quantitative or qualitative research methods. Success hinges upon a mixed-method approach that maximizes the value of both quantitative and qualitative data to synthesize an effective CRO strategy.
We work through four key areas of CRO research using this mixed research methodology:
- Acquisition and behavior
- Content effectiveness
- Conversion effectiveness
- Iterative improvement
Acquisition and Behavior
Acquisition and behavior research examines where visitors come from and how they behave. It looks at quantitative data that shows which actions are being taken and quantifies the occurrences of that action over a given period of time.
Google Analytics dominates this stage of the game. The user data they give allows you to get a snapshot of how many users actually become customers, and what path of pages led the most of them over the line.
Content effectiveness research looks at the value users assign to the words on a page, focusing on which words keep them engaged and which lead to an attention drop-off. Heatmaps rule this domain, and there are three types: click, scroll, and movement maps.
Click maps track where on the content the user is clicking during their session, and scroll maps measure how far along the page they scroll. Finally, movement maps track where the user’s cursor goes on a page. This is correlated with user eye movement.
Heatmaps help you give a comparative value to user behavior, which informs the design-based initiatives within a CRO strategy.
Conversion effectiveness research analyzes the gaps between user desire and the fulfillment of that desire. This is the domain of usability testing, which also makes use of both qualitative and quantitative data.
Along the x-axis is the level of moderation, which is reflective of the level of involvement from the researcher. An interview is on the extreme side of moderation, for example, because it cannot be done without human involvement.
Along the y-axis looks at whether the user testing platform requires in-person conditions to be valid or if it can be done remotely. Observation-based platforms have to be in-person, for example, because such qualitative data requires the behavior to be both natural and directly observable.
Finally, iterative improvements focus on how the learnings from the other three areas can be tested on and improved. A/B testing rules this area of research.
This is the area where design-based and copy-based CRO techniques merge into one cohesive CRO strategy that is designed to perpetually measure and analyze user behavior.
The key to executing such a mixed-method strategy is to know when to use quantitative research and when to use qualitative research.
When should you use quantitative vs. qualitative research?
I’ve touched on this in the process description above. Let’s go a little deeper, though, to make sure you and your team are clear on when to use quantitative research and when to use qualitative research.
Conversion rate optimization isn’t a one-stop fix for sales. It’s a series of events and actions occurring between the time a prospect first becomes aware of your company and the point where that prospect makes a purchase.
That means there isn’t just one conversion rate to consider, there are many. Here are some of the most common data collection points in conversion rate optimization that can help you identify trouble spots in your sales path:
- Bounce rate: Once a visitor arrives on your website, how long does that visitor stay?
- Opt-in rate: Of the visitors given an opportunity to subscribe to your email list, how many respond to the offer?
- Search rate: How many visitors take advantage of your on-site search engine?
- Click-through to a product detail page: How many of those who visit a category page go on to view a product detail page?
- Cart abandonment rate: How many visitors enter the checkout process but fail to place an order?
That list could be extended to point out dozens of potential ‘micro conversions’ your visitors can take on the way to becoming paying customers.
Website designers often focus on making your site pleasing to the eye. Your visitors care more about usability. (see graph above)
You can compare quantitative to driving on a highway. You’re clipping along at good speed when you come to a spot where traffic stopped completely. Is there an accident ahead? Construction? What’s the problem? Cars begin pulling U-turns and looking for a different route.
When visitors to your website are confronted by confusing or irrelevant sections of your path to the sale, they’ll bail out too. If you can uncover and fix those trouble spots, you’ll keep visitors happier and see sales revenue grow.
Once you’ve identified the areas of your website that are of special concern, you need to understand what’s going on and why your website visitors are leaving at that junction. That’s where qualitative research comes into play and user experience is closely examined.
Perhaps you set up live sessions with users who fit your preferred customer personas but have never visited your website. You give them a task to complete, then observe as they attempt to carry out the assignment. User experience studies can provide invaluable feedback.
Do visitors tend to get confused on a certain page? Is your on-site search engine returning irrelevant or misleading data? Is the information you provide clear and helpful? Those qualitative discoveries can help you determine what needs to change to keep visitors engaged and help them find the product that best suits their needs.
Other qualitative tools allow you to observe real-time actions of any visitor you choose, map out exactly where clicks occur (and don’t occur) on your site, see how far visitors are scrolling down each page, get feedback from customers, and more.The key to making a good forecast is not in limiting yourself to quantitative information – Nate Silver @NateSilver538 Click To Tweet
Which is Better: Quantitative or Qualitative Research?
Quantitative research tells you at a high level what activity is occurring on your website. Qualitative research helps you view your ecommerce website through your customers’ eyes to tell you why those things are happening.
By combining the two correctly, you get a wider view and deeper understanding of customer actions. Armed with that knowledge, you are better prepared to develop effective A/B testing hypotheses that will get quicker results and increase conversions. That is the essence of conversion optimization.
Now that you have a better understanding of quantitative vs. qualitative research, you can pass this article on to your team, discuss how it might inform your conversion rate optimization strategy, and take action to put these principles to work.
When you hit a snag or have questions, call The Good. Helping you streamline your path to sales and boost bottom line revenue is what we do best.
Enjoying this article?
Subscribe to our newsletter to get more just like it, sent straight to your inbox every week.
About the Author
Jon MacDonald is founder and President of The Good, a conversion rate optimization firm that has achieved results for some of the largest online brands including Adobe, Nike, Xerox, Verizon, Intel and more. Jon regularly contributes content on conversion optimization to publications like Entrepreneur and Inc. He knows how to get visitors to take action.