The Iteration Engine: How Focused Consumer Testing Leads to Compounding Growth

What is it that makes it so easy to purchase from Amazon? How have they achieved total dominance in the ecommerce space? Three words: focused consumer testing leading to compound growth.

They’ve figured out exactly what their customers want and then given them exactly that. They know that before making a purchase, potential customers want to read reviews, see multiple pictures, compare to similar products, have questions answered, and be able to seamlessly purchase. They’ve put all those things together into a single page, enabling their customers to research and purchase in one step.

Amazon determined what their customers wanted through consumer focused testing. They haven’t guessed or relied on gut feeling. They haven’t randomly tested variations on pages and products. They’ve strategically and systematically used data and consumer testing to determine what converts most effectively.

As Chris DeRose told Investor’s Business Daily:

As Amazon pursues growth through small experiments, they have tried to use data to trump intuition. The company culture embraces experimentation and Bezos recently said at a shareholder meeting that 99% of all innovations at Amazon are incremental. The company relies on “Testing in Production,” or TiP, methods such as A/B testing and ramped deployment. By showing user group “A” a different version of a Web page than group “B”, the company can measure how long they stay on the site, how much they spend, etc.

Amazon analyzes, hypothesizes, tests, and then iterates. They don’t try for home runs or stratospheric growth. They take small bets, leading to incremental improvements, ultimately creating compound growth.

Bottom line: Amazon has used consumer focused testing and then incremental consumer experience changes to create an amazingly user-friendly ecommerce experience, leading to high conversions and massive long term growth.

If you want to experience ecommerce success, you need to pursue the same strategy as Amazon. Today, consumers are more empowered than ever and if your consumers have trouble engaging with your site, they’ll simply move along and won’t buy from it.

We’ve grown accustomed to seamless research and purchase experiences, which means that if you can’t keep up and don’t provide an exceptional consumer experience, they’ll buy somewhere else.

If you can’t keep up and don’t provide an exceptional consumer experience, your consumers will buy somewhere else. Click To Tweet

So how do you create that beautiful ecommerce experience? Focused consumer testing. Focused consumer testing is designed to allow for incremental improvements to the consumer experience, which in turn increases conversions. Over time, this creates compound growth that leads to big revenue increases. It’s not fast and it’s certainly not sexy. It probably won’t get you featured in marketing blogs. But it will generate results, which is what really matters.

What Do We Mean By Growth?

If we’re not careful, we can land on the wrong definition of growth. So let’s start by talking about what we don’t mean.

One Time Spikes

Every marketer dreams of checking the numbers one morning and seeing a massive, mountainous spike in revenue or conversions. Maybe you were unexpectedly featured on a large website or magazine. Maybe an influencer shared your product on social media. Or maybe you accidentally wrote a viral blog post that led to a flood of traffic. And while spikes may get your heart racing, they’re not ultimately what you want. Spikes don’t last. They don’t create long term growth. Remember Pets.com, Drugstore.com, eToys? They all had massive spikes and significant funding, and yet in the end they all collapsed. Spikes aren’t enough to keep an ecommerce business going.

While spikes may get your heart racing, they’re not ultimately what you want. Click To Tweet

Nir Eyal puts it this way:

I actually think that virality by itself is not enough. Having your product spread virally or some piece of content spread virally is not enough. We have seen a lot of businesses that have achieved viral growth but then they can’t sustain engagement and we call those businesses leaky buckets. How valuable it is when there is a bunch of users checking out your product and then they leave, they don’t stick around. So, virality on itself is not good enough. You have to have high engagement, you have to build a habit-forming product or else your users don’t stick around.

What you’re after is sustainable, incremental, long term growth. You need to pursue steady improvement month over month, not one-time jumps. The worst-case scenario is a massive spike followed by a catastrophic crash. You need to pursue steady improvement month over month, not one-time jumps. Click To Tweet

Unaligned Growth

It’s also easy to get excited about growth that’s not ultimately aligned with business goals. For example, if your PPC strategy is working properly, you’ll probably see a growth in site traffic. However, unless traffic is an explicit business goal, it doesn’t do any good unless the traffic is accompanied by increased conversions. Cameron Chapman puts it this way:

If you only focus on raising your visitor numbers, you’re eventually going to hit a ceiling, where you just can’t bring in any more visitors, either because you’ve reached the vast majority of people in your niche or because your niche is so large that you can’t afford to reach any more of them. What can you do to prevent hitting this ceiling? It’s simple: focus on your conversions first.

Your email list is another prime example. If your lead magnet and opt-ins are converting well, your email list will grow. But if your email list doesn’t lead to increased revenue, then it’s relatively useless. The bottom line is that growth always needs to be tied to your consumer and business goals. You’re not after vanity metrics, you’re after revenue growth.

The Moneyball Mentality

The book (and film) Moneyball, tells the true story of how the Oakland Athletics created a winning team with very little money and in a small market. How did they do it? By focusing on the right kind of growth. Traditionally, baseball teams evaluated players based on a few key metrics (batting average, homeruns, RBI’s, etc.) and a lot of gut feeling.

The Athletics determined that the bottom line goal that would lead to wins was simply getting players on base. And so instead of pursuing big hitters who cost a lot of money, they went after players who could get on base in non-sexy ways like walks and getting hit by pitches. The end result was that they began winning like crazy. The lesson? Small, repeated improvements (getting on base) leads to big results over time.

Small, repeated improvements (getting on base) leads to big results over time. Click To Tweet

The same applies to you as an ecommerce marketer. You need to have a moneyball mentality, going after the small, incremental improvements rather than the massive, sexy home runs.

Why Testing?

When making improvements which you think will lead to increased conversions and revenue, it can be tempting to move fast. You want to get the changes in place so that you can start reaping the benefits immediately. This is a mistake on multiple fronts. Proper testing is often a somewhat plodding process. Why is it plodding?

It Allows You To Gather, Synthesize, and Learn From The Data

If you don’t take the time to gather, synthesize, and learn from your data you’ll probably implement an ineffective or even harmful change to your site. For example, let’s say you have a product page that isn’t converting well. Before you can make any conclusions as to why, you need to know where the traffic is coming from, where visitors are clicking, what the bounce rate is, where visitor loops are occurring, and a host of other metrics. Once you gather the data, you can create intelligent hypotheses and then test them. Without this kind of data gathering, you may make a change that actually results in fewer conversions.

Improvements Are Tailored To Your Audience

When you spend every hour of every day in your ecommerce site, it can be easy to make improvements you think your audience will like but that aren’t really tailored to them. You may think that your audience will love the new layout of a page, only to discover that it causes your conversion rate to fall through the floor. Creating a data-based hypothesis and then testing it allows you to determine if your audience actually responds to change. Incremental testing also keeps you from making big mistakes which are difficult to recover from.

It’s Data Driven Rather Than Gut Driven

Gut feelings and intuition are often wrong. If your ecommerce strategy relies primarily on your gut feelings, you’ll never succeed. Structured testing takes you out of the fuzzy arena of intuition and into the realm of hard and fast data. Hunches are often wrong, but the data never lies.

It Shows You How Customers Want To Use Your Site

Friction is any portion of your website where visitors get hung up on the path to purchase. Friction can be caused by confusing navigation, poor design, technological issues, or a host of other problems. Testing allows you to see how your customers want to use your site and then eliminate the areas of friction. Through tools like heat maps, Google analytics, and user testing, you can see where people are clicking, looping back, and even abandoning your site. If they’re not using a portion of your site like you intended, that’s an area to be thoroughly tested and incrementally improved.

Testing allows you to see how your customers want to use your site and then eliminate the areas of friction. Click To Tweet

Why Must Testing Be Consumer Focused?

It’s incredibly easy to get wrapped up in company goals to the point where you essentially forget about your customers. For example, you may want to redesign a particular feature of your website to match modern design best practices. But if this design doesn’t ultimately further the goals of your customers, you’re shooting yourself in the foot. Testing must always be consumer focused. Why?

Internally Focused Companies Will Cease To Innovate

Yahoo! was once a giant in the internet world, and yet now is a shadow of it’s former self. Why? Because they failed to understand what their customers wanted, which then led to stagnation. Their list of missteps and mistakes is long and well documented. They tried to remain a web portal for too long, not realizing that customers were moving toward social media and search engines exclusively. They missed out on the chances to buy Google and Facebook. They had a series of 24 different mission statements over the last two decades.

If your company becomes too internally focused, you’ll stop innovating. You’ll be blind to what your customers really want and eventually get passed by other companies. You can’t exist apart from your customers. Don’t forget them.

If your company becomes to internally focused, you’ll stop innovating. Click To Tweet

Customers Are Already Doing Their Own Research

Customers now have access to all the information they need to make informed purchasing decisions. They can read objective reviews, watch video demonstrations, and get commentary via social media. Jack Lowinger highlights the need to tightly tailor ecommerce to the desires of consumers.

While ecommerce platforms can present consumers with a wide variety of offerings (online-only coupons, exclusive items, express shipping), there is a significant lack of focus on building personalized relationships with shoppers and providing the interactive experience that comes with in-store shopping.

You need to be ready to give your customers what they want. Consumer testing is how you get there.

Consumers Are Your Best Feedback Loop

Your consumers are the absolute best measure of what is and isn’t working. Best practices can be helpful but the ultimate insights come from seeing what your customers are doing. This reveals the true problems – the places they get stuck, confused, or lost. It shows the biggest pain points for your customers and guides you in your incremental improvements.

Your consumers are the absolute best measure of what is and isn’t working. Click To Tweet

Additionally, consumer feedback loops give you a sense of overall trends. If multiple customers are having the same issue, it deserves your full attention.

The Pieces of the Solution

How do you create an iteration engine that leads to compounding growth? It’s a relatively simple, four-step process.

Gather The Data

Before you can make incremental improvements, you need to gather the essential data. Your data is scattered across:

• Analytics
• Customer surveys
• Feedback boxes
• Heat maps
• User tests
• And a host of other sources.

The disparate data must be gathered into a central location so that it can be appropriately evaluated and analyzed.

Analyze The Data

Once all necessary data has been collected, it should be thoroughly analyzed. The analysis should look closely for:

• Trends, both positive and negative
• Areas of friction causing consumers to get confused or stuck
• Pages with unusually high bounce rates
• Pages with unexpectedly low or high conversion rates
• Spots on your site receiving high and low click rates
• Pages where carts tend to be abandoned
• Common feedback from users
• Customer questions that repeatedly arise
• And more

The goal of analysis is to be able to form cogent, insightful hypotheses about problems and how they can be fixed. You’re looking for positive trends which you can double down on and negative trends that can be nipped in the bud. Remember, opportunities for revenue are hidden in the data.

Create Hypotheses

Once you’ve identified both positive and negative trends, it’s time to form hypotheses around these trends. You are making predictions about changes that can be made based on the data. For example, if you see a high number of cart abandonments on a page, you may hypothesize that the navigation is leading to confusion. Or, if you see one page converting at twice the rate of another, you may hypothesize that it is due to the copy differences. These hypotheses are not blind, random guesses. They are intelligent predictions rooted firmly in data analysis.

Perform Tests

After forming hypotheses, they need to be tested. The only way you’ll be able to determine whether a new page layout leads to less cart abandonment is by implementing consumer focused tests. Have trained users walk through the new page layout, giving you a running commentary as they go. Show the new page to a small, yet statistically significant selection of visitors to your site. Comparing the new page to the control will give you your answer. Once you’ve determined the winner, you can move on to the next test. It’s crucial not to go to fast through these tests. The big temptation is to jump the gun and push changes into the wild too quickly. This can create massive problems. Martin Goodson nails it when he puts it like this:

So why is statistical power important? First, if you don’t calculate the sample size required up-front you might not run your experiment for long enough. Even if there is an uplift you won’t have enough data to be able to detect it. The experiment will likely be a waste of time. But underpowered experiments have a much more insidious effect. It’s that any winning variants you do see are likely to be ‘false positives’. They appear to generate an uplift but will not actually generate any increase in revenue. Let’s say our A/B test will take two months to gain good statistical power. If we decide to save time by only running the test for only two weeks instead, this test will be underpowered. The result? Almost two-thirds of winning tests will be completely bogus. Don’t be surprised if revenues stay flat or even go down after implementing a few tests like these.

Repeat

The key to incremental, compounding growth is iteration. Perform a test, determine the winner, repeat. This systematic process allows you to move top-down from the most important to least important problems. It also gives you a long term strategy for sustained improvement.

The key to incremental, compounding growth is iteration. Click To Tweet

Look For The Strategic Wins

When A/B testing, it’s tempting to look for the big wins. The loud, splashy, silver bullet wins. Generating $300 million in added revenue through a simple button change. The reality is, big wins rarely if ever happen, and businesses that shoot for the moon often waste time and money without achieving noticeable success.

A much better solution is to strive for small, strategic, constant wins. These wins aren’t sexy and they may not seem to make much of a difference at first, but they quickly add up. By getting qualitative feedback from your users, you can predict the most important areas for improvement on your site. You can then test those predictions through A/B tests, choose the winners, and then move on to the next most crucial area. It’s an iterative process that builds upon small wins to produce big results.

Your Pace Of Growth Is Determined By Your Pace Of Learning

The speed of your growth is dependent on how fast you learn. The quicker you can create an iterative, incremental growth process, the faster you’ll grow. The process begins with ensuring the UX of your site is optimized for customer experience. Once your site is fully optimized, you can significantly accelerate your growth process. You can gather and analyze the data, create hypotheses, test those hypotheses, implement changes, and then repeat the process.

Iterative consumer focused testing is the key to compounding growth. It allows you to identify strategic wins, make incremental improvements, and see a significant improvement over the long run. Don’t be lured into trying to find one-time massive growth hacks that will change your business overnight. Small, steady, and consistent is the key. It’s not the flashiest, but it’s the most successful. Just look at Amazon.