10 Steps to Choosing the Best Testing Platform for Your Website

It’s not long after you recognize the need for CRO and begin defining your optimization strategy, that the question of selecting a testing platform arises.

But with so many to choose from, it can be difficult to know where to begin.

Everyone should test and keep testing. A/B/n and multivariate testing are integral parts of every high-performing ecommerce website – but what is the best testing platform?

At The Good, we get asked that question all the time.

A/B/n and multivariate testing are integral parts of every high-performing ecommerce website – but what is the best testing platform? Click To Tweet

And, the answer? That’s what we’ll talk about in this article. We’ll give you 10 touch points you’ll want to consider before deciding on one of the most crucial tools in your ecommerce toolkit.

Here’s a preview: spending more doesn’t mean you’re getting the best solution. It’s entirely possible you’d be better served to budget less on your testing tools, not more.

Save Time and Money When You Choose the Best Testing Platform

Peak performers, no matter the business, sport, or discipline, have one thing in common: they are masters of the fundamentals. Typically, there are just a handful of factors important enough to rank as absolutely necessary, but those factors are – to a point – essential to your success.

For instance, the best basketball players are excellent at dribbling, passing, shooting, screening, rebounding, and defense. They don’t need to be showboats at any of those skills, but they must know how to perform them at a high level.

Ecommerce success also requires adherence to the fundamentals: traffic, products, testing, conversion optimization, selling, and follow-up. We find that many ecommerce managers, though, are focused only on traffic and sales. That’s a mistake.

Every ecommerce strategy should include a testing platform aimed at increasing the conversion rate. Testing and conversion rate optimization (CRO) are the two ends of the stick called “Sales.” You can get sales without paying attention to testing and CRO, but you’ll never get the sales you could be getting if you forego the basics. You’ll always be a junior varsity player.

We’ve published extensively about conversion rate optimization elsewhere. We’ve also talked considerably about A/B and multivariate testing.

For now, we’ll suppose you’re convinced of the value regular testing can deliver to your bottom line results, and that you’re on the lookout for the best testing platform you can find.

Before you invest in a testing solution, though, read through the suggestions we offer in the 10 steps below and ask others on your team do the same.

Knowing which testing platform is best for your ecommerce website can both save you money and earn you money. That’s a tough combination to beat!

Spending more on a testing tool doesn’t guarantee the best solution. Check these 10 steps to getting it right. Click To Tweet

The 10 Essential Steps to Choosing the Best Testing Tool or Service for Your Needs

Use this checklist as the backbone of your strategy for choosing a testing platform or service. Spending some time to run through these questions with your team now will keep you from expensive mistakes and lost time later.

1. What level of skill does my team have?

Do you have backend capability for coding complex tests, or will you need to rely on a turn-key WYSIWYG product that requires no additional coding? Does your team already possess significant experience with setting up and interpreting tests, or will this be a maiden voyage? Questions like these help you determine the user-friendliness factor you’re going to require. Don’t get something so complicated you can’t operate it. That can lead to a disaster.

2. What level of IT requirements and resources are necessary?

You looked at your team in the first step, now look at the tool you’re considering. Is it simple to customize, or will it take a developer’s touch for every change? You’ll quickly see there’s a trade-off here: More power normally means you’ll need more technical expertise. Ease of use normally means you’ll sacrifice capabilities. It may be that your best path is to begin with the training wheels version, then upgrade after you’re comfortable with the testing process and procedures.

Those who reach too high, in the beginning, often get bogged down and give up on creating a rigorous testing program at all. Keep the bar low enough to handle easily. Begin testing. Get results. Build enthusiasm. That way, you’ll be able to identify champions on your team who understand the vision and are willing to put in the hard work necessary to conduct more complex tests. Don’t overload people in the beginning. Set them up to succeed, then stoke the fire.

The wrong testing tool can bog down the work and frustrate your team. Use these 10 steps to find your CRO sweet spot. Click To Tweet

3. How much support is available?

The best testing platform for you may be the one easy to learn and well-supported. More advanced teams may be ready to dive deeper and need very little hand-holding. Don’t just take “fully supported” to mean you’ll get what you need, though. What some companies call “full support” is really just a customer service person who can look up information in the knowledge base and read it to you. That’s more aptly known as a frustrating waste of time.

Find out the level of support offered by the vendor, how long it takes to speak with someone who can answer technical questions and guide you through the process, and how much the support you need is going to cost. Premium support usually comes at premium prices. One more consideration: what days and hours is technical support available, and do those times match your probable needs well?

Enterprise solutions often come with an appointed support liaison who can get you out of the weeds fast. This is a good place to balance your actual needs with your allotted budget.

4. Will it integrate with your other tools?

Especially important here is the ability for your testing tool to ‘talk to’ your analytics tool. You don’t want to have to transfer data by hand. There should be seamless integration between the two. This, of course, assumes you have a robust analytics program in motion – or at least Google Analytics – and that your team knows how to make it hum.

It’s also helpful to employ a testing protocol that directly integrates with your shopping cart and checkout procedure, your email marketing program, and other parts of your ecommerce business. And don’t forget that you’ll want access to usability testing and other user feedback tools to identify problems with your site.

Another part of the puzzle is making sure your testing tool and integrations don’t significantly hamper the loading speed of your pages. That can defeat the purpose of testing and skew results so badly you can’t trust them.

5. How much targeting and segmentation is available?

The more capable your testing tool is of functions like knowing where the visitor came from (referral source), how many times the visitor has been to your website before, which pages are browsed and for how long – the list of capabilities can get quite granular – the more you’ll understand about your prospects, your customers, and their behavior.

As your testing protocols become more advanced, you can even find out what happens (for instance) if someone ready to leave your site is suddenly presented with a pop-up offering a limited-time BOGO offer or other incentive to purchase. This is the turn in the road where you’ll find testing champions getting excited about the depth of possibilities with a well-oiled testing strategy.

To maximize the benefits of A/B/n and multivariate testing, find champions on your team who get excited about the potential testing offers! Click To Tweet

6. Will I be using all the features this tool offers?

Before you get too excited about all the bells and whistles the tool you’re considering offers, stop to reflect back on the first three steps and ask this sobering question: Do we have a real need for and are we capable of utilizing all the features available?

We often find companies where the tools far exceed the ecommerce team’s ability, needs, or time to utilize. That’s a breeding ground for discouragement and despair.

Your company’s requirements, culture, and goals may not call for all the features the tool you’re considering offers – or you may be better served with something more robust. Getting the right fit between your needs and the tool’s capabilities is a balancing act of primary importance. Keeping an eye on the sweet spot here will help you weed out the wrong tools and quickly get to a shortlist of candidates.

Versatility is a big issue here. Will you be conducting basic A/B testing, A/B/n testing, or will you be running multivariate tests as well? Don’t make your decision based on which tool offers the most. Base it on which tool offers what you need.

7. What are the reporting capabilities of the tool?

Look closely at the reporting functions of every tool you consider. Are reports easy to generate, and do they provide all the data you need for further decision making? Or are they difficult to configure and will need a developer’s help to customize? This was mentioned in the fourth step, but it’s way worth repeating: will the tool integrate directly with your analytics tool? That gives you an instant leg-up on better insight and makes analysis of your tests and results a whole lot easier.

8. Are you best served by a client-side or server-side tool?

An example of a client-side (front-end) process is a testing platform rendered by Javascript in the visitor’s browser. Potential downsides to client-side operations are infrequent security risks and when used incorrectly they can be a drag on page loading speeds. On the upside, client-side tools can be less expensive and easier to operate.

Server-side tools live and run on the server hosting your website. They can be more robust, run faster, and offer more security than client-side tools. They can also be quite expensive and complicated.

These are generalities, though. Look closely at the tools you’re considering to weigh the pros and cons.

9. Should you choose a managed tool, a self-service tool, or a CRO agency?

You’re getting right down to the nitty-gritty now. There are three primary options to consider:

• You can contract with a testing service to do all the work for you. If you’re still scratching your head after working through the first nine steps, that can be an excellent option. For best results, choose a contractor that also excels at conversion rate optimization (CRO). Testing and CRO go hand-in-hand (testing is actually a tool in the CRO arsenal). It’s better to have one overseer for both. For guidance on this option, contact The Good.
• You can choose a tool that offers hands-on help with setting up tests and managing the process. In most cases, this will be more expensive than working with a capable CRO provider, and it will give you less control over the process. Many companies choose this option, thinking they’ll get trained up in how to operate the tool, but find out the provider has little patience for training. Like the plumber told the homeowner when asked about special pricing: “It’s $90 per hour if I do it by myself and $150 per hour if you help.”
• If you’ve worked through the first eight steps and team confidence is running high, you’re a candidate for the self-service option. Choose the tool best-suited to your own needs and abilities, learn how to use it, and start testing. Remember, though, that you’ll need to dedicate in-house resources on a perpetual basis to keep your testing program running. You’ll also need to cover the chores associated with reporting and analytics, and setting and iterating on your testing plan.

For most non-enterprise ecommerce websites, it’s more efficient, more effective, and more cost-conscious to find and work with a qualified CRO company for testing and conversion rate optimization. That’s the shortest, quickest path to a return on investment. Working through this checklist will help you decide the best path for your business.

Many enterprise-level companies run their own server-side testing, but still contract with CRO providers to help them fine-tune conversions and bring a new perspective to their testing roadmap. Getting a fresh set of eyes on your operation is sometimes the way to find stuck points that have been hiding in plain sight.

10. How do the cost comparisons pan out?

You’ve taken an inventory of your capabilities and your needs. You’ve looked at the available tools and services from several critical angles. You’ve culled out the unsuitable options, and you’ve honed in on the five or so top contenders. Now list the pricing beside each. Be sure to include the total cost of the tool and tech support, the cost load on internal resources that will be committed to testing. Don’t forget training time and implementation time.

Pay close attention to the structured tiers provided by the vendors. Is there a significant discount for paying yearly versus monthly? What if you’re not satisfied with the tool or service—what is the cancellation policy and potential penalty?

Will you be charged a flat rate, or will your cost vary according to the amount of traffic your site gets and the number of tests conducted? Are some tests more expensive than others? And if you are engaging testing service provider like The Good, can they provide you with discounted partner pricing on the platform?

How much help is available on the conversion rate optimization end? Are you on your own, or will you have 24/7/365 support? Will the vendor help you see the project completely through to the next testing cycle, or will you get the report only, with the interpretation and next test design completely left up to you?

Now’s the time to ask every lingering question and make sure you fully understand the ramifications of each choice.

Now You’re Ready to Select the Best Testing Platform

It’s quite common for an ecommerce brand to purchase a testing platform that’s far more advanced (and more expensive) than current needs require. That’s not only a drain on resources, but can add unnecessary confusion to the testing process.

When it’s managed correctly, testing is an eye-opening, profit-boosting experience. When those charged with establishing tests and leveraging them for growth are overwhelmed by the tools and the demands, testing can become a hardship and a nightmare.

You’ve just escaped that fate, though. You’ve worked through the ten steps and you know exactly what will work best for your company.

Good for you. If you get stuck, or if you just want to call someone up and brag about the spectacular results you’re getting from testing, give The Good a call.

Testing tools and protocols are something we’re always pleased to discuss.