The Telegraph has distributed authoritative news for over 160 years. They were the first newspaper in Britain to produce an online website (the “Electronic Telegraph”), and, in 2017, they became the top quality UK news brand with over 25 million unique visitors. Today, they’re an award-winning multimedia outlet brand with 1,200+ employees managing print and online editorials on sports, business, finance, travel, and more.
With a large reader base, and ambitious acquisition growth plans, David Humber, Telegraph’s Head of Conversion, has no small task. A large part of his role involves converting anonymous visitors into engaged readers — and then subscription trialists. All within an era where studies indicate the UK’s trust in media is declining year-over-year.
Yet, David is:
- Drawing meaningful analyses out of large quantities of data
- Testing and implementing design changes in days, not weeks
- Recommending high-stakes tests with confidence and
- Boosting subscriptions toward an ambitious company goal
Here’s how we’ve helped David and his team set new benchmarks for subscriber growth, despite being in one of the most challenging markets in recent history.
Chasing a North Star Metric
The Telegraph has an ambitious goal to increase readership. “We have a set business goal to deliver a million subscribers by 2023,” David said, “and the whole business is focused on that.” To hit this north star metric, David and his team need to take inbound traffic on The Telegraph’s site and convert more of those visitors to subscribers.
This conversion is simple in concept but complex in execution. Here are some notable challenges The Telegraph team faces:
- Varied audiences mean no one-size-fits-all solution. The Telegraph draws a large UK and US reader base, plus audiences throughout the rest of the world. Readers are as varied in intent as they are in geography, too. “If someone comes through [search engine results], we know those users are probably going to be much cooler because they’ve seen a link somewhere,” David explained, “They might not even know who the Telegraph is.” Those “cool” readers may simply want access to a piece of content, compared to readers from social media, who are “warmed up” with brand context and may crave more engagement.
- Testing on a live site is expensive. Every live A/B test involves identifying a hypothesis, designing variants, coding the test, getting stakeholder signoff (on multiple levels), and more. These inputs mean each test risks time, resources, and revenue. So, running the wrong tests can incur extraordinary sunk costs.
“Running a live test is no simple thing.”David Humber, Head of Conversion at The Telegraph
- Test result interpretation. The Telegraph consists of clever experts in cross-functional teams. It is rich in data but felt a gap in interpretation— how to question and formulate tests in order to tell a story in the results from all the rich data they collected.
To address these challenges, David could either hire new team members, invest heavily in professional development, or engage an external expert who already had deep expertise in Conversion Rate Optimization.
“I wanted to bring some additional eyes in quickly to help us scale up and determine what is best from a reader perspective.”David Humber, Head of Conversion at The Telegraph
Finding a partner who melds with the team
Initially there were reservations. What if we came in, started the conversation, then disappeared for weeks before delivering something that wasn’t workable? Or, just as frustrating, what if we stomped all over the hard work The Telegraph was doing? David’s team didn’t want to sign up for a consultants vs. internal teams tug-of-war — they wanted an embedded partner.
“The Good came in as a partner and basically joined the team. They added value to what was already here in terms of staff expertise.”David Humber, Head of Conversion at The Telegraph
Right away, Natalie Thomas, our Director of Strategy, alleviated those concerns. “They’re fully integrated into the team, and we’re all working on it together,” David affirms. “We’ve got teams from different areas of the business, and Natalie’s team actually became part of that.” By meeting with David’s team on the front end and maintaining frequent touchpoints throughout each project, our team kept a pulse on The Telegraph’s goals and priorities. This perspective helped us make data-based recommendations within The Telegraph’s business model.
Improving paywall conversions and subscriber quality
The first stop in The Telegraph + The Good’s partnership was improving The Telegraph’s paywall-ed experience (showing a “subscribe to read” cta on an article).
In particular, The Telegraph wanted to:
- Simplify their governance of paywalls: The Telegraph’s various overlays were complex to manage.
- Improve user experience for readers: The user’s paywall journey wasn’t optimized by context (referral channel or number of visits) and could feel disjointed. “The user actually had to work quite hard to find all the information they needed,” David explained. For example, site visitors would sometimes see seemingly competing offers on the same screen.
At first glance, this project seems like a major conflict of interest. A user doesn’t want to land on The Telegraph’s site and get slammed with paywalls. But, The Telegraph needs paywalls to drive signup volume. How can user experience and business goals possibly coexist here?
Yet, over a decade of testing and experience with ecommerce brands has shown us something surprising. These goals can do more than coexist; they can work hand-in-hand. The trick is focusing on the user’s experience.
Using better user experience to drive improved conversion rates
Conversions and user experience have a one-way relationship. If you focus on improving experience, conversions follow. So do other metrics like customer satisfaction, retention, and referrals. (But the opposite — focusing on conversions to drive experience — doesn’t work. Hence, a one-way relationship.)
So, the first step in improving The Telegraph’s paywall experience was diagnosing visitors’ pain points. Senior CRO Strategist, Maggie Paveza, kicked off a Comprehensive Conversion Audit by reviewing quantitative data available through The Telegraph’s internal dashboards and reporting systems.
Initial findings included:
- Behavioral differences on desktop vs. mobile
- Conversion rate differences on old vs. new site content
- Content preferences for registered vs. anonymous visitors
Strategists also reviewed the visitors’ qualitative experiences using user testing, session recordings, heatmaps, heuristic analysis, and over-the-shoulder research.
They also compiled an in-depth competitive analysis and drew on our firm’s decade-plus experience to identify and prioritize critical improvements.
“If you’re purely looking at statistical data, in terms of website visits, etc., without interpretation…the numbers might tell you something that could lead you down the wrong road.”David Humber, Head of Conversion at The Telegraph
Our strategists summarized these improvements in a roadmap — a visual depiction of where visitors were getting lost or frustrated in their journey with The Telegraph, combined with our recommended approach to fixing each one. Suggestions included messaging, layout, paywall placement, timing, and other paywall journey improvements leading to the subscribe button.
“The decisions are made based on science, rather than it just being a gut-feel.”David Humber, Head of Conversion at The Telegraph
Making data-backed changes in days, not weeks
Once we had a roadmap, it was on to testing. Through a process called rapid testing, we worked with The Telegraph to design, test, and decide on implementations in a matter of days.
“You do less spinning of the wheels because you’re having somebody come in that’s got this additional expertise as their bread and butter.”David Humber, Head of Conversion at The Telegraph
Rapid testing refers to a collection of tactics we use to get quick feedback for operational decisions. This type of testing helps teams make agile decisions around design, copy, and other site elements. It’s ideal in scenarios where:
- A full A/B testing campaign is impractical in terms of budget, time, or resources
- A company needs to make fast changes during a seasonal time crunch
- A team wants to make interim changes to a broken site/process to bide more time for a complete conversion program
While rapid testing uses less qualified participants and smaller sample sizes than traditional A/B testing, the tradeoff is exponentially faster results. Rapid testing also delivers value by:
- Limiting politics of A/B testing (what to A/B test is informed by rapid test data vs. executive opinions)
- Narrowing down many ideas to the best ones
For example, one rapid test we ran for The Telegraph assessed site banner color and layout. When shown two banner variants, visitors had a clear preference — 78% of participants found content easier to read against a yellow background. Recall tests also showed visitors were more likely to remember key details in this variant as well, further supporting it as the preferable option.
We ran over 20 similar tests to assess cookie notification placement and design, desktop and mobile paywall presentation, brand headlines, offer messaging, and more. We did this in just a few weeks, something that would have been impossible to accomplish via a/b testing due to resource constraints. Overall, identifying small wins in numerous places added up to a significant impact for The Telegraph.
Reducing same day cancellations by 30% while improving subscriber quality and increasing acquisition rates
While comparing numbers pre and post-audit requires factoring in COVID impacts and overall business changes, conversion rates did increase due to our joint efforts. An increased percentage of readers upgraded from a free subscription to a paid one. And, just as significantly, the quality of those free-to-paid conversions improved, too.
“We had fewer people cancel and more people staying on the files. So, overall revenue for the business went up by a higher amount than volume did.”David Humber, Head of Conversion at The Telegraph
Formerly, some readers would sign up for a subscription, read a gated article, and then cancel before their credit card was billed. Through improved experiences, The Telegraph and The Good reduced this behavior by 30%.
It’s worth noting many brands attempt to improve retention by using deceptive design. They create experiences that make it difficult to cancel, which results in lower churn, but also leaves customers feeling frustrated or even angry.
That doesn’t align with our philosophy, nor the Telegraph’s, so instead we focused on improving subscriber quality and journeys.
Improving onboarding for customers
Bettering paywalls was an excellent first step, but it was only one part of a user’s subscription journey. Keep in mind the initial paywall project sat in the context of big business goals (one million subscribers) and complex inputs (varied visitors and journeys).
To get to one million subscribers, The Telegraph knew there was more work to do. So, the next project we partnered on, this time with their retention team, was improving the overall onboarding journey.
Quick clarification here:
- A paywall exists to encourage a website visitor to purchase a subscription or start a free trial. A paywall maximizes conversions.
- Onboarding exists to familiarize a visitor with the brand and product immediately after they buy a subscription or start a free trial. Onboarding maximizes retention.
For The Telegraph’s customers, onboarding begins the moment of sign up, right after a reader opts in for a free trial.
A successful Telegraph onboarding experience relies on key actions (sometimes called “Activiation Events,” “North Star Metrics,” or an “Aha Moment”). If subscribers take these actions, they’re more likely to stay subscribed long term. The most famous example of this comes from Facebook. They learned that if a new user was able to connect with at least 10 friends within one week, they were much more likely to stick around and remain active long term.
Telegraph has similar indicators of what separates a “tire kicker” from a “true fan.” Yet, only a subset of subscribers were hitting these indicators — our job in collaboration with the retention team, was to help more people reach that “Aha moment.”
Diagnosing problems before prescribing solutions
As with the first project, we started with a diagnosis — a thorough review of data. This data review informed key questions such as:
- What’s the best place for a new subscriber to onboard?
- What’s the best language, design, and number of actions required for onboarding?
- Why aren’t visitors completing this key action?
Next, we started forming hypotheses with the help of heatmaps and heuristics. Among other findings, we noted visitors were:
- Click-click-clicking through onboarding patterns to skip to what they wanted
- Overlooking much of the onboarding content
- Missing important celebratory milestones
We also invested in a competitive analysis, which looked at onboarding flows. And we didn’t just look at other publications — we researched other subscription-based purchasing and onboarding experiences, such as Netflix, to identify insights. This cross-industry review honed our strategists’ eyes for great experiences and inspired rapid testing options.
Overall, the data review, qualitative testing, and analysis revealed a substantial number of improvements The Telegraph could make. The next question was, out of all the possible roads we could take, where do both teams go from here?
Ideal customer experience standards to guide testing
To align both teams around what The Telegraph’s onboarding journey should look like, our strategists created ideal customer experience standards. These standards described what a phenomenal experience should feel like to a user. For example:
Every action within the onboarding journey should feel like a step towards sign-up/ account registration, rather than an advertisement.
This and ten other standards set the tone (and vocabulary) for onboarding improvements. Potential improvements were vetted against the standards and then assessed with rapid testing.
Rapid testing in this project looked like:
- Asking, “Which of these variants will be most effective at communicating the value of downloading the mobile app?”
- Gathering data from a small group of visitors, through various user testing methods (such as card sorting)
- Bringing findings to The Telegraph team, who implemented a live site change in days
“The main benefit to Dave and his team,” Natalie explains, “is they were able to hear some new research, take a recommendation, turn it into a design within a week — sometimes even three days.”
From David’s perspective, the audit and rapid testing allowed him to move quickly through internal pitches and approvals. “Bringing in The Good allowed us to leapfrog a lot of that process,” David explained. He could confidently say whether an idea had merit or would be problematic. This decisiveness helped him gatekeep The Telegraph’s focus and efforts. “It just speeds up the whole process, and there’s a lot of value in that,” he affirmed.
“When we get to the live test, we’ve already validated and done a lot of the work.”
Improving paywall conversions, subscriber quality, and the onboarding experience were notable results for The Telegraph. But those aren’t the only improvements that matter.
Shifting the internal testing mindset
The Telegraph has always been customer-centric, but things like complex business operations, market volatility, and changing consumer preferences sometimes make it difficult to keep the customer at the heart of every design decision. Now, they have the tools and processes to see changes from the customer’s perspective.
“Now a lot of the decisions are using a science layer — data, research, user testing — to get to the answers.”David Humber, Head of Conversion at The Telegraph
Before a change goes live, David’s team asks questions like, “how do we get this in front of readers?” and “does it make sense to site visitors?” Among other benefits, this line of questioning has a gatekeeping function — it halts changes that would otherwise go out and then be rolled back.
Seeing the customer’s experience with third-party assistance
One of David and his team’s biggest strengths is their insight into The Telegraph, journalism, and their subscription ecosystem. But our strengths are often our weaknesses, too. Telegraph’s correlating weakness is this: a close-up perspective means it’s hard to take a step back and see the customer’s journey with fresh eyes.
“It’s useful to have an external set of eyes to identify opportunities we may have missed being in the weeds”David Humber, Head of Conversion at The Telegraph
That’s where a third-party partner like The Good can add value. “It’s clearer to you,” David told us, “because you’ve got expertise doing it elsewhere.” We can see from the customer’s perspective because we’ve practiced doing so for over a decade with dozens of ecommerce and subscription brands.
“One of the reasons why someone like The Good can add value is because they’ve done this with other people, and they’re experts.”David Humber, Head of Conversion at The Telegraph
Increasing efficiency of internal teams
Many companies think that working with an external partner means slower communication and extended timelines, but our work with Telegraph has actually allowed them to improve their internal efficiency.
Instead of spinning their wheels, The Telegraph now harnesses a powerful flywheel: diagnosing pain points → fuels confident tests → which deliver valuable results → which fuel additional testing. Over time, these rolling wins add up to significant conversion gains.
“Having The Good come in allows you to validate your hypotheses, your thinking, and get an early read on whether you’re going in the right direction or not.”David Humber, Head of Conversion at The Telegraph
Winning testing bets with high confidence and fast approvals
Running a live test, as David said, is no simple thing. And to many marketing managers, live tests feel like shaky bets. You put time and resources on the line, hoping for a return. With our services, David took hope out of the equation. He didn’t cross his fingers and hope the tests would make an impact — he knew.
“The value of having The Good in to help us,” he explained, “is when we get to the live test, we’ve already validated and done a lot of work…we’re fairly confident we’re going to get a positive uplift.” Uplift, or an unexpected learning The Telegraph can tweak and adjust. Either way, the test delivers a valuable return — it’s a sure bet.
“The case for change becomes a no-brainer. All the senior team needs to do is tick it off and say, ‘yes.’”David Humber, Head of Conversion at The Telegraph
About the Author
Jon MacDonald is founder and President of The Good, a conversion rate optimization firm that has achieved results for some of the largest online brands including Adobe, Nike, Xerox, Verizon, Intel and more. Jon regularly contributes content on conversion optimization to publications like Entrepreneur and Inc. He knows how to get visitors to take action.