telegraph case study

How The Telegraph Improved Stakeholder Buy-In With Data-Backed Practices

“The case for change becomes a no-brainer. All the senior team needs to do is tick it off and say, ‘yes.’” - David Humber, Head of Conversion at The Telegraph

The Telegraph has distributed authoritative news for over 160 years. They were the first newspaper in Britain to produce an online website (the “Electronic Telegraph”), and, in 2017, they became the top quality UK news brand with over 25 million unique visitors.  Today, they’re an award-winning multimedia outlet  brand with 1,200+ employees managing print and online editorials on sports, business, finance, travel, and more. 

the telegraph homepage screenshot
The Telegraph’s homepage

With a large reader base, and ambitious acquisition growth plans, David Humber, Telegraph’s Head of Conversion, has no small task. A large part of his role involves converting anonymous visitors into engaged readers — and then subscription trialists. All within an era where studies indicate the UK’s trust in media is declining year-over-year.  

Yet, David is:

  • Drawing meaningful analyses out of large quantities of data
  • Testing and implementing design changes in days, not weeks
  • Recommending high-stakes tests with confidence and
  • Boosting subscriptions toward an ambitious company goal

Here’s how we’ve helped David and his team set new benchmarks for subscriber growth, despite being in one of the most challenging markets in recent history.

Chasing a North Star Metric

The Telegraph has an ambitious goal to increase readership. “We have a set business goal to deliver a million subscribers by next year,” David said, “and the whole business is focused on that.” To hit this north star metric, David and his team need to take inbound traffic on The Telegraph’s site and convert more of those visitors to subscribers. 

This conversion is simple in concept but complex in execution. Here are some notable challenges The Telegraph team faces:

  • Varied audiences mean no one-size-fits-all solution. The Telegraph draws a large UK and US reader base, plus audiences throughout the rest of the world. Readers are as varied in intent as they are in geography, too. “If someone comes through [search engine results], we know those users are probably going to be much cooler because they’ve seen a link somewhere,” David explained, “They might not even know who the Telegraph is.” Those “cool” readers may simply want access to a piece of content, compared to readers from social media, who are “warmed up” with brand context and may crave more engagement. 
  • Testing on a live site is expensive. Every live A/B test involves identifying a hypothesis, designing variants, coding the test, getting stakeholder signoff (on multiple levels), and more. These inputs mean each test risks time, resources, and revenue. So, running the wrong tests can incur extraordinary sunk costs. 

“Running a live test is no simple thing.”

David Humber, Head of Conversion at The Telegraph
  • Test result interpretation. The Telegraph consists of clever experts in cross-functional teams. It is rich in data but felt a gap in interpretation— how to question and formulate tests in order to tell a story in the results from all the rich data they collected.

To address these challenges, David could either hire new team members, invest heavily in professional development, or engage an external expert who already had deep expertise in digital experience optimization.

“I wanted to bring some additional eyes in quickly to help us scale up and determine what is best from a reader perspective.”

David Humber, Head of Conversion at The Telegraph

Finding a partner who melds with the team 

Initially there were reservations. What if we came in, started the conversation, then disappeared for weeks before delivering something that wasn’t workable? Or, just as frustrating, what if we stomped all over the hard work The Telegraph was doing? David’s team didn’t want to sign up for a consultants vs. internal teams tug-of-war — they wanted an embedded partner. 

“The Good came in as a partner and basically joined the team. They added value to what was already here in terms of staff expertise.”

David Humber, Head of Conversion at The Telegraph

Right away, Natalie Thomas, our Director of Strategy, alleviated those concerns. “They’re fully integrated into the team, and we’re all working on it together,” David affirms. “We’ve got teams from different areas of the business, and Natalie’s team actually became part of that.” By meeting with David’s team on the front end and maintaining frequent touchpoints throughout each project, our team kept a pulse on The Telegraph’s goals and priorities. This perspective helped us make data-based recommendations within The Telegraph’s business model. 

Improving the paywall experience and subscriber quality

The first stop in The Telegraph + The Good’s partnership was improving The Telegraph’s paywall-ed experience (showing a “subscribe to read” cta on an article).

The Telegraph free trial unlock prompt
The Telegraph’s free trial checkout experience as of March 2022

In particular, The Telegraph wanted to:

  • Simplify their governance of paywalls: The Telegraph’s various overlays were complex to manage. 
  • Improve user experience for readers: The user’s paywall journey wasn’t optimized by context (referral channel or number of visits) and could feel disjointed. “The user actually had to work quite hard to find all the information they needed,” David explained. For example, site visitors would sometimes see seemingly competing offers on the same screen.  

At first glance, this project seems like a major conflict of interest. A user doesn’t want to land on The Telegraph’s site and get slammed with paywalls. But, The Telegraph needs paywalls to drive signup volume. How can user experience and business goals possibly coexist here?

Yet, over a decade of testing and experience with ecommerce brands has shown us something surprising. These goals can do more than coexist; they can work hand-in-hand. The trick is focusing on the user’s experience.  

Diagnosing visitor pain points & improving user experience

Conversions and user experience have a one-way relationship. If you focus on improving experience, conversions follow. So do other metrics like customer satisfaction, retention, and referrals. (But the opposite — focusing on conversions to drive experience — doesn’t work. Hence, a one-way relationship.) 

So, the first step in improving The Telegraph’s paywall experience was diagnosing visitors’ pain points. Senior Strategist, Maggie Paveza, kicked off a Digital Experience Optimization Audit™ by reviewing quantitative data available through The Telegraph’s internal dashboards and reporting systems. 

Initial findings included:

  • Behavioral differences on desktop vs. mobile
  • Conversion rate differences on old vs. new site content
  • Content preferences for registered vs. anonymous visitors 

Strategists also reviewed the visitors’ qualitative experiences using user testing, session recordings, heatmaps, heuristic analysis, and over-the-shoulder research.

User testing screenshot
An example of our UserTesting in progress for The Telegraph

They also compiled an in-depth competitive analysis and drew on our firm’s decade-plus experience to identify and prioritize critical improvements. 

“If you’re purely looking at statistical data, in terms of website visits, etc., without interpretation…the numbers might tell you something that could lead you down the wrong road.”

David Humber, Head of Conversion at The Telegraph

Our strategists summarized these improvements in a roadmap — a visual depiction of where visitors were getting lost or frustrated in their journey with The Telegraph, combined with our recommended approach to fixing each one. Suggestions included messaging, layout, paywall placement, timing, and other paywall journey improvements leading to the subscribe button. 

“The decisions are made based on science, rather than it just being a gut-feel.”

David Humber, Head of Conversion at The Telegraph

Making data-backed changes in days, not weeks

Once we had the research plan and were armed with a strategic roadmap, it was on to the next phase of the Digital Experience Optimization Program™. We took hypothesized improvements and tested them with The Telegraph’s ideal audience to confirm they would move the needle before investing in implementation.

And it happened fast. Through a process called rapid testing, we worked with The Telegraph to design, test, and decide on the first phase of implementations in a matter of days.

“You do less spinning of the wheels because you’re having somebody come in that’s got this additional expertise as their bread and butter.”

David Humber, Head of Conversion at The Telegraph
Find out what stands between your company and digital excellence with a custom 5-Factors Scorecard™.

Rapid testing refers to a collection of tactics we use to get quick feedback for operational decisions. This type of testing helps teams make agile decisions around design, copy, and other site elements. It’s ideal in scenarios where: 

  • A full A/B testing campaign is impractical in terms of budget, time, or resources
  • A company needs to make fast changes during a seasonal time crunch
  • A team wants to make interim changes to a broken site/process to bide more time for a complete conversion program 

While rapid testing uses less qualified participants and smaller sample sizes than traditional A/B testing, the tradeoff is exponentially faster results. Rapid testing also delivers value by:

  • Limiting politics of A/B testing (what to A/B test is informed by rapid test data vs. executive opinions)
  • Narrowing down many ideas to the best ones

For example, one rapid test we ran for The Telegraph assessed site banner color and layout. When shown two banner variants, visitors had a clear preference — 78% of participants found content easier to read against a yellow background. Recall tests also showed visitors were more likely to remember key details in this variant as well, further supporting it as the preferable option. 

Banner color testing example
Two banner variants we ran for The Telegraph, the yellow was the winner

We ran over 20 similar tests to assess cookie notification placement and design, desktop and mobile paywall presentation, brand headlines, offer messaging, and more. We did this in just a few weeks, something that would have been impossible to accomplish via a/b testing due to resource constraints. Overall, identifying small wins in numerous places added up to a significant impact for The Telegraph. 

Focusing on the quality of free-to-paid conversions

While comparing numbers pre and post-program requires factoring in market impacts and overall business changes, conversion rates did increase due to our joint efforts. An increased percentage of readers upgraded from a free subscription to a paid one. And, just as significantly, the quality of those free-to-paid conversions improved, too. 

“We had fewer people cancel and more people staying on the files. So, overall revenue for the business went up by a higher amount than volume did.”

David Humber, Head of Conversion at The Telegraph

Formerly, some readers would sign up for a subscription, read a gated article, and then cancel before their credit card was billed. Through improved experiences, The Telegraph and The Good reduced this behavior by 30%.

It’s worth noting many brands attempt to improve retention by using deceptive design. They create experiences that make it difficult to cancel, which results in lower churn, but also leaves customers feeling frustrated or even angry.

That doesn’t align with our philosophy, nor the Telegraph’s, so instead we focused on improving subscriber quality and journeys. 

Improving onboarding for customers

Bettering paywalls was an excellent first step, but it was only one part of a user’s subscription journey. Keep in mind the initial paywall project sat in the context of big business goals (one million subscribers) and complex inputs (varied visitors and journeys). 

To get to one million subscribers, The Telegraph knew there was more work to do. So, the next project we partnered on, this time with their retention team, was improving the overall onboarding journey

Quick clarification here: 

  • A paywall exists to encourage a website visitor to purchase a subscription or start a free trial. A paywall maximizes conversions. 
  • Onboarding exists to familiarize a visitor with the brand and product immediately after they buy a subscription or start a free trial. Onboarding maximizes retention. 

For The Telegraph’s customers, onboarding begins the moment of sign up, right after a reader opts in for a free trial. 

The Telegraph free trial payment box screenshot
The Telegraph’s free trial checkout experience as of March 2022

A successful Telegraph onboarding experience relies on key actions (sometimes called “Activiation Events,” “North Star Metrics,” or an “Aha Moment”). If subscribers take these actions, they’re more likely to stay subscribed long term. The most famous example of this comes from Facebook. They learned that if a new user was able to connect with at least 10 friends within one week, they were much more likely to stick around and remain active long term. 

Telegraph has similar indicators of what separates a “tire kicker” from a “true fan.” Yet, only a subset of subscribers were hitting these indicators — our job in collaboration with the retention team, was to help more people reach that “Aha moment.”

Diagnosing problems before prescribing solutions 

As with the first project, we started with a diagnosis — a thorough review of data. This data review informed key questions such as:

  • What’s the best place for a new subscriber to onboard? 
  • What’s the best language, design, and number of actions required for onboarding?
  • Why aren’t visitors completing this key action? 

Next, we started forming hypotheses with the help of heat maps and heuristics. Among other findings, we noted visitors were:

  • Click-click-clicking through onboarding patterns to skip to what they wanted
  • Overlooking much of the onboarding content
  • Missing important celebratory milestones 
Newsletter signups heatmap
Heat map research results from our work with The Telegraph helped us identify user patterns

We also invested in a competitive analysis, which looked at onboarding flows. And we didn’t just look at other publications — we researched other subscription-based purchasing and onboarding experiences, such as Netflix, to identify insights. This cross-industry review honed our strategists’ eyes for great experiences and inspired rapid testing options. 

Pricing plans
Our research explored onboarding flows from big brands, here’s an example of plan comparison from Netflix

Overall, the data review, qualitative testing, and analysis revealed a substantial number of improvements The Telegraph could make. The next question was, out of all the possible roads we could take, where do both teams go from here? 

Ideal customer experience standards to guide testing 

To align both teams around what The Telegraph’s onboarding journey should look like, our strategists created ideal customer experience standards. These standards described what a phenomenal experience should feel like to a user. For example:

Every action within the onboarding journey should feel like a step towards sign-up/ account registration, rather than an advertisement.

This and ten other standards set the tone (and vocabulary) for onboarding improvements. Potential improvements were vetted against the standards and then assessed with rapid testing.

Rapid testing in this project looked like:

  • Asking, “Which of these variants will be most effective at communicating the value of downloading the mobile app?”
  • Gathering data from a small group of visitors, through various user testing methods (such as card sorting)
  • Bringing findings to The Telegraph team, who implemented a live site change in days

“The main benefit to Dave and his team,” Natalie explains, “is they were able to hear some new research, take a recommendation, turn it into a design within a week  — sometimes even three days.”

From David’s perspective, the audit and rapid testing allowed him to move quickly through internal pitches and approvals. “Bringing in The Good allowed us to leapfrog a lot of that process,” David explained. He could confidently say whether an idea had merit or would be problematic. This decisiveness helped him gatekeep The Telegraph’s focus and efforts. “It just speeds up the whole process, and there’s a lot of value in that,” he affirmed. 

“When we get to the live test, we’ve already validated and done a lot of the work.”

Improving paywall conversions, subscriber quality, and the onboarding experience were notable results for The Telegraph. But those aren’t the only improvements that matter. 

Shifting the internal testing mindset  

The Telegraph has always been customer-centric, but things like complex business operations, market volatility, and changing consumer preferences sometimes make it difficult to keep the customer at the heart of every design decision. Now, they have the tools and processes to see changes from the customer’s perspective. 

“Now a lot of the decisions are using a science layer — data, research, user testing — to get to the answers.”

David Humber, Head of Conversion at The Telegraph

Before a change goes live, David’s team asks questions like, “how do we get this in front of readers?” and “does it make sense to site visitors?” Among other benefits, this line of questioning has a gatekeeping function — it halts changes that would otherwise go out and then be rolled back.  

Seeing the customer’s experience with third-party assistance

One of David and his team’s biggest strengths is their insight into The Telegraph, journalism, and their subscription ecosystem. But our strengths are often our weaknesses, too. Telegraph’s correlating weakness is this: a close-up perspective means it’s hard to take a step back and see the customer’s journey with fresh eyes. 

“It’s useful to have an external set of eyes to identify opportunities we may have missed being in the weeds”

David Humber, Head of Conversion at The Telegraph

That’s where a third-party partner like The Good can add value. “It’s clearer to you,” David told us, “because you’ve got expertise doing it elsewhere.” We can see from the customer’s perspective because we’ve practiced doing so for over a decade with dozens of ecommerce and subscription brands. 

“One of the reasons why someone like The Good can add value is because they’ve done this with other people, and they’re experts.”

David Humber, Head of Conversion at The Telegraph

Increasing efficiency of internal teams 

Many companies think that working with an external partner means slower communication and extended timelines, but our work with Telegraph has actually allowed them to improve their internal efficiency.

Instead of spinning their wheels, The Telegraph now harnesses a powerful flywheel: diagnosing pain points → fuels confident tests → which deliver valuable results → which fuel additional testing. Over time, these rolling wins add up to significant conversion gains. 

“Having The Good come in allows you to validate your hypotheses, your thinking, and get an early read on whether you’re going in the right direction or not.”

David Humber, Head of Conversion at The Telegraph

Winning testing bets with high confidence and fast approvals

Running a live test, as David said, is no simple thing. And to many marketing managers, live tests feel like shaky bets. You put time and resources on the line, hoping for a return. With our services, David took hope out of the equation. He didn’t cross his fingers and hope the tests would make an impact — he knew.  

“The value of having The Good in to help us,” he explained, “is when we get to the live test, we’ve already validated and done a lot of work…we’re fairly confident we’re going to get a positive uplift.” Uplift, or an unexpected learning The Telegraph can tweak and adjust. Either way, the test delivers a valuable return — it’s a sure bet. 

“The case for change becomes a no-brainer. All the senior team needs to do is tick it off and say, ‘yes.’”

David Humber, Head of Conversion at The Telegraph

Now It’s Your Turn

We harness user insights and unlock digital improvements beyond your conversion rate.

Let’s talk about putting digital experience optimization to work for you.

Jon MacDonald smiling at the camera for The Good

About the Author

Jon MacDonald

Jon MacDonald is founder and President of The Good, a digital experience optimization firm that has achieved results for some of the largest companies including Adobe, Nike, Xerox, Verizon, Intel and more. Jon regularly contributes to publications like Entrepreneur and Inc.