The Power of Tree Testing in Optimization (With Expert Insights)

Creating an effective navigation hierarchy is a tricky process. Even when you follow best practices, there’s no guarantee that your categorization and labeling will make sense to your users.

How do you know what will make sense to your audience? You can test your navigation and general information architecture with a validation method like tree testing.

We spoke with two optimization experts to explore this valuable research method:

• Lexi Traylor, Digital Designer at The Good. She tackles intricate design challenges that optimize user engagement.
• Tristan Gamilis, Co-Founder and Chief Product Officer at Lyssna, a leading user experience research platform.

In this article, we’ll explain how you can use tree testing to assess whether users can find resources within your site or app.

What is Tree Testing?

Tree testing is a usability technique that helps you understand how users navigate through your website or app’s structure. It focuses on how well people can find information within a system.

“Tree tests may not be the first test that springs to mind when choosing a methodology for your research, but they can provide such a great insight into how easy it is for people to find information,” says Tristan Gamilis.

By stripping away visual elements and focusing solely on the structure (the “tree”), you can identify if the content organization makes sense or if users are getting lost.

Tree testing is often called “reverse card sorting.” While card sorting helps you understand how users would categorize information, tree testing evaluates how easily they can find items within an existing structure.

Both methods offer insights to improve UX and navigation, but tree testing gives you a clear view of how real users expect topics to be organized and clustered, making it a valuable step when designing better digital applications.

“Tree testing is particularly useful when you’re focusing on the information architecture (like a navigation structure of a site with several categories and subcategories),” says Lexi Traylor. “We use it when we want to evaluate how well users can find information within a proposed structure. It’s often used in the beginning process of creating a navigation structure, adding a lot of new categories, or overhauling an existing navigation structure.”

Qualitative vs. Quantitative Tree Testing

Tree testing can be done in two ways: qualitative and quantitative. The distinction lies in the type of data you collect and the feedback you aim to gain.

“Quantitative tree testing focuses on gathering numerical data to identify patterns and measure how well tasks are performed,” says Tristan Gamilis.

It involves collecting data from a larger group of users and gives you measurable results, such as success rates and completion times, which help identify broader trends and patterns. This approach helps validate the effectiveness of your site structure with data.

“Qualitative tree testing focuses more on understanding the ‘why’ behind user actions,” says Tristan. “By asking follow-up questions after your tree test tasks, you can add context and depth to your findings.” It’s great for understanding user behavior and uncovering deeper issues.

Both methods are useful, depending on whether you need detailed insights or broad patterns. A robust tree testing exercise should include some of both types.

What is Tree Testing Used For?

Tree testing evaluates whether users can successfully find specific items or sections within a site’s structure. It’s commonly used to test a website or app’s navigation, but it has other use cases.

For example, tree testing examines how easily someone can navigate to a particular product page or find a support section based solely on the category labels and hierarchy. This reveals whether the organization of content makes sense to users and if it aligns with how they think about the information.

Tree testing also tests your category labels and language. By removing the influence of design, it focuses on whether the terms used for menus and category labels are clear for users. If they consistently struggle with certain labels, it’s a sign that the wording might need to be adjusted.

Let’s look at some practical use cases of tree testing:

New Product Development

When launching a new product, tree testing helps ensure that the information architecture is clear and easy to navigate. It allows you to test the hierarchy before finalizing the design, which saves money, time, and resources before you invest in more development.

Website Redesigns

For redesigns, tree testing helps you identify whether changes to the navigation will improve or hinder usability. This ensures that users can still find what they need after a major update or restructuring.

Testing Information Architecture

Tree testing is useful for comparing different versions of a site’s structure. You can run tests to see which version is easier for users to navigate. This helps you make data-driven decisions on which architecture works best for your audience.

Optimizing Ecommerce Navigation

Tree testing helps ensure that product categories, filters, and subcategories make sense to users. By testing the hierarchy of products and categories, you can streamline the shopping experience and reduce friction.

Validating Taxonomy Changes

If you’re changing or introducing new categories or subcategories, tree testing helps validate whether these changes work for users. It allows you to test how easily users can adapt to a new taxonomy and whether they can still find information efficiently.

How to Run a Tree Test in 9 Steps (With a Case Study)

If any of these use cases sound like a good fit for your company and you’re ready to run one, it involves a few key steps. Let’s walk through the process. We’ll also look at an actual tree-testing study for one of our clients.

1. Identify What You Want to Test

Start by determining the goals of your tree test. Are you testing the usability of your navigation, evaluating labels, or understanding how users think about your product? Having a clear goal will help guide the rest of the process.

In our client test, we asked: How might we improve product discoverability in the mobile menu?

2. Build Your Tree Structure

Create a simplified version of your site’s information hierarchy (the “tree”). This should include the categories, subcategories, and any key items users would navigate through. The tree should represent your current or proposed site structure but without any visual design elements.

For our ecommerce client, we tested two variants:

• Variant A: Product categories are seen as the main level of navigation alongside account info, stores, etc.
• Variant B: Product categories are nested under “shop” to clean up navigation if users are looking for info rather than products

3. Create User Tasks

Develop tasks that represent common actions users would take on your site. These tasks should ask users to find specific information or complete certain goals. Tasks need to be realistic, simple, and reflect the way users would naturally interact with your site.

Here are some examples of tree-testing tasks you could ask users to complete:

• Find where you would go to update your account settings.
• Locate the page where you can learn about pricing options.
• Where would you go to find customer support or contact information?
• Where would you go to access the company’s privacy policy?
• Locate the section where you can download the app.
• Where would you find information about shipping and returns?
• Find the page that explains how to get started with a new service.
• Where would you go to view recent blog posts?
• Find the option to sign up for the company newsletter.

As you design tasks, it’s important to document the correct answers that correspond with the information in the tree. This helps you quickly calculate the access rate.

How do you come up with the tasks for a tree test? By identifying what we want to uncover and reverse-engineering the questions.

“I craft the scenarios based on how a user might interact with the website,” says Lexi Traylor. “I like to use language that doesn’t give away the answer in the task description. I also try to vary the difficulty of tasks to get a comprehensive view of the navigation’s effectiveness.”

4. Select the Tree Testing Method

Tree testing can be run in person or using remote software. The advantage of in-person testing (called moderated testing) is that you can ask questions that occur to you in the moment. You can also read people’s body language, facial expressions, or vocal expressions, which you don’t get in remote testing. The quantitative data could technically show the task was completed quickly and successfully, but there could be outliers with hesitations or subtle struggles that you wouldn’t have seen in remote testing.

Remote testing, however, is faster and easier to scale. Participants can take the test anywhere on their own time without your presence. Software tools like Lyssna also let you build a tree structure quickly by dragging and dropping blocks.

5. Recruit Participants

Find participants who match your target audience. “When we consider recruiting participants who match our target audience, we take an approach that starts with a bird’s-eye view (demographics) and then funnels further into a narrower scope (psychographics),” says Lexi.

“Since tree tests are more technical, we also like to recruit participants with at least average technical proficiency.”

You’ll need enough users to get reliable data, but the exact number will depend on your goals. Recruiting at least 20–50 participants is a good range for collecting meaningful insights, but more testers mean a higher confidence level.

If you have multiple customer segments, you’ll need to recruit participants from each segment. Make sure to compartmentalize them in your test so you learn how each segment behaves.

6. Consider a Pilot Study and Peer Review

It’s important to make sure everything works before you send your tree test to a large group of people. Two helpful ways to ensure this are with a pilot study or peer review.

In a pilot study, launch your tree test to a single user to make sure there are no issues with the tasks or instructions. For a peer review, have your team practice by going through the tasks as if they were real participants.

Both will help you catch any issues, refine instructions, and spot anything confusing or missing before the full test.

7. Run the Test

Once your tree test is set up and participants are recruited, run the test. Users will navigate through the tree structure to complete the tasks you’ve provided. The moderator or the software will track how long they take, where they click, and how successful they are in completing the tasks.

In our client study, we conducted a tree test with two cohorts of 40 participants recruited from our remote testing panel. Each cohort navigated through the information architecture (IA) of one of the two menu structures.

Participants were presented with multiple tasks, asking them to identify specific pages. They were also asked to rank the ease of each task and express frustrations or confusion.

8. Analyze the Results

After the test is complete, review the data. Look at task completion rates, the time taken to find items, and whether users followed a direct path or bounced between categories. This will help you understand where your structure works well and where it needs improvement.

“When analyzing tree test results, I look at success rates, paths, directness, and time-on-task,” says Lexi Traylor. “Lastly, I look at the qualitative insights, reviewing all the comments to identify patterns and trends in the responses. I group similar responses into buckets to identify patterns and use those patterns to inform any iterations made to the structure.”

• Success rate: The percentage of participants that completed the task.
• Directness: The percentage of participants who completed the task without hesitation and found the correct answer on their first attempt.
• Time: The time it took participants to finish a task.
• Path: The routes participants took before selecting an answer.

Let’s look at the results of our client study for the first task:

• Success rate: Users presented with Variant A successfully achieved their first task slightly more.
• Directness: Users presented with Variant A achieved their first task with more directness (less backtracking).
• Time: The participants presented with Variant B spent the shortest amount of time, on average, completing the first task.
• Path: Although Variant A had the highest success, it also had the lowest directness.

What Did We Learn?

Variant A: Participants often noted categorical confusion, label ambiguity, redundancy in naming, mismatch in location expectations, and excessive click depth for tasks. Some mentioned second-round tasks being easier due to already browsing the categories in the first tasks.

Variant B: Participants had feedback similar to that of Variant A Information Architecture, noting category confusion, label ambiguity, navigation and click depth, and general uncertainty on where to look.

9. Implement Site/App Improvements

Based on your analysis, adjust your site’s structure, labels, or navigation to address the issues uncovered during the test. You can then re-test to validate the changes before moving forward with your design.

In the case of our client study, our research suggests the two proposed navigation structures could be further optimized for improved clarity and product discoverability. A significant portion of participants had to rely on assumptions and multiple attempts to find the correct categories.

Here are the next steps we recommended to this client:

1. Clarify Category Labels
2. Cross-List Items
3. Improve Navigation Hierarchy
4. Generalize Parent Categories
5. Conduct Follow-Up Testing

The Limitations of Tree Testing

There is no one best kind of research or validation. There is a situation and use case for all methods, and knowing the honest limitations of each can help set your efforts up for success.

Based on the step-by-step process for running a tree test and the accompanying case study, you can see that running a test doesn’t always leave you with the answer to your question. Here are some other limitations to keep an eye on when conducting this research method.

• Potential for Bias: “Bias can happen when tasks are overly instructional or include labels that can unintentionally lead users to the right option,” says Tristan. “For instance, ‘Find the children’s bedroom section’ is too much information because the instructions match the category name. A better task would be, ‘Where would you purchase a bed for a child?’”
• Lacks Visual and Contextual Elements: Tree testing strips away all design elements and focuses solely on the structure of the site or app, which means the test doesn’t account for how users interact with the visual cues, page layout, or content on a live site. Real-world behavior can differ when users see the actual design.
• May Miss Real-World Complexities: In real-world scenarios, users navigate through your site with a combination of goals and distractions. Tree testing presents tasks in isolation, which doesn’t fully reflect how users would naturally behave when exploring the live site. This can result in slightly skewed insights that don’t capture the full complexity of user behavior. “When this happens, it’s a good idea to dive into other metrics to get a clearer picture of what’s really going on with the user experience,” says Tristan.
• Requires Thoughtful Task Design: The quality of a tree test depends on how well the tasks are designed. Poorly constructed tasks can lead to unclear or misleading results, so it’s crucial to create tasks that accurately reflect the way users would approach your product in real life.

Tree Test for a Better User Experience

Despite its limitations, we’re big fans of tree testing.

“Tree testing is great to validate information architecture early in the design process and nip navigational issues in the bud before investing time and resources into design and development,” says Lexi Traylor.

Whether you’re developing a new product, redesigning a site, or fine-tuning your current structure, tree testing helps you create an organized information architecture that aligns with your users’ mental models. It also helps you identify and fix issues that might frustrate users or hinder their ability to find important content.

As part of our Digital Experience Optimization Program™, we use tree testing alongside other testing and validation tools to help you build a better digital product. We bring a dedicated pod of experts in key optimization disciplines to help you unlock results that move the needle.

Contact us to learn more.