Author: Jandira Neto
Traditionally, SEOs have relied on advice and recommendations from Google in hopes that it would improve their organic traffic. Even for brands and agencies that rigorously follow these on-page SEO guidelines, the uncertainty can be tough to navigate, and sometimes you don’t even know where to start.
With SEO A/B testing, you can take the guesswork out of website changes and make informed, data-backed decisions. By designing an experimentation program, you can run small tests that insulate you from risk while identifying valuable opportunities to optimize your site.
In this article, I will show you how to build a testing strategy that enables you to run A/B tests on your meta tags so that you can enhance your competitiveness in the search results and bring in more organic traffic.
Table of contents:
A/B testing: The fundamentals
For those newer to A/B testing, here’s some crucial context that will guide you throughout the process and help you better explain it to teammates and stakeholders.
What is A/B testing for SEO?
In the context of SEO, A/B testing is a methodology in which you compare the impact a site change has on two statistically similar web pages.
A/B testing is not new to the world of marketing. American advertiser and author Claude Hopkins pioneered this method by conducting the first documented A/B test, in which he looked at the rate of return to measure the impact of his experiment with two distinct promotional coupons.
Fast-forward to the modern day: A/B testing has expanded to all types of digital marketing (including SEO), giving rise to a variety of SEO testing tools, including SearchPilot (the one I work for), SEOTesting, and more.
How A/B testing works in SEO
To begin A/B testing (or just about any SEO testing), you should first split your web pages into one of two subsets (also referred to colloquially as “buckets”):
One subset of pages are the control pages—you will not make any changes to these pages so that they can serve as a baseline for comparison.
The other subset of pages are the variant pages—you will test either an off-page or on-page SEO change to the pages in this group.
This way, you can see how these changes affect organic traffic in a controlled, repeatable way.
What’s the difference between user testing and A/B testing?
The main difference between user testing and A/B testing is that the A/B testing tests Googlebot.
User testing, on the other hand, uses cookies to test the behavior of your real-life site visitors.
In A/B testing, you cannot show Google different versions of the same page (as that would constitute cloaking). To avoid this, SEOs split pages, not users. Users will see the same page every time.
The benefits of A/B testing for SEO
More and more website owners and digital marketers are adding A/B testing to their SEO strategy, and here’s why: A/B testing helps you make data-driven decisions that benefit your business/website. Long gone are the days of relying on assumptions or half-baked competitor analysis to anchor your marketing strategy.
Instead, you can run a small scale A/B test to assess the ROI on a series of optimizations (enabling you to take on a fraction of the risk compared to implementing sweeping changes all at once without testing). If it goes well, you can bring a stronger case to your stakeholders to get their support for your recommendations.
Another thing you might really appreciate is the agility and creativity that comes with A/B testing: You can test on almost any part of your site—from title tags and meta descriptions to schema markup and URL structures.
Once you pinpoint an area of your website that you would like to optimize, you can run an A/B testing experimentation program. This is an iterative process that can help your site stay competitive and potentially improve your organic traffic.
A/B testing strategies for title tags and meta descriptions
Meta tags help Google understand what your website is all about. Google uses this important information to help determine what it displays in the search results.
To that end, Google has said that “it’s important to use high-quality title text on your web pages.” Likewise, it’s also important to use relevant, high-quality copy in your meta description:
“Google will sometimes use the [meta description] tag from a page to generate a snippet in search results, if we think it gives users a more accurate description than would be possible purely from the on-page content.” — Google, “Control your snippets in search results”
Meta tags are a great element of your site to test on, but how do you strategize it? Where can you start?
As an SEO consultant, I have tested on a wide variety of website sections with a wide range of customers, so I am no stranger to test ideation and building SEO testing strategies. When discussing strategies with customers, they normally come to me with their SEO problems and I, in turn, seek to prove the value of SEO by supporting them with a successful experimentation program.
Now let’s take a look at how you can build an A/B testing experimentation program for your own website.
Experimentation programs
As we dive deeper into SEO A/B testing strategies, a pivotal type of strategy emerges—experimentation programs. This strategic approach plays an important role in running SEO A/B tests that will extract learnings about what works and doesn't work for your website.
Let's explore what an experimentation program entails and how you can build one.
What is an SEO experimentation program?
An experimentation program is a structured approach to testing a range of SEO hypotheses in order to:
Learn more about how your site performs under different strategies
Improve key performance metrics
Make more informed, data-driven decisions
As opposed to testing to “just see what happens,” you are creating a personalized testing strategy to solve your SEO problems.
Your experimentation program will uncover useful insights into the organic performance of your site and the ROI of the SEO tools you are currently using. By the end of the program, you should be able to look back at a portfolio of tests and see how much time and money you saved by eliminating risks associated with negative changes. A drop in your organic traffic from negative changes could massively damage your sales/conversions. You will also be able to see how much time and money you saved your engineering team by only deploying winning changes.
How to create hypotheses for A/B tests
The best testing strategies are cohesive and strive for one solid goal. Even though SEO is ever-changing, the idea behind an experimentation program is that the testing strategy should be cohesive—not purely reactive.
You can create hypotheses based on existing strategies, known problem areas, or research in your industry. In that hypothesis, you should include:
What changes you plan on making
What pages you will change
The projected impact on organic traffic
Note: CRO testing measures users and their behaviors. This is a separate testing methodology and set of considerations, but you should consider conducting CRO testing if your conversion rate has decreased despite otherwise solid website performance.
Going back to our example of finding the most well optimized title tag, a good example hypothesis for this testing strategy might be:
“We want to find the most well optimized title tag (as the title tags on our site are low quality). We will test changing the title tag to feature more keywords on a subset of product pages, therefore improving our ranking for new keywords.”
This hypothesis is not just for one test, but for as many as you like. You can iterate until you land on the best one.
How to build a reliable SEO experimentation program
The entire point of building an SEO testing program is so that you can obtain repeatable results. You can achieve a reliable experimentation program in four steps:
Ensure your hypothesis is aligned with your high-level goals. If your digital marketing goals are to increase brand awareness via search rankings, for example, then your hypothesis might involve optimizing your title tags.
Come up with a handful of variations to achieve your goal. Create a couple of title tag variations (based on keyword research in your industry) that are likely to improve your organic traffic and, in turn, your existing search rankings.
Start SEO A/B testing, record observations, and analyze your data. Run your A/B tests for one to two weeks and observe the behavior of your variant page. Has organic traffic increased, decreased or stayed the same? Look at how your pages appear in the search results and take note of any visibility changes.
After you draw conclusions from your test, come up with some new ideas to iterate and improve. You can create a fresh variation of your test(s) (based on your findings) and repeat the process until you identify the best method for optimizing.
Identifying title tag test opportunities
Review your current title tags and page performance to identify whether there’s potential for improvement:
Does it align with Google’s recommendations?
Is it competitive?
And, think about how it could be better for Googlebot (more on this in the example below).
So for example, this VR arcade business has a title tag (“Page 1”) that gives no information about the content of the page and Google will have trouble understanding it.
There is an opportunity here to make the title tag more descriptive and reflective of the page’s content. It’s worth running an experimentation program and finding ways to optimize your title tag if it is:
Generic (like the example above)
Vague
Contains keyword stuffing
Does not reflect user search intent
There are a few automated tools that can help you spot title tags that are not performing well, like WebFX's SEO grader. This tool gives your title tag a score to gauge its effectiveness, providing a baseline for A/B testing.
However, I personally prefer manually inspecting title tags, comparing them to competitors and drawing on industry knowledge. There’s always room for improvement and optimization. By staying informed about your industry, you can determine if your title tag isn’t performing at its best.
Identifying meta description test opportunities
The same idea (discussed in the section above about title tags) also applies to meta descriptions.
Again, you’re looking for meta descriptions that are generic, vague, stuffed with keywords, and/or don’t reflect search intent. The example VR arcade businesses’ meta description (above: “Come read the Game Catalogue”) lacks specificity and fails to communicate the content of the page.
Crafting variations of title tags and meta descriptions
Now that you have identified the opportunities, you can craft variations for the experimentation program using keyword research, competitor analysis, and a user-first mindset.
Instead of these title tags… | Let’s test… |
“Page 1” | “Explore Virtual Realities at AI Arcade | Top VR Gaming in the UK | Multiple Locations for Endless Fun!” |
“Locations” | “Our AI Arcade Locations | Find Nearby Venues for Immersive VR Experiences” |
“The Wrestling Game” | “Step into the Ring: Immersive VR Wrestling Experience at AI Arcade” |
Instead of these meta descriptions… | Let’s test… |
“Welcome to AI Arcade, Best VR, Affordable Games, Top Quality” | “Welcome to AI Arcade: Immerse Yourself in Cutting-Edge VR Experiences at Multiple UK Locations. Affordable Games, Unparalleled Quality, and Endless Fun Await” |
“Come read the Game Catalog” | “At AI Arcade, We Have A Diverse Catalog for Thrilling Adventures and Immersive Experiences. Choose Your Next Virtual Journey Now!” |
“We Offer A Gladiator Wrestling Game In Our Five Game Options” | “Dive Into Thrilling Virtual Reality Combat As You Challenge Friends In A Gladiator-Style Arena. Experience Intense VR Wrestling Right In Your City!” |
Conducting A/B tests and gathering data
Similar to traditional SEO best practices, you will implement the change and monitor performance before deploying the new changes to your whole site. This will help ensure the integrity of your A/B tests.
Analyzing and interpreting test results
While testing, you should monitor performance to see how the change you made impacts your organic traffic. You can track these performance changes in Google Analytics 4, Google Search Console, your CMS’s built-in analytics, or various other third-party tools.
The example above is from the SearchPilot platform. You can see that the test ran for a number of days and resulted in a 5.2% uplift in organic traffic. The customer won 797 extra sessions by making this SEO change. The SearchPilot platform takes into account many external factors that might affect the organic traffic of your website, such as algorithm updates and seasonal changes.
Although you can use any of the SEO tools I mentioned (among many others), not all of them have features specifically to support A/B testing and sophisticated analyses. If your tools don’t offer those features, you can start off with before-and-after testing. Bearing in mind, this method does not factor in algorithm updates or seasonal changes, at the bare minimum you will be able to keep records on:
Baseline performance (your control group/web page)
The exact change you made and the date you made it
The date the test ended and the impact(s) on performance
Case studies and real-world examples
A/B testing can revolutionize website performance and prove the value of SEO (where you otherwise couldn’t via standard site changes). Let’s go through some successful examples that my team and I (at SearchPilot) have tested that you could try emulating on your site.
3 success stories from A/B testing title tags
Optimizing your title tags can impact the overall organic traffic your website brings in by influencing click-through rates and introducing new keyword rankings.
Once you see a change in your organic traffic, you can identify what influenced the change by reviewing search results for your industry. Here’s an example that transcends industries:
The hypothesis for this test was that adding “The Best” to the beginning of title tags could help produce better click-through rates and generate a positive impact on organic traffic.
We ran the test and saw a 10% uplift in organic traffic. By deploying the change, the site could see an extra 11,000 organic sessions per month. Google respected the change and showed the new title tags in search results (remember, Google may opt to rewrite your titles in search results if it thinks they’re not relevant for the user). This change had an amazing impact and is easy to implement on just about any site in nearly any industry.
An informational query is a search in which the user is looking for an answer to a question. Their intent is to know something (hence these types of searches are sometimes called “know queries”). If your site is content heavy, you should optimize it to align with informational queries so that you can establish your sites’ expertise in the industry. This, in turn, can help improve your organic traffic.
In conducting this test, we hypothesized that by increasing the occurrences of the targeted keyword and structuring queries as questions, we could enhance the page’s relevance and better align it with user search intent. The aim was to boost existing rankings and improve the organic click-through rate.
The change was small but mighty. We saw a 5% uplift in organic sessions. Google was able to answer users “know queries” because the question users were searching was in the title tag.
Small, agile changes can improve your organic traffic.
The hypothesis behind the test was that adding the brand name and location to the end of the title tag would help improve organic traffic by increasing the pages’ relevance, visibility, and trust. But, I did fear that the title tag would become too lengthy so Google would cut off the brand name and locale in the search results. This could have a negative effect on organic traffic. When title tags are truncated in search results by Google, the text can end up not being visible to the user.
Thanks to our customers’ clear and concise title tags, there were no cases where the search results got cut off and the results showed that this small change brought in an impressive 9% organic traffic uplift. This result backs up my idea that a small but agile change made the pages more noticeable. By including the location, Google was giving the correct information to prioritize our site for users in the locale. Also, adding the brand name didn't just make the site more relevant to local users; it also sent out strong signals of authority because the brand is well-known.
3 success stories from A/B testing meta descriptions
If you have good user reviews, you can make them work for your website.
The hypothesis for this test was that adding third-party reviews to the meta description could help enhance E-E-A-T signals for improved rankings and/or better click-through rates.
The meta description now helps instill an element of brand trustworthiness and allures users, potentially leading to an uptick in organic traffic and click-through rates.
Google respected the new meta description and showed it in the search results. Although the test was not statistically significant, it resulted in a positive correlation. This means that there were small uplifts in the organic traffic that were not strong enough to be a 95% positive test, but the customer classified it as a positive test and rolled it out on their site. Testing this out on your site could be the beginning of an iterative process.
Sometimes your meta tags can do you more harm than good (i.e., when they’re irrelevant or poorly optimized). An easy way to test this is by removing them altogether.
The hypothesis behind this test is that the site’s web pages featured low-quality and generic meta descriptions. So, we tested removing the meta descriptions entirely to allow Google to select a snippet from the page’s text that it deemed useful.
The test was positive at an 80% confidence level, meaning if there is a 20% chance that if this change was rolled out on the site, it wouldn’t have the same impact.
This might be the easiest out of all the examples that you can implement. You can learn what Google classifies as important on your page and therefore test putting all the variations of verbiage it populated as your meta description in an experimentation program.
To remain competitive in search results, you need to test different approaches: Most search listing snippets contain your standard text. How about adding some numbers to your meta description to make yourself look different and stand out?
In this test, we experimented with adding the lowest price deal into the title tag. Prior to this, the title tags did not feature any pricing details. The hypothesis was that by adding the price, we could help boost click-through rates by attracting users who are seeking the best deals and improve search results performance by including relevant information.
The test ended with a huge 12% increase in organic traffic. Test adding prices to your title tag; whether it’s the lowest price available or an average for the products you sell. This user-first approach could help you improve your organic search traffic from customers who search prices.
How to A/B test and optimize your meta tags on Wix
Wix website owners can see an overview of all their title tags, meta descriptions, page/product names, and URLs for a given page type (i.e., blog posts, products, events, etc.) in the Edit by Page section of the Wix dashboard (SEO > SEO Settings > [desired page type] > Edit by Page), as shown below.
From here, you can make changes to any of your title tags and meta descriptions (without having to open each page individually) by clicking on the three dots to the right of the desired page, as shown below.
You can also generate title tag and meta description suggestions based on your page content using our AI meta tag creator (accessible via each individual page or the Edit by Page section).
You can also use the AI meta tag creator to refine suggestions for your brand/audience, expanding ways you can test your meta tags for the best performance.
SEO A/B testing is your roadmap to iterative performance improvement
SEO A/B testing helps you answer a simple question: “What should I optimize next?” Although some creativity is involved with your optimizations, testing largely takes the guesswork out of your strategy, which keeps you on track to iterative performance improvements.
Once you’ve gained some experience with this process, you’ll be able to work smarter and faster by testing small (to manage risk) and applying those changes at scale, enabling you to get the most out of the SEO you’re already doing for your business (or your clients’ businesses).
