A/B Testing

Time and time again, businesses find that they’re bringing in more traffic, but not managing to get it to convert. Maybe you’ve launched some brand new services on your site, or you’ve managed to drastically grow your traffic through SEO, but if you don’t see those conversions coming in, it’s going to be difficult to convince stakeholders to continue to invest in this kind of work!


What is A/B Testing?

A/B testing is a digital marketing strategy in which websites display two different versions of a webpage to various visitors on your website, to see which one performs better.

Let’s say you’re considering changing your homepage design.

It’s crucial that you know for a fact whether or not your proposed redesign would actually perform better or worse. So, you would create your new version, and use an A/B testing tool to display it to a percentage of your traffic.

Often, businesses split their traffic evenly between the two variants. But you could also take a more cautious approach, e.g. display the default to 80% of traffic, while only 20% see the new version.

A/B testing is sometimes refers to as ‘split testing’, as the traffic to your site is ‘split’, with a certain percentage being shown the test variant (the ‘B’), and the remainder being shown the default (the ‘A’).


A/B Testing Goals

A/B testing is used to determine which version of a page ‘performs better’… but ‘better’ at what, exactly?

The goal of your A/B test is actually up to you. Many businesses choose to build tests to find out which version of a webpage has the better ecommerce conversion rate (i.e. which version generates the most sales), but that’s just one of the many options available to you.

In fact, looking exclusively at e-commerce conversion rate can give you a somewhat limited picture. For example, if you tested displaying cheaper products higher up on your page, you may well find that your ecommerce conversion rate increases (as you’re selling more products). However, you might be making less revenue overall. Therefore, it’s important to measure several KPIs whenever you run a test, so that you can make an informed decision at the end about whether or not to implement your test variant.


A/B Testing Conversions

It’s important to remember that a conversion doesn’t always have to refer to a transaction.

A conversion is really just a user completing an action that you want them to complete, and they’re definitely not limited to just e-commerce sites. Every website and application with sufficient traffic should be using A/B testing, whether they have a checkout or not.

Examples of Conversions

Examples of conversions that you might try to optimise for using A/B testing include:

  • Signing up to a newsletter
  • Completing a registration form
  • Completing a contact us form
  • Booking a consultation

Your registration form is an especially important element to test, as for many sites, this is something that every user on your site will have to complete on their user journey. Therefore, it should be one of the most tested areas of your site, but it’s often overlooked.

The results can sometimes be surprising – best practice would suggest that the shorter the registration process is, the less drop-off you’ll get. However, this isn’t always the case. One case study actually showed that registration completion (%) actually increased when the website made their form longer!

The point is, you won’t know how your specific users will react until you test the change.


Benefits of A/B Testing

Essentially, A/B testing is the cleanest way of getting real-world data about about which of two or more possible options is best for your business.

It removes the subjective element from decision making, and takes out the guesswork.

Say goodbye to arguments between teams, or even team members, about which option looks better or would perform better, and get your answer directly from your users!

Even a test variant performs abysmally, this is still great news! If you hadn’t tested this, you might have implemented something disastrous. Even a negative test is a good result. Either way, you’re learning directly from your users’ behaviour that the version you end up implementing is the right one.


Types of A/B Test

Simple A/B Testing (Split Testing)

This is the most basic kind of test.

It pits the default against a single alternative to see if this alternative performs better or worse than the page you have currently.

Traffic can be split 50:50, or in some ratio, depending on the business’s needs.

A/B/N Tests

A/B tests are great for finding out which of two variants perform better among your user base, but there’s no reason two stop at two variants. Often, it makes more sense to test several variants at the same time.

Theoretically, there’s no limit to the number of concurrent tests you can run. However, it’s important to remember that the more variants you’re testing at once, the longer your test will take to complete (as you have less traffic going through each variant).

The optimal number of variants for a test will depend on your website traffic, and the page being tested.

Funnel Tests / Multi-page Tests

In many cases, you’ll want to test site-wide changes, rather than changes limited to a specific landing page, for example.

A typical example would be changing the colour of your ‘Buy’ button everywhere on your site. It wouldn’t make sense for one product page to have a green CTA, while all of your other product pages have a red CTA!

To solve this, we can identify the element(s) that we want changes made to using CSS Selectors, for example, and then apply that change to as many pages as needed.

This approach is a great way to achieve faster learnings, and to ensure that any single result is not a false positive or false negative.

MVT Tests (Multi-Variate Testing)

Multivariate Testing is similar to A/B testing, but features a number of different variables. These variables have different versions, and an MVT test will display different combinations to users to find out the optimal arrangement.

For example, say you’re testing your call to action.

You might want to test colour (red vs blue), size (small vs large), and design (chevron or no chevron), all in one test to see which combination is the best for conversion. Rather than building eight different versions of your webpage for a massive A/B/N test, you could gain the same insight much more efficiently by building a Multivariate Test.


A/B Testing Services

If you’d like help coming up with an A/B testing roadmap, or you’d like help building the tests themselves, Optiminder here’s to support your business with your optimisation needs. Reach out today to find out how Optiminder can help you create and execute an actionable, data-led testing roadmap, and stop guessing what your customers want.