Ultimate Guide to eCommerce A/B Testing for Conversion Optimization

eCommerce stores are not a fire-and-forget business tool. They need to be maintained and improved upon over time if you want to keep your customers happy and coming back for more, as trends, needs, and intentions change over time during the year.

You don’t need to relaunch or redesign your entire website, just update your merchandise to keep your customers interested. The problem is that doing this properly requires adjusting for a lot of parameters: time of the year, current trends, customer segments, potential sales or clearances, and all the different marketing campaigns related your store. That is too much to test for even with a huge budget and loads of time.

Luckily, A/B testing allows us to change only one parameter and see the effect in real-time. Not only does this help make things easier to digest, it’s the only scientifically accurate solution for working with so many parameters.

Why should you A/B test your eCommerce store?

A/B testing is a lot of effort, but it’s the only way to truly optimize store conversions. It’s an iterative learning process, which means that each test helps inform the test that comes after.

Think of your last visit to the eye doctor. All of that “which is better, one or two” questioning you go through is effectively an A/B testing process. You compare two things, choose the winner, then test that against something else. And you keep iterating on tests until you get it as good as it can be.

In eyeglasses, that means finding the right prescription for your lenses. In eCommerce, there’s really no end. As trends change, continued testing is the only way to keep up with them.

It’s the only scientific, data-driven way to compare different settings and layouts in your ecommerce store, and can be used for all kinds of purposes, channels and metrics. For example, A/B testing can be used to understand customer needs and behaviors. Or it can be used to evolve your store’s appearance to its most appealing design.

How does A/B testing work?

A/B testing is basically setting up two different experiences for your customers, and splitting your visitors into two groups. Some customers would go to Experience A, while others go to Experience B. Each Experience would be different in one specific way—either design, wording, or functionality. This lets you isolate the differences and learn what version of the variant gets better results. That may sound complicated, but A/B testing is a simple process if you have the right tools in place.

Here is a quick guide to how we handle it at LimeSpot:

Step 1: Define an optimization goal

What is the objective of this A/B test? Is it to increase product views? Increase basket size? How are you going to measure success? Is it in number of clicks? Conversions? Sales?

You have to know what you’re measuring before launching your A/B test. Decide on how you will measure the success of your test, and collect actual data on existing performance so that you have a benchmark. This benchmark won’t just help orient you on your first A/B test; it will also be a milestone to see how much progress you’ve made after several months of A/B tests.

For this example, let’s say you want to explore how different discount codes are attracting customers—your goal would be to find a discount code that boosts conversion.

Step 2: Make sure you can track

A/B testing is useless if you have no way to track conversions or clicks. You can either get dedicated A/B testing software, or you can use apps or plugins that have A/B testing built in.

The LimeSpot Personalizer, for instance, can track based on which product a user clicked, what they clicked on to get to it (i.e. standard navigation vs Recommendation Box), and more. Some important analytics to keep track of are “View to Purchase Rate” (how many people view a product compared to how many purchase it) and “Time Spent” (how long users stay on a page). Tracking these metrics will allow you to see patterns in your customers’ behavior, then manipulate those patterns to improve sales.

Using the LimeSpot Personalizer, you would track what codes were used, and at what rates conversions happened.

Step 3: Determine which element to test

Pick a single variable that you think will have a significant impact on a user’s shopping experience.

Reminder: only A/B test one variable of your store at a time.

If you A/B test multiple variables at once, you will have no way of knowing which variable actually caused the change in results.

Here are some examples of what you can A/B test on your store, from both merchandising and placement points of view:

• Homepage banner
• Product collections
• Offers and promotions
• Product recommendations
• Upsell and cross-sell CTAs
• Cart Page

For our example, you’ll want to offer two different discount codes, one for 10% and the other for 20%.

Step 4: Set up your test

The exact process for this would depend on which A/B software you’re using. This is how you would setup a recommendation test using LimeSpot.

If you haven’t done so already, install the Personalizer App using the supplied instructions.

The LimeSpot Personalizer has a robust A/B testing feature to help you set up and compare two different layout variations (called Experiences). You can use the Personalizer manage and label them for easy identification.

In order to get started with your first A/B test, go to LimeSpot Admin Panel -> Personalizer -> Setup and then request help from Customer Support. Once you’ve created your first Experience, you can open the box designer using the Customize button to create the layout that you want to test.

For our discount code test, you’d set up an Experience for each of the discount codes. Then you’d run them to determine what’s most effective.

Repeat this process for any other Experience you want to A/B test.

The Personalizer also gives you the ability to add Weight (from 0-100) to each Experience. This allows you to control what portion of new visitors would land in each experience.

The higher the weight, the more likely the option is to show up during the testing phase. For example, if two Experiences have the same weight (5 and 5), they each have a 50% chance of showing up to a user. But if the weight is 5 and 15, then the former option will show up 25% of the time, and the latter will show up 75% time.

Once you’ve set up both Experiences to compare and assigned them the proper weights, Publish them and Activate them, and they will start to appear in your online store based on the weights you configured.

LimeSpot will then start measuring the performance of each experience so you can determine which one should be applied to the next iteration of your online store.

Pro-Tip: Multivariate Testing

Once you get used to running simple A/B tests, you can start running more complex tests that compare multiple variations. Known as multivariate testing, this is an extremely effective way to improve your business.

Let’s go back to our simple promo code test, described earlier, where we ran a 10% discount code vs a 20% discount code in a simple A/B test. To expand this into a multivariate test, we would just add additional codes to the mix. Using the LimeSpot Personalizer, it’s easy to split your traffic across more than two groups, so you could easily test several codes at once.

Try setting up five codes, say 10%, 20%, 30%, 40%, and 50%, and see what happens to conversions. While it’s easy to guess that the higher the discount, the more likely a shopper is to purchase, you might just find that there’s a certain point where increasing the amount of the discount doesn’t significantly increase the number of purchases. For example, a 50% discount code might not actually motivate people more than a 20% code does.

But there’s only one way to know for sure, and that’s to test it for yourself!

LimeSpot can help

LimeSpot is already a great tool for promoting products, highlighting offers, and increasing conversions on an online storefront. But we know that a truly successful store evolves with its users’ tastes and behavior.

That’s why A/B testing should be an important part of your eCommerce store management process. Regular, incremental improvements to your store over time will lead up to huge results to your bottom line. Especially if these improvements are driven by data collected through intelligent A/B testing.

Reach out to LimeSpot today and discover just how easy it can be to apply A/B testing to your online store!

‍