eCommerce Consultant London
Blog

"The truth about split testing for eCommerce"

If I had £10 every time I’ve been asked to change a button's colour.

I’d be on a yacht sipping margaritas, and if running split-tests this insignificant, increased sales as much as people hoped, let’s just say, I’d be out of the job.

Go big or go home

Jokes aside, split-testing can be great and move the needle when it comes to identifying opportunities to increase conversion rates on eCommerce sites, this however comes with a huge BUT!
If you’re making insignificant changes, you will yield insignificant results.

Already split testing?

Great, you’re running experiments on your eCommerce site and at the very least you’re reducing the risk of sinking time and money into big changes that could be difficult to reverse if they don’t have the desired effect. However, if you’re not ambitious with the tests you run, you’re still throwing time and money down the toilet.

Not split testing yet?

If you’re an established online retailer and you’re making sales as you are, it’s time to get serious and install a growth strategy into your business that sets you up to squeeze more sales out while minimising the risk along the way. Dear reader… STOP tweaking your website based on whims and assumptions. You need data and a controlled testing environment to inform the changes you should be rolling out to your eCommerce store. Here’s a simple set of rules you to follow… 1. Use raw data such as your visitor’s behaviour and use this to create hypotheses and experiments 2. Never make a permanent decision based on your interpretation of raw data! 3. Let raw data inform you in creating controlled experiments What is a controlled experiment? Let’s say you sell wine online… You can look at your visitor behaviour and see some patterns suggesting that your customers buy on average 12 bottles of wine per transaction. What can you hypothesise from this? When somebody buys wine online, they intend to get the most bang from their visit. so they STOCK UP. Purchasing enough wine to save them from having to shop for wine again for the foreseeable (or until the wine is all gone). How can we use this knowledge to close more sales of 12 bottles or more? That’s easy, productise 12 bottles of wine as “a case of wine” and offer incentives to your customers for reaching this threshold. For example, Buy a case of wine (any 12 bottles) and get FREE delivery on your order. What’s the final hypothesis? People will increase their average order value to qualify for this incentive. The experiment we will run: 50% of customers on the site for the 30 days will see an option to build a case of wine in exchange for FREE delivery on their order. I.e, 50% of users who surpass 12 bottles in their shopping cart, will qualify for FREE delivery. At the end of the 30 days, we will compare our sales and visitor data to the previous month. If the experiment wins. Get that change made as a permanent change. If it loses. Make a note of it, it either needs tweaking or abandoning. ====== Did you notice anything about the above hypothesis? That’s right, there’s no mention of changing the colour of a button or swapping out the image on a page. This swing at increasing sales based on raw data is a big swing and big swings are the best kind to run split-tests against. ====== Why do I joke about split-testing the colour of a button? Because even if you see an increase in conversion off the back of small experiments and changes on your website, it’s going to be a marginal gain at best and won’t explode your sales overnight. Why do eCommerce brands get caught in the trap of running these minuscule experiments and tests? Well, what could be easier than changing the colour of a button, changing your fonts or using a different photograph on your homepage? The reason why small experiments get priority is that they are exactly that, small super-easy to implement and seemingly low-risk. But what about all those success stories? You ask… The mistake here is comparing yourself to billion-dollar start-ups with incredibly large amounts of traffic where a micro change can make a vast difference. You’re not Google, Facebook, Amazon or Dropbox. There’s no colour you can make your button that makes your sales and average order value shoot through the roof. I know, I’ve tried. =====