What Does A/B Split Testing Have to do with Shoes?

Posted on August 18, 2011 by Jodi Kaplan

Image via Wikipedia

Some years ago, I was taking the garbage down the steps of death, and slipped. I fell down about half a flight.

Once I got outside and tossed the trash, I stopped for a second to check and see if I was bleeding or bashed up. As I was turning my legs this way and that a woman stopped, looked at me, and said, ‘Yes, those shoes are really nice.”

Huh?

I was checking for wounds, not admiring my footwear (there weren’t any, but I was pretty black and blue for a while).

Assumptions can be dangerous

You know what people say about assuming. Assuming that you know the best headline, or the right wording on your offer, can be a mistake. The best way to tell is to test it. Run an A/B split and find out which really works better. The idea is to just change one thing (and one thing only). That way, you know exactly what made the difference (if there is one). Change the headline, and the button text, and the color of the subheads all at once and you’ll never know what helped (or hurt) your results.

Think you already know the answer, without testing? Check out Which Test Won and see how good you are.

Add or subtract?

If you don’t test, you won’t know if that award banner is distracting, or adding credibility to your offer. The right graphic could increase your click through rate. The wrong one might send people running to the delete button. Testing the wording can make a big difference too.

And, always, walk carefully when going down the stairs with a big bag of trash.

Have you tried split testing? What results did you get? Share in the comments, or ask questions.

Are You Reading These Blogs?

Posted on February 28, 2011 by Jodi Kaplan

A roundup of some useful posts from around the web:

Spotting Difficult Clients – Ever have a contract or assignment you wish you hadn’t gotten? Here’s how to spot the “problem children” in advance.

Six Tips for E-Commerce Testing – Going to run an A/B split? Here’s the best (and worst) places to start testing.

Use A/B Split Testing to Write Better Copy

Posted on February 10, 2011 by Jodi Kaplan

Image by Yodel Anecdotal via Flickr

Would you like to improve your copywriting ? You can. And, you don’t necessarily have to write a single word.

There’s an old technique we use in direct mail (the kind with paper) that also works on the web. You can do it with copy, a button color, the wording on your call-to-action; anything you like.

Do a split-test

It’s called an A/B split. Essentially what you do is you divide your list into two pieces: A then B, then A, then B (etc.)

One thing at a time

Then you test something. Just ONE thing at a time. It could be a different headline. Or, a different call to action. Online, or in an email newsletter, it might be text link copy. Whatever you choose to test, the idea is to see which one gets a better result.

This could be more clicks, more orders, phone calls, whatever your goal is. The version with the most clicks (or orders) wins!

There are paid tools to do this (such as Unbounce), or you can use Google’s Website Optimizer (which is free).

Select something to test

Say you want to test a “buy now” button (A) against a “free demo” one (B) You make two web pages, one with each button. Then you tell Google (or whatever tool you use) to alternate which version people see when they visit your site. The first visitor gets a page with version A (the buy button). The second gets version B (the free demo one), the third gets A, etc.

When you have enough data, you check to see which version did better. Then, if you like, you can run the winner against a third version, to see if the results hold up.

One thing to remember is to make sure you have enough data before you stop the test, or declare a winner. You’ll need two more tools for this.

The first is a sample size calculator. This will show you how many responses you need before you have a valid result.

The second is a tool to calculate the statistical significance of your results and declare a winner. You plug in the number of visitors for each page, add the number of conversions (whatever you decide a conversion is) and look to see which had a better result.

Why split test at all?

We often think we “know” which one is better. Oh, the button that says “buy now”! Or, definitely the button that says “Free Demo” will beat “30-Day Trial.” Guessing can be wrong. Experience can be wrong. Often, the version you think is best isn’t. What works for one company, or even for one product, may not work for another. The only way to know for sure is to test it.