How To Split-Test Your Landing Pages For Better Results

by Pam Neely

Testing can make the difference between success and failure for your business. Does that sound too extreme? It’s not. Here’s why: What would happen if you doubled the conversion rate at some of the key points in your sales funnel? Dramatic improvements to your profits, right? It might mean you could finally escape your cubicle, or maybe you need some more guidance.

Let’s go in the opposite direction. What would happen if the conversion rate of key pages was cut in half? Not a pretty picture, eh? Instantly, your revenue is slashed. Your advertising rates double. That cubicle starts to look like safe haven.

Testing doesn’t usually yield such dramatic results immediately, but over time it can. That’s why every smart marketer runs tests. It’s why the “always be testing” maxim stuck.

One of the best things to test – and one of the easiest too – is landing pages. So, if you’re an email marketer who’s already discovered how great landing pages are for results, here’s how to take the next step forward. Testing landing pages isn’t hard, and it delivers big results.

Read more:

1) Understand what “statistically valid results” means

I know, I know. “Statistically valid results” sounds about as exciting as brown wrapping paper. But bear with me for a second. Not understanding this – and applying it – can erase any benefits you might otherwise have gotten from testing. In fact, doing testing without understanding this can actually reduce your results.

Alright – that’s the end of the dire warning section. Here’s what statistically valid results means: You need a certain number of conversions to have results that are trustworthy.

Here’s an example. Let’s say you have two landing pages, A and B. Landing page A gets three conversions. Landing page B gets six. Should you declare victory, send all your traffic to landing page B, and get ready to retire early? Absolutely not.

Why not? Because you haven’t gotten enough results to know for sure that landing page B is the winner. It looks like the winner, sure. But in the real world, there’s a lot of randomness running around. We have to account for it, and the best way is to run a bunch of tests (i.e. get a lot of conversions), and then use the principles of statistics to declare a winner. Otherwise, there’s a high chance we’ll pick the wrong option as the winner. And then, instead of retiring early, we’d end up working into our 90s.

A very simple tool like Perry Marshall’s SplitTester can show you whether your test is valid or not. It was designed to split-test pay per click ads, but it works for other tests too.

Even when we do have enough conversions to declare a winner, we’ll only have a certain level of confidence in the results. We can be reasonably sure, but rarely 100% sure.

Actually, testing experts tend to use certain standard levels of confidence – usually called “statistical confidence.” It’s typically reported as 80%, 90%, 95% or 99% confidence.

2) Play around with a testing calculator for a bit

Now that you understand how important this is, let’s give you a short cut. The good news is you don’t need to know much about statistics to run good tests and get good results.

Some nice people have built testing calculators that take care of all the math. They’re similar to Perry’s calculator, but a bit more advanced. Like this one from Visual Website Optimizer:


The best way to understand how this works is to start plugging in a few numbers for your test. For example, your “estimated existing conversion rate” is just what it sounds like – it’s the current conversion rate for what you want to test. So, say you’ve been running landing page A for a few months. It’s been converting about 8% of the people who arrive on the page. So you would enter “8%” into that field.

For the next item, “Minimum improvement in conversion rate you want to detect (%)” – this is how much of a lift you want to see to consider one page better than the other. The default here is 20%, which definitely a safe number. For the nitty gritty on how this works, see Optimizely’s help page.  Honestly, though, you might want to just leave this at 20%.

The next field is really important. “Number of variations / combinations (including control)” refers, in our case, to how many pages you’ll be including in your test. Because we’re doing a simple A/B test, we’ve only got 2 variations. If you were testing three variations, you’d put a 3 in that field – and your test would take significantly longer to run. (It might also be called an A/B/C test… I bet you can guess why.)

Next up is “Average number of daily visitors”. I bet you know what this means. It’s your traffic. This number is often a problem – especially for people just starting out. Some people don’t have enough traffic to do a lot of testing. They may have to wait for what feels like forever to get statistically valid results.

If that’s your situation, hang tough. Accept that you’ll need to run fewer tests, and thus test only things that really, really matter. For example, you’ll want to test entirely different pages – different layout, headline, offer etc. – instead of just testing different background colors on two pages.

Finally, for the last field, “Percent visitors included in test?” That refers to a special feature of some testing tools where you can direct only a small portion of your traffic to a new variation. This can minimize business losses if a new variation ends up not converting well.

Say you’ve got a ton of traffic, and you want to test two new pages, but you only want to send 10% of your traffic to each new page. Then you’d change this number. For the rest of us, just leave it at 100%.

The real result of playing around with this calculator is that you’ll realize how limited your testing opportunities are. It’s a lesson in testing only what really matters. To paraphrase some testing experts, “Test the forest, not the trees. And definitely not the branches.”

3) Decide which big things you’re going to test

There’re whole blog posts devoted to this topic, but to keep things simple, here’s a thumbnail list:

  • The price of your product
  • The offer on the landing page
  • The headline
  • The call to action
  • The product shot (aka the “hero shot”)
  • Whether a video helps or not
  • The form used to collect information (if you’re using a form)
  • The page layout

There’re dozens of other things you could test, but those items tend to affect conversion rates the most.

4) Set up the test

How you do this will depend on which landing page testing tool you’re using. For simplicity, I’m going to show you how to set up a landing page test in GetResponse.

  • First, create a landing page you want to use as the default, or “control” in your test. Or just decide on an existing landing page you want to test.
  • Once you’re logged in to your GetResponse account, go to the top navigation row, select “Landing Pages” and then select “Manage”.
  • Find the landing page you want to test on the next page.
  • Hover your mouse near the right of the landing page’s title. A tool icon with the word “Actions” will appear. Click it, and then choose “Edit page” from the pull-down menu.
  • The landing page editor will load and you’ll see your landing page. Near the top left of the page, notice the tab that says “Variation A”. There’s a plus sign next to it. Click that to create Variation B.
  • Edit Variation B the way you’d like, then click the blue “Next Step” button in the right-hand corner.
  • Adjust the settings like you would for your other landing pages, then click “Publish”.
  • When you go back to the listing of your landing pages, you’ll see a blue link that says “A/B Test” and a testing image icon. Click on that to see the results of your test.

5) Declare a winner

This is a little more complex than it sounds. After waiting all this time, you’ll probably want to just call the winner as soon as you’ve got statistically valid results.

You could do that, of course. But sometimes it’s wise to take those results with a grain of salt. Sometimes we can improve conversion rate for a certain step in the sales funnel… but not for the last step of the funnel.

For example, say landing page B ends up crushing landing page A. You do your happy dance. But then, you wonder how many people who converted via landing page B actually ended up becoming customers. Sure, more of them got through with landing page B, but were those conversions of the same value as the conversions from page A?

If you’re using a tracking program like Google Analytics (which works with GetResponse, by the way), you can add UTM codes to the links on your landing pages. That way – if you’ve goals set up in Analytics – you can just look at your goals reports and get the answer to this question.

Want to learn more about how to do this? See our blog post, “How To Set Up A Goal In Google Analytics: Why It’s the Most Important Thing You’ll Do This Month”.

6) Move on the next test

Congratulations. You’ve joined the elite group of people not only use landing pages, but that test them, too. Now you can either move on to your next test, or test a different step in your sales funnel – like your opt-in form, for example.

Just one word of advice. Don’t test more than one step of your sales funnel at a time. Otherwise you’ll get blurred results from both tests.


Testing is definitely the secret to better long-term results. But the key phrase there’s “long-term”. In all honesty, many of the tests you run won’t show dramatic results. Some tests may not even give you statistically valid results – the two pages just won’t perform much differently. And when you do run a test that shows an improvement, it may be small, like 10%. But every once in awhile – BOOM – you’ll hit the conversion jackpot and run a test that really blows the doors off.

We all love to talk about doubling conversion rates (hey, I’m guilty of it, too), but achieving that from just one test is rare. Usually it takes a year or more to deliver those kinds of results. However, if you never start testing… you’ll never get there at all.

Back to you

Are you running any landing page tests? What sort of results have you seen? Share your experience and expertise in the comments.

Subscribe for fresh tips & top articles

GetResponse S.A. needs data contained in this form to provide you with materials you requested. For more information, read Privacy Policy.

Thanks for signing up!

We'll send you an email to confirm this shortly.

Become a GetResponse Blog writer!

Write for us