October 02, 2006
Conversion Testing 101 - Chapter 2
Continued from Chapter 1
Okay, so now we have a basic understanding why it's important to test and how doing so can improve your Conversions. We've also got the ball rolling a bit by touching on what types of things you can test (we'll cover a lot more later, I'm keeping it to Headlines for now so that it's easier to grasp) and how to come up with our Final testing question.
It's time to start constructing our test, introducing a few new terms to help us keep it all straight.
Where we stand:
We have a site with a landing page that has a headline of:
World's Best Lederhosen Here!
This landing page is doing okay, but isn't converting at as high a rate as we think it can. For argument's sake, let's say our landing page is only convert 1% of all new unique visitors. So for every 100 people who hit the page for the first time, 1 of them ends up buying.
We've decided that maybe our Headline could be a bit better. It could better describe what we're selling; Or it could have a better call-to-action; Or it could stress the benefits of our product in a stonger manner. It could be anything really, but at least we've narrowed it down to deciding we want to test some different headlines.
So what's next?
Well, in the statistical world next is designing our test. Coming up with Variables, Values and Treatments. Let's take a moment to define what these three new terms mean.
A Variable is simply the General element we have chosen to test. In the case of our example, the Variable is our Headline. In other tests it could be a Price, the number of fields in a Form, the general coloring or layout of a page/site, etc. It's just the general thing we have chosen to test.
A Value is the specific option we intend to test. In our little example we have our current headline that will serve as the Control, plus I want to test three new headlines. Which ties back in with our Final Testing Question: We want to know which of the four headlines (1 control plus 3 new ones) will perform best.
The last definition is Treatments. In a nutshell, a Treatment is the more specific version of your Variable and Value. Meaning instead of saying we want to test the headline of our landing page (Variable), or we want to test to see which of four headlines performs the best (Value), we lay out exactly what we'll be testing.
So in the case of our example we already have one headline, which will be our control. So we need to come up with three other headlines that we think may have a chance to improve conversions. Our four treatments look like this:
Treatment 1 (Control): World's Best Lederhosen Here!
Treatment 2: Top Quality Goats' Hide Lederhosen
Treatment 3: Authentic Austrian Lederhosen
Treatment 4: Great Prices on Leather Lederhosen!
Do you see how we can test for different things all within a single test? We have the very generic control headline that may appeal to the masses. But then we're also testing against some different benefits shopping at our site can deliver, separating them to see which will have the most impact. Are people looking for lederhosen made from goat hide? Are they looking for authentic lederhosen, foregoing any knock-offs? Are they looking for the best price?
Even though we provide all of the above, which we stress as a benefit can have a large impact on how our site is viewed by our visitors, and how many of them convert from lookers into buyers.
So there we have it. We have our first test constructed and ready to go. We know what we're testing, both generally and specifically. And we've constructed our test in such a way that we can hopefully get a better idea of what Hot Buttons our potential customers may have that will naturally increase conversions.
See, it wasn't hard at all! With just a few tips and by following the above process, you'll be able to come up with tests for your own site(s), making sure you have a valid testing question.
For the record, the tool I'll be developing will help you to make sure you always have a valid final testing question, among other things. My goal in creating the tool is to take all of the guesswork out of the process, naturally and without anyone having to think about it too hard.
Next Up: How do we tell when we have a Sample Size that is sufficiently large enough to have confidence in our test results?