Split Testing Software — What You Need To Know.
You’ve probably heard that split testing elements of your website can improve your conversion rates. And that’s true. But there are many kinds of split testing out there. And there are many ways to set up split tests on your website.
I can’t discuss every feature of every testing tool out there, but I can give you a major overview of the different kinds of split testing software.
The first main division is between simple split testing (single variable) and multivariate testing.
Single variable split testing is very simple. You just try stuff and measure the results. And all you need is a tool that rotates and keeps score.
Google offers a single variable split testing tool, but you have to have an Adwords account to use it.
With multivariate testing, you can test many factors at the same time on your page. So, instead of just testing your headline, you can test your headline, your offer, the P.S. statement, and your lead image all at the same time. And with multivariate testing, if it’s set up right, you can use the same amount of traffic testing many variables that you would use to test a single variable.
There are further distinctions among multivariate split testing tools. There are hosted solutions, and solutions you install on your server. Typically a split testing program can run a little more quickly if it’s on the same server as your webpage. Installed programs tend to be less expensive, too. You also will have complete control of your data. On the other hand, with a hosted solution, you don’t have to do an installation.
Google Website Optimizer is a hosted plan. But it requires you to have an Adwords account and an analytics account. There is also a reporting delay. Other hosted plans include SiteSpect, and Offermatica. The Split Test Accelerator an MuVar are installed solutions.
Taguchi testing is only one kind of multivariate testing. There are also “full factorial” tools, and random independent rotators. When you run a Taguchi test, you run only a small fraction of the possible combinations of your factor levels. You might, for instance, run only 18 pages to test 4374 combinations. This is made possible by orthongonal arrays. This allows you to find the best combination with out running through all of them. You do have to be careful about your factors interacting negatively with each other, but if you take care as you design your test, you can avoid the bulk of the problem with these. The Split Test Accelerator, Vertster, and Kaizentrack are Taguchi testing tools.
When you run full factorial tests, as you would with Google’s Website Optimizer, you create a few factors, and a few options for each factor, and the software cycles through each combination. So, if you have 5 factors with 2 options each, you would cycle through 32 different pages. FFT can be used like a Taguchi test for small test designs. Or, if you run a lot more traffic to it, you can deal even more effectively with interaction effects by just letting all the interactions (good and bad) play out. Note that you can’t run as many combinations with full factorial tools as you can with a Taguchi too.
Some multivariate tools randomly serve the options for each factor independently from what is served for the other factors. This can lead to more flexible test design, but also to more interaction effects and other statistical noise, leading to longer tests.
Now that’s just the overview. There are many other features to compare and questions to ask. Does a tester allow you to exclude bots and spiders? Does it allow real time access to data? What kind of reports does it show? What kind of information does it track for each visitor, and is the visitor-level data available for viewing. Etc.
Dominic DeLong has used split testing tools for years to improve web sites.
Processing your request, Please wait....
