Regarding Website Optimization Software

The solution you choose will depend on the volume of traffic you have. Taguchi type methods are based on what is called a fractional factorial test and use statistical techniques to reduce the volume of traffic to get a significant test. The Google Website Optimizer is based on a full factorial test and will require much more traffic to get a significant result.

Let’s say you have 15 factors with 2 variation per factor. Using Taguchi based methods you have 18 page variations in the test. Using GWO (full factorial) you have 32,768 pages to test. To get a valid test you have to send around a 1000 visitors to each page. The fractional factorial test (Taguchi) would require 18,000 visitors whereas GWO requires 32,768,000 visitors. Now GWO does have methods of turning off various variation that appear to not be doing well, but even doing that you have to have significant traffic to the page to determine that.

So you may ask, “Why would Google do it that way?” They realized the error made by other software using the Kowalick modification of standard Taguchi arrays. Kowalick wanted to test a much larger number of variations for each factor. To do this he combined factors in the standard Taguchi array. The assumption he made was that each factor was independent of the other factors and interactions could be ignored. In some test you can do that but in general having all factors independent is not possible. Using full factorial you do not have to worry about that because all possible variations are tested. Google understands that a headline may interact with a “call to action” or a header image may interact with buy button image. Any interaction like that makes the factors dependent and Kowalick array cannot be used.

Now you may ask, “Can Taguchi method be used?” Yes it can. You have to use the standard Taguchi arrays, not Kowalick arrays, and assign factors to array columns so that interaction are accounted for during test. Using standard Taguchi arrays allows you to directly test for interactions and also the impact that noise factors have on your test. Noise factors could include browser type, traffic source, age of visitor or anything else you cannot control or do not want to control during test.

KaizenTrack is based on the Kowalick arrays. If you use it you have to keep the independent factors assumption in mind. Also, KaizenTrack is not SEO friendly. That is you have to isolate test pages using robots.txt file to keep from hurting your rankings.

SplitTestAccelerator.com uses Kowalick arrays and some standard Taguchi array. Problem there is they do not take into account the linear graphs so you can know how to assign columns correctly and test interactions. They also require a lot of code cut and paste.

Some people try using Google Website Optimizer to implement a Taguchi test. It can be done but you really need to understand the Taguchi method to do it correctly.

Our Taguchi Split Test software uses the original Taguchi arrays and will test interactions. The program picks the array to use and assigns factors to the array based on the linear graph for that array. It will do this in the background so users of the software do not have to learn anything about the Taguchi Method.