Product Management Insight

Web Analytics is not just pulling numbers from the Web Analytics tools and reporting numbers, it is much more than that. As an Analyst, your task is to report the numbers in a presentable and descriptive manner with analysis but also give suggestions on how to continuously improve the website in terms of appeal and ease of use. This article will touch base on the 3 major website conversion testing methods: AB Testing, Multivariate Testing and Segmentation Testing.

AB Testing or Bucket Test as you might call it is a way to test 2 variations of a page in order to determine which page is more successful in terms of pulling people towards a website goal. We split traffic on each page based on a defined proportion (50/50, 90/10). Usually the first step where there are most chances of convincing a user to buy something is the homepage as this is the page that defines a website in terms of visual appeal, product listing, ease of navigation, color scheme and even text. These things go a long way in generating a sense of trust in the users. For e.g. we have a website whose business model is selling books and there are 2 AB Test pages with the first page being the Control (original) having a discount offer banner on the top and the second page doesn’t have any discount offer banner but a slideshow listing the top selling books. We will then put tracking codes on both these pages and identify each page with a unique identifier in order to measure each page’s success. This success is usually measured in terms of the Conversion ratio which can be measured as: Visitors on the Conversion page/Visitors on the Test/Home Page. Some tools that can help you achieve AB Testing are Google Website Optimizer, Verster, SiteSpect and SplitAnalyzer. Below is a screenshot depicting a typical AB test report.


The second and the most efficient method to optimize your page is known as a Multivariate Testing. In this method, we don’t use multiple pages to determine a higher conversion rate but we test the same page by changing the elements around. For e.g. we can test headlines, colors, buttons and images on the same page by correlating each element with each other to find the best possible combination/element. The method used to achieve this is known as the Taguchi Method. This method takes into consideration the sample size and how long a test should run in order to have a clear winner. Google Website Optimizer uses this method to measure a multivariate test. This tool lets you view the various combinations and elements separately and then based on the Taguchi method decides the winner. All you need to do is add a tracking code on to your page and Google Website Optimizer will then split the traffic equally on each element separately. Please look at the below screenshot to understand how Google Website Optimizer measures a multivariate test. Some other tools that can help you achieve multivariate Testing are Google Website Optimizer, Optimost, Memetrics, Verster, Maxymiser and Omniture Test&Target.


Another form of testing which can be used to target specific segments like country, keywords, referring domains and direct traffic is known as Segmentation. In this we split the traffic based on a defined segment and then track the success factor/behavior pattern of each segment. This is the first step to achieve Behavioral Targeting which is in itself a very powerful way to understand user patterns. Please look at the below mockup segmentation test report screenshot.
Again the best KPI to measure the success is the Conversion rate. Some tools that can help you achieve Segmentation are Google Analytics (Free), Kefta and Maxymiser.