Essential A/B and Multivariate Testing Tools.

Use these testing tools to help structure your experiments and boost your conversion rate

A/B tests allow you to run one or more different versions of a page which may include one or more of different features such as a call to action, a hero banner or other page elements compared to a control version. Multivariate testing enables you to test multiple elements at the same time. This requires more traffic and more time spent on set up, but gives the benefit of being able to isolate the impact of each element on conversion and average order value.

Improving leads or sales through driving more users down the conversion funnel is the primary goal of these tools. They simplify the process of running these tests, allowing control over the test variables (e.g. how much traffic is sent to the various test versions), provides statistical analysis of the outcomes and provides comprehensive reporting.

Key things to consider before purchasing and using these tools:

You need enough traffic to test each variant in order to achieve statistical significance (i.e. complete the test and determine a winner). This is unlikely to be a problem for your high traffic pages or for testing the clickthrough rate of your campaign landing page. However, traffic volumes may be an issue in the conversion funnel which is where these tools can bring the greatest value leading to a long time for test to complete (many weeks or months in the worst cases).
Not everything needs to be tested so be careful not to fall into that trap. Setting up and running a test requires time and resource, as do interpreting and acting on the results. Ensure each test has a clear rationale and hypothesis

Convert

Convert allows A/B and multivariate testing using a visual WYSIWYG editor after the addition of asynchronous code. It is a SaaS application allowing unlimited projects and users, and seamlessly imports Google Analytics goals. The easy integration with Google Analytics is a handy addition to this tool that is not evident from others, but which will enable the reporting of tests and the website behaviour from the same interface.

Key features

Multi-domain A/B and multivariate testing and tracking, development tools for jQuery, JavaScript and CSS, with comprehensive reports
Create and edit tests quickly, test e-commerce products and category pages
Integrates with Google Analytics for real-time data and extended segmentation.

Visual Website Optimiser

VWO allows A/B and multivariate testing using a point-and-click editor after the addition of asynchronous code, but is designed for client side only. It can be used to test full and fractional factorial MVT depending on the requirements. Simple and intuitive to use requiring little technical skill. The tool also offers additional products such as behavioural targeting, heatmaps, usability testing etc. giving a holistic view within the one platform.

This tool is one of the market leaders within smaller businesses and therefore offers the usual features expected from an A/B or MVT tool. It also includes a feature called ‘The Idea Factory’ which is a repository of best-practices ideas to test to help users get the most from their subscription if they are lacking expertise to devise tests themselves.

Key features:

Target and experimentation tools
Research and user feedback
Analytics and reporting including heatmap reports
Targeting and personalisation

Optimizely

Optimizely provides A/B and full factorial multivariate testing with easy integration. Unlike Convert, this tool cannot be self-hosted and appears to only be offered as a self-service tool. Although plans can be customised at enterprise level, there is no full-service option available. The basic subscription is limited in functionality offering restricted access, but the only other option seems to be the customised plan. There is, therefore, no clarity on the price of premium subscriptions since this tool makes most of its revenue from larger enterprise businesses and therefore their tool is less likely to focus attention on servicing the smaller end of the market.

Key features:

Easy installation with visual editor
Testing tools – A/B and MVT
Personalisation experimentation
“Always-valid” analytics – to see results at any time

Maxymiser

Seen to be the leading provider of cloud-based software for testing and personalisation, Maxymiser offers a wealth of sophisticated tools and a significant knowledge-base. Since the acquisition by Oracle, it is now part of the Oracle Marketing Cloud, which means it is integrated with other products and can be used simultaneously for closer testing and analysis if required. It does, however, mean that it is a premium enterprise service.

This tool is possibly the best-known and most sophisticated tool for larger
companies that require a more bespoke testing approach. It allows HTML emails
to use the functionality and test designs with the option of integrating with its
CRM platform to collate and analyse big data for greater insight.

Key features:

A/B and MVT tools
Content personalisation
Mobile and app optimisation
Customer insight data
Big data integration (CRM)
Can integrate with most analytics providers

Adobe Target

Adobe Target is a comprehensive enterprise A/B, multivariate testing and personalisation platform working across all devices. Similar to Maxymiser, Target is now part of the Adobe Cloud which offers full integration with Adobe Analytics and content management products. Includes visual editor for micro-tests.

Target is popular with larger e-commerce sites for the real-time automated self-learning personalisation engine. It allows you to use pre-set or customised rules, including visitor location and specific audiences based on real-time data. Target also has an extensive range of out-of-the box targeting rules for you to choose from. Most recommended for larger enterprise businesses given the cost.

Key features:

A/B and MVT
Automated personalisation tools
Mobile app optimisation
APIs can integrate Target with data from web analytics,
CRM, partner and other 3rd parties to enable further customised segmentation and targeting.