Is A/B testing dead?

Download the report free to find out:

• The most common A/B testing misconceptions

• The big differences between testing on hotel websites and testing on OTAs

• How to avoid an invalid test

Our latest report tackles the most overused term in digital marketing: A/B testing.

It has become received wisdom in the hotel industry and beyond that A/B testing should be employed on every website in order to optimize performance. In reality, the majority of websites do not have the required traffic or resource to run a statistically significant A/B test.

In February 2018, A/B testing platform Optimizely called time on their free A/B test tool in a move heralded by many as the death knell of indiscriminate split-traffic testing. For many years, businesses and individuals have turned to A/B testing as the catch-all solution for conversion optimization, the 'scientific' nature of the test leading many to believe that they were gathering ironclad evidence for the profitability of whatever change they were trying to introduce.

What many marketers ignored, however, was the fact that setting up scientifically valid experiments is a complex process. Not mincing their words, Venture Beat have suggested that "trained statisticians would tear the average marketer's A/B test to shreds."

As providers of a software platform to some of the world's largest hotel groups, we are regularly asked to set up tests on the websites of our clients. With accuracy being such a core element of our value proposition, performing the right kind of testing is something we spend a lot of time thinking about. And, often, the right kind of testing is indeed a properly hypothesized, correctly set up, professionally monitored A/B test.

So, we wouldn't agree that A/B testing is dead. We do however celebrate the growing acceptance of the fact that A/B tests are not a catch-all solution that anyone can benefit from. The over-simplified version of A/B testing that businesses have been sold for the best part of a decade encourages them to look at their data in the wrong way.

Download the free report and rethink everything you thought you knew about A/B testing.