July 2018

Marketers and business owners have become fond of A/B testing over the past few years. We had a client recently come to us for consulting help who had run a series of three split tests and redesigned their website to improve conversions. What happened, however, was exactly the opposite. After the design went live, conversions dropped dramatically. The site under-performed and they were in trouble very quickly. They hired us to do an independent audit of their site and changes to see what went wrong. When we dove into the test data, it was fairly obvious: the tests were never setup to create a valid winner. Problem 1: Not Enough Data The third & final test completed was rushed. The client was pushing the agency to go live so the agency cut the test very short. They only saw about 200 combined visitors to