Join Brad Batesole for an in-depth discussion in this video What is A/B testing?, part of Marketing Foundations: Growth Hacking.
- The key to powerful growth hacking is to establish the discipline of A/B testing your efforts. An A/B test is really just testing one variation against the current option. You might A/B test an advertisement where you try a different call to action, say buy now versus buy today. Or you might try a new element to the user interface, in hopes of increasing the time spent on site. Every A/B test should have a clearly defined goal. What do you want to see happen, and what data do you need to measure to understand if the idea worked? With A/B testing, your ideas can be tested in real time.
Using some relatively simple tools, you can drive a fraction of your users to a different version of the page they were trying to access. There, you can measure their behavior and compare it to the existing solution. What's great about A/B testing is that you don't need to have equal groups of people. Instead, you need just enough volume that your results are statistically significant. This basically means the results you're seeing are unlikely to have happened by chance. To show you what I mean, let's look at an example of the results from a landing page test.
In Test A, we had 10,000 visitors and 50 conversions. This equals a .5% conversion rate. In Test B, we had 25,000 visitors and 150 conversions. This equals a .6% conversion rate. At a glance, you'd be inclined to call Test B a winner. For a marginal traffic increase, it looks like the conversions are higher, and the conversion rate is better. But we can use a statistical significance calculator to determine the truth behind this data. So here's a look at the output.
We're only 88% confident that Test B performs better than Test A. So we would need to let the test run longer, until we had enough data to determine it was really the winner. You're looking for a confidence value in the neighborhood of 95%. Now take a look at this same set of data, but this time we'll assume Test A had 80 conversions, and Test B still had 150. It's hard to tell at a glance. Test A has a higher conversion rate, but Test B has more conversions. When we plug the data into that calculator, we find that Test A converted 33% better than Test B, and our results have a 98% confidence rating.
Therefore, we would call Test A the winner. It's important to build A/B testing into your growth-hacking efforts. You can run A/B tests on your own, but if you need a little extra structure, take a look at the resources provided by Optimizely and Visual Website Optimizer. All A/B tests should start with a great hypothesis, and remember, a failed test isn't a bad thing. Learn from it and explore other alternatives.
The course concludes with growth-hacking case studies, highlighting the growth-hacking techniques that helped propel such companies as Airbnb, Uber, and Tinder to explosive growth.
- What is growth hacking?
- Understanding the funnel
- Setting up tracking and analytics
- Leveraging customers and existing users
- Testing ideas
- Generating an audience
- Creating an incentive strategy
- Real-world examples of successful growth hacking