As a marketer you'll likely be running a lot of A/B tests, but how do you calculate if they're statistically significant? Walk through a simple example in Excel.
- [Voiceover] As a marketer, you'll be running…a lot of A/B tests.…It's important to know how to calculate if the result…of your test was significant or just a random coincidence.…If you'd like to follow along, go to…3.10- Calculating statistical significance…in the exercise files.…To work out statistical significance, you first need…to know how many visitors were involved in your test…and how many conversions you drove.…For both the control and the variation you were testing.…The first step is to work out conversion rate.…
So, this one's easy.…Equals conversions divided by visitors.…2.08% here, we can drag this across,…and 1.46% for the variation.…Now was this significant?…Did the variation actually perform worse than the control?…To work this out, let's keep going.…To work out the standard error, we just need to go…square root of the conversion rate…times 1 minus the conversion rate in brackets,…divided by the number of visitors.…
And drag this across.…Okay, Step 3 is to work out the z-score.…So for this one, a little bit more complicated,…
Author
Michael Taylor
Released
10/4/2017Michael begins with the basics, showing how to work with marketing data sources and explaining basic formulas for analysis. Next, he walks through how to build a data-driven marketing plan, including mapping out your marketing funnel, reviewing volume vs. cost tradeoff, and running A/B testing. Michael shows how to analyze a campaign's performance, working with PivotTables, charts, and dashboards. Finally, he demonstrates how to set up marketing campaigns in bulk, including manipulating text for ad copy generation and automating the most boring, repetitive tasks with macros.
- Cleaning up data formatting
- Using formulas for analysis
- Building a data-driven marketing plan
- Top-down vs. bottom-up marketing plans
- Running A/B tests
- Analyzing campaign performance
- Plotting data visually
- Building report templates and dashboards
- Building bulksheets for bulk campaigns on AdWords and Facebook
Skill Level Beginner
Duration
Views
-
Introduction
-
Welcome56s
-
Exercise files35s
-
-
1. Excel Basics for Marketers
-
Sources of marketing data1m 58s
-
Navigating large data sheets2m 49s
-
Common marketing metrics3m 13s
-
-
2. Build a Data-Driven Marketing Plan
-
3. Analyze Your Campaign Performance
-
4. Build Marketing Campaigns in Bulk
-
Conclusion
-
Next steps27s
-
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.
Share this video
Embed this video
Video: Calculating statistical significance