Join Jeff Toister for an in-depth discussion in this video Understanding data-analysis techniques, part of Instructional Design: Needs Analysis.
I want to let you in on a secret. I'm not exactly a statistics wizard. Truth is, most of us instructional designers aren't. Now if you are really good at math and statistics, then consider yourself lucky. You'll have a huge advantage when it comes to data analysis. However, no matter what your skill level, there are techniques that you will find useful when it comes to analyzing your data. You might even have fun. Analyzing data can be like being detective making a case. You might have a theory or two, but you need to find concrete evidence to support it.
It's up to you to search for clues and then put all the pieces of the puzzles together. Let's go back to the interviewing skills example that we've used throughout this course. One of our research questions was, what are the causes of turnover? Our theory was that high turnover rates among new employees is a result of poor hiring decisions. How could we test that theory and prove that a poor hiring decision was a cause of turnover? We could start by looking at why people left the company. Fortunately, this information can be found in the company's Human Resources information system.
Here's what the data looks like. This is called a Pareto chart. It's an analysis tool that reveals the leading causes of a particular problem such as new employee turnover. The chart appears to support our conclusion. When people don't like a new job, they often resign or just stop coming. If they aren't capable of doing their job, they might be let go early in their employment. Based on the chart, it seems like 85% of new employee turnover might be related to poor hiring decisions. We can't really prove our conclusion because we don't know the exact reasons why employees left.
This might be helpful to know, but these employees have already left the company, and there aren't any exit interviews in the files. Sometimes we have to weigh the time and expense of doing a more thorough analysis against what benefits it might yield. In this case, we might consult our subject matter expert in Human Resources to see if they agree with our conclusion. There are other occasions when visualizing data helps answer our question. One of the theories for our interviewing skills example was that an inexperienced supervisors had higher turnover rates than our experienced supervisors.
We can create a graphic that compares supervisor experience to turnover rate to tell us whether this theory is accurate. The supervisor's years of experience is on the x-axis, and their individual turnover rates are on the y-axis. If inexperienced supervisors had high turnover rates, we'd expect to see this on our chart. Instead it looks like there's no connection between experience and turnover rate. Statisticians call this graphic a scatter plot. It's a useful tool for comparing two variables like experience and turnover to see if there's a connection. The scatter plot chart answers one question, but it raises another.
If experience isn't a factor, then what is causing high turnover rates? We'll have to continue analyzing our data to find out. If you'd like to know more about creating scatter plots, you can download the reference sheet that accompanies this video. It contains links to helpful resources on scatter plots, Pareto charts, and other commonly used data analysis techniques. The reference sheet also highlights where in this course you can see an example of each one.
- Setting project objectives
- Identifying the target audience for training
- Selecting data sources
- Facilitating focus groups and interviews
- Designing effective surveys
- Identifying participant needs
- Defining learning outcomes
- Presenting results to project sponsors