Evaluation methods

- I now want to get into the nuts and bolts of running your L&D program. Whether you're a team of one or a hundred, there are some common elements to address. Theses may flex in terms of size, scope and budget, but the basic elements and considerations are the same. First, consider how you'll evaluate the impact of your learning. Did you know that there's actually five levels of evaluation? The success of your program depends on your ability to demonstrate all five. These were initially identified by Donald Kirkpatrick but were updated by Jack Phillips who runs the ROI Institute. As we go through them, let's apply them to our example with John and Maria at the hospitality company. John has contacted Maria because his marketing team isn't delivering the right level of quality on time. Maria runs L&D and built a learning solution to address his needs. Let's look at the first most basic level of evaluation which is how satisfied the participants were. Did they like it or not? After the training, Maria would ask participants how much they liked or were satisfied with the program. Some people use net promoter score or NPS which is an example of this lowest level of evaluation. I also like to ask learners to comment on how valuable the learning was to them and whether they improved their knowledge and skill. The second level measures comprehension and whether people learned the content. This can only be truly measured through an assessment since self-reports can be inaccurate. But it's very easy to use survey tools like Survey Monkey or interactive videos like HapYak to assess understanding. Maria could assess comprehension on executing the project plan, time management and quality control. Even better, she could roll out post-event booster material if the result showed low comprehension on an element. The third level is implementation and measures if people were able to deploy that learning on the job. I'm sad to say that this is where a lot of L&D programs fall down which is why thinking like a consultant is so important. It allows you and your stakeholders to get clear about what behaviors you need to see on the job and by when. You can track this by observing their behaviors or by looking for changes in the metrics you identified. The fourth level is about impact and whether that behavior change moved the needle on the metrics that matter. In our example, we should see the percentage of on-time delivery of marketing materials increase at the same time we see percentage of errors decrease. If that's not happening, Maria would work with John to adjust the learning solution or add more elements until the results are achieved. The bottom line is if your organization is investing time and money on learning, then it needs to demonstrate the necessary results period. The last level is to calculate ROI or return on investment. It's a measure of the cost spent on the learning solution as it relates to the savings created by the behavior change. All kinds of costs and benefits can be explored. You can look at cost, output, time, quality, and energy, all of which can be translated to financial data. And there are other people indicators that have a crucial impact on your business too including customer service, creativity or innovation, the development of your talent, and the culture of your organization. So be thorough when you explore the metrics that matter and how you can show the impact of your learning solutions. As learning professionals, we must design learning to deliver on all five levels of evaluation and track the results. This not only helps us show real ROI on learning but we can course correct as needed so that we're always delivering the metrics that matter to our organizations.