This video explores the price of perfection, the problems with using metrics that can be perfected, and the problems with driving employees to be perfect.
- Let's consider some of the more basic metrics that we see on a day-to-day basis. Customer service levels, school grades, on-time arrivals for airlines. In each case we utilize this data to help us make decisions. In reality, though, each of these metrics has flaws. Consider customer service levels in car dealerships. Car dealerships love to gather data about your satisfaction levels, but in most dealerships employees will coach the customer on how to answer the survey, and they will tell them if the rating is going to be less than perfect that the customer should discuss this with the manager before filling out the survey.
This sort of defeats the entire purpose of the survey in the first place, and what you find is that in many cases, there are employee incentives tied to getting perfect scores on these surveys. A bad survey process, unethical actions surrounding the survey, negative consequences associated with low scores, and the quest for perfect scores. This metric is extremely flawed, and in the end provides meaningless data.
How about school grades? Some classes give multiple-choice exams. Some require detailed research papers. Professors sometimes grade differently. In some cases grades are subjective. It would be difficult to argue why one paper deserves an 85%, and another paper only deserves an 84%. One math class might have 20 students, but another math class covering the same topics might have 250 students.
The level of variability, the subjective nature of grades in some classes, and the objective nature of grades in other courses. What we basically find is that not all grades are created equal. Next, let's look at on-time arrivals for airlines. In an effort to make all of their flights arrive on time, airlines will often artificially inflate the period of time a flight is supposed to take.
A flight might realistically require three hours, so the airline will add 15 minutes to the arrival time. If flights leave on time and arrive early, no problem. If the flight leaves late, there might still be a chance that the flight arrives on time. This is the airline working to distort the number in their favor. On the other hand, what happens if the airport is congested, and the flight cannot land on time? Or what happens if weather delays a flight? In these cases, the airline is subject to the power of outside forces.
As you saw, each of these metrics is flawed. Nonetheless, each is not likely to be eliminated any time soon. Should we completely ignore these metrics in evaluating service at a dealership, student performance, and airline reliability? Probably not. But it doesn't mean we can't look to improve the measurement process. It doesn't mean we should avoid discussing these flaws when the metric is being used to help make a decision.
So, back to our tune-up. We have a long list of metrics. The list is well-rounded. Now it's time to understand how each metric will be observed and recorded. Consider making improvements to the collection process, or consider eliminating the metric that might be too unreliable. Remember, these metrics are going to be used to make important decisions, and unreliable metrics can only lead to unreliable decisions.
- Metrics and human behavior
- Common corporate errors in measuring
- Developing a good metric
- Using the performance measurement tune-up
- Avoiding redundancy
- Using dashboards, infographics, and other data visualization tools