Join Todd Dewett for an in-depth discussion in this video Data collection tools and methods, part of Performance Review Foundations.
We've talked so far about what to measure, that's traits, behaviors, and outcomes. We've also considered different sources for each of these from self reports to 360s. Now it's time to talk about the specific tools and methods used to collect data. What are the measures we use? I'll start by being honest. There is no perfect measure. Several are popular and useful. I'd like to familiarize you with the most common approaches. Earlier we mentioned that performance diary, which is full of critical incident observations.
We also noted objective measure, such as goal attainment. But the most common type of measure, by far, is some form of graphic rating scale. You'll also see some use of rankings and checklists. But rating scales dominate most appraisal systems. A graphic rating scale gives the evaluator a list of dimensions, each one representing some aspect of the employee's effectiveness. Each dimension has its own multi-point rating scale, most often a five-point, or a seven-point scale.
On the positive end of the scale, you'll see labels such as very strong, or excellent. Similarly, on the negative end of the scale it might say very poor, or unacceptable. Typically in the midpoint is then labeled average, or satisfactory. We often refer to these measures as Likert Scales. They're tremendously popular, because they're easy to use and cost little to develop. Though popular, there are several problems with Likert Scales, or nearly any type of graphic rating scale.
First, they don't provide a lot of actionable feedback. For instance, on an item measuring a positive attitude, what does a rating of three on a five point scale tell an employee he or she can do to improve? In a second these ratings are generic feedback, one specific feedback would be more helpful. For example, you can tell someone their rating for dependability or you can share with them, the fact that 4 colleagues and 2 customers actually complained about their responsiveness to emails and phone calls.
In addition, Likerts and other graphic rating scales are known to have accuracy problems. This stems from the fact that raters can easily interpret the points on the scale differently. For example, what average means to one person is not necessarily what it means to the next. In an attempt to overcome these issues researchers later developed behaviorally anchored scales, or BARS. With BARS raters still rate people on several dimensions but the difference is that instead of using numbers and adjectives for the anchors, BARS use specific job behaviors that reflect varying levels of performance.
Thus one anchor might say Follows directions without reminders, and another might say Capable of assisting others while maintaining workload. BARS is an interesting step forward, but suffers a few limitations as well. First, construction valid BAR scales is far more difficult compare to simple Likert scales. Second, while they do provide a bit more actionable feedback to employees, research has failed to show that they are in fact a superior way to measure performance.
Now, a related type of measure called a Behavioral Observation Scale or BOS, has been meet with some initial success. A BOS takes the approach of identifying desired behaviors in the job and having the rater indicate how frequently these behaviors occur. Like BARS, it suffers from complexity and cost, in terms of constructing a good instrument. Having said that, research is emerging that suggests that BOS approach is valid and reliable, and that bosses and employees both like it since it focuses on desired behaviors and thus serves as useful source of feedback.
As you think about all the complexities I've begun to note, it's no wonder that people gravitate to simple approaches like Likerts scales. In fact, another very simple approach that's still quite popular is to use simple checklists. A checklist, similar to the BOS approach we discussed, will state several specific desirable behaviors. Allowing the rater to indicate the presence of the behavior with a check mark. This process can get more complex with the use of weights to differentiate the relative importance of different items on the list.
In any case, checklists tend to be useful, and efficient, but somewhat rudimentary. One final category is rankings and comparisons. These approaches are polarizing, with a strong minority of practitioners who love them and a majority who tend to oppose them. Because they're so debated and contested, we'll be addressing them more completely in a subsequent part of the course. To summarize all of this, think of measures this way. They're a trade-off between the time and money required to create them, so they're reliable and valid, and their ability to effectively measure employee performance.
By now you might have guessed the punchline. There are no perfect measures. So your best bet is to use a variety in order to capture the best aspects of different approaches.
Lynda.com is a PMI Registered Education Provider. This course qualifies for professional development units (PDUs). To view the activity and PDU details for this course, click here.
The information contained in the following course is provided with the viewer's understanding that the course should not be used as a substitute for consulting a human resource professional at your company for specific guidance. Lynda.com and LinkedIn expressly disclaim liability for any damages, loss, or risk, incurred as a direct or indirect consequence, from the use and application of any content herein.
The PMI Registered Education Provider logo is a registered mark of the Project Management Institute, Inc.
- Understanding the performance cycle
- Setting performance goals
- Collecting performance data and feedback
- Writing the review
- Discussing performance with an employee
- Using a performance improvement plan (PIP)<br><br>
- The PMI Registered Education Provider logo is a registered mark of the Project Management Institute, Inc.