Join Wayne Winston for an in-depth discussion in this video Examples of good and bad metrics, part of Wayne Winston on Analytics.
Well one of my favorite examples of analytics, is the story of Derek Jeter's fielding. And a lot of the analytics people get furious with the fact Derek Jeter won several gold gloves, which is the award they give to the best fielder in Major League Baseball. Because the analytics people would say Derek Jeter is not a good fielder. So for about 100 years, how did we measure good fielding? With a metric which I think is really bad, called fielding percentage. So a fielding percentage of 985 means that every out of every thousand balls hit near you, you would field 985 with those balls successfully and Derek Jeter looks pretty good on fielding percentage.
But what's the flaw on fielding percentage? If I never move, I'll have a good fielding percentage. The balls that go five feet to the left of me and five feet to the right of me, I'll never get to, and they'll never be called an error. So Bill James, who you may have heard of, who's sort of the father of saber metrics, started as a night watchman in Lawrence, Kansas. And there's not much crime in Lawrence, Kansas so all he did at night was analyze baseball box scores. And he could tell by watching games that Derek Jeter was not a good fielder. So basically he said, what metric can I use to show the rest of the world that Derek Jeter is not a good fielder.
He came up with a simple but brilliant metric called range factor. So range factor uses publicly available statistics for a short stop. And so what's out there? There's assists which means how many balls a short stop got to. That basically were thrown to the first baseman to get somebody out. And then, there is putouts which means basically, how many fly balls you caught. So if you are a good shortstop, you should get to a lot of balls. You should have a lot of assists and a lot of putouts. And so basically, this stuff was publicly available back to the 1800s And so, Bill James did a range factor which is the average number of assists plus putouts per inning for Derek Jeter, divided by the average number of assists plus putouts per inning for an average shortstop.
And Derek Jeter's range factor came out to be 0.9. That means Derek Jeter got to roughly 10% less balls than the average shortstop. So he was a below average shortstop. So basically he didn't deserve that gold glove when you look at the numbers but you had to see that good metric that Bill James developed called range factor. Now the best shortstop of all time, most people believe, is Ozzie Smith of the Cardinals. And he has a fantastic range factor above 1.2. He would get to 20% more balls than the average shortstop. So here a simple change in the metric made a big difference in how you evaluate an important quality in baseball.
There are some other measures in fielding that are available now, particularly on a website called fangraphs.com. Ultimate Zone Rating is one of the more sophisticated metrics, if you want to look at that. So one more sports example here, of where we have a bad metric that's used a lot. How do we measure the effectiveness of a goalie in hockey? Usually by save percentage, which is what percentage of the shots on goal does the goalie save? Well I think you can see the problem there, from what we just discussed. If a goalie faces easy shots, he'll save them. If he faces hard shots, he may not have as good a save percentage.
So you need to adjust, basically, the save percentage for the difficulty of shots that the goalie faces.
- What is analytics?
- Predictive vs. prescriptive analytics
- What do you need to know to become an analytic professional?
- Looking at examples of good and bad metrics
- How mobile devices will shape analytics