Join Keith McCormick for an in-depth discussion in this video Work with subject matter experts, part of The Essential Elements of Predictive Analytics and Data Mining.
- [Tutor] There's an art in working with subject matter experts. A colleague once told me a story about the early days of the data-mining software that I use. A banking client wanted to put them to the test so the client said, "Here are some unlabeled variables." In other words in the data set there were no column headings. They couldn't tell what was going on. "We're going to keep the variables secret. If your software is so great, tell us which are the best predictors of variable x and if you answer correctly, then we'll buy the software." What a terrible idea.
The reason this doesn't work is you can't model outside of context. You're trying to solve a business problem. You're not just solving some kind of algebra equation here. The data-mining algorithms play an important role in guiding the model building process. But only the human partner in the process can be the final arbiter of what best meets the need of the business problem. You need that context. The nature of the data requires it. That context might come from doctors, engineers, call center managers, insurance auditors, or from a whole host of other specialties.
These folks are busy. There are some common mistakes with how modelers try to collaborate with subject matter experts. They go to the SME and the SME usually wants to tell them what to look for, where to look, and they might even have specific variables in mind. This is not a good idea. They will have a wealth of experience and that's why you want to work with them. But humans simply don't have mental models with dozens of variables. Their mental models only have a handful, probably just two or three key variables.
Instead the modeler should go to them and ask a simple question, "Am I leaving anything out?" So let the computer narrow the search. Let the SME help you widen the search. Also, they'll help you fair it out, interpretation problems, and problems with data quality. So don't let anyone convince you that you can work without subject matter experts, but to take full advantage of their time and yours, you have to do it right.
- What makes a successful predictive analytics project?
- Defining the problem
- Selecting the data
- Acquiring resources: team, budget, and SMEs
- Dealing with missing data
- Finding the solution
- Putting the solution to work
- Overview of CRISP-DM