From the course: Advanced Predictive Modeling: Mastering Ensembles and Metamodeling
Join today to access over 22,600 courses taught by industry experts or purchase this course individually.
Understanding model error: Classification - SPSS Tutorial
From the course: Advanced Predictive Modeling: Mastering Ensembles and Metamodeling
Understanding model error: Classification
- [Instructor] Now we're going to talk about accuracy when you're predicting a category. So what I've done is I've run three decision trees on the famous titanic dataset, and I can see that my train accuracy for my CHAID, C&RT, and C5 models are all in the high 80s. Notice the substantive drop between the train performance and the test performance. That's indicative of a lack of stability, in other words I've got high variance. So let's see how the ensemble does. The ensemble is the top performer on the train data, but not by much at 89.8%. And the ensemble also seems to have fairly high variance in that it's somewhat unstable and dropping quite a bit to the test. So this is what most folks look at first, overall accuracy. But when you're doing ensembles, there's a lot of moving parts and you really want to up your game. It is not enough to look at just overall accuracy, we really have to look at the confusion matrix, so let's take a look. Here in the confusion matrix what we can do…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.