Learn how to measure the accuracy of a regression algorithm using calculated error.
- [Instructor] After training a machine learning model, the next step is to measure how well the model performs. Let's open up train_model pt4.py. To check the accuracy of our model's predictions, we'll use a measure called mean absolute error. Mean absolute error looks at every prediction our model makes, and it gives us an average of how wrong it was across all the predictions. Scikit-learn provides a simple mean absolute error function that we can use to do this. To use it, we pass in the y values, or the correct answers for our training data set.
Then we call model.predict on X, our training features. This will generate a prediction using our training model for each entry in our training data set. Scikit-learn will compare the predictions to the expected answers and tell us how close we are. Now let's do the exact same calculation for our test data set. The only difference here is to make sure to pass in the test data instead of the training data. Let's run this and see the result. Right-click, choose Run. For the training set, the mean absolute error is $48,727.
That means our model was able to predict the value of every house in our training data set to within $48,000 of the real price. Considering the wide range of houses in our model, that's pretty good. For the test set, the mean absolute error was a bit higher at $59,225. This tells us that our model still works for houses it has never seen before, but not quite as well as for the training houses. The difference in these two numbers tells us a lot about how well the model is working.
- Setting up the development environment
- Building a simple home value estimator
- Finding the best weights automatically
- Working with large data sets efficiently
- Training a supervised machine learning model
- Exploring a home value data set
- Deciding how much data is needed
- Preparing the features
- Training the value estimator
- Measuring accuracy with mean absolute error
- Improving a system
- Using the machine learning model to make predictions