From the course: Machine Learning and AI Foundations: Value Estimations

The basic workflow for training a supervised machine learning model

From the course: Machine Learning and AI Foundations: Value Estimations

Start my 1-month free trial

The basic workflow for training a supervised machine learning model

“

- [Instructor] Let's walk through the basics workflow for training a supervised machine learning model. No matter which supervised machine learning algorithm we use, the basic process will be the same. We'll cover each step of this workflow in more detail later in the course, but it's very helpful to understand the basic workflow before we go into more detail. The first step to using machine learning is to get data. For our project of predicting home prices, we'll collect data about all the houses sold in one area for a year. For each home sold we'll collect as much as possible, things like the number of bedrooms, the size of the house and so on. The next step is to clean and pre-process the data. The data we start with will almost always need a little bit of work before it can be used for machine learning. We'll cover this in detail, but the basic idea is to convert any text-based data into numbers and choose which pieces of data are most useful to include in a model. Next, we'll take all the data we have and shuffle the order of the data. We want to make sure the data is in the random order, so that the machine learning algorithm doesn't pick up on the pattern that isn't really there because the data happen to be in the certain order. Next, we'll take the shuffled data and split it into two groups, usually with 70% of the rows of the data in the first group and 30% in the second group. The first 70% of data is what we'll use to train our model. This is called our training data set. The machine learning model will be created based only on this data. The last 30% of data is our test data. This data will not be used to train the model, instead the data will be used to check the accuracy of the model. It's very important that we don't check the model's accuracy with the same data we used to train it. Now that we have clean, shuffled and split data, we're ready to train the machine learning model. But each machine learning algorithm has a few parameters we have to set. These parameters will control things like how fast the model should learn patterns and the data and how complex of a pattern to look for. These setting are called hyperparameters. Next, we train the machine learning model by showing it training data and the expected output. When the model is trained, we need to check it's accuracy against the test data set. This proves if the model actually works well on new data, or if it only works on the training data. If it can predict accurate values for the test data set, it means the model's ready to use. If it can't predict accurate values for the test data set, we need to try again. We can either change the training data we are feeding into the model, or change the model's hyperparameters until we get the results we want. And finally, once the model is trained and evaluated we can use it. We can feed the new data and it'll give us new predictions.

Contents