From the course: Neural Networks and Convolutional Neural Networks Essential Training

Unlock the full course today

Join today to access over 22,400 courses taught by industry experts or purchase this course individually.

Gradient descent

Gradient descent

- [Instructor] When we looked at the perceptron, or artificial neuron, we said that it was made up of weighted sum of inputs. The inputs are x1, x2, all the way up to xn, and the weights are w1, w2, all the way to wn. This is basically a binary classifier, because what we're seeing is that if it exceeds a certain threshold, the neuron fires and returns a one, otherwise a zero is returned. With a classification problem, for a given input, we know what the actual output is. And when we pass the same input into our model, we then have a predicted output. We call the difference between these two ouputs the loss, and to improve our model's prediction capability, we want to minimize this loss. We can change our predictions by changing our weights within the model. Now one of the things we need to do is to learn a neuron's weights and parameters. These are the weights and the biases. We can use a loss function to quantify the loss. We take the gradient of this loss function to determine…

Contents