From the course: Transfer Learning for Images Using PyTorch: Essential Training
Join today to access over 22,600 courses taught by industry experts or purchase this course individually.
Learning rates
From the course: Transfer Learning for Images Using PyTorch: Essential Training
Learning rates
- [Instructor] Transfer learning is a key technique for improving the accuracy of deep learning models. There are a couple of other tricks that we can employ to boost the performance of our model. Let's take a look at two of them. Finding a good learning rate is crucial. Too slow, and your network takes forever to learn. But if it's too high, each step you take will go over the minimum and you'll never get to an acceptable loss. Research Scientist Leslie Smith provided an approach for finding an appropriate learning rate. This was made popular within the deep learning community by Jeremy Howard. The idea's quite simple. Over the course of an epoch, start off with a very small learning rate and increase to a higher learning rate over each mini batch, resulting in a very high rate at the end of the epoch. Calculate the loss for each rate and then looking at a plot, pick the learning rate which gives the greatest decline.…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.