From the course: Faster pandas
Join today to access over 22,600 courses taught by industry experts or purchase this course individually.
Profiling
- [Instructor] I love the rules of the optimization club. Profiling rule number two will help you find where you're code is spending your time. I've been optimizing code for a living for more than 20 years, and still the initial profiling results tend to surprise me. Bottlenecks are not where you expect them to be. Even veterans like Rob Pike agree with me. The standard recommended profiler in the standard library is C profile. Let's see how we can use it. We have a function that tokenizes text. It gets on text, starts a list of empty tokens and then uses a regular expression to find the tokens, convert them to lowercase, move them to a canonical form with the STEM function defined line four, and then check if the token is not in stop words which are words ignored by search engines, such as in, the, and others append it to the tokens and returns them. So, we're going to run Ipython, and we're going to use the run magic to load the code into our IPython session. And now let's take an…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.