From the course: Apache PySpark by Example

Unlock the full course today

Join today to access over 22,400 courses taught by industry experts or purchase this course individually.

Importing

Importing - Spark DataFrames Tutorial

From the course: Apache PySpark by Example

Start my 1-month free trial

Importing

- [Instructor] Now it's really important to remember that if it's been a couple of hours since you used your notebook on Google Colab, you won't see your files there anymore. Google Colab is a virtual machine, and it just gets recycled if you've left it idle for a while. So that's not a problem, we just need to re-run a couple of the cells and we're done. The start of each notebook will have a Download and install Spark section. You just need to run the ls command, and if you see the spark files and the Chicago's Reported Crime Data file there, you don't need to download them again. As you know, comments and pythons start with a hash character, and extend to the end of the physical line. I've put comments at the start of each line so you don't download the data again by mistake. If you do need to download the data, all you need to do is remove the hash character at the start of each line. For each notebook, you will need to run the setup environment section, and the downloading and…

Contents