Join Dan Sullivan for an in-depth discussion in this video What you should know, part of Introduction to Spark SQL and DataFrames.
- [Instructor] Now, I make some assumptions about background knowledge. I assume that you're familiar with SQL, at least the SELECT statement within SQL. We'll be using that quite a bit and I won't go into a lot of introductory comments about the structure of SELECT statements. I also assume you're familiar with Python programming or at least able to read Python code. And then finally, I assume you're comfortable working with the command line, in particular, navigating between directories, running commands and using an editor.
- Installing Spark and PySpark
- Setting up a Jupyter notebook
- Loading data into DataFrames
- Filtering, aggregating, and saving data
- Querying and modifying DataFrames with SQL
- Exploratory data analysis
- Basic machine learning
Skill Level Intermediate
1. Introduction to Spark DataFrames
2. Installing Spark
3. Getting Started with Spark DataFrames
4. SQL for DataFrames
5. Data Analysis with Spark
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.