Join Dan Sullivan for an in-depth discussion in this video Joining DataFrames with SQL, part of Introduction to Spark SQL and DataFrames.
- [Instructor] One of the most useful features of SQL … is the ability to join tables. … We can join in Spark SQL as well. … So in this case, I've created a new Jupyter notebook … and I've loaded our utilization data. … And of course, the first thing I will do … is I'll create a table for working … with this DataFrame using SQL. … In this case, I called it df util … just to distinguish it from another DataFrame … we'll create in a moment. … And using df util I will create … or replace temp view utilization. … Now, I want to do a join, … so I'm going to need some additional data. … So I'm going to load another file, … a file with server names. … And so the first thing I'm going to do is … specify a path to my data directory … and I'll call that cfdf path. … And that's going to be the combination … of my data path variable plus the file I want to load, … which in this case is called server name.csv. … And this is included with the exercise files. … And then I'll create a data frame with this server data …
- Installing Spark and PySpark
- Setting up a Jupyter notebook
- Loading data into DataFrames
- Filtering, aggregating, and saving data
- Querying and modifying DataFrames with SQL
- Exploratory data analysis
- Basic machine learning
Skill Level Intermediate
1. Introduction to Spark DataFrames
2. Installing Spark
3. Getting Started with Spark DataFrames
4. SQL for DataFrames
5. Data Analysis with Spark
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.