From the course: Introduction to Spark SQL and DataFrames
Join today to access over 22,600 courses taught by industry experts or purchase this course individually.
Joining DataFrames with SQL
From the course: Introduction to Spark SQL and DataFrames
Joining DataFrames with SQL
- [Instructor] One of the most useful features of SQL is the ability to join tables. We can join in Spark SQL as well. So in this case, I've created a new Jupyter notebook and I've loaded our utilization data. And of course, the first thing I will do is I'll create a table for working with this DataFrame using SQL. In this case, I called it df util just to distinguish it from another DataFrame we'll create in a moment. And using df util I will create or replace temp view utilization. Now, I want to do a join, so I'm going to need some additional data. So I'm going to load another file, a file with server names. And so the first thing I'm going to do is specify a path to my data directory and I'll call that cfdf path. And that's going to be the combination of my data path variable plus the file I want to load, which in this case is called server name.csv. And this is included with the exercise files. And then I'll…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.