From the course: Cloud Hadoop: Scaling Apache Spark

Unlock the full course today

Join today to access over 22,600 courses taught by industry experts or purchase this course individually.

Spark SQL

Spark SQL - Apache Spark Tutorial

From the course: Cloud Hadoop: Scaling Apache Spark

Start my 1-month free trial

Spark SQL

“

- [Instructor] Now that we've worked through some examples, we're going to next look at using additional libraries with Spark. We're going to go to our workspace and we're going to import our Spark SQL notebook. So we're going to work with a hypothesis over the next few notebooks. It is rather detailed, and it models some of the work that a data scientist would do, and it kind of showcases data science explorations in these types of notebooks. We're not really working with data of any volume, because we're in the community edition, and again I'm just wanting to get you familiar with the different libraries and how they work at this point. This example is rather long. It's adapted from the one that Databricks has, and I broke it into pieces so that it's more consumable. So we're going to start by looking at the scenario, and then we're going to see why we want to use the Spark SQL library in this case and how to…

Contents