Learn how to use Spark SQL, a SQL variant, to process and retrieve data that you've imported.
- [Instructor] Now that we have some data…in a table,…we can actually use SparkSQL to query that data.…So what we're going to do…is start with just a basic select statement,…then do an aggregation,…and lastly, join some different tables together.…Here in Databricks, I have 3.2 loaded…and I'mma attach this to the cluster…and take a look at the file…that we just created a minute ago called Cogsley Sales.…Just doing a simple select statement,…you can see all the data that's loaded here.…Everything looks good,…even where we have a comma embedded…in the actual column name,…which is nice that it parsed that automatically for us.…
If I wanted to do an aggregation,…I can do some basic sum operations…on the sales amount and round that out.…Click Play on this cell.…There we go.…And next,…if we wanted to do a join,…we need to actually upload the other dataset here…that's being used, Cogsley Clients.…So let's go do that now.…I'll go over to Tables,…Create Table,…click on where it says Drop File…and I want to load Cogsley Clients CSV.…
- Understanding Spark
- Reviewing Spark components
- Where Spark shines
- Understanding data interfaces
- Working with text files
- Loading CSV data into DataFrames
- Using Spark SQL to analyze data
- Running machine learning algorithms using MLib
- Querying streaming data
- Connecting BI tools to Spark
Skill Level Intermediate
Amazon Web Services for Data Sciencewith Lynn Langit3h 56m Intermediate
1. Introducing Apache Spark
2. Analyzing Data in Spark
3. Using Spark SQL to Analyze Data
4. Running Machine Learning Algorithms Using MLlib
5. Real-Time Data Analysis with Spark Streaming
6. Connecting BI Tools to Spark
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.