Create a StreamingContext to enable real-time analysis.
- [Instructor] So in order to use Spark streaming,…we need to actually set up a streaming context.…And what we're going to do here is first download some files,…and we're going to download a zip file…and we're going to read those files in one by one.…So we'll sort of simulate streaming here.…Then we'll perform some basic analysis.…One of the things that we're doing here…in Spark 2.0 is called structured streaming,…so what it's going to be is we're going to define the structure…after we've done some basic analysis,…then set up our streaming job.…All right, let's switch over now,…and here in Databricks I have the 5.2 file loaded.…
This is from the exercise files.…And I need to attach this to a cluster.…Well, if I don't have one, I can create a cluster here,…and I'm just going to call it 2B, 2C,…you can call it anything you want.…The important part is that Spark 2.0.1…is the edition you're using.…We're not using Scala, so it doesn't matter here.…And I'll click Create Cluster.…As that runs, I'll go ahead and choose it…
- Understanding Spark
- Reviewing Spark components
- Where Spark shines
- Understanding data interfaces
- Working with text files
- Loading CSV data into DataFrames
- Using Spark SQL to analyze data
- Running machine learning algorithms using MLib
- Querying streaming data
- Connecting BI tools to Spark
Skill Level Intermediate
1. Introducing Apache Spark
2. Analyzing Data in Spark
3. Using Spark SQL to Analyze Data
4. Running Machine Learning Algorithms Using MLlib
5. Real-Time Data Analysis with Spark Streaming
6. Connecting BI Tools to Spark
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.