Learn how to set up and configure Spark on your local machine.
- [Instructor] Now so far we've been doing everything…in the DataBricks cloud…but if you're working in an enterprise environment…you most likely have a cluster…which you need to actually run things on.…So, when you development it's common…that we want to actually run things locally first.…And so, what we're going to do here…is look at setting up Spark locally.…The first thing I want to do…is download Spark, then run it from the shell.…So, here on spark.apache.org/downloads…and if that link doesn't work…go to spark.apache.org…and look for this big green download Spark button,…then find the version you want.…
So, for this I'm going to use 1.6.3…and the prebuilt version for Hadoop 2.6…and I just click on the link here…and it'll download it.…Once I've done that, just unzip it.…Also, you'll want to download the Java SDK here,…the version eight is the current version.…You'll need to accept the license agreement…and then choose whatever platform you're working on.…Once I've done that and I've installed the JDK,…what I want to do is take this actual Spark folder…
- Understanding Spark
- Reviewing Spark components
- Where Spark shines
- Understanding data interfaces
- Working with text files
- Loading CSV data into DataFrames
- Using Spark SQL to analyze data
- Running machine learning algorithms using MLib
- Querying streaming data
- Connecting BI tools to Spark
Skill Level Intermediate
1. Introducing Apache Spark
2. Analyzing Data in Spark
3. Using Spark SQL to Analyze Data
4. Running Machine Learning Algorithms Using MLlib
5. Real-Time Data Analysis with Spark Streaming
6. Connecting BI Tools to Spark
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.