Learn what Spark is and why it has become the most popular big data platform for data science in recent years.
- [Narrator] First let's take a look…and understand what Apache Spark really is.…In short, Apache Spark is a fast…and general engine for large-scale data processing.…It really was designed more specifically for data science,…but has evolved to support even more use cases…and including real-time stream event processing.…There are four main reasons to use Spark:…speed, ease of use, generality,…and we'll get into what that really means,…and also because it runs everywhere…it's platform agnostic.…You can run Spark on top of Hadoop…and all kinds of other platforms…so we'll take a look at that here in a second.…
Spark has an advanced Dag engine…that can achieve up to a hundred times faster processing…than traditional MapReduce jobs on Hadoop.…This is largely due to how Spark distributes the execution…across it's cluster…and performs many of its operations in memory.…This chart here, for example,…shows the run times of logistic regression,…a common task we need to perform in data science,…both on Hadoop using MapReduce and on Spark.…
- Understanding Spark
- Reviewing Spark components
- Where Spark shines
- Understanding data interfaces
- Working with text files
- Loading CSV data into DataFrames
- Using Spark SQL to analyze data
- Running machine learning algorithms using MLib
- Querying streaming data
- Connecting BI tools to Spark
Skill Level Intermediate
Amazon Web Services for Data Sciencewith Lynn Langit3h 56m Intermediate
1. Introducing Apache Spark
2. Analyzing Data in Spark
3. Using Spark SQL to Analyze Data
4. Running Machine Learning Algorithms Using MLlib
5. Real-Time Data Analysis with Spark Streaming
6. Connecting BI Tools to Spark
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.