Access the Spark shell in an EMR Hadoop cluster. Work with data using Spark commands.
- [Narrator] In a previous movie, we set up an instance of…Amazon EMR or Elastic MapReduce.…Now we're going to work with it and see how the Spark…service is used to process data.…In order to do that we need to connect to the…EMR master node using SSH.…And then we're going to use the Spark shell.…Now there's lots of other ways to run…Spark processes on an EMR cluster,…but this is kind of like the hello Spark…or the hello world.…It's the simplest possible way to verify connectivity…and to get some sense of working with Spark.…
So you can see here's our cluster.…We're going to click on the name.…And we'll see that we can SSH in.…Now we have a couple of preparation steps…before we can SSH into the master node.…The first step is we're going to need to set up our security…rule so our client machine can SSH in.…We're going to scroll down,…and you can see here's our security group.…Now I've actually already set up this rule,…but I'm going to show you where and how you would do that.…So I'm going to click over here,…and here's our security group for our master instance.…
- Explain the difference between files and databases.
- Identify examples of batching, micro-batching, and streaming.
- Prepare helpful data visualizations with QuickSight.
- Recognize the different types of analytics available in AWS.
- Demonstrate how to set up AWS CLI.
- Describe common analytics architecture patterns.
Skill Level Intermediate
Amazon Web Services for Data Sciencewith Lynn Langit3h 56m Intermediate
1. Analytics on AWS
2. Analytic Services
3. AWS Code Tools for Analytics
4. Advanced Analytics
Next steps1m 20s
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.