Connect to an EMR cluster with Spark and Livy using a SageMaker notebook. Run Spark Scala code on your cluster by using your client Jupyter notebook.
- [Narrator] So the next service we're going to look at…is Elastic MapReduce, which is managed Hadoop,…Spark, and other type of library clusters…of virtual machines.…So, the question is, why should we use virtual servers…when we have API's, docker containers,…and many other options?…Well, the answer is, you shouldn't always.…But there are situations for which…you need the level of control.…Could be security requirements.…Could be custom setup steps.…
Could be the amount of data.…I've been working with some bioinformatics customers,…and in processing genomic sequencing result output,…the data is huge, and taking advantage…of the economies of spot pricing on EC2 is really critical…for some machine learning workloads.…Amazon Elastic MapReduce is platform as a service.…It's Hadoop clusters, so master and worker nodes,…that are customized EC2 instances,…that are designed to run Hadoop…and its associated libraries,…such as Spark, SparkML, or machine learning,…and other workloads.…
Many data processing libraries such as Spark and SparkML,…
Author
Released
4/5/2018- How machine learning is used in analytics
- AWS AI servers vs. platforms
- Predicting using Polly text-to-speech
- Predicting using Rekognition for video
- Using Lex to build a conversational application
- Using the AWS Machine Learning service to train, host, and predict
- Working with MXNet in Databricks
- Working with EMR for machine learning
Skill Level Intermediate
Duration
Views
Related Courses
-
Amazon Web Services for Data Science
with Lynn Langit3h 56m Intermediate -
Machine Learning & AI Foundations: Recommendations
with Adam Geitgey58m 7s Intermediate -
Amazon Web Services: Data Analytics
with Lynn Langit2h 49m Intermediate
-
Introduction
-
Welcome1m 12s
-
-
1. Machine Learning on AWS
-
AWS AI servers vs. platforms2m 38s
-
2. Machine Learning API Services
-
3. Machine Learning Platforms
-
Understanding ML platforms3m 53s
-
Understanding SageMaker3m 54s
-
Advanced use of SageMaker2m 37s
-
-
4. Machine Learning Virtual Servers
-
Understanding deep learning2m 36s
-
Databricks on AWS7m 2s
-
5. Machine Learning Architectures
-
AWS ML service for IoT apps2m 12s
-
Conclusion
-
Next steps1m 10s
-
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.
CancelTake notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.
Share this video
Embed this video
Video: Work with EMR for machine learning