Connect to an EMR cluster with Spark and Livy using a SageMaker notebook. Run Spark Scala code on your cluster by using your client Jupyter notebook.
- [Narrator] So the next service we're going to look at…is Elastic MapReduce, which is managed Hadoop,…Spark, and other type of library clusters…of virtual machines.…So, the question is, why should we use virtual servers…when we have API's, docker containers,…and many other options?…Well, the answer is, you shouldn't always.…But there are situations for which…you need the level of control.…Could be security requirements.…Could be custom setup steps.…
Could be the amount of data.…I've been working with some bioinformatics customers,…and in processing genomic sequencing result output,…the data is huge, and taking advantage…of the economies of spot pricing on EC2 is really critical…for some machine learning workloads.…Amazon Elastic MapReduce is platform as a service.…It's Hadoop clusters, so master and worker nodes,…that are customized EC2 instances,…that are designed to run Hadoop…and its associated libraries,…such as Spark, SparkML, or machine learning,…and other workloads.…
Many data processing libraries such as Spark and SparkML,…
Author
Lynn Langit
Released
4/5/2018- Describe business scenarios that benefit from machine learning.
- Identify the different types of algorithms used in machine learning.
- Explain how Rekognition is used to predict image and video labels.
- Demonstrate how to use custom machine learning algorithms with SageMaker.
- Compare and contrast deep learning and traditional learning.
- Summarize how VariantSpark is used when working with genomic scale data.
Skill Level Intermediate
Duration
Views
-
Introduction
-
Welcome1m 12s
-
-
1. Machine Learning on AWS
-
AWS AI servers vs. platforms2m 38s
-
2. Machine Learning API Services
-
3. Machine Learning Platforms
-
Understanding ML platforms3m 53s
-
Understanding SageMaker3m 54s
-
Advanced use of SageMaker2m 37s
-
-
4. Machine Learning Virtual Servers
-
Understanding deep learning2m 36s
-
Databricks on AWS7m 2s
-
5. Machine Learning Architectures
-
AWS ML service for IoT apps2m 12s
-
Conclusion
-
Next steps1m 10s
-
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.
Share this video
Embed this video
Video: Work with EMR for machine learning