Review an EMR-based Spark application pipeline which uses integration with AWS Service Catalog.
- [Instructor] In this next example,…we're going to use yet another AWS service…to build our complex pipeline…and that's the Service Catalog.…But before we cover that, let's cover the basic pipeline.…For infrastructure, we're going to use S3 files, YAML files,…the familiar pattern of CloudFormation templates.…For app code, we're going to use CodeCommit…and of course we're gonna have an infrastructure repo…and an app repo as well.…Pipeline tools, we'll use CodeBuild and CodeDeploy…and tests will be run there.…
The new capability in this pipeline is the Service Catalog…and you may not be familiar with it.…The Service Catalog is a set of tools…that allows administrators…to assign and approve application code.…So in this case, we have a Spark application…and the integration with Service Catalog…provides an additional level of verification.…Of course, we are using Lambda,…as you've noticed every single one of these…advanced CD pipelines uses.…Probably you're familiar, it's gonna be used as a trigger,…so let's look at how this was written up.…
- Understanding high availability (HA)
- Preparing for HA
- Designing for HA
- Understanding continuous deployment (CD)
- Types of verification tests used in CD
- Server mutability and CD
- Implementing CD
- Advanced CD pipeline techniques
- CD pipeline with Step Functions
Skill Level Intermediate
1. Understanding High Availability
2. Design for High Availability
HA for a big data pipeline5m 51s
3. Understanding Continuous Deployment
4. Implementing Continuous Delivery
Next steps1m 6s
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.