In this video, learn how to work with Cloud Dataproc for managed Apache Spark.
- [Instructor] To see these services, … if you click on the menu in the GCP console, … and you scroll down to the Big Data section, … you'll see we have Dataproc, Dataflow, … Data Fusion, which we covered previously, … and Composer. … So Dataproc, as mentioned, is managed Hadoop Spark clusters. … What I've done for the sake of time, … is I've set up a cluster, … so we can just look at the interface … and get a basic understanding of how you'd interact with it. … The objects here are Clusters, Jobs, and Workflows. … So here's my cluster. … I set up a very small cluster just for demonstration. … In production, I've had as many as 1,000 … or even 10,000 worker nodes. … The idea with these types of services … is that they are massively distributable for huge workloads. … Again, I've been using this in genomics, … but previously I used it in finance, … and I've used it in social media sites, … where you have just tremendous amounts of data, … flowing through the pipelines. … Notice there's an associated Cloud Storage staging bucket, …
- Enterprise concerns
- Enterprise scenarios
- Setting up your organization’s account
- Managing billing
- Enterprise compute services
- Enterprise storage and database services
- Enterprise data pipelines
- GCP developer and DevOps tools
Skill Level Intermediate
Working with cloud services1m 13s
1. GCP for the Enterprise
2. Enterprise Setup and Security
3. Enterprise Compute
4. Enterprise Storage and Database
5. Enterprise Data Pipelines
6. Dev and DevOps Tools
Next steps1m 20s
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.