Get introduced to a new public-cloud managed Hadoop distribution from Google. Lynn talks about GCP with Dataflow, which includes key Hadoop libraries, such as Apache Spark.
- [Instructor] So I've been talking about Modern Hadoop,…let's see it in action.…So here we are inside of the Google Cloud Console.…So if you want to try this on your own, as a new customer,…you would sign up for a Google Cloud account…and you get up to $300 in usage credit as a new customer.…And so long as you spin up your Hadoop cluster,…try it out in a couple of hours maybe,…and then spin it down you really shouldn't have any charges.…But I will give you the caveat, here,…that you're trying a commercial service…and in order even to sign up,…even though you have the $300 you have…to supply a credit card.…
So please do turn this off if you're just trying it out,…I don't want you to get any unexpected charges.…So how do we spin up a Hadoop cluster on the Google Cloud?…Click on the menu, and then scroll down, and go to Big Data,…and in Big Data this is called Dataproc,…weird name but that's what it is,…and click on the blue button to Create a cluster.…And what this is going to do is this is going to provision,…
Author
Lynn Langit
Released
7/5/2017- Relate which file system is typically used with Hadoop.
- Explain the differences between Apache and commercial Hadoop distributions.
- Cite how to set up IDE - VS Code + Python extension.
- Relate the value of Databricks community edition.
- Compare YARN vs. Standalone.
- Review various streaming options.
- Recall how to select your programming language.
- Describe the Databricks environment.
Skill Level Intermediate
Duration
Views
-
Introduction
-
Welcome53s
-
-
1. Hadoop Core Fundamentals
-
Modern Hadoop1m 53s
-
Hadoop libraries1m 23s
-
Run Hadoop job on GCP1m 52s
-
Databricks on AWS2m 32s
-
-
2. Setting Up a Hadoop Dev Environment
-
Load data into tables1m 51s
-
3. Hadoop Batch Processing
-
Processing options1m 2s
-
Resource coordinators1m 30s
-
Compare YARN vs. Standalone1m 30s
-
-
4. Fast Hadoop Options
-
Big data streaming1m 57s
-
Streaming options1m 10s
-
Apache Spark basics1m 46s
-
Spark use cases1m 2s
-
5. Spark Basics
-
Apache Spark libraries3m 24s
-
Spark shell1m 53s
-
-
6. Using Spark
-
Tour the notebook5m 29s
-
Import and export notebooks2m 56s
-
Calculate pi on Spark8m 19s
-
Import data2m 50s
-
Transformations and actions4m 43s
-
Caching and the DAG6m 49s
-
7. Spark Libraries
-
Spark SQL8m 34s
-
SparkR6m 11s
-
Spark ML: Preparing data4m 21s
-
Spark ML: Building the model3m 50s
-
MXNet or TensorFlow2m 30s
-
Spark with GraphX2m 12s
-
-
8. Spark Streaming
-
Spark streaming4m 21s
-
9. Hadoop Streaming
-
Pub/Sub on GCP3m 59s
-
Apache Kafka1m 26s
-
Kafka architecture1m 6s
-
Apache Storm1m 30s
-
Storm architecture1m 36s
-
-
10. Modern Hadoop Architectures
-
Conclusion
-
Next steps26s
-
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.
Share this video
Embed this video
Video: Hadoop on Google Cloud Platform