Learn about the concept and importance of Spark architecture and features.
- Spark brings general purpose computing to Hadoop.…It can handle both real-time and batch processing,…which makes Hadoop more flexible.…In a cluster mode,…which involves multiple computers working together…to get a job done,…you need to designate a Master Computer.…The rest of the computers in the same cluster…are known as Workers.…When you submit a job to the cluster,…one of the Workers starts a Driver Program,…which in turn creates a SparkContext object.…
The SparkContext object then connects to a cluster manager…on the Master Computer,…which could be a Sparks Default Resource Manager or YARN.…Next, Spark acquires Executors…on Worker computers,…which either perform computation or…handle data for the driver.…You can run Spark in a single-node mode too.…Regardless of what mode you are using to run Spark,…you can connect to the Master Computer interactively…and issue commands.…
This interactive environment is called Spark Shell.…Spark is written in Scala.…The Spark Shell gives full access to…all the latest Scala APIs.…
- Enabling technologies in data science
- Cloud computing and virtualization
- Installing and working with Proxmox, Hadoop, Spark, and Weka
- Managing virtual machines on Proxmox
- Distributed processing with Spark
- Fundamental applications of machine learning
- Distributed systems and distributed processing
- How Hadoop, Spark, and Weka can work together
Skill Level Beginner
Course organization1m 17s
1. Introduction to Data Science
2. Cloud Computing
3. Distributed File Systems
4. Distributed Processing
5. Machine Learning
6. Case Study
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.