Learn how to use pyspark.
- [Instructor] Spark offers other alternatives…for its default shell,…and PySpark is one of them.…Before we try PySpark, let's first make sure…that Python is installed.…Type python in the terminal window and press enter.…If you get a message like what you see here,…you need to install Python.…Type apt, hyphen, get install, Python.…Press enter.…
Type Y and press enter.…Now try Python again.…Type Python and press enter.…Looks like Python has been successfully installed.…Type exit, E-X-I-T, open parentheses, close parentheses yes.…To start PySpark, type PySpark and press enter.…
If you see the screen like this, you're good to go.…Now, let's try to create an RDD,…by typing lowercase text, T-E-X-T,…uppercase F, lowercase I-L-E,…space, equals sign, space, lowercase S-C,…dot, lowercase T-E-X-T,…uppercase F, lowercase I-L-E,…open parentheses, double quotation mark,…read me, R-E-A-D-M-E all capital, dot lowercase M-D.…
Double quotation, close parentheses.…Here, S-C stands for Spark Context.…Press enter, let's try an action.…
- Enabling technologies in data science
- Cloud computing and virtualization
- Installing and working with Proxmox, Hadoop, Spark, and Weka
- Managing virtual machines on Proxmox
- Distributed processing with Spark
- Fundamental applications of machine learning
- Distributed systems and distributed processing
- How Hadoop, Spark, and Weka can work together
Skill Level Beginner
Course organization1m 17s
1. Introduction to Data Science
2. Cloud Computing
3. Distributed File Systems
4. Distributed Processing
5. Machine Learning
6. Case Study
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.