From the course: Data Science Tools of the Trade: First Steps

Unlock the full course today

Join today to access over 22,600 courses taught by industry experts or purchase this course individually.

Hadoop cluster: Installation

Hadoop cluster: Installation

From the course: Data Science Tools of the Trade: First Steps

Start my 1-month free trial

Hadoop cluster: Installation

“

- [Instructor] Setting up a Hadoop cluster is much more involved than installing Hadoop in a standalone mode. It requires a tedious process of installing underlying software, adding environment variables, and changing configuration files. Much of this can, of course, be automated by writing shell scripts or by cloning virtual machines. In this demo, I have two Proxmox virtual machines, an Ubuntu desktop VM, shown here. It's configured to be my master node, while an Ubuntu server VM, shown here, serves as a slave. One additional software requirement for a Hadoop cluster, is secure shell server, which needs to conduct public key authentication. For example, I created a new user called hduser for both my Hadoop master and slave. I also edited their /etc/host and /etc/hostname files so that I can do this. Let's first log in as hduser on my Hadoop master VM. Type su - hduser as your user name, and then type your password. Here, the dash allows me to log in to the home directory of the…

Contents