Hadoop cluster: Operation

- [Male] There are a number of steps involved in starting a Hadoop Cluster. Since you're starting the cluster for the first time, issue a command to format the name node. Let's change our directory to slash user slash local slash hadoop. Now type HDFS space name node space dash format.

Press enter. You can start your name node by first typing dot slash 'sbin' slash hadoop hyphen daemon dot SH space dash dash 'config' space dollar hadoop underscore 'conf' underscore DIR space dash dash script space HDFS space start space name node.

Next, just change name node into data node. Press enter. This will start your data node. Lastly, type dot slash 'sbin' slash start hyphen DFS dot SH script, as I'm doing here. Press enter. Okay, I've made an error here. Let me retype that very quickly.

Now let's open another terminal window. Now, let's sign in as HD user. So type su, SU, space dash space HD user. Press enter. And then enter your password. Now, you'll need file system that will work with Hadoop. So type HDFS space DFS space dash MKDIR space slash user.

So now recall the command and then add slash HD user. Press enter. This completes the process of starting our distributed file system. Once your HDFS is up and running, you can access it through a web interface. Open the web browser and type master five zero zero seven zero. You can see some useful information here, such as free disk space available on your HDFS or when it's started.

You can also see some details on your data node. Click on data nodes. Now go to utilities. Choose browse the file system. And you can see the directory we just created. Click on HD user. Now let's go back to our terminal. Let's create a new directory called test on our newly created Hadoop distributed file system.

To do that, just type HDFS space DFS space dash MKDIR space slash user slash test. Press enter. Oops, I made a little typo here. Let me do that again. You can check if the new directory has been created or not by using the Hadoop explorer web interface we just visited.

Click on utilities, and then browse the file system. We put our test directory under user, so click on user. Here is our new test directory. Do you see it? Okay, now go back to the terminal window again. Let's see if we can remove the directory. Type HDFS space DFS space dash RMDIR remove directory space slash user slash test.

Go back to the file browser and refresh. The directory is gone. Next, let's try Yarn.