Explorr HDFS through bash, creating directories, and learn about a typical Hive structure.
- [Instructor] The first thing I want to cover…is organizing files in HDFS…and I know that when I first started working with Hadoop…this really threw me for a couple loops…that I couldn't really get past…so I wanted to start off just by showing you…some tips that I've learned over the years…on how to work with these directories…and work with the data in HDFS.…I have here open the first script file…from our exercise files that we downloaded…and I'm going to open up a terminal window…where we'll do most of our work.…First I want to create a new folder…for these exercise files…and then I'm going to copy them over.…
So I'll type mkdir, or make directory, hadoop_for_ds.…Now if I take a look I can see that that directory is there…in my user's home directory…and I'm just going to copy the stuff over…from the exercise files into this…so I no longer need to worry about accessing them remotely.…So I'll do copy, just cp -rf, so recursive enforce,…and the exercise files that I linked to in my shared folder…is under media and sf_ and I'll just hit Tab to complete…
- Explain which commands are used to make changes in HDFS.
- Identify the commands used to upload data from the command line to the HDFS.
- Recognize two operations the HDFS performs when a user moves files.
- Summarize how to remove files recursively in HDFS.
- Recall how to select and implement partitions.
- Explain how to flatten a Struct data type in HiveQL.