From the course: Cloud Hadoop: Scaling Apache Spark

Unlock the full course today

Join today to access over 22,600 courses taught by industry experts or purchase this course individually.

Hadoop and Spark libraries

Hadoop and Spark libraries - Apache Spark Tutorial

From the course: Cloud Hadoop: Scaling Apache Spark

Start my 1-month free trial

Hadoop and Spark libraries

“

- [Instructor] When we're thinking about working with modern Hadoop, we need to think about the Hadoop libraries that we're going to be using. So of course, as I mentioned in a previous movie, we're working with Apache Hadoop version 2.7 or greater as of this recording, which will include normally MapReduce, Hive, Pig, so on so forth, the core libraries. Now the newer libraries that we're going to be focusing on in this course are Apache Spark and Apache Storm, and as we get into those configurations, I'll make sure to note the versions of Spark and Storm. One of the ongoing challenges with working with the Hadoop ecosystem is matching all the versions of all the libraries to make sure that when you're moving code into production, it'll actually run when you move it from development to production. So really, it's just a tip. You heard it in my other course and I'll say it to you again, it's actually even more important…

Contents