Processing options

- [Instructor] As we continue our tour into the world of modern Hadoop, there have been some updates to processing that I want to mention. Now although we will shortly be diving into this core update around in-memory with Apache Spark, pipelining has been impacted because Spark and other libraries allow not only for the typical batch type of data ingest, but also streaming ingest. In addition to that, batch processing, of course, is the central part of Hadoop, and stream processing is the new player in town.

The highest level is deciding whether to use MapReduce or Spark, and that's why shortly we're going to be diving deep into Spark. Also, new libraries that integrate with the core libraries, such as machine learning libraries, graph data bases, SQL on ACT, so SQL on Spark, for example, are really impacting what you can do in the pipelines that modern Hadoop is a part of.