From the course: Big Data in the Age of AI

Unlock the full course today

Join today to access over 22,400 courses taught by industry experts or purchase this course individually.

Distributed storage and processing

Distributed storage and processing

From the course: Big Data in the Age of AI

Start my 1-month free trial

Distributed storage and processing

- [Instructor] One of my great accomplishments is that a few years ago my family and I traveled around the world using just carry-on bags. Now, we had to be very, very selective about what we took with us, and occasionally we had to buy stuff when we were there. But a few weeks ago, we went on a short trip, and our car ended up looking more like this. It's not that we don't know how to pack anymore, it's just that our trip required a lot more things. That's the same thing about data. Big data, of course, is big. Often because of the extraordinary volume of the data, you can't fit it all in a single file, or a single drive, or a single computer. Nor can you process it that way. Instead, you frequently have to turn to distributed computing and processing. Now, several years ago, Apache Hadoop was the first major solution to distributed storage, and it really ushered in the big data age. And, for that reason, Hadoop…

Contents