What is big data?

- [Instructor] Welcome back. Let's get started. Today, we're going to talk about business decision making. In particular, for any business that's out there, there's two ways of making decisions. Intuition or data. Each of these two methods has its own adherents and they've both been around for centuries, right? There's some people that will say, "Oh well, I make decisions around my firm "based on my gut feeling, "based on my past experience, "and what's going on in the business." That is a form of data in itself, but these people would say, "You can't really capture business decisions with data, "it's too complex." Other folks would say, "Well, in order to avoid subjective decision making, "in order to make the best decision in each case, "I like to look at data.

"I like to look at what's happened "and then make an informed decision." So each of these two methods has its own adherence, and data is really just another way of talking about business intelligence. That's the focus for this course, as we're going to see. You might ask why is big data so important? Why do I keep hearing about this? For those of you asking that question, I have one thing you should remember. Avoid the HiPPO. In the absence of data, business decisions are usually going to be made by what I call the HiPPO.

The highest paid person's opinion. Granted, I'm not here to say that your boss, or your boss's boss's boss, doesn't know what they're talking about. Certainly not. The highest paid person's opinion has value, just like all of our opinions have value. But even the best business decision makers can sometimes be swayed by their subjective viewpoints. Making decisions in business is a balance. It's a balance between data and our best professional judgment.

We want to take a combination of those two things and put them together in order to make an informed and reasonable decsion for any big data question that we face. Now, let me tell you a little story about two friends of mine. Jack and Diane. Two business executives doing the best that they can on behalf of their companies. Jack and Diane are representatives that we're going to talk about throughout this course. They're going to be trying to use big data to make predictions specific to their particular jobs.

As we'll see, they've got very different challenges ahead of them. Before we get to Jack and Diane's specific challenges though, let's talk about how big data is used in industry in general. Some industries are going to use big data more than others. For example, in insurance big data has been used for centuries when it comes to pricing risk. Since the time of George Washington, insurance companies have been trying to price risk on for example, ship accidents. The risk that a ship sailing across the ocean sinks. Today, insurance companies use data to look at the risk of a car accident, or a fire at a house.

Insurance is clearly very data intensive. Kind of on the other end of the spectrum we have manufacturing. Manufacturing firms have traditionally been reluctant to use big data. They've not relied on that area as much as they have on efficient operations. Then somewhere in the middle is something like banks, where they have data on say, people borrowing money for houses. But it's only in the last couple of decades they've really been able to capture that data effectively. Now, all big data projects are going to follow the same set of steps.

It doesn't matter whether we're a banking industry executive, an insurance executive, a manufacturing firm executive, or someone else all together. All big data projects, all business intelligence projects, are following the same set of steps. Jack and Diane are in very different industries, but they're going to follow the same four steps. Step one is to gather and clean our data. Step two is to analyze our data. Step three is going to involve us testing our choices with data.

Step four will be making a decision.