Join Barton Poulson for an in-depth discussion in this video Understanding big data for consumers, part of Big Data Foundations: Techniques and Concepts.
- Most of the time when you hear people talk about big data, they're talking about it within the commercial setting about how businesses can use big data in advertising or marketing strategies. But one really important place that big data is also used is for consumers, and what's funny about this is that while the data is there and the algorithms are there and as incredibly sophisticated processing it's nearly invisible. The results are so clean they give you just a little piece of information, but exactly what you need.
What I want to do is show you some common applications of big data for consumers that you may be using already without being aware of the sophistication of the big data analysis that's going into it. The first one is if you have an Apple iPhone or iPad is what Siri can do. So for instance, aside from saying what's the weather like, and Siri actually knows what it is you mean, and where you are, and what time you're talking about, it can do things like look for restaurants of a particular kind of food and see if they have reservations available.
It can do an enormous amount of things that requires the recommendation of other people, awareness of your locations, awareness of the changes over time of what is most preferable for people. Another one is Yelp. A lot of people use this to find a restaurant, and again, it draws on millions and millions of reviews from users and from other sources to make a very small recommendation. Here I'm searching for Thai food in Carpinteria, California, which is where Lynda.com is located.
I've got Siam Elephant and Your Place Restaurant as my first two hits. The next one, you might be familiar with recommendation engines. This is an idea of software that is able to make a specific suggestion to you. Yelp is an example, but people are more familiar with things like movies, and books, and music. Here's my Spotify account, and Spotify knows what I listen to when I'm on Spotify, and what I listen to all the way through, what I add to my list, what I skip through, and it's able to make specific suggestions to help me find new artists that I wouldn't otherwise know about.
I love some of the stuff that Spotify comes up with. Similarly, Amazon.com makes recommendations for books. For instance, here's a book, Principles of Big Data, one of my favorites, by Jules Berman, and if you scroll down you'll see that they have a list of other books that are recommended. This is generated by Amazon's recommendation engine, and you see several other books on big data, and in fact, it's a great list, I own about half of them. It's the same general principle here.
A lot of people use Netflix to get movies. Netflix makes specific suggestions for other movies you might like. What's interesting is that a few years ago they had a major contest called the Netflix Prize where they wanted to see if anybody could improve the accuracy of their predictions, meaning would you actually like this. If they could improve those predictions by 10% it was a million dollar prize and it was incredibly sophisticated analysis that went into this, but the end result again is a very simple thing, you get recommended a hand full of movies, and usually you pick one and you like it.
In another context there's the app called Neighborland, which is designed to help you collaborate with people to make your city work a little bit better. That's a simple goal, but what Neighborland uses is photos, and data, and APIs from Twitter, and Google Maps, and Instagram, and agencies that report on real estate parcels, it uses transit systems, it uses three one one complaints, an enormous set of data that really highlights the variety of big data. The other ones, for instance, with Spotify and Yelp, show the volume, but this one shows the variety of integrating data from so many different places and so many different formats to help people collaborate on something simple about working together to improve their neighborhood.
Finally, the last one I want to show you is Google Now, and what Google Now does is it actually makes recommendations before you ask for them, especially when it's linked up to your calendar and it's linked up to the location sensing on your phone. It knows where you are, it knows where you need to be, and it can tell you about things like traffic or the weather before you even ask for it, and this is based on, again, an enormous amount of information about the kinds of information people search for, and it provides it in a sort of preemptive manner.
So for consumers, big data plays an enormous role in providing valuable services, but again, with the irony that it operates invisibly by taking a huge amount of information from several different sources and distilling it into just two or three things that give you what you need.
- Evaluate the demand for data science in business, research, and consumer technology.
- Assess the careers and skills in data science.
- Review the ethical issues in data science.
- Explore data visualization with graphing tools.
- Discover how data scientists use tools such as Hadoop and Excel.