In this video, Keith recounts the story of the vanishing terabyte to explain why many projects discover that they have too little data and not too much data.
- [Instructor] I want to revisit one of the themes…of the course.…Sometimes, you'll actually discover…that you have too little data.…Years ago, the documentation for a software package…that I sometimes use…told the tale of the vanishing terabyte.…The name alone communicates the basic idea.…The data miner in the story was told that their client…was terrified that their systems…couldn't handle the volume.…This occurred back in the '90s, by the way.…Then they began the actual act…of choosing the relevant data only to discover…that they only had a few hundred instances of fraud.…
So what are some of forces at play…that causes your data to get smaller and smaller?…One, not all the cases are relevant,…so for instance, we've already seen…that if you're going to market by email…and you don't have everybody's email,…customers without emails might be left out.…Another one, going too far back in history…sometimes causes problems.…Why, well, nine years ago,…you might have had a very different business…than you have now.…
Chances are good that you don't want to go back that far,…
Note: This course is software agnostic. The emphasis is on strategy and planning. Examples, calculations, and software results shown are for training purposes only.
- Evaluating the proper amount of data
- Assessing data quality and quantity
- Seasonality and time alignment
- Data preparation challenges
- Data modeling challenges
- Scoring machine-learning models
- Deploying models and adjusting data prep and scoring
- Monitoring and maintenance
Skill Level Beginner
Machine Learning and AI Foundations: Recommendationswith Adam Geitgey58m 7s Intermediate
Deploying Scalable Machine Learning for Data Sciencewith Dan Sullivan1h 43m Intermediate
Defining terms1m 48s
1. The Phases of a Machine Learning Project
2. Designing a Machine Learning Dataset
3. Data Prep Challenges
4. Modeling Challenges
7. Monitoring and Maintenance
Next steps1m 1s
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.