Fit two models using different parameters for evaluation.
- [Instructor] Alright, now let's actually take a look…and perform some machine learning.…We're going to build a linear regression model here.…Step one we're going to import…the machine learning library,…then we're going to define our algorithm,…we'll fit two different models…and then we'll make a prediction.…So, back here in DataBricks I've loaded 4.3…from the exercise files.…I need to attach this to my cluster…running Spark 1.6…and then I'm going to execute the first cell…which will download the results.…I'll execute the second one…which reads in and cleanses them,…the third one, which aggregates and converts…and the fourth one, which creates the DataFrame…with features and labels.…
So, we'd previously went over all that,…that's why I didn't go over it here.…In this step we're just going to build…the linear regression model.…So, we need to import the linear regression method…from PySpark dot ML dot regression.…Then we need to define our linear regression algorithm,…so it's as easy as saying LR equals linear regression.…
- Understanding Spark
- Reviewing Spark components
- Where Spark shines
- Understanding data interfaces
- Working with text files
- Loading CSV data into DataFrames
- Using Spark SQL to analyze data
- Running machine learning algorithms using MLib
- Querying streaming data
- Connecting BI tools to Spark
Skill Level Intermediate
1. Introducing Apache Spark
2. Analyzing Data in Spark
3. Using Spark SQL to Analyze Data
4. Running Machine Learning Algorithms Using MLlib
5. Real-Time Data Analysis with Spark Streaming
6. Connecting BI Tools to Spark
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.