From the course: Azure Spark Databricks Essential Training

Unlock the full course today

Join today to access over 22,600 courses taught by industry experts or purchase this course individually.

Use ML Pipelines API

Use ML Pipelines API

From the course: Azure Spark Databricks Essential Training

Start my 1-month free trial

Use ML Pipelines API

“

- [Instructor] Here we are in our ml-cluster and we have two notebooks attached. So we're going to work first with the binary classification. Now, I uploaded an attached and run everything just to make it simpler when we work with it because it does take a little bit of time to run. So you'll notice that this is an example and it shows the Pipelines API and it's using binary classification. The idea is, an email spam or not spam, for example. So in this case, we're going to work with UCI public data, and it's about people and their incomes. So this is their dataset here. So we're going to load the data into the file system and then we're going to create a table using SQL, so it just has lots of features and it has headers. So here, we'll see that in command eight, we can just take a look at what are the columns, so that's the schema and we can display it. So we're going to try out different algorithms. So we're going to need to pre-process data, so that we can compute against it…

Contents