From the course: Azure Spark Databricks Essential Training
Unlock the full course today
Join today to access over 22,600 courses taught by industry experts or purchase this course individually.
Understand ML Pipelines API
From the course: Azure Spark Databricks Essential Training
Understand ML Pipelines API
- [Instructor] As we work with a product, I have found that a lot of things are called ML and I wanted to clarify terms. So some are associated to Spark and some are Databricks. So Spark has what are called MLlib, that's an API, to simply machine learning on Spark. So it contains algorithms and pipelines as you see below, so MLlib algorithms called ML algorithms and those are algorithms for classification, regression, typical machine learning problems. So they would be something like decision trees for example. Spark MLlib pipelines, which we're going to be exploring next, is an API to create and tune models and it supports hyper-parameter optimization and many other features to work with machine learning models. So in the Spark world we have MLlib, MLlib algorithms, which are called ML algorithms, and ML pipelines. In the Databricks world, we have the Databricks runtime ML which we just saw in the previous movie. That is a cluster type which supports third-party machine learning…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Contents
-
-
-
-
-
-
Use Databricks jobs and role-based control5m 37s
-
(Locked)
Use Databricks Runtime ML2m 52s
-
(Locked)
Understand ML Pipelines API4m 16s
-
(Locked)
Use ML Pipelines API8m 39s
-
(Locked)
Use distributed ML training9m 59s
-
(Locked)
Understand Databricks Delta3m 41s
-
(Locked)
Use Databricks Delta5m 10s
-
(Locked)
Use Azure Blob storage2m 41s
-
(Locked)
Understand MLflow7m 34s
-
-
-