From the course: Building Recommender Systems with Machine Learning and AI
Unlock this course with a free trial
Join today to access over 22,400 courses taught by industry experts.
Movie recommendations with Spark, matrix factorization, and ALS - Python Tutorial
From the course: Building Recommender Systems with Machine Learning and AI
Movie recommendations with Spark, matrix factorization, and ALS
- [Instructor] So open up Spyder inside your Rexus environment. And from your course materials go to the scaling up folder and open up SparkALS.py. This is adapted from one of the examples that comes with Apache Spark. They actually use movie lenses in example as well. But I made some modifications to get it to work with our specific dataset and to generate top end recommendations at the end. It's surprisingly small, right? One reason Spark is so fun to work with is because you can do really powerful things with very little code. Let's dive in. First we start by importing the packages we need from pyspark itself. As we mentioned modern Spark scripts use Spark SQL as their primary interface. And that means we have to set up something called a Spark Session for our driver script. It's similar to a database session in Spirit but we're not going to use it as a database. We're also going to import regression evaluator which will let us measure RMSE in our results. And most importantly ALS…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Contents
-
-
-
-
-
-
-
-
-
-
-
Introduction and installation of Apache Spark5m 49s
-
Apache Spark architecture5m 13s
-
Movie recommendations with Spark, matrix factorization, and ALS6m 2s
-
Recommendations from 20 million ratings with Spark4m 57s
-
Amazon DSSTNE4m 41s
-
DSSTNE in action9m 25s
-
Scaling up DSSTNE2m 14s
-
AWS SageMaker and factorization machines4m 24s
-
SageMaker in action: Factorization machines on one million ratings, in the cloud7m 39s
-
-
-
-
-