From the course: Building Recommender Systems with Machine Learning and AI

Unlock this course with a free trial

Join today to access over 22,400 courses taught by industry experts.

Movie recommendations with Spark, matrix factorization, and ALS

Movie recommendations with Spark, matrix factorization, and ALS - Python Tutorial

From the course: Building Recommender Systems with Machine Learning and AI

Movie recommendations with Spark, matrix factorization, and ALS

- [Instructor] So open up Spyder inside your Rexus environment. And from your course materials go to the scaling up folder and open up SparkALS.py. This is adapted from one of the examples that comes with Apache Spark. They actually use movie lenses in example as well. But I made some modifications to get it to work with our specific dataset and to generate top end recommendations at the end. It's surprisingly small, right? One reason Spark is so fun to work with is because you can do really powerful things with very little code. Let's dive in. First we start by importing the packages we need from pyspark itself. As we mentioned modern Spark scripts use Spark SQL as their primary interface. And that means we have to set up something called a Spark Session for our driver script. It's similar to a database session in Spirit but we're not going to use it as a database. We're also going to import regression evaluator which will let us measure RMSE in our results. And most importantly ALS…

Contents