From the course: Cloud Hadoop: Scaling Apache Spark

Unlock the full course today

Join today to access over 22,600 courses taught by industry experts or purchase this course individually.

Calculate Pi on Spark

Calculate Pi on Spark - Apache Spark Tutorial

From the course: Cloud Hadoop: Scaling Apache Spark

Start my 1-month free trial

Calculate Pi on Spark

“

- [Instructor] All right, we've covered the mechanics of this environment, and now we need to start doing some work on our Spark cluster. So to do that, we're going to click on Workspace, and we're going to import our first notebook that performs some Spark jobs. So we're going to click on Workspace and import, and once again we're just going to make this a little smaller. And now we're going to use the exercise files and we're going to start with the first one, calculate pi, and we're going to import. All right, so we've got a couple of things going on here and let's just take a look at it. The idea of this is to run one or more jobs so that we can start to understand when we run a job, how we look at the steps of the job and evaluate the results. The key aspect of working with fast Hadoop and new Hadoop technologies is having appropriately sized clusters so that you can get get the work done, first, at all so the jobs…

Contents