Learn how to define custom functions in Scala.
- [Instructor] Now we're going to discuss…Scala classes and objects.…I'm going to start the Scala REPL.…Classes are definitions of structures…and operations on those structures.…Recall that Scala is an object oriented language…and variables are all objects.…When we create a variable or value of a particular data type…we can access the operations that are defined on that type.…For example, when we create an array…we get access to a method,…which can sort the elements of the array.…
Let's create a value, we'll call it y,…and we'll say that it's an Array.…And let's give it some names of countries, like England,…Liberia, Haiti,…Australia, and Sweden.…Now the Array, of course, will order the elements…according to how we entered them,…but I can say y and call the sorted method on the Array y…and it will return a sorted version,…in this case alphabetical order version,…because it's strings.…
Now we get this because the sort method…is defined for Arrays.…Now often when working with data sets…it can help to define our own classes.…
Dan also focuses on using Scala with Spark, a distributed processing platform. He first describes how to work with Resilient Distributed Datasets (RDDs)—a fundamental Spark data structure—and then explains how to use Scala with Spark DataFrames, a new class of data structure specially designed for analytic processing. He wraps up the course by providing a summary of advantages of using Scala for data science.
- The advantages of Scala for data science
- Scala data types
- Scala arrays, vectors, and ranges
- Parallel processing in Scala
- Mapping functions over parallel collections
- When and when not to use parallel collections
- Using SQL in Scala
- Scala and Spark RDDs
- Scala and Spark DataFrames
- Creating DataFrames
Skill Level Intermediate
Java for Data Scientists Essential Trainingwith Charles Kelly2h 43m Intermediate
1. Introduction to Scala
2. Parallel Processing in Scala
3. Using SQL in Scala
4. Scala and Spark RDDs
5. Scala and Spark DataFrames
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.