From the course: Introduction to Spark SQL and DataFrames

Unlock the full course today

Join today to access over 22,600 courses taught by industry experts or purchase this course individually.

Basic DataFrame operations

Basic DataFrame operations

From the course: Introduction to Spark SQL and DataFrames

Start my 1-month free trial

Basic DataFrame operations

“

- [Instructor] Now let's look at some other basic Dataframe operations. Now from here on out, I will start by having the data already loaded. Now we could do all of these exercises in a single notebook. But I am separating them out so that you can jump into any lesson without having to depend on steps from previous lessons. So each of the exercise files will have all of the commands that are needed to load the data that is required for that particular exercise. In this case, I have our JSON file with the utilization data already loaded. Let's just take a quick look at that. I've loaded the data into a Dataframe called df2. And I want to show the first 10 rows. So again, this has our event date time, free memory, cpu utilization and so on. And let's just check the count. And there's 500,000, which we expected. Now sometimes you might want to work with or view the list of column names. To do that, you can specify…

Contents