From the course: Apache PySpark by Example
Join today to access over 22,600 courses taught by industry experts or purchase this course individually.
Working with dates - Spark DataFrames Tutorial
From the course: Apache PySpark by Example
Working with dates
- [Instructor] Working with dates can sometimes be a little confusing if you're new to Spark. The best way to get started is with an example. So, let's head over to our notebook. This time, we won't be using the reported crimes CSV file. So, don't worry if you don't have it downloaded, but you'll still need to set up your Spark environment. When you create a simple data format object in Java you specify a pattern string. The contents of the pattern string determines the format of the date and time. So, let's take a look at the simple date format. You can see that we'll use different abbreviations or letters based on what week, month, or day we want to represent. To make sure we understand this, let's head back to our notebook and let's convert Christmas Day 2019, so that's the 25th of December 2019 at 1:30 in the afternoon using multiple date formats. Now remember that all we'll be doing is we're going to be using the format table from the simple date format class from the Java docs…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.