Learn how to import data from a JSON file and create a temporary table from it in a Databricks notebook.
- [Narrator] So let's take a little bit deeper look…into actually creating tables in Spark.…There is some syntax we're going to use here…and we're actually using the SQL syntax…so it's Spark's SQL and it's almost identical…to the ANSI standard SQL.…So you can see here there's some key pieces.…Create table using.…You can create partitioning,…you can do clustering, you can do bucketing.…A lot of the same stuff you can…do with Hive you can do with Spark.…So what we're going to do is upload a CSV…and create a table.…
Manually create the table with SQL.…And create a table with the schema.…So here in my Databricks environment…I have 3.1 creating tables notebook up.…This is an actual SQL notebook.…So it's unlike the other ones where…we've been doing everything in Python.…Now we're actually going to use Spark SQL.…So first we need to attach our notebook to the cluster.…Then I'll take a look and make sure that our…CSV file exists here.…Under the population versus price,…and it looks like it does.…So what we can do now is we can…
- Understanding Spark
- Reviewing Spark components
- Where Spark shines
- Understanding data interfaces
- Working with text files
- Loading CSV data into DataFrames
- Using Spark SQL to analyze data
- Running machine learning algorithms using MLib
- Querying streaming data
- Connecting BI tools to Spark
Skill Level Intermediate
Amazon Web Services for Data Sciencewith Lynn Langit3h 56m Intermediate
1. Introducing Apache Spark
2. Analyzing Data in Spark
3. Using Spark SQL to Analyze Data
4. Running Machine Learning Algorithms Using MLlib
5. Real-Time Data Analysis with Spark Streaming
6. Connecting BI Tools to Spark
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.