Run a notebook with uses the Databricks structured streaming API to capture and process streaming input data.
- [Instructor] In this example notebook,…we're going to take a look at working with data…in two types of input methods.…The first method is batched, or regular input,…and in the second method,…we're going to take a look at the new…structured streaming API.…This is available for Python DataFrames.…So as we did in previous examples,…what I've done is I've just imported this.…And you can see, here it is.…
And then of course, you'll remember…that to interact with a cluster, we're going to attach it.…So this is a relatively new API,…and it's been very popular with my customers.…The idea is that you can do complex distributed…data processing in your cluster…on both batch and streaming data.…It does require, as it says here in command one,…that you attach it to a Spark 2.x or greater cluster,…as we've done.…So, we're going to work with some new type of data.…We're not going to use the diamond dataset anymore,…we're going to use the event dataset.…
So let's take a look at that.…Now, you'll notice that we are in a Python runtime notebook…
- Business scenarios for Apache Spark
- Setting up a cluster
- Using Python, R, and Scala notebooks
- Scaling Azure Databricks workflows
- Data pipelines with Azure Databricks
- Machine learning architectures
- Using Azure Databricks for data warehousing
Skill Level Intermediate
1. Big Data on Azure Databricks
2. Core Azure Databricks Workloads
Use a notebook with scikit-learn11m 29s
3. Scaling Azure Databricks Workloads
4. Data Pipelines with Azure Databricks
5. Machine Learning Architectures
Next steps1m 1s
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.