Review architectural patterns for data warehousing scenarios which include serving and migrations.
- [Instructor] For our first reference use case…let's look at data warehouses and databricks.…Typically data warehouses need data…that is prepared for loading…and there's some sort of processing.…And we also could be looking at a migration.…So let's start with loading and processing.…One possible reference architecture…could be the one shown here…where we're loading data both using the streaming pattern…and the batch pattern and we're using something like Kafka…to load via a stream and Data Lake Store…to load via a batch.…
Of course there's other components that we could be using.…The role of Azure Databricks in this scenerio…would be to perform the extract transform and load or ETL…for the incoming data and of course Databricks…is running on Spark which is very, very easy…to parallelize and runs in memory so any sort…of data cleaning, transforming, aggregating,…that sort of activity can be done much, much…more rapidly given the volume of data…that you may be working with.…
And then the data can be shipped from Azure Databricks…
- Business scenarios for Apache Spark
- Setting up a cluster
- Using Python, R, and Scala notebooks
- Scaling Azure Databricks workflows
- Data pipelines with Azure Databricks
- Machine learning architectures
- Using Azure Databricks for data warehousing
Skill Level Intermediate
1. Big Data on Azure Databricks
2. Core Azure Databricks Workloads
Use a notebook with scikit-learn11m 29s
3. Scaling Azure Databricks Workloads
4. Data Pipelines with Azure Databricks
5. Machine Learning Architectures
Next steps1m 1s
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.