Build code for processing EU sales data in the solution.
- [Instructor] In this video,…we're going to see how to build a pipeline…for Europe sales data.…This code is available…in Code_04_04 Publishing EU Sales Data.sh.…On the right side, we can look at the content…of the eu_sales.book_sales,…and it has two records for two books.…And we also see the current status…of the exit summary table,…which only has data updated from the U.S. sales.…Now let us first try to create…the connect-eu-sales-data.properties.…
It is very similar to the other example we saw…except for the database connection being different.…(instructor typing and clicking mouse)…So that is now saved.…Let us go and run the Kafka Connect task…for publishing this data.…Okay, Kafka Connect has now started publishing.…We will use the console consumer to make sure…data is getting correctly published…for this topic, use-case-book_sales.…
And you'll see that there are two records,…both json strings, for books…that are already getting published.…At this point, we see that the pipeline…is already set up for publishing the data.…
- What is data engineering?
- Spark and Kafka for data engineering
- Moving data with Kafka and Kafka Connect
- Kafka integration with Apache Spark
- How Spark works
- Optimizing for lazy evaluation
- Complex accumulators
Skill Level Advanced
Big Data Foundations: Program Managementwith Alan Simon1h 11m Intermediate
1. Data Engineering Overview
2. Moving Data with Kafka
3. Spark High-Performance Processing
4. Use Case Project
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.