Do an exercise to use Kafka Connect to read from a JDBC source.
- [Instructor] In this video, I'm going to show you…how to set up Kafka Connect…to…listen to a JDBC source…and publish that data to Kafka.…The code for this is available in the file…code_…02…_04…Building a JDBC Source, so let's first…take a look at the properties file…that lists the task parameters.…We start on line 24 with name demo-jdbc-source,…any name you can give.…
Then, on line 26, there is the connector.class…which is pointing to a confluent library.…This needs to be, again, acquired from a third party…but we have already copied the required files…for jdbc also into the include folder,…as a part of the earlier video.…Then comes, on line 28, the connection URL…to the database that gives you the MySQL…connection parameters, the name of the database,…usernames and passwords.…On line 30, there is the table.whitelist…that gives you the list of tables…that you need to look and publish,…so it can be one or many tables,…count separated by a comma.…
You can even have a table.blacklist…if you want to publish every other table…
- What is data engineering?
- Spark and Kafka for data engineering
- Moving data with Kafka and Kafka Connect
- Kafka integration with Apache Spark
- How Spark works
- Optimizing for lazy evaluation
- Complex accumulators
Skill Level Advanced
Big Data Foundations: Program Managementwith Alan Simon1h 11m Intermediate
1. Data Engineering Overview
2. Moving Data with Kafka
3. Spark High-Performance Processing
4. Use Case Project
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.