Learn how topics and brokers work inside of Kafka.
- [Narrator] So what is a topic anyways?…I like to think of a topic as a category…of something or a feed of changes.…We talked about customer information being a category,…which would be a great topic.…Another could be sales orders…or customer visits to the website.…All of these are great use cases for topics.…Essentially each one is a separate log in Kafka…and these topics as I mentioned in the architecture…are partitioned across multiple brokers,…which allows them to be resilient.…
If one broker goes down or one hard disk fails,…the topic can still be read and in fact when that happens,…which we'll test coming up here in a minute,…the previous broker which was handling only the Reads,…now will start handling the Writes.…So this way operations can continue without…any gap in functionality or capacity.…If we take a look at the Anatomy of a Topic,…there are three partitions here.…There is partition zero, one, and two…and they start with the index of zero.…
So that's important to know as we start to…set things up and actually work with the data.…
- Understanding the Kafka log
- Creating topics
- Partitioning topics across brokers
- Installing and testing Kafka locally
- Sending and receiving messages
- Setting up a multibroker cluster
- Testing fault tolerance
Skill Level Intermediate
Transitioning from Data Warehousing to Big Datawith Alan Simon1h 50m Intermediate
Big Data Foundations: Program Managementwith Alan Simon1h 11m Intermediate
1. Why Use Kafka?
2. Core Concepts
4. Installing and Testing Kafka Locally
5. Real-World Examples
6. Distributions and Packaging
Next steps1m 4s
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.