Learn how consumers interact from Kafka to read messages.
- [Instructor] When it comes to consumers,…we're focusing on the apps that are reading data from…our Kafka Cluster, and they're actually…a little bit more complicated than the producers.…First off, consumers are organized into consumer groups,…and these groups, they have all the partitions of a topic…that they're reading from, divided among all the consumers.…So, if there are 30 partitions and 30 consumers…in the group, each one will have…their own partition assigned to them.…Now, as consumers come and go from the consumer group,…the consumer group needs to be rebalanced.…
Now, this is handled by Zookeeper and there is…a newer version of Kafka which has a Broker coordinator,…but just know that there is essentially another part…of the system that is monitoring these consumer groups,…making sure that they don't become overly bloated…or that partitions are not being assigned appropriately.…So, for the configuration, when you set up…the consumer group, you need to add the consumer group ID,…which is essentially the name or the identifier…
- Understanding the Kafka log
- Creating topics
- Partitioning topics across brokers
- Installing and testing Kafka locally
- Sending and receiving messages
- Setting up a multibroker cluster
- Testing fault tolerance
Skill Level Intermediate
Transitioning from Data Warehousing to Big Datawith Alan Simon1h 50m Intermediate
Big Data Foundations: Program Managementwith Alan Simon1h 11m Intermediate
1. Why Use Kafka?
2. Core Concepts
4. Installing and Testing Kafka Locally
5. Real-World Examples
6. Distributions and Packaging
Next steps1m 4s
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.