Learn how to monitor the Kafka cluster.
- [Instructor] Monitoring in Kafka is pretty robust.…I'm going to show you the latest and greatest here.…There is an older API that uses…something called Yammer metrics…where it spits out a ton of details,…and statistics about your cluster.…But all of those metrics as well as a framework…that you can use to do long-running tests…and regression analysis have been packaged up…and released by LinkedIn…into something called Kafka Monitor.…This is an open-source component…that you can use to run these long-running tests,…and monitor your Kafka Cluster.…
Before, the problem was that you may be looking…at something right now and everything's fine.…But what happens is some of these problems…creep up over time.…So they're not immediately obvious.…You need to set up tests…that can be run over longer periods of time,…to see the impact in the overall cluster.…In the Kafka Monitor, what a user can do…is set up these different tests,…and have services that use those tests…to interact with your cluster.…They give feedback to the user in this long-running test.…
- Understanding the Kafka log
- Creating topics
- Partitioning topics across brokers
- Installing and testing Kafka locally
- Sending and receiving messages
- Setting up a multibroker cluster
- Testing fault tolerance
Skill Level Intermediate
Transitioning from Data Warehousing to Big Datawith Alan Simon1h 50m Intermediate
Big Data Foundations: Program Managementwith Alan Simon1h 11m Intermediate
1. Why Use Kafka?
2. Core Concepts
4. Installing and Testing Kafka Locally
5. Real-World Examples
6. Distributions and Packaging
Next steps1m 4s
- Mark as unwatched
- Mark all as unwatched
Are you sure you want to mark all the videos in this course as unwatched?
This will not affect your course history, your reports, or your certificates of completion for this course.Cancel
Take notes with your new membership!
Type in the entry box, then click Enter to save your note.
1:30Press on any video thumbnail to jump immediately to the timecode shown.
Notes are saved with you account but can also be exported as plain text, MS Word, PDF, Google Doc, or Evernote.