Learn how to send and receive some messages in a test Kafka topic running locally on a machine.
- [Instructor] Now, let's take a look at using Kafka to send and receive messages. So, what we need to do is have a couple more terminal windows here and in one, what we're going to have is the producer and in the other one we'll have the consumer. And then we'll type messages into one and see it pop up in the other window. So, we already have Zookeeper running and Kafka server running in the background. And I'm going to minimize this just for now, so we have more real estate to work with here. Now, we created a topic for test.
And so, what I want to do first is just run the producer by calling the script file that Kafka gives us. And I'll just copy this from the exercise files here. Now with that running, anything I type into this command will actually be sent into my Kafka cluster. But in order to make sense of that, I'm going to create a new terminal window here, I'm going to open up a new one. And just below it, what I'll do is I will set up the consumer. That way, any messages I type in, you'll see them instantly pop into the consumer.
So, to do that, we need to run the Kafka consumer script, and I'm going to tell it which topic to look at, the topic of test, and where the bootstrap-server is running. And that's localhost at port 9092. Then, I'm going to tell it where to read from, as I mentioned before. And in this case, you can pass in an offset, but here I'm saying from-beginning. So, I'll paste this in. Make sure you're in the right directory. And that'll be wherever you've extracted Kafka to.
And so far, so good. I have it running, however there's nothing going on because I haven't sent any messages. So, I'm going to go up to the top one and I'll just type in message 1. And you can see it instantly, or nearly instantly, appeared in the one down below. So I have the producer on top, which sends the message over to Kafka and it does the whole thing where it has a consumer group and then the consumer down below is receiving that. So I can type something like, hello world, how are you? So on and so forth.
So this is essentially the very basic way that we can send messages to Kafka using the producer script and also read from them using the consumer script. This is just a really good example here to learn about how this works. And I wanted to keep it simple so that way we're not getting too deep into the weeds of the configuration, and you can just see it in action and get a sense for how it actually works.
- Understanding the Kafka log
- Creating topics
- Partitioning topics across brokers
- Installing and testing Kafka locally
- Sending and receiving messages
- Setting up a multibroker cluster
- Testing fault tolerance