Learn how to test the fault-tolerance of the system by killing one node and monitoring its status.
- [Instructor] Now that we have our multi-broker cluster up and running, and our replicated topic, I thought it'd be good for us to test the Fault Tolerance of it, and actually see what happens. I'll open up a new terminal window here, and I'll just resize this a little bit, so you can read it better. And first, what I want to do is, set up my producer. So, this is the thing that's going to send messages to my topic. Make sure that you're in the correct folder, which is wherever you extracted kafka.
And, I'm going to run the Console Producer. Tell it where the broker list is, and what the topic I want to produce messages to is. Now that that's there, I'll shrink that down, and I'll create a new window, which is going to be my consumer. So, we'll shrink this one down, and resize it a little bit, so it's easier to read. Make sure that we're in the correct directory here. And we're just going to run the consumer, and point it at my replicated topic. So now, if you have set this up before, and done anything, you'll have some messages in there, but it doesn't matter because the producer's going to send new ones to it.
So, if I say test 7. You'll see it pop down below. Test 8. If you're running this for the very first time, you won't see any of those initial messages. But, don't worry, it just means that it is running, and that there were already messages in there. So, you can continue on from here. Now, the next thing to do is, actually to test the Fault Tolerance. This is where we need to go find where one of our servers is running. So, I'm going to use this command here, which will search for that. I'll open up a new terminal window, and again resize it so you can see a little bit easier.
And here, I'm just going to paste this in. And, it's basically just going to search for the processes using the server-1.properties. You'll get this big, long mass of text, and if I scroll all the way up to the top, the first number here, in my case, 49297. And it'll be different for every machine. This is the one that is the process ID of our first server. So, if I want, I can go kill that process, by typing kill -9 and pasting it in.
Now, if you're on Windows, there's some different commands that you'll need to use. And I've listed them here for you. But again, you'll want to make sure that you're using the correct process ID. So, the process ID I list, won't be the same one that you have on your local machine. Now we can check which node is the leader for our topic. I'll copy the command here, using the topic script. First, let me just clear the screen, jump over to where I have kafka installed, clear it again.
If I expand this out, you can see what's going on. So, I have one partition and a replication factor of three. The partition zero is the first one, and then the leader is partition two. And you can see the other replicas there. Now, if we try reading a message again from this. Do a new window. Cd to the downloads, where I have kafka, and I will run that command to read the consumer from that.
You can see that all of the messages were still in there. So, we essentially shut down one of our servers, but the system continued to persist. In fact, all the data was still there. We didn't lose any data. And that is a demonstration of, essentially, the resiliency of this system. And, a big reason why you want to consider using kafka.
- Understanding the Kafka log
- Creating topics
- Partitioning topics across brokers
- Installing and testing Kafka locally
- Sending and receiving messages
- Setting up a multibroker cluster
- Testing fault tolerance