Redis does not have traditional indexes. In this video, learn about Redis storage structures.
- [Instructor] Welcome to Redis Storage Structures, part of the course Build Complex Express Sites with Redis and Socket.io. In the previous video, we looked at many of the commands for Redis. In this video, we're gonna take at look at designing a data we'll put into Redis. This is important as it is a different paradigm than a classic database. We will do this by discussing what is different and then how that changes what we do. First off, let's make sure we are on the same page.
When discussing the classic database, we are talking about things like relationships and indexes. Now let's use the example we've been using for this entire section of setting up a user database. In the classic database, we would have a user table which would have columns that hold user ID, username, password, when they signed up, et cetera. Then we would have a person table that would hold the user's name, date of birth, stuff like that. Then these would be joined by an ID. We would need to look up users by ID and username, so we would put an index on those fields so that any queries would execute quickly.
Now let's compare this to Redis. We could essentially mimic a table by using hash. This would allow us to add columns or fields in Redis to store similar information, but what about relationships and joints? There is nothing that matches that in Redis. Also, what about looking up a user by either ID or username? We cannot query a hash by field. Everything in Redis can only be looked up by its key. This can definitely create problems if we are not designing our data to match how Redis stores data.
So let's talk about how to actually do this. First of all, using a hash to store information about an object, or user in our case here, is fine. We just need to make sure we use a good key. For example, we will use a user ID, so this would mean that a key would be something like user:1, with one being the ID, and then as different users are stored, that ID would be diff. Remember that when looking up data in Redis, when you know the key, the lookups are pretty much instantaneous.
It is very, very fast when you know the key. So that kind of mimics what we're talking about with the database in that we can have a row with a bunch of columns of data, but what do we need to do if we need to search by username? We can't take that hash and then look up inside of it and say, "I don't know the user ID "which we would have as a key. "I just know the username." Well, we would do this by creating another key that then would point back to the hash. So if for example the username was josh, we would create a key called user:josh and then we would set the value of that to user:1.
We would then look up the username key and then use that to look up the user information. We could do the same thing if we had, for example, like a person hash that we'd look up. Now what's great about this is this essentially is pretty scalable. Any time we wanna index any field in a hash, we just create another key namespace. We would just have to make sure that we name these key namespaces unique and be able to look them up later. And all we'd have to do for the value is always just point back to the official hash or the official row of the data that we want.
So let's go ahead and fire up Redis and take a look at how to do this. We're at the console. Again, I want to highlight that we are using Docker and all the Docker commands are in a text file at the root of this section. So here to run it, we're gonna run our Docker command that connects up to our Redis server that we have set up and we can run Redis commands. Okay, so we're ready. And just like the Docker commands, all these commands that I'm gonna run now are going to be in a file called Redis Commands that you can use.
So let's set up a couple of those user. We will use hmset, which again is hash multi-field setup. So that's our key. This is saying this is the user namespace, user key namespace, if you will, and this is user:1. And I'm gonna do username josh and gender male. Okay. We will do a couple more. We will do user:2, set the same thing.
We will say username chris, and we will say gender female. And now we're going to do a similar thing. This time, we're gonna say user:3. We're gonna say username is chris2 and that the gender is male. Okay. If we do hgetall, we'll do user:1, we can see everything is set. So here, we have a simple three-row user namespace.
If we know the user's ID, we can easily look it up. We just say hgetall user, and then the user ID. So how do we look up by username? Well, we're essentially gonna have to create some more keys to point back to these rows. So we will do username, and then josh. And then we will point it to the correct row. And then we will do this for our other users, so we have a chris as a username, and that is two.
And we have a chris2, and that is three. Now we have these indexes, so now we can run a get and then we can say username:josh, which returns user:1. Then we would run hgetall user:1 and now we have the user's information. So for example, if someone was logging in and then we got their username passed back in from a form and now we have it in Node.js, we can now then say, "Well okay, "we know they typed josh in as their username.
"We can run get username:josh "and look up the key that will then point us back "to the user row." So hopefully this starts to make sense. If we ever want to index on something else, we just go ahead and add rows with that. And again, the key thing is making unique keys that we can then replicate when we have to look up the data. So we have a username, so we put username, as a namespace essentially, colon josh. Let's do a little more complex lookup.
Let's say you want to look someone up by their first name. So we really have no real way of adding a key for every name. Again, this is something where people can have the same name, so we can't have a key that says this is the chris. That is not gonna work. So what we need to do is use a set. The set would store the name and then we would just keep adding the user keys of people that have the same name. So let's do that.
We will do the set add and we will create one called names:josh 'cause we have someone named josh, and we would set the user key. Then we would do, again, set add, and we would say names, we would say chris, and here, we have user:2 and we have user:3, so there. So we can look at these and if we needed to say who is everyone named chris, we just run it, and there we go, we have user:2 and we have user:3.
So let's get even more complex. Let's say we have a question that says we need to know someone that is named chris and that is a male, well how would we look this up? Again, we can't just query the hash. So to do this, we're gonna have to add two more sets. Again, because multiple people can be male and multiple people can be female, we need to use a set because we can't use just a key to tie that together.
So we will set gender, we'll say male, and user:1 is male and user:3 is male. So we will do the same this time, say two, and we wanna say female. So now we have our set set up. Remember when we talked about commands that when you have sets, you can run set theory, and what is a set theory? If we need someone that is both named chris and a male, well we're looking for an intersection.
So that's what we'll run. sinter, and then we say names:chris intersected with gender:male. We run that, we get one user back, and that should be the one that matches. If we do an hgetall, user returned from that, okay. Now technically, we didn't set like a first name field here but we know that username chris2 is named chris and gender male.
So this is how we can even get even more complex queries out of data that is sitting in Redis without having to set indexes on fields. Finally, we will look at one last use case. So let's say we need everyone that has signed up in the last week. Again, everything we have covered so far would not be sufficient for tracking this. We can't set a key. We can't really create a set that stores every weeks of sign-ups because this question is really arbitrary.
And then once we've set up a sorted set, we can use the sorted set rangebyscore and then figure out who has signed up in the last week. So let's look at an example of this. Now I've copied some of these commands because they have specific timestamps. So this is real timestamp of time right around now. Remember zadd is sorted set add.
So at this point, we have three items in our set here and if we say who signed up in the last week, user:1 and user:3 should return. So to look that up, again I have this copied because it has specific timestamps in it to make this work. We are gonna say zrangebyscore, look in the set logins and then the first time is roughly seven days ago.
And then the other timestamp is roughly just a few minutes ago. So between these two, user:1 and user:3 should come back. So we ran it and we see that is correct. Now here, we were just using timestamps but again, this could be anything that we could create a number for and rank people by and then once we have that number and we can compute that number within a range, we now have complete control of how we look it up.
And again, we have user:3 and user:1, so we can now immediately go to the hashes and determine what that is. Again, knowing that when you have that key, the lookup is extremely fast. In summary, in this video, we discussed the difference between classic databases and Redis. Then we covered ways to correctly put data in Redis so that we can look it up later. Now, this is the last video of this section, so in summary of this section, we have laid a nice foundation for Redis.
We have discussed that Redis is an in-memory data structure store, and what that means. Then we discussed the data structures, then we moved on to the commands, and finally, we talked about how to design data to go into Redis in a usable way. In the next section, we will discuss another great feature of Redis, which is messaging, or as it is known in Redis as Pub/Sub, Publish-Subscribe.
This is a great way to send messages between different servers and processes.
Note: This course was created by Packt Publishing. We are pleased to host this training in our library.
- Using Redis
- Structures, channels, and subscribing
- Adding messaging
- Integrating Redis and Node.js
- Working with geospacial indexes
- Using Node.js with Socket.IO
- Working with rooms
- Creating namespaces
- Using Socket.IO and Express
- Sharing states
- Using routes