In this video, look at the Redis commands related to sets and sorted sets as implemented in the StackExchange.Redis library.
- [Instructor] Returning to Visual Studio, let's use C# to manipulate a set on Redis. I'll copy and paste the last file, rename it for the chapter, 06_03, rename the class, and I'll delete all of the content in the method.
(typing) And this'll be all about a set. Now scrolling up, just be reminded from the previous exercise, we have a Redis connection string from Azure; and we have a configuration object for which we go and set an additional flag here. We ask for allow admin, and then we flush the database before we run these tests. So this method as a test method will have an empty Redis database to start. So going back to our idea of having a short message blogging service or a Twitter clone or something like that, one of the features people expect is to be able to follow other users, whether it's an RSS feed or a Twitter-like following of a feed of posts from another user.
We would need to keep track of who's following whom, and so we can ask Redis on a basis of sets to create a set where we add members to a set; and so I'll create the key, and I'll structure the key internally again. So this is our short message blog; this is our application prefix for the key, and we'll have the user Anton. And I'll have Anton following some users.
So we'll create the set, so key value pair where the key points to a data element of type set; and in that set, we will list all of the users that Anton is following. So, in this case, so it's keyAntonfollowing. And given that key, we're saying SetAdd. And so we're adding elements to an existing set, and it's just Redis values, just strings; and I'll do one value.
And so this'll be a user name. I'll just call it userA, and I can do that multiple times; and, of course, we have some overloads. Comma, down arrow, and notice the Redis value array. (typing) And let's just assume the user Anton is actually following both userA and userB. Of course, I'm making assumptions that these two users do exist; but that's up to our authentication authorization system.
For right now, we're just going to keep track of who Anton is actually following as users. Now the trick here, of course, is that we're saying in one data structure that Anton is following these two users; but there's no way to go back in reverse. If I am userA, who is following me? Well, let's stick to the Anton user for a moment; so Anton following and Anton follower. (typing) So now we to have a second key; and in this case, who would be the followers of Anton? Well, we would again repeat the user name here; and we would have a new set.
(typing) And I'll do an array again, just making the assumption there would be more than one value. That's my ego you're seeing in action. And so what we're saying is depending on which perspective we take; so looking at the user, Anton, which other users is Anton following? So pointing out to those users interested in content from those users; but there may be people following Anton, so Anton's followers, might be other users interested in Anton's content.
(typing) But what you probably noticed now is that this is all from the point of view of the user Anton; and so, at this point, we are going to have to do some duplication because what about userA? What about userB? They, themselves, have their own perspectives, their own points of view; and so now to add a follower to a user's content actually will be two operations.
To add to following and to add to followers. So let's start with the first one here. So if the Anton user exists and the Anton user is following user A and B, then userA should have its followers collection updated; and userB should have its followers collection updated. (typing) So this'll be the short message blog for userA. That user will have followers, and that's, that user's follower is Anton.
And so in the same way, userB has the follower Anton; and so these two operations is essentially the same one operation. If you want to say Anton is now following content from userA and from userB, you should also go and update the collections for userA followers and userB followers; and so if you're a relational database-type person, that's how you're currently thinking, you will ask, well, why isn't there a relationship between these two? Well, we don't have a table and we don't have entities that are linked with relationships at all.
So we get incredible speed with additional complexity, but we have to manage the structure of our keys and the structure of our data within the constraints of these data structures. And so let's do the same again down here for the following. (typing) So if Anton has the followers userC and userD, then userC and D is following Anton.
So small message blogging platform, (typing) userC has the following users following content from userC; and, similarly, small message blogging service for userD (typing) has the user Anton following userD's content. And so you'll see it kind of reverses in the key names here, and this is one of the things that you will have to start designing is how do you structure your keys, what content are held within a key as your primary means of accessing data, and how do you link back and forth? Is there a link in both directions, and do you need to maintain data structures to be able to walk that link in both directions.
So Anton following A and B, A followers, B followers being Anton, and Anton followers C and D; so C following Anton, D following Anton. And you can imagine, yes, you can still draw this on a white board; this is still worth designing for. Now in the case of Twitter followers, a set probably appropriate because we don't care about the order; we just care that there's many of them. If we are insisting on unique user names, this'll work. Remember, the elements of a set have to be unique; but if we do allow duplicate names, then we should probably consider a scheme by which we use identifiers of some kind that is unique within the system.
So I've not implemented that here, but let's just run a couple of checks to make sure our unit test is working. And now we're looking for the cardinality of the set. In the C# library, this is called set length. So you're not going to find the word "cardinality" in here. So by the key, key Anton following; and so this number will check that in Anton following count is equal to two in this case.
And here we'll do something similar, var nUserAFollowers. So set operation, get the length for the key, copy, paste, and check that in UserA followers is equal to one.
Build. Right-click on six and three, run test; and we should be okay. Feel free to add more check constraints in your unit test checker asserts to make sure your thinking matches what's happening in the background. But essentially we have a set operation here where we're just looking at sets to create structure so we can determine who follows who in terms of content, and you notice very clearly that suddenly it seems like we have twice as much going on than we might have in a relational database.
But not really; everything is accessed by key. So essentially a primary key lookup; and so if you think of a key value store you might have in a SQL database, we would have two entries. In this case, we have an entry for the one direction; and we have an entry for the opposite direction. So all relationships here are determined by key value pairs. A set is very handy for data like this where ordering does not matter, but we also have ordered sets. (typing) And so now, for data where the ordering does matter, we can use an ordered set with a score on each element to help determine the ordering of the elements within the set.
So let's create a list of posts or messages, blog entries by a user. (typing) So for assorted set, let's add an element; and we need a key, and then we want the elements in the set. So new, sorted, set entry array; and we'll have an array of elements listed down here. Now, firstly, for the key, bear in mind that I'm going to do this on a per user basis. So I'll have a set of blog posts or messages by an individual user in this design.
And so the key for Anton posts would be structured again, so short message blogging service for user Anton. This'll be all the posts by Anton. So that key variable here; provide it here. We have a single set, and the elements within that set will be sorted. And so a new sorted set entry would consist of the blog post message itself plus, notice the second element there is a score for this entry.
Now, this is just a number; but this can get really interesting. So let's start this with one and two, and we'll do hello world, welcome to short message blogging service and so forth. So as we repeat this sorted set entry, the order in which we add them to the set is really not relevant; the ordering will be determined by the score on each element. So, in some cases, it might be a matter of time.
So if you think of today, so we can ask date, time, control dot if we're using system, .utcnow.ticks. That's a nice big number. So it'll be a long, and it'll give us a unique number. This is a Windows thing, of course. A unique number of ticks from when the clock is considered to have started, and it just gives us a long that helps us order. So if this is today's post, welcome to the SMB service, (typing) then we can copy and paste this for tomorrow maybe or yesterday.
So DateTime.UtcNow.adddays or just minus one in there. So that becomes yesterday. So hello world might have been posted yesterday; (typing) and the welcome message with, maybe, your logging in instructions, et cetera, might have been posted today. I do need to add ticks in there. I meant for that to be a double. So now these two variables, long and long, that can cast to the double that is the second parameter over here.
And so now we've added two elements to this set, and the set can be manipulated in order of elements. So let's do this again so we can see the power of this server side. So I'll create a new key; so this'll now be userX, (typing) and that user will have posts as well. Copy and paste. So now userX key and userX can say hello world and userX can say welcome to the SMB service.
So now we have two separate sets, two separate key value pairs, and each set contains two elements. Each set uses a number to determine the date time and thus ordering of these particular elements. And now just to be clear, I do want to make sure these are offset somewhat. So if hello world was posted yesterday by Anton, it's going to be yesterday plus one, plus one tick. So a very, very small time interval there; but making it clear that userX posted their hello world second.
And similarly for today; I'll just offset that ever so slightly. Now having done this means we have two keys on the server. When we look at the data structure for each key, it'll be a sorted set; and when we look at the set itself, there's two elements and they're ordered. So if I want a timeline, so if I want the front page of WordPress.com or the front page of LinkedIn, I want to see a series of posts; and in that series of posts, I want to see content from multiple users and probably in chronological order.
So in that case, we want to ask the server to combine these sets for us. So I can say dear Redis, for sorted set, I know that you have more than one; so please combine two sets for me. And look at the cleverness here. Its combine and store, so the server will do the server operation; and there will be resulting data structure on the server. Great, you don't have to fetch it if you don't want all of it. So the first thing we'll do is we'll say, okay, I will combine and store.
So what is the operation that you want to do? The set operation can be a difference, so minus one from the other; an intersect (mumbles) show me what is common for bother of them; or a union, which is what we want from a timeline. So given the data we have, we want a union. We want a combination of all posts by all users, and we'll provide a new key for the new set that'll be the result of this operation. I'll just call it timeline, or maybe better homepage.
Okay, and what is the input for the combined operation or the union operation? Well, it'll be the key for each of the sets we want to operate on. So the first set and the second set for this operation as identified by their keys. So these two sets will be union, and a third set will be created with the name homepage; and so now the homepage will contain all of the elements from both the sets of key userX and key Anton.
So let's first see how many elements end up being in that homepage. So I'll ask Redis, hey, how many elements in that set? Sorted set length, and we provide the key homepage. Now this is just for my unit test, (typing) and I really hope that'll be four.
Save All, Ctrl + Shift + B for build, run our ordered set test. I have a green outcome, so homepage post count is equal to four. So these two sets have been combined into a set called homepage, and that homepage set contains four elements. So we want to inspect it. (typing) We can actually fetch all of it.
Instead of fetching everything, you can imagine a homepage will have a limited length. Just consider the limited bandwidth of your user; so can ask, hey, sorted set, get me the elements from the set. So as a range like we had for a list, but by rank. So now look at the overload here. So it's sorted set, get a range of elements by rank. So starting at zero, and you can go all the way to the end. So the last one, minus one; second last one, minus two; and so forth. Or you could just say, you know what, I would like to have for the key from its set, the first element; and maybe the first two elements is as much as I can realistically display for my homepage.
And so when I hit the break point here or I click debug, not run, or I click debug the test. Look at the contents of this variable here. The homepage posts collection has two elements: hello world and hello world from userX. And so what we've asked for is a union to be done server side, purely in memory operation on Redis, no additional traffic over the network.
And after the union completes, only then I picked out the elements that I wanted from that set. In this case, the two elements that I could handle in homepage. Click Stop; keep an eye on your test. You might need to click cancel, and I'll remove the break point. So with a set, we have all the set operations we could desire and an unordered collection of items. With ordered sets, we have to keep track of the idea, not just of the key with our elements within the set, but also here, the second parameter when we create an element for the set, a score.
So now we have to carry two pieces of information for each element in the set, and the second one we can use for ordering and also for retrieving subsets of elements from our set. Union still works, intersect still works, and so forth; but then as a result, the outcome, the resultant set can again be used as an ordered set. And so now we have a very powerful tool to build applications that essentially rely on set operations server side, and the results of those sets are what we're going to be retrieving client-side.
- What is Redis?
- Installing Redis on a Windows machine
- Deploying the latest version of Redis into a container
- Connecting with C# clients
- Working with data types in Redis
- Key expiration and expiration policies
- Working with lists, sets, and hashes in Redis
- Working with batches and transactions
- Using Redis with ASP.NET web apps