Learn how to create an SNS topic in AWS.
- [Tutor] Another really powerful capability of S3 is the ability to send notifications to other AWS services when something happens in the bucket. You can send messages to SNS, the Simple Notification Service, to SQS, the Simple Queue Service, or to Lambda where you can actually execute functions in response to something that happens in S3. For purpose of this demo, I'd like to set up a SNS topic and show you how to connect it to the S3 bucket to receive notifications. So let's actually leave the Amazon S3 section here, go out to the main menu by clicking the top left icon here, and go visit Simple Notification Service.
You can always get there by typing SNS and choosing the option that pops up. Here in SNS, if you've never created a topic before you'll see a big blue button that says Get Started. Otherwise just come in here and choose Create Topic. We're going to give our topic a name. We'll call it S3DemoNotifications. Here under the Display Name, we don't have a lot of characters to work with, so we're going to just call this s3notice. I think after eight or 10 characters it will stop us. We'll Create Topic.
Now that the topic is created, we have one more step. SNS implements a publisher subscriber model, meaning that you can have multiple subscribers to the one publisher that is the message topic. So right now we don't have any subscribers. We need to create a subscription on this topic so that we can receive email whenever S3 has a notification for us. I'll click Create subscription. Here you see the Topic ARN. That's the unique identifier for this SNS topic and we'll need it in the next step. For now we'll choose the Protocol and you can see that there are a number of ways that SNS can actually ping someone.
It can hit an HTTP endpoint, it can talk to Email, it can send a message to SQS, the Queue Service, it can run an AWS Lambda, or it can even do a text message via SMS. That's a fun demo but for purposes of this video we'll choose Email. Type in an email address here and click Create Subscription. Now at this point, if you look below, you'll see that this subscription is in a pending state because what I need to do is go to my email, and confirm the subscription. So here we already have an email from the S3 notice notification topic, and if I click the confirm subscription link, we'll get a subscription confirmation for the email, and we can head back to the SNS tab in the web browser.
Now if we reload, we can see the pending message has gone away, and there's an actual ID for this subscriber. The last thing that we want to do is confirm that everything that we just did worked by using the manual Publish to topic button. If we click in here, we can give a subject - testing topic - and a message, let's say, hello world, scroll down and click Publish message. This is exactly the same way that later on when we set things up, S3 will publish to this topic. Of course, its messages will be a lot more complex.
Here in my inbox, you see we have a new email - testing topic. So the notification for my email works. But before we can have S3 start to send notifications to this topic, there's one more thing that we need to do. Head back to the tab containing the SNS console. We need to set a policy on this topic that will allow the S3 service to actually publish to the topic. We'd find that under Other topic actions + Edit topic policy, and now you can see there's a little bit of a gooey here that will let us select some options, but we don't want to authorize a person or an IM user, we want to authorize a service.
So we're going to need to go to the Advanced view and write some json. This is AWS's json policy language and it can be a little bit complicated but don't worry, we've got a template for you in the example files. So I'm going to head over to my favorite text editor. Under chapter two, we have sns_publish_policy_template.json. Open it up in your favorite text editor and have a look at what's inside. So this might look a little bit complicated but I'll take you through what's inside here. So the first two lines here are boilerplate that you'll find at the top of every AWS policy document, so you don't need to worry about that.
What's important is inside these Statement section. This Sid is like a name or an identifier so this can be descriptive, it doesn't really have any execution effect. So we named it allows-publish-from-s3-demo-bucket. The Effect is going to be an Allow. That's what this policy is going to do; it's going to allow something. And it's going to allow that something to be done by the S3 service. Later on we'll narrow that to say not just the S3 service, but a single bucket. The something that this bucket is going to do is the action.
That's the SNS:Publish action. And we need a target. It's not just publish on any topic but on a specific topic; that's our Resource. The value here will be the ARN, the unique identifier of the SNS topic. So in a minute we'll need to go get that and paste it in. Finally there's this Condition section which says the ARN of the principle needs to be a certain ARN. In this case, the identifier of the S3 bucket that's going to be performing the notifications. So clearly we need to go fetch some ARN's and paste them into this document.
Let's head back to the SNS console and grab the ARN of our topic. We can Cancel out of this screen for now and select the Topic ARN from right here. Copy that and head back to the text editor. I'll delete to the end of the line here, paste in the ARN, closed quotation marks, and make sure to have the comma there. Now I need to head back to AWS one more time. We're going to exit SNS and head back into the S3 console.
Now we're going to do these notifications from the demo-primary bucket, and if I click here I'll go into the details of that bucket. But there's a handy way to get the ARN without going that far. If you click the icon of the bucket next to this name, you'll get this sidebar and you'll notice there's a button up here called Copy Bucket ARN. If we click that, we'll have the ARN in our clipboard for pasting. So now that it's clicked, I'm going to head back to the text editor. Again I'll delete the place holder, and paste in the value. Now I have what I want, I'm going to save and exit, and then print out the final result.
Making sure to highlight from one curly brace to the final curly brace, I'm going to copy this policy and head back to the AWS console. We're still in S3 so we'll exit out, go to SNS, browse to our topic using the Topics button here on the left, click the ARN to go into our topic, and choose Edit topic policy. And then under Advanced view, I can highlight and delete what's already here and paste in the topic policy that we just created.
Scroll through to make sure that it's the one that you just made. Now we can click Update Policy and you can see that the topic has been updated successfully. In the next video, we'll go back to S3 and we'll set up the actual notifications, and see them in action.
Join AWS architect Brandon Rich and learn how to configure object storage solutions and lifecycle management in Simple Storage Service (S3), a web service offered by AWS, and migrate, back up, and replicate relational data in RDS. Find out how to leverage flexible network storage with Elastic File System (EFS), and use the new AWS Glue service to move and transform data. Plus, learn how Snowball can help you transfer truckloads of data in and out of the cloud.
- What is data management?
- AWS S3 basics
- S3 bucket creation
- S3 upload and logging
- S3 event notifications
- S3 data lifecycle configuration
- Working with Amazon Elastic Block Store volumes
- Creating and mounting an EFS
- Creating an AWS RDS instance
- RDS backup and recovery
- Moving data with AWS Database Migration Service
- Moving data with Data Pipeline and Glue