What is the difference between SNS, SQS, and Kinesis? - Amazon Web Services (AWS) Tutorial

- [Instructor] In this chapter, we are going to use SQS to send a message to trigger another AWS Lambda. And because of that, you might be wondering, why we don't use Kinesis, or why don't we use SQS for the whole system? In this video we are going to start explaining the difference between the different messaging services offered by AWS. And later in the chapter we will continue with the project. What is SQS? Amazon Simple Queue Service is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and server-less applications. The sender sends the message to the queue, messages are not pushed to receivers, receivers have to call the SQS service to receive the messages. Messages can be received by multiple receivers at the same time. When one receiver receives a message, processes, and deletes it, then the other receivers will not be able to receive the same message. Calling inherently introduced some latency in message delivery in SQS, unlike SNS where messages are immediately pushed to subscribers. Why use SQS? A message queue is one of the best tools to decouple and scale microservices, distributed systems, and server-less applications. Using a queue you can send, store, and receive messages between software components at any volume without losing messages or requiring other services to be always available. Building applications from individual components that each perform a discrete function improve scalability, reliability, and is the best practice design for more applications. A reliable queue placed between components allows you to leverage many integration partners for connecting service. Queues are great to decouple server-less applications. We can chance the receiver A for another receiver and the whole application will keep off working as before. Sender A doesn't need to know about this change. Limitations with SQS. Single consumer. SQS, once a consumer has processed a message from the queue, that message is removed and no other consumer can read that message message replay ability. As messages are removed after they are processed, SQS does not support replaying messages that have been already published. SNS is another kind of delivery service that AWS offers. Amazon Simple Notification Service is a highly available and fully managed pub/sub messaging service. SNS is used in general to decouple microservices, distributed system, and server-less applications. SNS provides communications from many publishers to many subscribers. A publisher can fan out messages to large numbers of subscribers for parallel processing. Also, SNS can be used to trigger notification in end user using push notification, SMS and email. So let's talk about Kinesis in the same lines as we talk about SQS and SNS. The primary use case is collecting, storing, and processing real time data streams. Data streams are data that are generated continuously by thousands of data sources which typically sends the data records at the same time. Kinesis streams provide streaming capability for events. There can be multiple data producers and multiple data consumers. Each consumer can have their own progress of reading events, and can also go back in time and read old events as many times as they need. So how does Amazon Kinesis data streams differ from Amazon SQS and SNS? Amazon Kinesis data streams enables real time processing of streaming big data. It provides ordering of records as well as the ability to read and or replay records in the same order to multiple Amazon Kinesis applications. Kinesis deliver all records for a given partition key to the same record processor, making it easier to build multiple applications, reading from the same Amazon Kinesis data stream, for example to perform counting, aggregation, and filtering. Amazon SQS offers a reliable, highly scalable hosted queue for storing messages, it's travel between computers. Amazon SQS lets you easily move data between distributed applications component, and help you to build applications in which messages are processed independently such as automated work flows. SNS is a fully managed pub/sub. The messages are pushed immediately to the consumers. For our use case, Kinesis is great, as we expect to have a lot of tech orders, and we need them to be processed in real time. And when our system is in production we will have different consumers for the events. For example, we could plug the accounting system to the order place event. Also, we don't want the messages to be sent automatically to the consumers, because in the future if we have a lot of orders we might want to batch the order processing to make the system more efficient.