In this video, Sharon provides an overview of the Azure Event Hub service. She explains the purpose and benefits of using this service, and also outlines a sample workflow.
- [Instructor] To wrap up this chapter, let's go ahead and take a look at Event Hubs. Azure Event Hubs are essentially a data streaming service that receives and processes millions of events per second. This makes this service a very high throughput system with very low latency, and you may be wondering what type of applications would need such a large scale solution? Well, think of the millions of IoT devices out there, such as your health trackers or your mobile devices that provide the best route for us to get to our destination based on real time data, and sometimes you'll hear the Event Hub referred to as the event ingestor.
There are several common uses for the Azure Event Hub including logging and telemetry. Think about dashboards in this case. Those dashboards can be providing you with realtime data about your maybe truck fleet, or all the elevators that you service. Another common use would be for transaction processing, or for data archiving, or capturing your data and storing it in Azure Storage. Let's take a quick look at the Event Hub Workflow as it relates to Azure.
In our first column, over to the left, we have an event producer, and these event producers could be devices or applications. All of these events need to be collected and directed to the Event Hub, and this can be done through gateways or the event producer may directly speak to the Event Hub. Next, this is where the Event Hub comes into play. It ingests all of those events, and then passes those events along to the appropriate processing service, and that could be a service bus, Azure databases, HDInsight, Azure Storage, or Azure Data Lake, and from there, the data could then be pushed out to a presentation or action such as a search or to a dashboard.
Now, let's break down the previous workflow, and look at the Event Hub architecture itself. I'll be using terminology here that we haven't covered yet, that will be following. First, over to the right, we have our event producers. Again, I would like to think of the event producers as IoT devices. The event producers publish the events using either HTTPS or the advance message killing protocol, AMQP. This is passed to the Event Hub where we have partitions that separate the data so each consumer group which is the consuming application, can only read from a specific partition, and then that data is then pushed out to the event receiver.
Now, let's explore some of the bits and pieces that make up the Event Hub and clarify some of that terminology. First of all, let's start off with those protocols. I just mentioned that you can use HTTPS or AMQP. If you use AMQP, you need to be aware that it requires a persistent bidirectional socket, TLS, Transport Level Security, or SSL/TLS is required, but AMQP does have a higher performance than the HTTPS protocol.
We also mentioned partitions which are a logical sequence of events that are stored in the Event Hub, and newer events are added at the end of the partition. We have our older events, and as new events are added, they're added to the end of the partition. We also mentioned consumer groups, and consumer groups are processes that consume messages from the Event Hub, and the partitions can only be accessed via consumer groups. The event receivers can not transverse and grab information from the partition directly, and Microsoft recommends only one receiver on a partition per consumer group.
And finally, let's talk about the pricing tiers. You'll notice our image here only has two pricing tiers. Now I'll talk about that in a moment, but we do have three pricing tiers that we can choose from when using Event Hubs. We have the basic, standard, and dedicated tiers. Both the basic and standard tiers will allow for a message size of 256 KB. The basic tier will allow for one consumer group whereas the standard will be 20 consumer groups. The basic allows for 100 broker connections, and the standard, 1000 broker connections.
Each will also support one day message retention, whereas in the standard tier, we can have up to seven additional days, and we do have that third tier, dedicated, and this is only available in the EA or Enterprise Agreements only. The message size in the dedicated tier is one meg, and you have a seven day retention. Event Hubs are best used to process streaming data, and as I said during the lesson, I always think of Event Hubs and IoT devices. Event Hubs are to consume massive amounts of data and pass it along.
- Creating compute-intensive applications
- Creating long-running applications
- Implementing messaging systems
- Azure Service Bus relays
- Using Azure Storage queues
- Creating an Azure Event Hub
- Creating Azure WebJobs
- Managing cloud environments with Azure Active Directory Domain Services