Azure Event Hubs and Event Grid are services that assist with delivering event messages throughout a solution. Yes, Event Hubs and Event Grid are similar only when we hear them , but these two Azure services are quite different as far as their actual purpose is concerned. Both has its own significance in the Azure Serverless world and there is a reason for their existence.
This blog will help you find the key difference between Azure Event Hubs and Azure Event Grid, and hence to understand which one to choose for your application. In many cases, the messaging services are complementary and can be used together.
Before jumping into the difference straight away, it is worth to know about Events and Messages.
An event is a lightweight notification of a condition or a state change. The publisher of the event has no expectation about the consumer and how the event is handled. The consumer of the event decides what to do with the notification.
Event is of two types:
Discrete events report state change and are actionable. For example, if we created an invoice for our order, an event could indicate that the invoice has been placed on a storage account. Discrete events can be used to start an action, without indicating what type of action is expected. Each system which receives a discrete event will decide for itself what type of action it will take.
Series events report a condition and are analyzable, which represent a stream of events belonging together. Here we can think of telemetry readings from an IoT device, or logging events from our application as perfect examples.
In short, a message is raw data produced by a service to be consumed or stored elsewhere.
With the above understanding of the Events and Messages, let us now jump into the actual topic.
Event Hubs is a fully managed, real-time data ingestion service that is simple, trusted and scalable. It streams millions of events per second from any source to build dynamic data pipelines and immediately respond to business challenges.
Azure Event Grid allows you to easily build applications with event-based architectures. First, select the Azure resource you would like to subscribe to, and then give the event handler or Webhook endpoint to send the event to.
When to Use
This service can be used when your application deals with the series of events and when you think your application might need a massive scale at least in the future, say a million events and to handle the data that also comes along with the event.
This service can be used when your application deals with discrete events. Predominantly, when there is a need for your application to work in a publisher/subscriber model and to handle event but not the data, unlike Event Hubs.
Let us consider, a real-time e-commerce scenario and see where these two services, Azure Event Grid and Event Hubs fit in.
Azure Event Hub is used more for the telemetry scenarios. Let’s say if every component that’s been used in the Enterprise for this e-commerce application emits telemetry data like Log4Net or Log4J and you want to capture it, then Azure Event Hub can be used. A great example is Azure Application Insights, under the hood it uses Azure Event Hub to capture the telemetry information.
When you’re actually shipping and moving stuff, where you are reacting to things like item has been pulled from the shelf, item was bought to postal area, item has been shipped, or this shipment was rejected and so on, in this case instead of doing a central control workflow you would do in an Event-based model running active workflows for each product item. Here you are reacting to real changes in real-time. Azure Event Grid is ideal for these kinds of reactive scenarios.
Monitoring options for Event Hubs and Event Grid
You can monitor metrics overtime in the Azure portal. The following picture depicts the view of successful requests and incoming requests at the account level:
Also, it is possible to monitor the metrics via namespace of the Event Hubs.
Serverless360 (Out of the shelf product)
Event Hubs Monitoring
Monitor Event Hub State
If the intention is to monitor the state, say the business demands the Event Hub to be always active, Serverless360 can monitor Event Hub state. By associating an Event Hub to Status Monitor or Threshold Monitor, it is possible to monitor the state, compare against the expected state and alert through configured notification channels.
Monitor Event Hub on its Metrics
If the intention is to understand the efficiency, reliability, capture utilization of the Event Hub then the choice should be a Serverless360 Data Monitor. Data Monitoring can be configured for an Event Hub on an extensive set of metrics. Event Hub data monitoring with appropriate warning and threshold values can be set as in figure below:
Event Grid Monitoring
Data monitor can be configured with appropriate metrics to monitor Event Grid Topics and Subscription in various perspectives like efficiency and performance.
The publisher can be anything which sends telemetry of events to the Event Hubs.
The event source of the Event Grid can be of any one of the following:
- Azure Subscriptions (management operations)
- Container Registry
- Custom Topics
- Event Hubs
- IoT Hub
- Media Services
- Resource Groups (management operations)
- Service Bus
- Storage Blob
- Azure Maps
Having multiple handlers or listeners in Event Hubs listening to the same partition is a bit tricky. If you straight away assign all the recipients to the same consumer group that listen to the same partition, then duplicate events will be received by the event handlers. You need to assign each listener to a unique consumer group.
The event subscribers of the Event Grid can be of any one of the following:
- Azure Automation
- Azure Functions
- Event Hubs
- Hybrid Connections
- Logic Apps
- Microsoft Flow
- Queue Storage
- Service Bus (Preview)
- Webhooks (anything)
Using the batch option available in Event Hubs, one can send a new batched message event to an Event Hub. Batching reduces the number of messages that are transmitted by merging information from multiple messages into a single batch of messages. This reduces the number of connections established and network bandwidth by minimizing the number of packet headers that are sent over the network.
When using a custom topic, events must always be published in an array. This can be a single batch for low-throughput scenarios, however, for high volume use cases, it’s recommended that you batch several events together per publish to achieve higher efficiency. Batches can be up to 1 MB. Each event should still not be greater than 64 KB (General Availability) or 1 MB (preview).
The Advanced Message Queuing Protocol 1.0 is a standardized framing and transfer protocol for asynchronously, securely, and reliably transferring messages between two parties. It is the primary protocol of Azure Service Bus Messaging and Azure Event Hubs. Both services also support HTTPS.
Azure Event Grid provides security for subscribing to topics, and publishing topics. When subscribing, you must have adequate permissions on the resource or event grid topic. When publishing, you must have a SAS token or key authentication for the topic.
Geo Disaster Recovery
When entire Azure regions or datacentres (if no availability zones are used) experience downtime, it is critical for data processing to continue to operate in a different region or datacentre. As such, Geo-disaster recovery and Geo-replication are important features for any enterprise. Azure Event Hubs supports both geo-disaster recovery and geo-replication, at the namespace level.
Event Grid now has an automatic geo disaster recovery (GeoDR) of meta-data not only for new, but all existing domains, topics, and event subscriptions. If an entire Azure region goes down, Event Grid will already have all your event-related infrastructure metadata synced to a paired region. Your new events will begin to flow again with no need for any intervention by you.
It stores the event data for the maximum period of 7 days and the retention days can be customized if needed.
When Event Grid can’t deliver an event, it can send the undelivered event to a storage account. This process is known as dead-lettering. By default, Event Grid doesn’t turn on dead-lettering. To enable it, you must specify a storage account to hold undelivered events when creating the event subscription.
There is no out of the box filtering option in Event Hubs, but we can play a trick here. We shall use the selected partitions in the Event Hubs to only store particular events and assign the listener to those partitions who is interested in it.
One of the significant capabilities of Event Grid is routing using filters and event type. Furthermore, Event Grid offers advanced filters making the routing even more powerful. An event subscription can contain prefix, suffix and event type filters. The event type filter will filter on the event Type property. You can add multiple event types separated by a semi-colon, wild cards do not work. The prefix and suffix filters will filter on the subject property.
Community ContentYou can also refer: Azure Event Hubs vs Service Bus
I believe you find this article useful. Do follow us for more updates on Azure Integration space. Happy Learning!