Free Trial Book a Demo

Azure Event Hubs and Event Grid better managed and monitored using Serverless360

Nishanth Prabhakaran

6 Mins Read | Published on Jul 11, 2019

event-hubs-grids-better-managed-monitored

In our previous webinar, “Azure Logic Apps better managed and monitored using Serverless360” we discussed the various capabilities of Serverless360 in managing and monitoring Azure Logic Apps with the Real Time Cab Booking Scenario. In this Webinar, we will be focusing on management and monitoring capabilities for Azure Event Hubs and Azure Event Grid in Serverless360.

Key Takeaways

  • Understanding Azure Event Hubs and Event Grids and their application
  • Managing Azure Event Hubs and Event Grids with deep integrated tooling in Serverless360
  • Consolidated monitoring from Serverless360 to monitor Azure Event Hubs and Event Grids
  • Tracking event processing activities done by the users in Event Grid with Serverless360
  • Achieving advanced security in dealing with Azure Event Hubs and Event Grid

What is Azure Event Hub?

Azure Event Hub is a big data streaming platform and event ingestion service. It can receive and process millions of events per second. Event Hub represents the “front door” for an event pipeline.     

                                                                                    Event-Hub

                                                                Azure Event hubs – Event Ingester

What is Azure Event Grid?

Azure Event Grid is a fully managed event service that enables you to easily manage events across many different Azure services and applications. Made for performance and scale, it simplifies building event-driven applications and serverless architectures. It consists of Event Grid Topic and Event Grid Subscription.

                                                    Event-Grid-Topic                                                                  Event-Grid-Subscription

                           Event Grid Topic – Event Publisher            Event Grid Subscription – Event Subscriber

Where are they used in Business?

Azure Event hubs and Event Grid are used in business orchestrations for event storage and handling. Let’s take a simple Vehicle availability management scenario for better understanding.

In the orchestration above, the Event Hub Capture feature automatically writes batches of captured events into the Azure Storage blob containers and enables timely batch-oriented processing of events.

The Event Hubs emits an event to the Event Grid when the capture file is created. These events are not strongly correlated and don’t require processing in batches. Hence, Event Grid is selected to provide a reliable event delivery at massive scale.

Event Grid delivers the event to the Azure Relay which securely exposes the service that runs in the corporate network to the public cloud. Therefore, the actual business logic to process the telematics data in the storage blob container for decision-making analytics resides in the on-premise service.

Challenges in the Azure portal

We have seen how powerful the Azure Event Hubs and Event Grids are but there are some challenges in managing and monitoring them in the Azure portal.

Here are some business challenges that Azure users face in managing and monitoring Azure Event Hubs and Event Grids in the Azure portal.

  • No Integrated tooling
  • No consolidated monitoring
  • No dead-letter event processing in Event Grid
  • No Auditing

Obviously, the solution to solve these challenges would be Serverless360, the one platform to manage and monitor the Azure Serverless Application 

Integrated tooling

In the FlyWheelCabs system, there might be a requirement like importing an Event Hub from Non-production Namespace to Production Namespace. There is no straight forward solution available in the Azure portal to achieve this. Serverless360 solves this challenge by bringing in the capability called import entities. With this capability, the user can import their Event Hub entities from Non-production to Production Namespace in a single click

Import-Entities

It is also possible to manage the shared access policy for Azure Event Hubs, view the properties of the Azure Event Hubs and Event Grids and create consumer group for Event Hubs.

Event Processing

In our scenario, there is a chance that the events would become dead-lettered when the Relay attains the maximum number of listener count. Those events may be business critical events and they need to be processed. To solve this business-critical problem. Serverless360 comes with the solution called Dead-letter Event Processing with which user can repair & resubmit the event to the Event Grid Topic. We can also view the event details

Event-Processing

Activities

When there is a need to test our business orchestration, we need to send some events to Event Hubs so that event will send to the Endpoint through the subscription. But in the Azure portal, it is not possible to test the orchestration. To solve this challenge, Serverless360 brought in the capabilities called Activities with which user will be able to send events to both Event and Event Grid Topic. It is also possible to schedule these Activities too.

Activities

Read more on this feature here.

Monitoring

Azure provides the entity level monitoring on their metrics, but the actual need would be Consolidated monitoring at the application level. 

For monitoring Azure Event Hubs and Event Grids in multiple perspectives, Serverless360 has three types of monitors: Status Monitor, Threshold monitor and Data monitor.

Status Monitor

Choose Serverless360 status monitor to get application health reports at a specified time in a day representing the state of Azure Event Hubs against the desired values of its state.  It is also possible to monitor the partition of the Azure Event Hubs.

Status

Threshold monitor 

Monitor your Azure Event Hubs when their state violates desired values for a specified period, say few seconds/minutes. For Event Hubs, It is possible to monitor the status and size of the partitions in Threshold monitor

Threshold

Data monitor

There would be needed to monitor the performance of the Event Hubs, Count of events in the Capture of the Event Hubs or Destination processing duration of Event Grid Subscription. To monitor the entities on their extensive set of metrics, Serverless360 has brought in the capability called data monitor which provides the calendric view of historical alerts.

Data-monitor

User management

Though there is a Role Based Access Control system available in the Azure portal it is challenging to come up with an appropriate custom role with need only permissions on the entities in the business application. Custom roles in Serverless360 are simple and straight forward. Provide permissions at the application level. Define a granular user access policy defining actions the role can perform.  For Azure Event Hubs and Event Grids, it is possible to give or restrict access to manage, monitor and Event Processing for Event Grid.

User-management

Governance & Audit

When multiple users are authorized to access the Azure Event hubs or Azure Event Grid, it is necessary that their actions on the Azure applications are audited. Serverless360 tracks every user action on the Azure entities associated with it. For Event Hubs, we can track the CRUD operations. For Event Grid, it is possible to track the Event processing. It is also possible to export the reports to excel or pdf format. 

Governance-Audit

Conclusion

In this webinar, we saw what is Azure Event Hubs and Event Grids? And how they can be better managed and monitored using Serverless360? Watch the recording of the webinar below.

 

Tags: ,

Azure Event Grid Monitoring Management CTA
Advanced Cloud Management Platform

Serverless360 helps to streamline Azure monitoring, distributed tracing, documentation and optimize cost.

Free Trial
Integrate 2023 On Demand Videos