In our previous webinar, “Azure Logic Apps better managed and monitored using Serverless360” we discussed the various capabilities of Serverless360 in managing and monitoring Azure Logic Apps with the Real Time Cab Booking Scenario. In this Webinar, we will be focusing on management and monitoring capabilities for Azure Event Hubs and Azure Event Grid in Serverless360.
Key Takeaways
- Understanding Azure Event Hubs and Event Grids and their application
- Managing Azure Event Hubs and Event Grids with deep integrated tooling in Serverless360
- Consolidated monitoring from Serverless360 to monitor Azure Event Hubs and Event Grids
- Tracking event processing activities done by the users in Event Grid with Serverless360
- Achieving advanced security in dealing with Azure Event Hubs and Event Grid
What is Azure Event Hub?
Azure Event Hub is a big data streaming platform and event ingestion service. It can receive and process millions of events per second. Event Hub represents the “front door” for an event pipeline.
Azure Event hubs – Event Ingester
What is Azure Event Grid?
Azure Event Grid is a fully managed event service that enables you to easily manage events across many different Azure services and applications. Made for performance and scale, it simplifies building event-driven applications and serverless architectures. It consists of Event Grid Topic and Event Grid Subscription.
Event Grid Topic – Event Publisher Event Grid Subscription – Event Subscriber
Where are they used in Business?
Azure Event hubs and Event Grid are used in business orchestrations for event storage and handling. Let’s take a simple Vehicle availability management scenario for better understanding.
In the orchestration above, the Event Hub Capture feature automatically writes batches of captured events into the Azure Storage blob containers and enables timely batch-oriented processing of events.
The Event Hubs emits an event to the Event Grid when the capture file is created. These events are not strongly correlated and don’t require processing in batches. Hence, Event Grid is selected to provide a reliable event delivery at massive scale.
Event Grid delivers the event to the Azure Relay which securely exposes the service that runs in the corporate network to the public cloud. Therefore, the actual business logic to process the telematics data in the storage blob container for decision-making analytics resides in the on-premise service.
Challenges in the Azure portal
We have seen how powerful the Azure Event Hubs and Event Grids are but there are some challenges in managing and monitoring them in the Azure portal.
Here are some business challenges that Azure users face in managing and monitoring Azure Event Hubs and Event Grids in the Azure portal.
- No Integrated tooling
- No consolidated monitoring
- No dead-letter event processing in Event Grid
- No Auditing
Obviously, the solution to solve these challenges would be Serverless360, the one platform to manage and monitor the Azure Serverless Application
Integrated tooling
In the FlyWheelCabs system, there might be a requirement like importing an Event Hub from Non-production Namespace to Production Namespace. There is no straight forward solution available in the Azure portal to achieve this. Serverless360 solves this challenge by bringing in the capability called import entities. With this capability, the user can import their Event Hub entities from Non-production to Production Namespace in a single click
It is also possible to manage the shared access policy for Azure Event Hubs, view the properties of the Azure Event Hubs and Event Grids and create consumer group for Event Hubs.
Event Processing
In our scenario, there is a chance that the events would become dead-lettered when the Relay attains the maximum number of listener count. Those events may be business critical events and they need to be processed. To solve this business-critical problem. Serverless360 comes with the solution called Dead-letter Event Processing with which user can repair & resubmit the event to the Event Grid Topic. We can also view the event details
Activities
When there is a need to test our business orchestration, we need to send some events to Event Hubs so that event will send to the Endpoint through the subscription. But in the Azure portal, it is not possible to test the orchestration. To solve this challenge, Serverless360 brought in the capabilities called Activities with which user will be able to send events to both Event and Event Grid Topic. It is also possible to schedule these Activities too.
Read more on this feature here.
Monitoring
Azure provides the entity level monitoring on their metrics, but the actual need would be Consolidated monitoring at the application level.
For monitoring Azure Event Hubs and Event Grids in multiple perspectives, Serverless360 has three types of monitors: Status Monitor, Threshold monitor and Data monitor.
Status Monitor
Choose Serverless360 status monitor to get application health reports at a specified time in a day representing the state of Azure Event Hubs against the desired values of its state. It is also possible to monitor the partition of the Azure Event Hubs.
Threshold monitor
Monitor your Azure Event Hubs when their state violates desired values for a specified period, say few seconds/minutes. For Event Hubs, It is possible to monitor the status and size of the partitions in Threshold monitor
Data monitor
There would be needed to monitor the performance of the Event Hubs, Count of events in the Capture of the Event Hubs or Destination processing duration of Event Grid Subscription. To monitor the entities on their extensive set of metrics, Serverless360 has brought in the capability called data monitor which provides the calendric view of historical alerts.
User management
Though there is a Role Based Access Control system available in the Azure portal it is challenging to come up with an appropriate custom role with need only permissions on the entities in the business application. Custom roles in Serverless360 are simple and straight forward. Provide permissions at the application level. Define a granular user access policy defining actions the role can perform. For Azure Event Hubs and Event Grids, it is possible to give or restrict access to manage, monitor and Event Processing for Event Grid.
Governance & Audit
When multiple users are authorized to access the Azure Event hubs or Azure Event Grid, it is necessary that their actions on the Azure applications are audited. Serverless360 tracks every user action on the Azure entities associated with it. For Event Hubs, we can track the CRUD operations. For Event Grid, it is possible to track the Event processing. It is also possible to export the reports to excel or pdf format.
Conclusion
In this webinar, we saw what is Azure Event Hubs and Event Grids? And how they can be better managed and monitored using Serverless360? Watch the recording of the webinar below.
