← Return To Home

Ensure Active Client Device Interaction through Azure Event Hubs Monitoring using Serverless360

Scenario

A Logistics solution provider needs telematics powered fleet management solution to empower their business and provide them with accurate telematics information on location, tracking, route, fuel level, engine idling, delivery status of goods.

Telematics fleet management solution would offer the following advantages

  • Real-time vehicle tracking
  • Asset tracking
  • Route optimization
  • Fuel consumption tracking
  • Improved operational efficiency
  • Improved fleet safety
  • Compliance verification
  • Automatic trip logging
  • Risky driver identification
  • Vehicle maintenance issues alerts

The Fleet Management System

Microsoft Azure is the selected platform provider to implement the above-mentioned fleet management system. Below is the orchestration of the fleet management solution built using Azure Serverless technologies.

fleet management solutions

Before proceeding further, let us understand every entity in the orchestration in detail,

The telematics device installed in the vehicles will stream telematics data. Thousands of devices across the world will stream telematics data, which is being ingested by Azure Event Hubs. In addition,  Azure Event Hubs is designed with a focus on the data point event scenario. In other words, Event Hubs is an “event ingestor” that accepts and stores event data and makes that event data available for fast “pull” retrieval.

This fleet management system requires to handle multiple data streams from different publishers in parallel. Event Hubs support a partitioning model that allows keeping related events together while enabling fast and highly parallelized processing of the individual streams that are multiplexed through the Event Hubs.

 The Event Hubs Capture feature automatically writes batches of captured events into the Azure Storage blob containers and enables timely batch-oriented processing of events.

The Event Hubs emits an event to the Event Grid when the capture file is created. These events are not strongly correlated and don’t require processing in batches. Hence, Event Grid is selected to provide a reliable event delivery at massive scale.

Event Grid delivers the event to the Azure Relay which securely exposes the service that runs in the corporate network to the public cloud. Therefore, the actual business logic to process the telematics data in the storage blob container for decision-making analytics resides in the on-premise service.

Serverless360 to Manage and Monitor the Fleet Management System

Fleet Management System can be represented as a Composite Application in Serverless360. The Azure entities participating in the solution can be logically grouped into the Composite Application. Once the entities are associated with the Composite Application, management and monitoring of them become easy.

Event Hubs plays a critical role in the above orchestration. It is important to ensure client devices interacting with Event Hubs are active. In other words, this can be achieved using Serverless360 as the monitoring tool. Analyzing deep into the orchestration, below are the requirements arrived at to ensure the telematic devices on the vehicles interacting with the Event Hubs are active;

  1. Monitor incoming requests to the Event Hubs to ensure if all vehicles across the world are active
  2. Monitor Event Hubs partitions to ensure vehicles in every region are active
  3. Detect any errors in the Event Hubs ensuring the Event Hubs is available for the client devices
  4. Simulate a real-time environment to check if Event Hubs receives the telematics data

Let us see how Serverless360 can meet all the above requirements.

Pre-requisite

The pre-requisite to manage and monitor Event Hubs in Serverless360 is to associate the Event Hubs in a Composite Application. Create a Composite application to represent the Fleet Management System with all its Azure Serverless entities associated.

CA card

Get instant representation to the health of entities that participate in the Composite Application representing the Fleet Management System. The best part here is these entities can be from different Azure Subscriptions, Azure Regions and Resource groups. Get to manage all the entities at one place which is a challenge in the Azure Portal.

CTA - composite application
 

CA-Entity dashboard

Monitor Incoming Requests

Let us assume some numbers for better understanding. Consider there are 2000 vehicles across the world expected to send telematics data once in every 15 mins. If the requirement is to check if all the 2000 vehicles are active, i.e. all devices send the telematics data in the required frequency.

Serverless360 Data Monitor can be configured to track incoming messages every 15 mins. Further, alerts can be triggered if the number of incoming requests in 15 mins duration is less than the number of vehicles. When an alert is received it reveals not all telematic devices in the vehicles are active.

Detect Errors

When our intention is to ensure client telematics devices interact with Event Hubs, it is necessary that the Event Hubs is available. The Event Hubs would be unavailable when it encounters Server Errors, User Errors, and Quota Exceeded Error. Serverless360 Data Monitor can be configured to monitor the above-mentioned metrics and alert whenever an error is detected.

The above 2 cases can be clubbed together in a single Data Monitor to detect the availability of client devices and the Event Hubs. Below should be the configuration to achieve the same. Moreover, the Data Monitor can be configured to send alerts on the supported Notification Channels to keep the stakeholders informed.

CTA- Event Hubs monitoring
 

DM

 
Below is the calendar view of the Data Monitor that reports every 15 mins as per the monitor configuration. The report represents details of the selected metrics with status arrived based on the warning and error threshold values defined in the configuration.

DM Success Report

The alert would be triggered even if one of the metrics fail to meet the expected configuration ensuring the client telematic devices are active and the Event Hub is available hence the orchestration works as expected.

DM-failure

Monitor Event Hub Partitions

Event Hubs provides message streaming through a partitioned consumer pattern in which each consumer only reads a specific subset, or partition, of the message stream. In other words, This pattern enables the Fleet Management System to handle telematics data from vehicles segregated based on their region. Each partition in this scenario represents a specific region.

Event Hub retains data for a configured retention time that applies across all partitions in the event Hub. Events expire on a time basis; you cannot explicitly delete them. Because partitions are independent and also contain their own sequence of data, they often grow at different rates.

This demands to monitor all the partitions of the Event Hubs on their Size. This can be achieved by using a Threshold monitor. Furthermore, violation persistence duration in the threshold monitor configuration can be defined based on the business scenario.

 In this Fleet Management system, the number of partitions in Event Hubs are 32 representing 32 regions. Exclusively for such cases, bulk monitoring configuration for multiple Event Hubs partitions was brought in in Serverless360. With this capability, the user can quickly define monitoring configuration for several Event Hubs partitions at a go.

An important consideration while creating an Event Hub. The number of partitions is specified at creation and must be between 2 and 32. Similarly, the partition count is not changeable, so you should consider long-term scale when setting partition count. You can increase the number of partitions beyond 32 by contacting the Event Hubs team. Refer to this MSDN documentation for more details.

PartitionMonitoring

Simulate Telematic Devices

As the orchestration is being built, there could be a need to test it by sending sample requests. Send Event activity can help to simulate the real-time environment. This activity can be configured to send events to a specific partition of the Event Hubs in batches, with a specified think time between the batches. Moreover, it is possible to send events in parallel simulating a real-time environment. Also, configure the frequency and schedule this activity to stream real-time data to Event Hubs and test if the orchestration works as expected. Below is a sample activity configuration to send events.

CTA- Event Hubs operations
 

Activities

 

With Serverless360, monitor Event Hubs in the business orchestration in various perspectives and also ensure the application meets the business need. Serverless360 capabilities on managing and monitoring Event Grid, Relay and also Storage blob which are a part of the Fleet Management System will be covered in the upcoming blogs.

Author: Ezhilarasi Chezhiyan

Ezhilarasi Chezhiyan, Technical Lead for the product, Serverless360, at Kovai Systems India Pvt Ltd. She has over 8 years of experience in Software development with knowledge spectrum spread across various domains. Problem-solving is her passion and she believes in the quote by Albert Einstein- "Intellectual growth should commence at birth and cease only at death"