Hybrid Application Tracking Using Serverless360 BAM

Introduction

In this blog, we will see how Business Activity Monitoring in Serverless360 can be used to monitor and track the Hybrid application constituted by On-Prem and Azure components.

Hybrid Application Tracking

Hybrid Applications

Hybrid applications are the applications designed to solve business problems using a combination of Cloud and On-Prem components. Some of the cloud components provided by Azure are Service Bus, Logic Apps, Azure Function apps, and Web apps. These components can be integrated into On-Prem components like .NET applications either through .NET libraries or Http API calls.

Tracking a Business Flow in Hybrid Applications

To ensure the continuity of a business, it is necessary to monitor the business transactions or business flows that happen through the Hybrid business application. The factors that can be tracked are the time taken for each step in a business transaction, the status of each step whether it is success or failure, and the business data that flows through the application.

Why Tracking Is Important?

The transactions happening in a business organization may fail due to any of the following reasons

Incorrect or invalid data
Unavailability of resources to process the data
Business logical errors

The transactions failing due to any of the above reasons can be found easily if they are tracked and monitored simultaneously. If such tracking is done, we can quickly diagnose the reason for failure and fix them.

Employee Onboarding Scenario

Now let us consider a simple hybrid application solving an employee onboarding scenario. Whenever an employee joins an organization, he must be registered with the payroll application.

This employee management scenario is built using a hybrid application constituting a .NET core On-Premise application that collects the required employee details. All the employee details are then sent as a Message to a Service Bus Queue.

A Logic App listens for the messages in the Service Queue and adds the required employee details into the payroll storage table.

The Logic App designed to listen to the message and store them in the storage table can be implemented as follows

The sample code to send the employee details to the Queue is as follows

using Kovai.Serverless360.Bam;
using Microsoft.Azure.ServiceBus;
using Newtonsoft.Json;
using System;

namespace Initial_Onboarding
{
  class EmployeeController
  {
[HttpPost]
    static void Post([FromBody] EmployeeDetail employeeDetails)
    {

      QueueClient queueClient = new QueueClient("NamespaceConnnectionString", "Queuename");

        Message message = new Message();
        message.UserProperties.Add("Name", employeeDetails.Name);
        message.UserProperties.Add("Department", employeeDetails.Department);
        message.UserProperties.Add("Designation", employeeDetails.Designation);
        message.UserProperties.Add("DateOfBirth", employeeDetails.DateOfBirth);
        message.UserProperties.Add("Id", employeeDetails.Id);
        queueClient.SendAsync(message).GetAwaiter().GetResult();
      
      Console.ReadLine();
    }
  }
  public class EmployeeDetail
  {
    public string Name { get; set; }
    public string Id { get; set; }
    public string Department { get; set; }
    public string DateOfBirth { get; set; }
    public string Designation { get; set; }
  }
}

Serverless360 BAM

In any kind of integration solution, there is a need for the organization to build an end-to-end traceability/monitoring solution that can be used by the business users. Often this is an afterthought, and the operational support team struggles to run the solution that has been implemented. Sometimes the delivery team will work on a solution but often it is bespoke and difficult to use across projects and teams spend about 20-30% of their time addressing this challenge, building custom solutions like logging components, and web dashboards. Most of the time the solution will be premature, since building an enterprise-grade tracking/monitoring solution requires enormous effort.

That’s exactly the challenge we are complementing in Serverless360 with the BAM feature.

Business Activity Monitoring

It is a functional end to end business activity tracking and monitoring product for Cloud Scenarios. With Serverless360 Business Activity Monitoring(BAM), get full visibility of your end-to-end business process flow across your Azure resources.

Terms in Serverless360 BAM

Business Process – A collection of 1 or more related business transactions
Transaction – An operation of the business which represents a set of related business activities or actions
Stage – An activity or action within the transaction. A transaction can contain 1 or more stages.

The below diagram explains the business terms used in BAM.

How to Integrate Serverless360 BAM with Hybrid Application

Serverless360 BAM provides the following components which can be plugged into your business flow to track the transactions happening in them.

Custom connectors for Logic Apps and Microsoft Flow
.NET Library for Azure Functions and OnPrem APIs

Creating a Business Process in Serverless360

The prerequisite for creating a Business Process in Serverless360 is that you should have an account in Serverless360. Once the account is created, you can create a Business Process by following the below steps.

Associate an Azure SQL Database connection string and Azure Storage Account connection string in the BAM section of Serverless360. The SQL Database and Storage account will be used to store all your tracked data and messages.
Create a Business Process by providing a Friendly name and description.
Create a Business Transaction within the created business process, in our case, it should be order validation.
Once the Business Transaction is created, the business flow can be designed by adding the required stages in the transaction. Each stage should represent the activities in the business flow as shown below

Instrumenting Hybrid Application with Serverless360 BAM

Three different components in BAM

Download the Serverless360 BAM project from here. Include this project in your solution. Get the BAM function app URL and API key from the access keys section of the Serverless360 portal. There are three available methods in the library.

StartTransaction
CheckPoint
CorrelationCheckpoint

StartTransaction

On passing the required parameters to this function, the transaction instance id and stage instance-id will be returned.

Checkpoint

On passing the required parameters along with the transaction instance id, the stage instance id of that stage will be returned.

CorrelationCheckpoint

A checkpoint with correlation can be used when the transaction instance id is not known. We should pass the already tracked property name and the value of that property as parameters to this function. This stage instance will be associated with the transaction instance that matches the provided properties.

Integrating BAM into Angular

import { Injectable } from '@angular/core';
import { HttpClient } from '@angular/common/http';
import { environment } from '../../environments/environment';

@Injectable({
  providedIn: 'root'
})
export class EmployeeService {
  headers = new HttpHeaders({ 'Content-Type': 'application/json' });
  options = ({ headers: this.headers });
  constructor(private http: HttpClient) {
  }
  setHeaders() {
    this.headers = new HttpHeaders({ 'Content-Type': 'application/json' });
    this.options = { headers: this.headers };
  }
  setBAMHeaders() {
    this.headers.append('SL360-BusinessProcess', 'Employee Mangement');
    this.headers.append('SL360-Transaction', 'Intial Onboarding');
    this.headers.append('SL360-Stage', 'Get Employee Details');
    this.headers.append('SL360-ArchiveMessage', true);
    this.headers.append('SL360-StageStatus', 'Success');
    this.options = { headers: this.headers };
  }
  fetchEmployeDetail(string id) {
    this.setHeaders();
    return this.http.get(environment.api.baseUrl + "/Employee/" + id, this.options).subscribe((employee: EmployeeDetail) => 
  var response = this.startTransaction(employee); // This response will be sent to Back end API to track underlying stages
  );
    startTransaction(employee: EmployeeDetail) {
      this.setBAMHeaders();
      var body: any = {
        messageBody = employee,
        messageHeader = this.headers
      }
      return this.http.post('https:///api/StartTransaction?code=', body, this.options);
    }
  }
}

.NET Code after Integrating BAM

Tracking Azure Logic App

With the custom connector for BAM being deployed to Azure, users can track the stages in Azure Logic Apps and Power Automate. After making the connection with the BAM connector, the user can pass on the necessary details like Correlation Id to correlate with the previous stage, which is the .NET Application, Message Body, Message Header, Stage Name, and more. Users can even log the exception using SL360-Exception and SL360-ExceptionCode parameters if there is a possibility of failure occurs in any of the stages.

User-Friendly Querying

Tracked Transactions can be queried for specific results based on the transaction status and tracked properties in a much user-friendly manner.

The queries also support LIKE operator and we can also group multiple search query clause using AND – OR Operators which helps us retrieve more focussed records.

For example, if I want to retrieve only the employees from the development branch, I can use the Query Branch = “Development” which returns only the required records.

Reprocessing

It is also possible to reprocess the failed or the required transactions. Serverless360 provides 4 types of reprocessing endpoints. They are as follows.

Http Endpoints
Service Bus Queue
Service Bus Topic
Event Grid Topic

The message body archived at the selected stage will be sent to the configured reprocess endpoints. This can be used to reprocess the failed instances by investigating the reason for failure at a later point in time.

Dashboards

We can also set up dashboards to visualize the trend of our business process. We can create multiple widgets with different queries. It also helps in understanding our business by viewing the transactions grouped by tracked properties. This kind of business dashboards can be created in Serverless360 BAM.

Monitoring

We can also set up Monitors to monitor the transactions occurring in a flow. There are two types of monitors available for BAM. They are as follows

Query Monitors
Exception Monitors

Query Monitors

Query monitors can be used to monitor the transactions by periodically querying the transactions with the configured query and validating the results against the configured threshold error and warning values. Once the configured error or warning thresholds are not met, an alert will be sent. It will also store the monitoring results upto 1 month for future references.

Exception Monitoring

Users can also get instant alerts in the configured notification channels whenever an exception is logged in the stages of a business transaction.

This can be achieved by sending the required exception code and exception message in the connector actions or can be sent as a parameter in .NET Library. Below is the configuration of Exception monitoring in BAM.

Serverless360 provides integration with the following notification channels.

Advantages of using Serverless360 BAM

Tracked data can be viewed in a business point of view rather than the technical point of view
It is possible to track the properties at different stages in a Microsoft flow
It is possible to filter or search the transaction instances based on the configured properties in addition to the status of the runs or transaction instances
It is possible to get alerted whenever there is an exception logged in the stages of the Microsoft flow
It is possible to set up a custom user access policy so that users having only required privileges can access the message content tracked in the Microsoft flow
We can set up a customized dashboard to monitor the transactions happening in a Microsoft flow
It is easier to find the exact stage where the transaction instance has failed and fix it with minimal cost
It is also possible to track the stages that constitute the same business flow but is present in a completely different infrastructure using the Checkpoint with Correlation action of BAM

Conclusion

Hope this blog explains how a hybrid application consisting of Angular, .NET Core application, and an Azure Logic App can be tracked and monitored using Serverless360 BAM. For more knowledge, checkout Serverless360.

You can also take a look at our pillar page dedicated to Distributed tracing to get better insights on the topic.