The Microsoft Azure Platform offers you various services to build cloud-native integration solutions. With Logic Apps, Service Bus, Functions and Event Hubs you have a few services to your exposure to develop these solutions. Once you build these solutions and deploy them in a production environment, you will need to monitor them. In this blog post, we will discuss monitoring such a solution with Serverless360. Moreover, we will examine them through a sample composite cloud-native integration solution.
Composite cloud-native integration solution
Assume you require a solution to ingest the latest exchange rates every hour for United States Dollars from a public API (https://openexchangerates.org).
You build a Logic App with a recurrence trigger set for every hour (time zone GMT +1 Amsterdam), followed by a scope containing an HTTP action. The scope includes a parse action for parsing the response from the endpoint, a call to a function to set the epoch to date time, enrich the message with the DateTime in a compose action. Finally, you send the message to a queue if actions in the scope do not fail.
Subsequently, you build a second Logic App listening to the queue. Hence this Logic App starts with a Service Bus Queue trigger. The trigger is followed by a variable to set a GUID, a second variable to set the message from the service bus queue to JSON Object. Next, a scope with an action to compose a JSON document, adding an ID with the GUID (variable), and insert trigger action to enter this document into a Cosmos DB collection. When the scope succeeds the message will be completed in the queue – otherwise, the message ends up in the dead-letter queue.
Azure Monitoring Support system
Next, to the two Logic Apps, you set up monitoring for the solution. You can add Application Insights for the function and an OMS workspace to monitor the Logic Apps and Cosmos DB instance. Furthermore, you can leverage KUDU to get more in-depth insights into your Functions and set up Serverless360 to forward notifications to your support team.
The above diagram shows the solution and the monitoring set up around it.
In my previous blog post – ‘Different options available for Azure Serverless Monitoring’, I have discussed several Azure monitoring services and Serverless360, which can play a role in monitoring your Azure Serverless components, why they are complementary, and add value to the health of the overall solution.
In the following paragraphs, we will discuss how Serverless360 can help in monitoring this sample cloud-native integration solution.
Composite Application
A Composite Application, in Serverless360 context, is a logical entity that you can use to group Azure Services like Service Bus, Logic Apps, Event Hubs etc that constitute a Line of Business Application in your Integration Solution Architecture. These resources can be from different Azure Subscriptions and namespaces.
To monitor the composite application with Severless360 you will need to create a Service Principal. A service principal is an application within the Azure Active Directory. This can be authorized to access resources or resource group in Azure. Serverless360 uses the authentication tokens of the service principal to manage the resources.
Once you have created a service principal in your Azure subscription you can add it to Serverless360 (you will need the Subscription Id, Tenant Id, Client Id, Client Secret available). Next, you can use the default composite application card, and click on it and start adding (associate resources) Azure resources like Logic Apps, Service Bus queues or topics.
Manage
When we have our composite application in Serverless360, we can manage the components of our solution – two Logic Apps, one Service Bus Namespace (queue) and one Function.
With the queue & topics we can perform operations like viewing the actual messages to know more details, process the dead letters messages to repair and resubmit or merely purge them. and we can do operations like disabling the queue, send messages, and look and properties. Next, we can enable dead letter analytics on the queue or topic subscriptions to understand the classification of dead letters in them.
Note: Azure Portal is a versatile platform to manage services in your subscription. But, it is so massive and complex that brings its own challenges in using it. Moreover, as you drill down at a resource level, you end up only having infrastructure level capabilities and no operation level capabilities. For eg: Message processing – you either end up writing and maintaining your custom code or use third-party applications like Serverless360.
We can examine the Logic Apps in our solution by selecting Logic Apps in our composite application. Next, we can choose a Logic App from the list and look at the trigger and run history. For instance, we can look at a particular run and its details.
Furthermore, we can trigger a Logic App, look at its details, and disable it.
Composite Application can also examine Azure Functions. By clicking Functions, we can look into the Function details, its runs, and details.
Alarms & Monitoring
With Serverless360 you can detect and be alerted about violations occurring in your composite integration solutions. Through the monitor tab, you can access configure alarms. This feature enables you to add an alarm by type – health or threshold. Furthermore, you can specify with threshold alarm, for instance, in our solution to be alarmed (notified) when the active messages are more than 5 (warning) or 10 (error) and when dead-letter message count exceeds 5 (warning) or 10 (error).
There are different ways in which you can monitor your services in Serverless360:
Threshold Monitoring
It is possible with Serverless360 to monitor your services at their property level in their current state. For eg: You may want to monitor the Active / Dead letter message count of your queue/topic subscription. If the value of any property persists over or below the desired value for a given period of time say 5 minutes, you may want to be notified.
Health Status Reports
For the Composite application scenario in this discussion, you may be interested to receive periodic status reports. Say, at a certain point of the day to see if all your entities are in the healthy state. Rather than receiving an alert when something abnormal happens, it is like a report generated for periodic assessment.
This level of monitoring at the property level is only available in Serverless360. Whereas Azure Monitoring services provide you monitoring at time-based metric points of your Azure resources. This is similar to performance counters in a Windows Server. Serverless360 provides this monitoring capability also with more simplicity and intuitiveness as Data Monitoring.
Data Monitoring
Unlike Threshold monitoring and Health status reports, which monitors your entities at the current state of their properties, Data Monitoring helps you to assess the overall health of your entities with a rich set of metrics data. It helps you to monitor the incoming and outgoing messages within a specific time window. You can also assess the rate of run action failures in logic apps and invocation failures in your Functions. This helps to determine if the overall application is meeting our business SLA.
In this scenario, I expect at least 1 message to pass through the queue, Logic App to be triggered once and Azure Function to have 1 invocation every 1 hour. In another perception, I can expect 24 messages in a queue, logic app triggered 24 times and Azure Function invocated 24 times over a 24 hour period.
Watch Monitor
In this scenario, where we have a couple of logic apps and an azure function, it will be nice to receive an instant alert if any of the run action in the logic app or invocation in the azure function fails. There could be scenarios where the trigger would have been successful but one of the run actions failed. This would break the workflow. Getting notified on the failure of Logic App run action and Azure function invocation in near real-time is now possible with Serverless360.
API endpoint monitoring
To keep this sample business scenario functional, the availability of the external Public API is very much important. Serverless360 provides the capability to monitor a set of APIs in the composite application for its response and responsiveness. You can check the response data or get notified when the API takes too long to respond.
Get Notified through your preferred notification channels
Next, step in configuring an alarm is the schedule and notification channel. The notification channel can be email (SMTP), Webhook, Slack, PagerDuty, Microsoft Team, and OMS. With our solutions alarms, we have been set up will notify us if service is healthy or not and there is a threshold alarm for the queue if it exceeds certain thresholds.
Summary
In this blog post, we discussed Serverless360, which can play a role in monitoring a composite cloud-native integration solution. There are differences and overlap between what Azure offers with their monitoring services and what Serverless360 offers. There are various services in the Azure portal to help you in managing and monitoring your entities. But, the ease of use of Serverless360 versus the more versatile, feature-rich Azure Services is one thing that stands out.
With Serverless360, you have a more intuitive setup. It’s a direct available SaaS solution, providing access to your complete composite application altogether. It provides a holistic approach to consider your entities together as one single solution than monitoring in bits and pieces. Furthermore, Serverless360 is a non-intrusive service, a SaaS solution. While the PaaS services in Azure are part of the platform and require extensive knowledge and experience to apply correctly to monitor any solution in Azure.
