In a recent blog “10 new gems added in Serverless360“, we introduced the recent capabilities added to Serverless360. This blog is more focused on management and monitoring capabilities in Serverless360 for Service Bus Queues and Topics.
• Understanding Azure Service Bus and its applications
• Deep integrated tooling from Serverless360 for better management of Service Bus
• Consolidated Monitoring from Serverless360 to monitor Service Bus in various perspectives
• Achieving Advanced Security in dealing with Azure Service Bus
What is Azure Service Bus?
Azure Service Bus is a messaging service on cloud used to connect any application, device or service running in the cloud to any other application or service. Handles messages in the FIFO format. It comprises of two entities Queues and Topics.
Linear Messaging Model Publisher-Subscriber Model
Service Bus in business?
Azure Service bus has become the most integral part of the Serverless applications as a messaging service. Let us consider a familiar real-time application, Cab booking system – FlyWheelCabs, for better understanding on Azure Service Bus.
For every new booking submitted by a customer in the FlyWheelCabs application, a Service Bus message is pushed into the bookinghandler Queue. Service Bus Queue is selected here to meet the need to have a linear messaging model. Logic App – BookingProcessor listens to the Queue for any new message. Logic App will pick up any new booking message arriving in the bookinghandler Queue, at a defined frequency. With the help of an Azure function, ValidateBooking, the booking request is validated to contain valid mandatory information. Based on the verdict from the Function, the message will be pushed into Topic with label ‘booking confirmed’ or ‘booking rejected’. Service Bus Topic has been selected here to meet the published- subscriber model. The messages pushed into the Service Bus Topic should get auto-filtered to the respective Topic subscriptions, Valid and Invalid.
Challenges in the Azure portal
Though Azure Service bus is powerful, there are some challenges in managing them through the Azure portal. Here are some business challenges that most Azure users face in managing and monitoring Azure Service bus in the Azure portal
- No Deep Integrated tooling
- No Message visibility
- No Message processing
- No Consolidated monitoring
- No Auditing
So, what would be the solution to solve these challenges?
The one platform to Manage and Monitor the Azure Serverless application is Serverless360
In the FlyWheelCabs system, there is a requirement to auto-filter messages into subscriptions which can be achieved by defining a rule in the topic subscriptions. There is no straight forward solution available in the Azure portal to achieve this. Serverless360 facilitates creating boolean, correlation and SQL filters in the Service Bus Topic Subscriptions. If not for Serverless360 one should use the .Net SDK and implement a custom application to achieve this task.
Many such critical business tasks can be achieved at ease using Serverless360 without writing even a single line of code.
There would be a need to move the resources from staging namespace to production namespace, Serverless360 handles this challenge in an efficient way with import capability. With this capability, one or more Service bus Queues or Topics can be moved from one namespace to another. The exact definition of the Queues and Topics including the Topic subscriptions and even rules will get replicated saving considerable testing and development effort.
With Serverless360, we can also create, delete and change the state of a Service Bus Queue or Topic.
Say the Logic App listing to the Queue in the FlyWheelCabs application went down for some reason this will have an impact on the Queue. Dead Letter Messages might pile up due to Time to Live Expiry. This is just one scenario, in business applications the messages would be dead lettered due to the various system or custom reasons.
In the Azure portal, it is not even possible to view the properties and message details. Whereas the real-time need is to process those messages to restore the business.
With Serverless360 Message Processing, it is possible to reprocess and resubmit a business-critical message to another queue or topic.
It is also possible to back up those business-critical messages into a storage blob associated in the composite application.
When a huge number of messages in multiple Service Bus Queues and Topic Subscriptions need to be managed at ease without any manual intervention, Service Bus activities are the choice.
Auto-process left behind messages in Service Bus
The Logic App listening to the Service Bus Queue might leave behind certain messages unprocessed. These messages need to be moved to another Service Bus Queue to accomplish failover processing. This must be carried out at a defined frequency to ensure none of the cab bookings go unserved. This critical business requirement can be achieved using automated activities to process active messages in Service Bus.
Read more on this feature here.
Auto-process Dead Lettered Messages
Piled up dead letter messages need to be appropriately processed. Manual processing of them is time-consuming and prone to errors. Use Dead Letter Message processing activities to reprocess or purge the dead letter messages by the dead letter reason. Schedule this activity to auto-manage the dead letter messages in Service Bus Queues and Topics.
Read more on this feature here.
What is required is an Application level, consolidated monitoring in various perspectives what is offered by Azure portal is entity-level monitoring on its metrics.
For monitoring Service Bus in multiple perspectives, Serverless360 has three types of monitors: Status monitor, Threshold monitor and Data monitor
Choose Serverless360 status monitor to get application health reports at a specified time in a day representing the state of Service Bus Queues, Topics and Topic subscriptions against the desired values of on state and basic properties
Monitor your Service Queues, Topics, and Topic subscriptions when their state or properties violate desired values for a specified period, say few seconds/minutes. Autocorrect their state to restore the business without any manual intervention. Detect violations in important properties like Dead Letter Message Count, Size and much more.
When there is a need to monitor the performance, Incoming and outgoing requests, and connection state, the better solution would be a data monitor. With the extensive set of metrics, it is possible to monitor the above needs and even more.
Though there is a Role Based Access Control system available in the Azure portal it is challenging to come up with an appropriate custom role with necessary permissions on the entities in the business application. Custom roles in Serverless360 are simple and straight forward. Provide permissions at the application level. Define a granular user access policy defining actions the role can perform. When it comes to Service Bus a role can even restrict the user to resubmit message only to the same Queue or Topic.
Sharing the Namespace connection is no longer required to manage and monitor the Service Bus Queues and Topics.
Read more on this feature here.
Governance and Audit
When multiple users are authorized to access the Azure Service Bus Queues and Topics, it is necessary that their actions on the Azure applications are audited. Serverless360 tracks every user action on the Azure entities associated with it. It is also possible to export the reports to excel or pdf format.
You can watch the webinar below