Microsoft Azure Service Bus

Acquire all necessary knowledge on Azure Service Bus in this definite guide. In a nutshell, it covers basic Service Bus concepts, comparison on the Azure messaging services, properties and metrics of Service Bus entities, tools available to manage and monitor Azure Service Bus and much more. This guide also justifies how Serverless360 turns out to be the best tool to manage and monitor the Azure Service Bus. This guide will answer all your questions related to the Azure Service Bus and provide useful links to explore further. This guide will be continuously revamped to stay up-to-date.

hero image

Explore out of box capabilities of Serverless360 for Azure Service Bus

Book a Demo

We promise to never spam or give away your email.

Service Bus Overview

This pillar page attempts to address all that one needs to know about the Azure Service Bus. For a better understanding of the concepts, a real-world business scenario is considered throughout the article, FlyWheel Cab booking application. This application is built using Azure Serverless services like Service Bus, Logic Apps, Event Hubs, Event Grid, Relay, Storage blob, and Azure Functions. There are two main components in the application, the Vehicle Telematics application to collect real time data of the cabs to provide necessary information for booking. The Booking application to facilitate cab booking by customers. The service map below illustrates how does each service interact with each other to build the business application.

Azure Service Bus Overview to booking applications

This article would focus on the highlighted Service Bus Queues and Topics in the booking application. Let us start.

What is Azure Service Bus?

Azure Service Bus is a messaging service on cloud used to connect any applications, devices, and services running in the cloud to any other applications or services. As a result, it acts as a messaging backbone for applications available in the cloud or across any devices.

Compare Azure Messaging Services

Azure messaging services have evolved over time with multiple options. It is important to understand all the options to make the right decision when to use what service. At the face of it, the technologies might look like they are doing pretty much the same thing, but there might be significant differences and there is a reason for their existence.

Azure Storage Queues

Azure Queue storage is a service for storing large numbers of messages that can be accessed from anywhere in the world via authenticated calls using HTTP or HTTPS. A single queue message can be up to 64 KB in size, and a queue can contain millions of messages, up to the total capacity limit of a storage account.

In systems that require intense processing, handling it all in a single process is virtually impossible. Possible processing failures, outages, hardware limitations, or cost constraints prescribe to divide work into chunks and distribute to processes that can handle it. This also helps with the responsibilities concern, where a process doesn’t have to assume all the responsibilities and turn into a monolithic system. The approach is to divide work among processes and distribute or delegate among other processes.

While this approach sounds straight-forward, several considerations need to be made such as temporal coupling, outages, workloads, and more.

Azure Service Bus

While Storage Queues service provides the necessary means for queueing, there are domains where you require real messaging. Usually, that means supporting features that involve compute and processing behind the scenes on behalf of the service. That’s where Azure Service Bus (ASB) service shines. The features it provides are genuinely impressing. Some of those are:

  • Scheduling
  • Dead-lettering
  • Transactional processing
  • Message ordering
  • Expiration (TTL)
  • Duplicate detection
  • Deferring
  • Pub/Sub
  • Filtering
  • And much, much more

Event Hubs

So far, the focus was on individual messages where each message is discrete and important. With ASB there’s support for messages that represent events, but those are not streams of events. Scenarios such as telemetry require a different approach to handling messages. Ability to capture events that arrive at high volumes. That’s where a third Azure message service comes in, Event Hubs. This is a service that is specifically built to deal with event streams.

With exceptional ingress rates, it can take on many incoming messages storing those internally offering no queue semantics. While it might sound like a significant drawback, it is highly optimized for the one job it has – store events to allow later processing by the consumers on their terms and conditions. Consumers are the ones that manage events reading and decide wherever to move the reading cursor forward or go back to already read messages. With this level of power comes the responsibility to manage the cursor. Often, data read is streamed into other services for analysis or persistence. Event Hub, Capture feature helps with that by allowing periodically export events to Storage Blob service using Avro format.

The service has processing power and is expressed in Throughput Units, where each TU is capable of up to 1MB/s or 1,000 msgs/s of ingress and 2MB/s or 2,000 msgs/s of egress. While TUs must be pre-allocated, EH service offers Auto Inflation feature, preventing message ingestion from ever been throttled. Dedicated tier of the service also offers an extended event retention period. EH is making a conscious preference in favor of throughput over features.

Event Grid

Event Grid (EG) is one of the relatively latest additions to the Azure messaging services family. This toddler service is making big splashes and at times feels like a threat to its older siblings. After all, the questions it raises are valid.

  • What was missing there that a fourth messaging service had to be introduced?
  • How is different from Azure Storage Queue, Azure Service Bus Queue, and Event Hub?
  • Is Event Grid going to replace some of the other messaging services?

To answer these, let’s see what problems already exists that messaging services didn’t address that Event Grid can.

The world we live in is reactive. Events take place, and we respond to those events. The reactive nature of Azure services is no exception to this paradigm. A newly updated blob to Storage service needs to be processed to import data into CosmosDB. A provisioned resource needs to be tagged. Ability to notify subscribers scaled-out between multiple regions with non-ubiquitous receiving mechanisms (webhooks, queues, streams, etc). Reacting to the events taking place on a cloud-scale is not what the messaging services covered earlier were built for. That’s what Event Grid was created for.

Event Grid’s whole purpose is to allow publishing and handling events crossing datacentre boundaries cheap and straightforward, helping with building reactive architecture.

As of today, Event Grid is already supporting several Azure services capable of publishing events.

Those services are:

  • Blob Storage
  • Storage (v2)
  • Event Hubs
  • Service Bus (Premium)
  • IoT Hub
  • Container Registry
  • Resource Groups
  • Azure Subscriptions

Additionally, Event Grid supports Custom Topics, expanding this paradigm outside of Azure. When it comes to subscribing to Event Grid topics and handling events, currently supported options include the following:

  • Custom WebHooks: Use webhooks for customizable endpoints that respond to events.
  • Azure Functions: Use Azure Functions for serverless response to events.
  • Logic Apps: Use Logic Apps to automate business processes for responding to events.
  • Storage Queues: Use Queue storage to receive events that need to be pulled. By sending events to Queue storage, the app can pull and process events on its own schedule.
  • Event Hubs: Use Event Hubs when your solution gets events faster than it can process the events. 
  • Azure Automation: Use Azure Automation to process events with automated runbooks.
  • Hybrid Connections: Use Azure Relay Hybrid Connections to send events to applications that are within an enterprise network and don’t have a publicly accessible endpoint.

The list is short, but it will expand to support all the Azure services. And eventually, not just Azure services, but beyond that. With already supported CNCF (Cloud Native Compute Foundation) CloudEvents v0.1, Event Grid can interchange messages with any system supporting this open standard for events.

Event Grid’s ability to process millions of messages and being platform or language agnostic, it is the go-to messaging service to distribute notifications and build reactive solutions on top of it. Despite its HTTP nature, the service is engineered for reliability. Built-in retries and dead-lettering offer uncompromisable guarantee that events are not lost. Reasonably priced to allow massive scale event publishing and distribution.

Messaging Services for the FlyWheel application

Let us consider a Fly Wheel Cab booking System described in the overview, where customers hire cabs and do payments to drivers. This business scenario demands for transactions and those bring in the need to use Azure Service Bus. It is also required to have instantaneous consistency, temporal control like if this is not processed in x amount of minutes then the message should be pushed into the dead letter or elsewhere for an escalation. When the requirement is these kinds of orchestrated (workflow) stuff to do state transitions then Azure Service Bus is an appropriate solution.

It’s really like IBM MQ or BizTalk Server, however at cloud scale. In a nutshell, when you need robustness and cannot afford to lose any messages, you’ll go for Azure Service Bus.

The Vehicle Telematics Application requires to collect real time telematics data from thousands of vehicles across the world. The best messaging service that would fit into this requirement would be ‘Event Hub’. Azure Event Hub is designed with focus on the data point event scenario.

This application requires to handle multiple data streams from different publishers in parallel. Event Hubs support a partitioning model that allows keeping related events together while enabling fast and highly parallelized processing of the individual streams that are multiplexed through the Event Hubs.

The Event Hubs Capture feature automatically writes batches of captured events into the Azure Storage blob containers and enables timely batch-oriented processing of events.

When reactions occur in the business like, the driver had accepted the booking, the driver is moving towards the pick-up point, the customer or driver cancelled the booking in this case instead of doing a central control workflow you would do in an Event-based model running active workflows. Here you are reacting to real changes in real time. Azure Event Grid is ideal for these kinds of reactive scenarios. 

Concepts

With this understanding on when to choose a messaging service, let us now focus on Service Bus Queues and Topics in this article. Following are the need to know Azure Service Bus terminologies.

Namespaces

A namespace is a scoping container for all messaging components. Multiple queues and topics can reside within a single namespace, and namespaces often serve as application containers.

Queues

Messages are sent to and received from queues. Queues enable you to store messages until the receiving application is available to receive and process them.

Azure Service Bus queues

Messages in queues are ordered and timestamped on arrival. Once accepted, the message is held safely in redundant storage. Messages are delivered in pull mode, which delivers messages on request.

Topics

While a queue is often used for point-to-point communication, topics are useful in publish/subscribe scenarios.

Azure Service Bus Topics

Topics can have multiple, independent subscriptions. A subscriber to a topic can receive a copy of each message sent to that topic. Subscriptions are named entities, which are durably created but can optionally expire or auto-delete. In some scenarios, you may not want individual subscriptions to receive all messages sent to a topic. If so, you can use rules and filters to define conditions that trigger optional actions, filter specified messages and set or modify message properties.

Moreover, using Service Bus Queues and Topics bring in the following benefits:

Competing Consumer

Queues offer First In, First Out (FIFO) message delivery to one or more competing consumers. That is, receivers typically receive and process messages in the order in which they were added to the queue, and only one message consumer receives and processes each message.

Temporal Decoupling

A key benefit of using queues is to achieve “temporal decoupling” of application components. In other words, the producers (senders) and consumers (receivers) do not have to be sending and receiving messages at the same time, because messages are stored durably in the queue. Furthermore, the producer does not have to wait for a reply from the consumer in order to continue to process and send messages.

Load Levelling

A related benefit is “load levelling,” which enables producers and consumers to send and receive messages at different rates. In many applications, the system load varies over time; however, the processing time required for each unit of work is typically constant. Intermediating message producers and consumers with a queue means that the consuming application only must be provisioned to be able to handle average load instead of peak load. The depth of the queue grows and contracts as the incoming load varies. This capability directly saves money on the amount of infrastructure required to service the application load. As the load increases, more worker processes can be added to read from the queue. Each message is processed by only one of the worker processes.

Load Balancing

Furthermore, this pull-based load balancing allows for optimum use of the worker computers even if the worker computers differ regarding processing power, as they pull messages at their own maximum rate. This pattern is often termed the “competing consumer” pattern.

Loose Coupling

Using queues to intermediate between message producers and consumers provides an inherent loose coupling between the components. Because producers and consumers are not aware of each other, a consumer can be upgraded without having any effect on the producer.

Azure Service Bus Queues Vs Topics

In contrast to queues, in which each message is processed by a single consumer, topics and subscriptions provide a one-to-many form of communication, in a publish/subscribe pattern. Useful for scaling to large numbers of recipients, each published message is made available to each subscription registered with the topic. Messages are sent to a topic and delivered to one or more associated subscriptions, depending on filter rules that can be set on a per-subscription basis. The subscriptions can use additional filters to restrict the messages that they want to receive. Messages are sent to a topic in the same way they are sent to a queue, but messages are not received from the topic directly. Instead, they are received from subscriptions. A topic subscription resembles a virtual queue that receives copies of the messages that are sent to the topic. Messages are received from a subscription identically to the way they are received from a queue.

By way of comparison, the message-sending functionality of a queue maps directly to a topic and its message-receiving functionality maps to a subscription.

Service Bus Queue and Topic in the FlyWheel Application

In the FlyWheel Application when a customer books a cab, the new booking made through the web application is submitted to a Service Bus Queue. The Logic App that is listening to the Queue will pick up the message for further processing. In this case, the Web App and the Logic App are loosely coupled. Temporal decoupling is achieved as the Producer (Web App) and the Consumer (Logic App) don’t have to send and receive messages at the same time.

After the messages are processed by the Logic App with the help of an Azure Function based on the telematic information collected by the Vehicle Telematics Application, the message is pushed into Service Bus Topic. The topic had multiple subscriptions to handle approved and rejected messages by regions. The subscriptions use correlation filters to restrict the messages they are expected to receive.

Service Bus Queue and Topic in the FlyWheel booking application

Service Bus Properties

This section covers the must know Service Bus properties to create the Queues and Topics with the appropriate properties to meet the business needs.

Message sessions

To realize a first-in, first-out (FIFO) guarantee in Service Bus, use sessions. Message sessions enable joint and ordered handling of unbounded sequences of related messages.

Auto-forwarding

The auto-forwarding feature enables you to chain a queue or subscription to another queue or topic that is part of the same namespace. When auto-forwarding is enabled, Service Bus automatically removes messages that are placed in the first queue or subscription (source) and puts them in the second queue or topic (destination).

Dead-lettering

Service Bus supports a dead-letter queue (DLQ) to hold messages that cannot be delivered to any receiver, or messages that cannot be processed. You can then remove messages from the DLQ and inspect them.

Scheduled delivery

You can submit messages to a queue or topic for delayed processing; for example, to schedule a job to become available for processing by a system at a certain time.

Message deferral

When a queue or subscription client receives a message that it is willing to process, but for which processing is not currently possible due to special circumstances within the application, the entity has the option to defer retrieval of the message to a later point. The message remains in the queue or subscription, but it is set aside.

Batching

Client-side batching enables a queue or topic client to delay sending a message for a certain period. If the client sends additional messages during this time period, it transmits the messages in a single batch.

Transactions

A transaction groups two or more operations together into an execution scope. Service Bus supports grouping operations against a single messaging entity (queue, topic, subscription) within the scope of a transaction.

Filtering and actions

Subscribers can define which messages they want to receive from a topic. These messages are specified in the form of one or more named subscription rules. For each matching rule condition, the subscription produces a copy of the message, which may be differently annotated for each matching rule.

Auto-delete on idle

Auto-delete on idle enables you to specify an idle interval after which the queue is automatically deleted. The minimum duration is 5 minutes.

Partitioned

Service Bus partitions enable queues and topics, or messaging entities, to be partitioned across multiple message brokers and messaging stores. Partitioning means that the overall throughput of a partitioned entity is no longer limited by the performance of a single message broker or messaging store. In addition, a temporary outage of a messaging store does not render a partitioned queue or topic unavailable. Partitioned queues and topics can contain all advanced Service Bus features, such as support for transactions and sessions.

Duplicate detection

If an error occurs that causes the client to have any doubt about the outcome of a send operation, duplicate detection takes the doubt out of these situations by enabling the sender to re-send the same message, and the queue or topic discards any duplicate copies.

Service Bus Properties in FlyWheel Application

The Service Bus Queue in the FlyWheel Application which handles the customer cab bookings is session enabled to ensure the cab are allocated in first come first severed basis. It is also duplicate detection enabled to discard any duplicate bookings from the same user.

The Topic subscriptions are defined with correlation filters to auto filter messages based on the labels ‘status’ which contains ‘Approved’ or ‘Rejected’ and the region of booking.

Understand Service Bus metrics

Service Bus metrics give you the state of resources in your Azure subscription. With a rich set of metrics data, you can assess the overall health of your Service Bus resources, not only at the namespace level but also at the entity level. These statistics can be important as they help you to monitor the state of Service Bus. Metrics can also help troubleshoot root-cause issues without needing to contact Azure support.

Request metrics

Counts the number of data and management operations requests.

Metric Name Description
Incoming Requests The number of requests made to the Service Bus service over a specified period. 

Unit: Count 
Aggregation Type: Total 
Dimension: EntityName
Successful Requests The number of successful requests made to the Service Bus service over a specified period.

Unit: Count 
Aggregation Type: Total 
Dimension: EntityName
Server Errors The number of requests not processed due to an error in the Service Bus service over a specified period.

Unit: Count 
Aggregation Type: Total 
Dimension: EntityName
User Errors:

1.Client-side errors (In HTTP that would be 400 errors).

2.Errors that occur while processing messages, such as MessageLockLostException
The number of requests not processed due to user errors over a specified period.

Unit: Count 
Aggregation Type: Total 
Dimension: EntityName
Throttled Requests The number of requests that were throttled because the usage was exceeded.

Unit: Count 
Aggregation Type: Total 
Dimension: EntityName

Message metrics

Metric Name Description
Incoming Messages The number of events or messages sent to Service Bus over a specified period.

Unit: Count 
Aggregation Type: Total 
Dimension: EntityName
Outgoing Messages The number of events or messages received from Service Bus over a specified period.

Unit: Count 
Aggregation Type: Total 
Dimension: EntityName
Messages Count of messages in a queue/topic. 

Unit: Count 
Aggregation Type: Average 
Dimension: EntityName
ActiveMessages Count of active messages in a queue/topic. 

Unit: Count 
Aggregation Type: Average 
Dimension: EntityName
Dead-lettered messages Count of dead-lettered messages in a queue/topic. 

Unit: Count 
Aggregation Type: Average 
Dimension: EntityName
Scheduled messages Count of scheduled messages in a queue/topic. 

Unit: Count 
Aggregation Type: Average 
Dimension: EntityName

Connection metrics

Metric Name Description
ActiveConnections The number of active connections on a namespace as well as on an entity.

Unit: Count 
Aggregation Type: Total 
Dimension: EntityName

Service Bus messages

Applications and systems communicate with each other by passing information back and forth. Information is passed as messages. A message is a packaged information; a data record passed from sender to one or more receivers. There are different types of messages with a very different intent and usage patterns. Regardless of type, all messages have a few things in common. Messages carry a message body as well as metadata, in the form of key-value pair properties, describing the message body and giving handling instructions to Service Bus and applications. Occasionally, that metadata alone is enough to carry the information that the sender wants to communicate to receivers, and the message body remains empty.

Message Body

Data is the whole reason a message is created and sent. Consider a customer sign up in Fly wheels Cab booking system, it will involve an action say ‘CreateCustomer’ carrying customer details. This data is the message body.

Metadata

For the receiver to interpret the message correct it is required to pass the message metadata, a supplemented information which would include message properties or headers. Service Bus message contains 2 sets of properties. The broker properties are predefined by the system. The user properties are a collection of key-value pairs that can be defined and set by the application.

Explore out of box capabilities of Serverless360 for Azure Service Bus

Book a Demo

We promise to never spam or give away your email.

Tools

Service Bus Explorer

The Service Bus Explorer allows users to connect to a Service Bus namespace and administer messaging entities in an easy manner. The tool provides advanced features like import/export functionality or the ability to test topic, queues, subscriptions, relay services, notification hubs and events hubs.

Serverless360

Serverless360 is a One platform solution for Azure Serverless monitoring and management. In a real-time scenario, the integrated cloud applications are not built with a single technology stack, it typically involves multiple Azure Services. Currently, Azure Portal is designed more on vertical technology silos and it’s difficult to visualize and manage such connected solutions. Serverless360 is one tool that you can depend for Azure Serverless monitoring and management from one place. Serverless360 is crafted with capabilities to complement the Azure Portal.

Serverless360 eases out achieving real time critical tasks in managing Serverless Applications with its management capabilities, which would be of use to the Azure Developers, Testers and DevOps teams.

Monitoring solutions offered at Application level would let Support, DevOps team to access and monitor Serverless Application from various perspectives.

Service Bus Explorer Vs Serverless360

Below are the strong reasons why Serverless360 should be chosen over Service Bus Explorer

One Tool to Manage and Monitor Serverless Applications

In real time Azure Serverless services are put together to build orchestrations those solve critical business needs. What is required is a management and monitoring solution for these applications. Why juggle between Service Bus Explorer for management and Azure portal for monitoring. Get almost everything required at one place – Serverless360.

Much broader Scope

Scope of Serverless360 is much broader when compared with Service Bus Explorer. Service Bus Explorer is restricted to managing Service Bus, Event Hub and Relay namespaces and entities. However, Serverless360 offers management and monitoring capabilities for Logic Apps, Function Apps, Event Grid, Storage Account, Web App along with Service Bus, Event Hub, Relay. This list is continuously expanded to accommodate all Azure Serverless Services.

Out of box Message Processing options

One key purpose of Service Bus Explorer is processing the messages in Service Bus Queues and Topics. Serverless360 offers much better message processing capabilities. View and process Active and Dead Lettered messages. Resubmit or Repair and Resubmit or Delete with the option to back up messages to an associated storage blob. Deferred mode of message processing ensures the messages are not lost due to any network outage or if a user doesn’t perform resubmission after receiving the message from the dead letter queue. Also, download a copy of the Service Bus messages in compressed file format.

Automate message processing with Scheduled Activities

Process active and dead lettered messages in the Service Bus queues or topics with scheduled activities. Schedule purging of dead letter messages at specified times in a day on recurrence, automatically resubmit messages to another destination periodically. Simulate a real-time test environment using send messages activity for Service Bus Queues and Topics. Send events activities for Event Hubs and Event Grid Topics.

No more concerns sharing Connection Strings with distributed team

Concerned about sharing the namespace connection string with manage claims with the Support team to manage the Service Bus, Event Hub or Relay entities? Serverless360 provides much secure access to the Support team on the Azure entities. Associate entities through Service Principals or namespace connection strings and provide only necessary access to the team members Managing the access keys becomes central and need not share them with other or redistribute whenever regenerated.

Not just a Service Bus Management Tool

Scope of Serverless360 is much beyond Service Bus Explorer. Later is built just for managing Service Bus, Event Hub and Relay. With Serverless360, manage entity properties, define Topic subscription rules, import Service Bus, Event Hubs and Relays from one namespace to another, manage shared access policies, process messages in Service Bus, Storage Queues and Event Grid Subscriptions and do much more on monitoring these entities too.

Azure Serverless Monitoring and Reporting

Serverless360 is not just a management tool, it comes with variety of monitors to enable monitoring Azure Serverless Applications in various perspectives. Serverless360 provides a consolidated report on all the Azure entities those participate in the business.

Keep stakeholders informed through Notification Channels

Serverless360 supports a wide range of Notification channels like Pager Duty, Slack, OMS, Microsoft Teams along with Webhook and email channels for keeping the stakeholders informed on the entity monitoring status.

Manage Teams with need only permission

Business needs would demand teams to manage and monitor Serverless Applications. It is required to define a granular user access policy on these applications to the team members. Serverless360 provides convenient User Management with the option to integrate Azure Active Directory too. There is no user management or restricted access in Service Bus Explorer.

Continuous Improvement with dedicated Customer Support

Serverless360 is continuously improved with expansion in the scope of Azure services those can be managed and monitored. We pride ourselves in providing excellent customer service. As you intend to manage and monitor your Azure Serverless Applications in production, the right choice would be a commercial platform over a community tool.

Prefer SaaS or Web App over Desktop executable

One of the primary challenges faced by users of Service Bus Explorer is that it is a Desktop executable. Upgrading or redistributing latest version of the executable to all team members, exporting the configuration and distributing access keys/connection strings is quite difficult.

There is always a possibility that the application goes unresponsive. For some tasks like message retrieval or resubmission, there is a possibility that application hangs and messages are lost.

How nice it would be to manage your team members by allowing them controlled access to a web application, security and management become easy. The updates are available for all at the same time and taken care of by the platform provider. The team just focuses on their day to day business activities.

Serverless360 is crafted with capabilities to complement the Azure portal

This article would substantiate the above points with technical proofs and provide the reader clarity on why to choose Serverless360 over Service Bus Explorer.

Manage Service Bus in Serverless360

Modify State

Modify state of a Service Bus Queue/ Topic/ Topic Subscription from Serverless360. Set one of the following status to meet business needs which is not possible through the Azure portal.

  • Active
  • Disabled
  • Send Disabled
  • Receive Disabled

Manage Shared Access Policies

Do applications interact with your Service Bus queues in your orchestration? Need to provide connection string with limited permissions on the designated Queue to the applications? Create and manage Shared Access Policies of Service Bus Queues in Serverless360 itself. Also, regenerate the primary and secondary keys as and when required.

Manage Shared Access Policies of Service Bus Queues in Serverless360

Manage properties

When messages accumulate in one of the queues, you can view the queue properties to get more information about the possible causes of this accumulation. The Queues list in Serverless360, contains a context menu, Properties, to fetch and display all properties of the Queue from the Azure portal. If the business demands redefinition of one or more properties of the queue, these property values can be modified using Serverless360.

Manage proporties using Serverless360

Import

Let’s consider the FlyWheel Application with the following Service Bus Entities:

  • Queue with the following properties:
    • Message time to live set to 20 minutes
    • Duplicate detection enabled
    • Duplicate detection history set to 10 minutes
    • Maximum delivery count set to 10
    • Maximum size 1GB
  • Topic with 2 subscriptions with rules defined to segregate messages by their label into accepted and rejected

The above-mentioned Service Bus Queue and Topics are created in the UAT environment and tested to work fine. Now the need is to replicate the same in the production environment. One option is to recreate the Queues with the same settings either in the Azure portal or using Service Bus Explorer. But this would need considerable development and testing effort. You have an option to import an Azure Service Bus Queues and Topics from one Service Bus namespace to another using Serverless360.

Serverless360 provides a simple solution by allowing the users to import the Queues between namespace to make the process much easier. Hence saving considerable development and testing effort.

When Topics are imported from one namespace to another, the Topics would get exactly replicated with the same set of properties and rules if any. All the Topic Subscription rules (Boolean filters, Correlation filters, SQL filters) would get exactly replicated along with a default role in the target namespace.

Serverless360 allows the users to import the Queues between namespace

Process Service Bus Message

The real-world domains require real messaging, that means supporting features that involve compute and processing behind the scenes on behalf of the service. That’s where Azure Service Bus (ASB) service shines. The features it provides are genuinely impressing. Some of those are:

  • Scheduling
  • Dead-lettering
  • Transactional processing
  • Message ordering
  • Expiration (TTL)
  • Duplicate detection
  • Deferring
  • Pub/Sub
  • Filtering
  • …and more

The service goes beyond simple queueing and exhibits all the necessary features commonly found in the enterprise line of business application and systems.

The real challenge in dealing with Service Bus Queues/ Topics in the Azure portal is there is no visibility on the messages, the Service Bus Queues or Topic Subscriptions hold. In real time, there is an immense need to access and process the Service Bus messages to keep the business going.

Serverless360 provides out of box message processing capabilities to overcome the challenges in dealing with Service Bus messages in the Azure Portal

Retrieve messages

In Serverless360, execute queries to retrieve messages from Queues in ‘peek lock’ mode. When the query is executed, Serverless360 retrieves the list of messages in peek lock mode along with necessary message information such as sequence number, size, label, enqueued time, expiry time, message details and properties. It is possible to retrieve both active and dead letter messages in Service Bus Queues and Topic Subscriptions.

Process messages

Message processing capability in Serverless360 supports processing of active and dead letter messages. Execute queries to retrieve active/ dead- letter messages from Queues in both ‘peek lock’ and ‘defer’ mode. Besides, the service call will retrieve other information related to the message such as message id, sequence number, size, label, error reason, enqueued time, expiry time, message details and properties.

Different modes of message retrieval possible on Service Bus Queues and Topic Subscriptions in Serverless360.

Message Type / Mode Peek Lock Deferred
Active / Dead-lettered Messages In this mode of retrieval, the messages are peeked. It does not increase the delivery count of the message.

  • Resubmit - A copy of a message will be submitted to the target Queue/ Topic Subscription. Message in source Queue/ Topic Subscription will not be deleted.

  • Repair and Resubmit - A copy of a message with updated content and properties will be submitted to the target Queue/ Topic. Message in source Queue/ Topic Subscription will not be deleted.

Retrieving messages in this mode will move the message to a 'Deferred' queue and it can be processed later. This process increases the delivery count of the message.

  • Resubmit – A copy of a message will be submitted to the target Queue/ Topic Subscription. Message in source Queue/ Topic Subscription will not be deleted.

  • Resubmit and Delete – A copy of a message will be submitted to the target Queue. Message in source Queue/ Topic Subscription will be deleted.

  • Repair and Resubmit - A copy of a message with updated content and properties will be submitted to the target Queue/ Topic Subscription. Message in source Queue/ Topic Subscription will not be deleted.

  • Resubmit and Delete after Repair - A copy of a message with updated content and properties will be submitted to the target Queue/ Topic Subscription. Message in source Queue/ Topic Subscription will be deleted.

  • Delete – Message will be deleted from the Queue/ Topic Subscription.

Below is an illustration to process the active messages in the Service Bus Queue/ Topic Subscription associated with a Serverless360 Composite Application.

Service Bus Queue/ Topic Subscription associated with a Serverless360

Automate Service Bus message processing

Enterprises use Service Bus entities and Event Hubs to communicate between different services. As a part of integrating the services together, it is important to ensure that the message flow across the services is healthy. Serverless360 provides a capability to automate the process of sending, deleting and resubmitting messages to Service Bus entities and facilitate the above-mentioned requirement.

Consider a scenario where an application which listens for messages from a Service Bus Queue or Topic Subscription went unresponsive. Now, the number of messages inside the queue will start increasing gradually. Depending on the Time to Live value of the Service Bus entity either the number messages to be processed would drastically pile up or the messages would get dead-lettered.

When the application resumes its operation, it might either get overloaded due to the increased number of messages to be processed or fail to process the dead- lettered messages.

Here Serverless360 activities can be used to take necessary actions like:

  • purge the active messages to reset the system
  • perform dead- letter activities like, Resubmit or Resubmit and Delete as an active message to be processed by the resumed system
Automate Azure Service Bus message processing

Monitor Service Bus in Serverless360

Monitor state

If the intention is to monitor the state, say the business demands the Queue to be always active, Serverless360 can monitor Queue state. By associating the Service Bus Queue to Status Monitor or Threshold Monitor, it is possible to monitor the state, compare the current state against the expected state and alert through configured notification channels. Threshold monitor is now equipped to auto correct the state of Service Bus Queue, Topic and Topic subscriptions.

Azure Service Bus Monitoring in Serverless360

Monitor properties

It is also possible to monitor the Queue based on the following set of properties:

  • Transfer Dead Letter Message Count
  • Size in Bytes
  • Message Count
  • Scheduled Message Count
  • Active Message Count
  • Dead Letter Message Count
  • Transfer Message Count

Service Bus Queue can be monitored on multiple properties mentioned above with appropriate warning and threshold values. Please refer extensive documentation on Status Monitor and Threshold Monitor to choose the right monitor to meet your business need.

Service Bus Monitor properties in Serverless360

Monitor metrics

If the intention is to understand the message flow, reliability of the Service Bus Queue then the choice should be a Serverless360 Data Monitor. Data Monitoring can be configured for a Service Bus Queue/ Topic on an extensive set of metrics. Service Bus Queue data monitoring with appropriate warning and threshold values can be set as in represented below:

Service Bus Monitor metrics in Serverless360

The table below lists down the combination of metrics to monitor the Service Bus queue in various perspectives using Serverless360 Data Monitor.

Monitoring Purpose Metrics to Choose
Check if the Service Bus Queue is available. Please refer this article on ‘Monitoring Azure Service Bus Queue Availability’ Server Errors (Count), User Errors (Count), Size (Bytes)
Monitor Incoming Requests over a time duration Incoming Requests (Count)
Monitor Incoming and Outgoing Messages over a time duration hence check if Upstream and downstream applications are active Incoming Messages (Count), Outgoing Messages (Count)
Check if applications interacting with the Service Bus Queue are generating any exceptions User Errors (Count)
Monitor Throttled Requests over a time duration Throttled Requests (Count)
Check if expected number of messages are being processed by the interacting applications. Check message count by type Count of messages (Count), Count of active messages (Count), Count of dead-lettered messages (Count), Count of scheduled messages (Count)
Detect anomaly in abnormal increase in Queue size Size (Bytes)

Webinar

This webinar covers in detail the capabilities of Serverless360 to facilitate managing and monitoring Azure Service Bus Queues and Topics. All concepts are dealt with real-world business use cases for better understanding.

This content will be continuously revamped to stay up to date.

One platform solution for your Azure Serverless Monitoring and Management

Sign up for free

Free 15-day trial – No credit card – Easy setup