How Event Grid could enhance your listening activity to your messaging entities

|  Posted: September 4, 2018  |  Categories: Azure

 Azure Event Grid

With Event Grid, we have the option to react to events, both for Azure entities as well as beyond Azure. As the team that works on Service Bus also built Event Grid, it is not surprising that these services work together very well. The announcement of integration between these two services was on February 26 and allows us to listen for messages without available listeners, as well as dead letter messages in the various Service Bus entities. This means we no longer need to do any polling on our Service Bus topics and queues. But instead can have an event-driven architecture.

Especially for scenarios where messages come in intermittent intervals, it allows us to react on these, without listeners continually checking for new messages. Instead, we can place Azure Event Grid in between, which triggers whenever new messages arrive, implementing a push model instead of the pull model that we usually use when listening on Service Bus entities.

Service-Bus-with-Event-Grid

Placed on the namespace level of Service Bus, Event Grid will look for messages on all queues and subscriptions within, and trigger an event for each of these entities where they are available. Do keep in mind that, currently, Azure Event Grid is only available for the premium tier of Service Bus. Furthermore, if the Service Bus entity did not have any messages, and a new message arrives, Event Grid will immediately trigger. However, once an event triggers, new messages coming in right after that can take up to two minutes to trigger another event, so this will be something we have to keep in mind while setting up our architecture.

Extending our messaging scenarios

Let us have a look at how we can use Azure Event Grid to enhance our messaging scenarios. For this, we will tackle a common use-case, where our partner places messages onto a Service Bus topic, and depending on the country property; we will route it to one of our subsidiaries.

 

Adding Event Grid to scenario1

In this case, we have a Logic App which polls on their Service Bus topic subscription for one of the subsidiaries, while the other uses a custom client to retrieve the messages. However, as our partner is not continuously sending messages, most of the polling actions will yield no results. In Azure Logic Apps, we pay per execution of operations, and thus we are spending money for nothing in many of these. In the custom client, we are wasting unnecessary CPU cycles and calls to Azure. Would it not be nice if instead of continually having to check if our partner placed a message on the topic, we would get a notification when a message is available? This is where Azure Event Grid comes in, as it allows us to send out an event whenever messages become available on Service Bus. This new implementation looks like the following.

Adding Event Grid to scenario2

The Logic App and custom client no longer poll on their subscriptions, but instead, an event triggers them. Once an event triggers one of the subsidiary systems, it will go out to their corresponding Service Bus topic subscription, and retrieve all available messages. This means that we no longer will be doing any polling, but instead, work with an event-driven way.

Implementation

Setting up Event Grid integration with Service Bus is simple, so let us have a look at how we can implement the use-case we discussed in the previous paragraph. We start by creating the various components needed for our scenario. The first one is a Service Bus namespace.

Note: Only premium namespaces allow for Event Grid integration. Thus we will need to use this here as well. When doing this implementation just for learning or demo purposes, we recommend deleting this when finished, as the premium namespace does come with some costs, which are excellent for production but probably are a bit much to keep running just for test or demo purposes.

In the Azure portal, create a new Service Bus premium namespace, making sure to set all the correct properties.

Create premium namespcace

Next, we will create a topic in the namespace, which will receive the messages from our partner.

Create topic

Now, add two subscriptions for routing to the subsidiaries. For now, we will not change the filters, so messages end up in both subscriptions; however, in a real case, these would use properties to route to one or the other.

Create subscriptions

We now have our prerequisites set up, so let us add an Event Grid subscription for the first subsidiary on it, which will be the custom client. To do this, we go to the namespace and select the Events blade.

Open Events blade

As we will find, there are already several out of the box sample scenarios, which allow us to get started quickly with Event Grid. However, for our use-case, we will not use these samples but add an Event Grid subscription.

Add event subscription

For the Event Types select ActiveMessagesAvailableWithNoListeners, as this will trigger when new messages are available on the subscriptions. For the endpoint, create a RequestBin endpoint by clicking Create a RequestBin and copying the Bin URL (change HTTP to HTTPS). On the Create Event subscription blade, set the Endpoint Type to WebHook, and set the endpoint to the RequestBin URL we just retrieved. Give the subscription a name, and switch over to the Additional Features tab.

Set event subscription properties

As this is the subscription for Subsidiary A, we only want an event to trigger when a message is available in its corresponding Service Bus topic subscription. For this, on the Additional Features tab, set the Subject Ends With filter to the name of the topic subscription. This is also the place where to set up dead lettering for the Event Grid subscription, as well as configure the retry policy. Once we have set everything up, click the Create button to add the Event Grid subscription to the Service Bus namespace. The first time this will trigger an error, but this is to be expected, see the next part for this.

Set additional features

Serverless360-Free-Trial

Be aware, that Event Grid needs to validate our endpoint before we can use it. To do this using RequestBin, after clicking the Create button, refresh the RequestBin, which will now show a message from Event Grid like the following.

[{
"id": "7cb7c0fb-e403-4e7c-b0c5-d16f396813b8",
"topic": "/subscriptions/fdf3a3a3-c8f5-472f-8367-6a9a4a6c11a9/resourceGroups/EnhanceMessagingWithEventGrid/providers/Microsoft.ServiceBus/namespaces/ServiceBusEldert",
"subject": "",
"data": {
"validationCode": "7A01AEA4-4B1A-4D13-8843-1C4A19E56689",
"validationUrl": "https://rp-westeurope.eventgrid.azure.net/eventsubscriptions/subsidiaryasubscription/validate?id=7A01AEA4-4B1A-4D13-8843-1C4A19E56689&t=2018-09-01T11:57:41.2214389Z&apiVersion=2018-05-01-preview&token=atNzU8gt8YQP4%2f%2f8aDzrHU9%2bEW1cPmT7enqP7v8LfkA%3d"
},
"eventType": "Microsoft.EventGrid.SubscriptionValidationEvent",
"eventTime": "2018-09-01T11:57:41.2214389Z",
"metadataVersion": "1",
"dataVersion": "2"
}]

The important part here is the validationCode and validationUrl fields. The validationCode field is what we would use if this were an implementation in an actual client. Which would then need to respond with this code to the request from Event Grid. However, as we are using RequestBin, this is not an option. Therefore, we use the validationUrl field instead. This we open in a new browser window. So Event Grid knows we own this endpoint. This will result in a message that Event Grid successfully validated the endpoint. If instead, we receive a message indicating an internal server error occurred, click on the Create button once more in the event subscription blade, and get the new validationUrl in RequestBin, as this will happen when the validation URL has timed out. Once we have validated the endpoint, our new Event Grid subscription will show up in the Events blade.

Event-Grid-subscription-has-been-created

For the second subsidiary, we will create a new Logic App. This Logic App does the processing of the messages in the corresponding topic subscription.

Create Logic App

Once the Logic App finished creation, open the designer and select the When an Event Grid event occurs trigger. This will allow us to listen for Event Grid events and start a Logic App each time these occur.

Add Event Grid trigger

Sign in to the Azure Event Grid connector using your Azure account.

Sign in to Event Grid

Set the parameters of the trigger to get events from the Service Bus namespace we created earlier. Make sure to include the Event Type and Suffix Filter here as well to only listen for available messages on the topic subscription for subsidiary B. This will make sure we receive an event once messages are available. So now we need to read the messages from the Service Bus topic subscription. To do this, we will add a Get messages from a topic subscription action, which retrieves all the messages available. Subsequently, we loop through these messages and do our processing of each message. Finally, remember to save the Logic App.

Process messages after event comes in

Now, when we switch back to the Service Bus namespace. Open the Events blade, we will find a new subscription for the Logic App as well.

Second Event Grid subscription added

Testing

To test the solution, we will use Serverless360’s capabilities to send messages into Service Bus. If you do not have a subscription yet, you can create one for free. To try out this great tool for monitoring and working with Azure’s serverless services. In Serverless360, we add the Service Bus namespace we created earlier, so we can manage it from here. To do this, go to the Namespaces blade, and choose to associate a namespace using a connection string. Grab the connection string on the Shared access policies blade in the Azure portal, and associate it with Serverless360.

Associate Service Bus namespace with Serverless360

Next, go to the Dashboard blade. Associate the topic we implemented earlier with the default composite application in Serverless360. Thus allowing us to send messages to this topic.

Associate topic with composite application

We now go to the Activities blade for Service Bus and create a new Send Messages activity. Configure the parameters, select the topic we just associated, and save the activity.

Create activity

Finally, run the activity from Serverless360, which will place a message in the Service Bus topic. The topic then routes the message to the topic subscriptions for both subsidiaries, triggering the corresponding Azure Event Grid subscriptions. Consequently, Azure Event Grid sends an event to both RequestBin as well as the Logic App we created. In RequestBin, we will see the body of the message coming in, which will look something like the following.

[{
"topic": "/subscriptions/fdf3a3a3-c8f5-472f-8367-6a9a4a6c11a9/resourcegroups/EnhanceMessagingWithEventGrid/providers/Microsoft.ServiceBus/namespaces/servicebuseldert",
"subject": "topics/subsidiaries/subscriptions/subsidiarya",
"eventType": "Microsoft.ServiceBus.ActiveMessagesAvailableWithNoListeners",
"eventTime": "2018-09-01T12:36:53.4387835Z",
"id": "e707ac59-df41-4507-92da-2f1a12da8afc",
"data": {
"namespaceName": "servicebuseldert",
"requestUri": "https://servicebuseldert.servicebus.windows.net/subsidiaries/subscriptions/subsidiarya/messages/head",
"entityType": "subscriber",
"queueName": null,
"topicName": "subsidiaries",
"subscriptionName": "subsidiarya"
},
"dataVersion": "1",
"metadataVersion": "1"
}]

In Logic App run history, we will see that it received the event. And, then the messages from the Service Bus topic subscription.

Azure Event Grid

Conclusion

In conclusion, we have seen today how easy it is to extend our Service Bus entities with Azure Event Grid. By adding the capabilities that Azure Event Grid provides, we no longer need to worry about polling for new messages arriving on Service Bus. But can instead use events to notify us. This opens many options for us, both inside of Azure, but also well beyond. This allows for a truly open event-driven architecture.

Serverless360 is a one platform tool to operate, manage and monitor Azure Serverless components. It provides efficient tooling that is not and likely to be not available in Azure Portal. Try Serverless360 free for 30 days!

Free-Trial

Author: Eldert Grootenboer

Eldert is a Microsoft Integration Architect and Azure MVP from the Netherlands, currently working at Motion10, mainly focused on IoT and BizTalk Server and Azure integration. He comes from a .NET background, and has been in the IT since 2006. He has been working with BizTalk since 2010 and since then has expanded into Azure and surrounding technologies as well. Eldert loves working in integration projects, as each project brings new challenges and there is always something new to learn. In his spare time, Eldert likes to be active in the integration community and get his hands dirty on new technologies. He can be found on Twitter at @egrootenboer and has a blog at http://blog.eldert.net/