This blog is an extraction of the session “Manage Azure VM’s using Azure Event Grid and Serverless” presented by Jeff Hollan, Bahram Banisadr, and Ziv Rafalovich in Microsoft Ignite, Orlando.
This blog gives a basic understanding of Serverless and also discusses how to use Azure Functions, Logic Apps, and Event Grid to monitor Azure VM and apply IT policies such as asset tagging, maintenance window and many more.
How do I Monitor my Azure VM’s using Event Grid and Logic Apps?
Step 1: Choose the VM to which you want to monitor the change
Step 2: Instantiate a brand-new Logic Apps in the Azure portal and make sure to create it in the same resources of you VM.
Step 3: Add Event Grid to the newly created Logic App
Step 4: Add a condition to the Logic App to get notified on particular operations
Step 5: Add notification channel where you prefer to get notified
Let’s dive deeper to understand how to monitor Azure VM’s in detail.
To monitor and respond to specific events that occur on Azure resources or third-party applications, you can create and run tasks such as workflow by creating Azure Logic App that uses minimal code.
These resources can publish events on the Azure Event Grid.
Next, the Event Grid pushes those events to subscribers like WebHooks, or Event Hubs as final locations. As a subscriber, your Logic App can wait for those events from the Event Grid before starting the automated workflow to perform tasks.
Create a Logic App that monitors changes in the virtual machine and sends emails about those changes. When you create a Logic App with an event subscription for an Azure resource, events flow from that resource through an Event Grid to the Logic App. The tutorial walks you through building this Logic App
I hope you would have prior experience with provisioning Logic Apps. So, this article is not intended to explain the Logic Apps Initialization process.
Once you initialize a Logic Apps, now is the time to design the workflow.
Note: It is recommended to create the Logic App, Event Grid in the same resource group of your VM.
- Add Azure Event Grid trigger to the newly created Logic App
- On the designer, in the search box, enter Event Grid as your filter. From the triggers list, select the When a resource event occurs trigger.
- When prompted, sign into Azure Event Grid with your Azure account credentials. In the Tenant list, which shows the Azure Active Directory tenant that’s associated with your Azure subscription, check that the correct tenant appears, for example
- Now subscribe your Logic App to events from the publisher. Provide the details about your event subscription as described in the following table, for example
- Save your Logic App. On the designer toolbar, select Save. To collapse and hide an action’s details in your Logic App, select the action’s title bar.
- If you want to your Logic App to run only when a specific event or operation happens, add a condition that checks for the Microsoft.Compute/virtualMachines/write operation.
- In Logic App Designer, under the Event Grid trigger, select New step.
- Under Choose an action, in the search box, enter condition as your filter. From the actions list, select the Condition action.
- The Logic App Designer adds an empty condition to your workflow, including action paths to follow based whether the condition is true or false.
- Create a condition that checks the event body for a data object where the operationName property is equal to the Microsoft.Compute/virtualMachines/write operation.
- In order to achieve this, On the first row under And, click inside the left box. In the dynamic content list that appears, select Expression. In the expression editor, enter this expression, which returns the operation name from the trigger, and select OK
triggerBody()?[‘data’][‘operationName’] - Once it is done, add outlook connector in the Yes condition to receive notification whenever the specified condition is met.
Common challenges in managing Azure resources
- Making sure assets are compliant and not just security
- Cost and utilization optimization – An enterprise want the engineering teams, business teams to be fast and quick. At the same time, it needs to be aware and also control the expected cost
- Self-service
- There are too many external systems available for integration and orchestration
- It is quite difficult to keep up with legacy management platforms
Adding automated operations can save you up to 65% in operational cost. Being an IT admin or a developer, you hate anything which will slow you down. However, Microsoft has not yet rolled out the ‘perfect’ settings which would take you out of your miseries.
Why do we need Serverless for IT management?
The following points justify the question:
- It allows the developers and operation teams to focus only on their need. It totally abstracts server management, capacity planning, and availability
- You pay only when the code, workflow, or the event gets fired. Even an automation process would only cost you a few nickels
- The agility to develop and bring the product to market is tremendous
- There are a lot of connectors available. For instance, out of the box, there are about 200 connectors available in Azure Logic Apps which can also be used to integrate with external services like Pager Duty
What is Serverless Computing?
Abstraction of server
It allows the developers to focus only on their code and completely abstracts server management, capacity planning, and availability.
Event-driven scalability
Application components react to events and triggers in real-time with virtually unlimited scalability.
Pay-per-use
The billing is typically calculated on the number of function calls, code execution time and also memory used.
Benefits of Serverless Computing
- Focus – It allows you to focus only on business problems and ignore the involved technology behind it
- Efficiency – It provides better efficiency by providing service stability, development and testing management, shorter time to market and many more
- Scalability – It allows you to scale at your pace and hence it is a natural fit for microservices
Major services available in Azure Serverless platform
Azure Event Grid
This is one of the new entrants in the Azure platform. It basically acts as a network between all the services which lets each to communicate with one another and also notifies the user when an event occurs. For instance, if a new device is added to the IoT Hub, it would notify the system admin or the concerned authority about this event. Also, it is customizable to notify on the custom events.
Azure Functions
It is a solution for easily running a small piece of code or function in the cloud. You can write just the code you need for the problem at hand, without worrying about a whole application or the infrastructure to run it.
Logic Apps
It helps you automate and orchestrate tasks, business process, and workflows when you need to integrate applications, services across enterprise or organization both on-premise and cloud.
What is new with Azure Functions and Logic Apps?
Azure Functions
- Version 2.0 is made generally available. Now, it supports python language which would be much helpful in the automation process
- Key Vault secret integration is now available for authentication purpose
- Python private preview
Logic Apps
- Introduced an Integration environment which allows you to run the Logic Apps in an isolated environment with Virtual Network
- Managed identities
- The smart design feature which uses machine learning for service recommendations
To know more look at our blog where we covered the latest announcements of Azure Functions and Logic Apps.
Event Grid Roadmap
Event Domains
It is a multi-tenant way to manage subscription in Azure Event Grid.
Advanced Filters
String
- In/Not in
- Begins with/Ends with
- Contains
Numerical
- In/Not in
- Greater than/Less than
- Greater than or equal/Less than or equal
Boolean
Publishers
- Compute
- Key Vault
- Azure Container Registry
- Device Provisioning Service
- Azure Data Lake Store
Identity and Authentication
Here, let us try to understand how identity and authentication work. Consider a scenario, where you want all the resources in the subscription to follow a naming convention. So, you can define a custom code which executes and checks the name when the resource gets created. If the name is not compatible the script will automatically rename it. To do this action, the script needs authentication and authorization. The script needs permission to act on the subscription.
Azure Active Directory
This is an identity fabric of Azure Resources and Microsoft Graph Entities. It gives service identity, user identity and much more. There are a few ways by which you can authenticate the services.
Managed identity authentication
This is one of the new concepts in Azure. This feature provides Azure services with an automatically managed identity in Azure AD. You can use the identity to authenticate any service without any credentials in your code. Azure takes care of rolling the credentials that are used by service instance.
Service principle authentication
It is a way to create a special account in Azure AD for automated authentication. For Instance, if you are creating a “Name-checking application”, you can generate identity and a secret key for that application. The application gets a token and would be authorized by the service principle where the required information is stored. The action which takes place in the “Name-checking application” can be tracked using an auto log.
User-based authentication
This is one way of authentication which we usually do in Microsoft office and Azure portal using username and password.
Non-Azure AD
- API keys and Anonymous.
Scenario: Azure Logger using Cosmos DB
This scenario is based on creating Azure Logger using Cosmos DB. The goal is to store a snapshot of the resource group state each time a change is made.
An event raised from the ARM (Azure Resource Manager) would reach Event Grid and triggers a Function. The Azure Function investigates all the resource state and also takes a snapshot. In the next phase, the snapshot sends to CosmosDB.
Azure Alert to Serverless Code/Workflow
The alerts in Azure could be fired due to many reasons. It may be a planned or unplanned one. One of the unplanned alerts is called resource health alert. The health alerts can be provisioned for the resources which you want to monitor.
The alert condition configuration requires two parameters,
- Target selection
- Alert criteria definition
Once you configure the alert, point it to Logic Apps or Azure Functions in the desired subscription.
Scenario: Automated Tags Sync
Consider a scenario, where you want to synchronize mandatory tags set on your resources. Out of the box, there is a Logic App which runs on a recurrent basis. It investigates all the resources and resource groups in the ARM and also triggers the Azure Function. In the next phase, the Azure Function checks the Cosmos DB which contains the necessary tag information that the resource should hold. In the absence of the tag, it automatically syncs the tag to the resources in the ARM.
The above picture represents the code to check into the Cosmos DB for tag information and compare it with the existing Resource tag.
This picture shows the Logic Apps workflow. There is a scheduler which runs over an interval of 10 minutes. Then, there is an ARM action which retrieves all the resources of the defined subscription. In the next step, there is a ‘for each’ action which checks all the resources in the subscription against the Azure Function code. If the tags do not match the resources, it publishes an event to Event Grid.
Scenario: VM scale set management
Consider a scenario, where you want to simplify the management of large-scale VM. There is a Logic App which runs on a recurrent basis and triggers the Azure Function every three minutes. In the next phase, Azure Function checks the Cosmos DB for the number of VM instances and passes the value to Power BI.
This dashboard in Power BI shows the historical data on the VM instances that generates over a period. This real-time dashboard will be helpful to the operations team to get insights on the VM instances.
Scenario: Cleaning up the resources
This scenario is based on how on to remove the resources once it meets the expiration date. Initially, there is a scheduler which runs over a defined interval of time. Then, there is a Function which checks each resource for its ‘project’ and ‘tag’ name. In the absence of any of these names, the Azure Function appends it to the resource. Once it meets the expiration date, it cleans up the resource based on the tag from the subscription which is present in the ARM.
React to Azure Scheduled Events from outside the VM
The above Function runs an infinite loop and whenever it finds a scheduled event in VM, it gets pushed to the Event Grid and “autoAck” attribute acknowledges to the Azure portal.
The HTTP request triggers the Logic App and stores the event in Blob Storage. Then the Logic Apps sends the Email notification to the user.
Community Content
Recently, Microsoft has published a brief tutorial on monitoring virtual machine changes with Azure Event Grid and Azure Logic Apps.
Wrap up
In this blog, we discussed how to monitor Azure VM’s using Event Grid and Logic Apps, some common challenges in IT, and also gave a basic understanding of Serverless. Furthermore, few use case scenarios were explained to leverage the available Azure resources. Stay tuned for further updates. Happy Learning!
