Azure Durable Functions – Patterns & Best Practices

|  Posted: January 2, 2019  |  Categories: Serverless

In a previous post on Serverless360 blog, we introduced Durable Functions and later discussed when to choose Durable Functions over Logic Apps. In this post, we like to dive into some of the patterns you can apply with Durable Functions and some of the best practices we consolidated from our online research.

Jan-webinar-cta

Durable Function Patterns

When you look at Durable Functions, you learn you can program workflow and instantiate tasks in order (sequential) or simultaneously (parallel) – or you can build a monitor, or support a human interaction flow (approval workflow). The patterns you can think of are chaining functions to control your flow, fan-in and fan-out scenario’s, correlating events, flexible automation, and long-running processes, and the earlier mentioned human interaction — patterns that are hard to implement with only functions or with Logic Apps.

Chaining Functions

The most natural and straightforward pattern to implement with Durable Functions is chaining functions together. You have one orchestrator function, that calls multiple functions in the order you desire. With functions alone and using Service Bus Queues you can achieve pattern too, however, you will face some challenges:

  • No visualization to show the relationship between functions and queues.
  • Middle queues are an implementation detail – conceptual overhead.
  • Error handling adds a lot more complexity.

Using Durable Function, you will not run into those challenges. With an orchestrator you can have a central place to set the order of calling the functions (relations), require no management of queues (under hood the Durable Functions use Storage Queues, and manage them), and central error handling (when an error occurs in one of the activity functions then the error propagates back to the orchestrator).

Fan-out/Fan-in

An alternative to the previous patterns is the fan-out/fan-in pattern – one you can use when you need to execute one or more functions in parallel and based on the results you run some other tasks. With just functions, you cannot implement such a pattern. Moreover, you will also face the challenges mentioned in the previous pattern. However, with Durable Functions, you can achieve the fan-out/fan-in pattern.

See also the Call to Action section in the end with links to samples.

Would you like to get notified on the failure in any invocation of an Azure Function?

HTTP Async Response

With Functions, it is possible you call another API and do not know the amount of time it would take before a response is returned. Latency, volume, and other factors can lead to not understanding the time it would make the API to process the request and return a response. A function can time-out (Consumption Plan), or state needs to be maintained (undesirable for functions, as they need to be stateless). Fortunately, Durable Functions provides built-in APIs that simplify the code you write for interacting with long-running function executions. Furthermore, the state is managed by the Durable Functions run-time.

See also the Call to Action section in the end with links to samples.

Actors

Another pattern is the monitor pattern – a recurring process in a workflow such as a cleanup process. You can implement this with a function. However, you will have some challenges:

  • Functions are stateless and short-lived.
  • Read/write access to external state needs to be carefully synchronized.

With Durable Functions, you can have flexible recurrence intervals, task lifetime management, and the ability to create multiple monitor processes from a single orchestration.

See also the Call to Action section in the end with links to samples.

Human Interaction

Within organizations, you will face processes that require some human interaction such as approvals. Interactions like approvals require the availability of the approver, and thus the process needs to be active some for time and need a reliable mechanism when process times out. For instance, when an approval doesn’t occur within 72 hours, an escalation process must start. With Durable Functions, you can support such a scenario.

See also the Call to Action section in the end with links to samples.

Sample Implementation: Chaining using Durable Functions

Assume you have a message resembling an order, which needs to trigger several tasks such as persisting the order in a database, store it in blob storage (archiving), and sending a notification (via a queue and Logic App). With Durable Functions, you can coordinate and control the flow of tasks (see diagram below).

chaining-using-Azure-durable

The Orchestrator client is a function that can be triggered when a message is sent to the Orchestrator client. This client (function) will call the orchestrator and pass the order message.

The orchestrator will on its turn receive the order and call the activity functions.

Each of the activity functions will perform a task – in this case, store the order in a document collection in a CosmosDB instance, store (archive) the message is stored, and send a message to the queue to send out a notification via a Logic App.

Best Practices

With Durable Functions there are a few best practices you need to consider:

  • Use Azure App Insights to monitor running instances and health (this also account for using Azure Functions).
  • Function App also exposes HTTP API for management (status, update, terminate), also discussed with the HTTP Async Response. With the API methods, you can influence the course of action of your Durable Functions.
  • Version your durable function consciously.
  • Side-by-side deployment (update the name of your task hub on deployment), (see Durable Functions Blue Green Deployment Strategies).

Call to action

Here is a list of resources you can leverage to get some hands-on experiences with Durable Functions or see some of the patterns in action:

Wrap up

In this blog post, we hope you have a better understanding of what implementing some of Durable Functions patterns can bring and what value it offers. Durable Functions, in the end, give you the ultimate control over a workflow not achievable with alternative technologies such as Logic Apps or Functions alone. Together with some of the best practices, we consolidated you should now be able to build sustainable solutions with Durable Functions.

Serverless360 is a one platform tool to operate, manage and monitor Azure Serverless components. It provides efficient tooling that is not and likely to be not available in Azure Portal. Try Serverless360 free for 15 days!

Serverless360-Blog-CTA

Author: Steef-Jan Wiggers

Steef-Jan Wiggers is Azure Technology Consultant at Codit, is all in on Microsoft Azure, IoT, Integration, and Data Science. He has almost 20 years’ experience in a wide variety of scenarios such as custom .NET solution development, overseeing large enterprise integrations, designing and building API's and cloud solutions, managing projects, experimenting with data, SQL Server database administration, and consulting. Steef-Jan loves challenges in the Microsoft playing field combining it with his domain knowledge in energy, utility, banking, insurance, healthcare, agriculture, (local) government, bio-sciences, retail, travel, and logistics. He is very active in the community as a blogger, book author, InfoQ editor, and global public speaker. For these efforts, Microsoft has recognised him as a Microsoft MVP for the past nine years. Steef-Jan's can be found on twitter at @SteefJan.