How and when to consider migration from BizTalk to Serverless or Azure Integration Services

|  Posted: September 20, 2018  |  Categories: Azure Serverless

Let’s face it, integration solutions are not solely anymore present in the server world. These days we see hybrid solutions, a BizTalk solution communicating with a SaaS solution like Salesforce, or a complete cloud integration solution with Logic Apps. Thus, you might wonder does BizTalk still have a future with the cloud, and should you start migrating your integration solutions to Azure full-stop? Or perhaps lift-and-shift your existing BizTalk group to VMs in Azure? There are no direct answers to these questions with a simple ‘yes’ or ‘no’. It depends on your organization’s strategy, current investments, and status of the IT-landscape.

Introduction

What are the options? Stay on-premise with your existing IT, because the infrastructure is still capable of pushing the workloads, is not end-of-life, and the integrations run internally with other systems like SAP, Oracle, or others. The only potential worry you might have is the support for BizTalk Server. However, if you have the latest edition 2016 or migrate to it, then mainstream support continues until the 1st of November 2022 –  four years from now. Furthermore, the Pro-integration Product Team from Microsoft recently announced another major BizTalk version in VNext. Hence, there will be another version coming – pushing to the support for the product for another few years beyond 2022.

BizTalk-vNext

With BizTalk Server 2016 and its feature packs your organization can benefit from the up-to-date cloud-based adapters such as Service Bus and Logic Apps. Furthermore, the feature packs for BizTalk Server 2016 Enterprise provide you with more capabilities to interact with the Azure Platform like analytics through Application Insights and PowerBI, push tracking data to Event Hubs, and publishing orchestration endpoints using API Management. BizTalk Server 2016 gives you the ability to support hybrid integration solution by leveraging the cloud-based adapters, on-premise data gateway, and REST/JSON support.

Another option for your organization might be to move to the cloud entirely. With digital transformation, it is well possible that your organization IT future lies with the cloud. The workloads, business process, applications are suitable enough to run in the cloud or easy to refactor or migrate to the cloud. Furthermore, your integrations are not too complicated and can be re-engineered using Logic Apps, Service Bus, API Management, and Azure Functions. Also, it is possible that the BizTalk footprint or investment is low, you only have a few integrations running. Refactoring these integrations in the cloud is not a significant investment.

What’s going to be my strategy moving forward?

With enterprises transitioning or adopting the cloud, what should the strategy be for integration in the next years to come? You have several options depending on what the overall strategy of your enterprise is. You can lift-and-shift the entire server infrastructure to Azure, transition to support a hybrid integration environment, or go to the cloud full stop.

Lift-and-shift to Azure

One of the possible strategies could be to lift-and-shift your existing BizTalk environment and other servers to virtual machines (VMs) in Azure. You could consider this approach because you can benefit from:

  • Better availability of your server infrastructure, which can span multiple data centers or through availability zones.
  • A secure environment by leveraging, for instance, the Azure Security Center. This service identifies security threats and provides you with an investigation path when security incidents occur. Thus allowing you to take automated actions.
  • A reduction of costs if set-up properly that is shifting all servers to Azure. Leaving a few on-premise and move, for instance, the BizTalk server to the cloud is not a good option. You will be dealing with networking challenges and manage both on-premise and VMs in the cloud.

When considering lift-and-shift to Azure VMs think about the following aspects:

  • Security – the VMs will need to be running in a domain setup in the Azure Active Directory (AD) when lifting your complete server infrastructure to Azure VMs. You can join Windows Virtual Machines to Azure AD – and create accounts in Azure AD for your service accounts, and user accounts. Note that those user accounts are necessary to assign a role to them and for Role-Based Access Control (RBAC) purposes.
  • Networking – you can create a network in Azure by creating VNETs – a representation of your network in the cloud. And within this network, you can have machines communicate with each other – and isolate them from the public internet.
  • Monitoring & Alerting – although Microsoft hosts the VMs you still need to manage the operating system and installed services on them. There are several services available to aid you in monitoring your VMs like Azure Monitor, and Application Insights. For a better understanding of monitoring and alerting your VMs see Monitoring Azure applications and resources
  • Backup & Restore – in case of a failure in your VM you restore them from a previous restore point when you use the recovery service vault service in Azure. This service enables you to take snapshots of your VM at defined intervals.

To decide for a lift-and-shift of your server infrastructure, you can read the Lift and Shift Guide from Microsoft.

Upgrade to the latest version

An excellent strategy will be to upgrade to BizTalk Server 2016 if you are still on an older version of BizTalk. Why? Well, the 2016 edition is the most cloud-enabled with adapters for messaging service in the cloud – adapters for the Azure Service Bus, Logic Apps adapter, and Feature Packs with specific cloud capabilities. Furthermore, the support for 2016 as described earlier has support until 1st of November 2027 – thus it can be valuable for your business for almost another decade while exploring alternatives in the cloud.

In case you have a BizTalk 2010, 2013, or 2013 R2 environment, then you can upgrade to BizTalk 2016 leveraging the BizTalk Server Migration Tool made available by the MSIT. This tool is useful once you have a new BizTalk 2016 Group set up and tested its performance (maximum sustainable throughput). It helps to migrate your applications, hosts, host settings, and so on from your current environment to the new BizTalk 2016 environment.

Migrating BizTalk to Serverless

Hybrid Integration Environment with BizTalk 2016

With BizTalk Server 2016 you can support a hybrid integration environment and have on-premise systems communicate with Azure Services and (third party) SaaS solutions. Azure Services and many modern SaaS applications offer an API layer, through which you can easily connect your on-premise systems to leveraging BizTalk Server. BizTalk Server 2016 provides the connectivity for you and brings all the other capabilities you are familiar with, like mapping.

Serverless360-Free-Trial

The table below shows an overview of the capabilities necessary in a hybrid scenario.

 

Capability

Description

Connectivity

BizTalk 2016 Adapters

Logic App Adapter, Service Bus Adapter (SB, and Relay bindings), Office365(1)

Network

On-Premise Data Gateway

Separate service that you need to install on a BizTalk machine. It acts as a bridge between the on-premise IT environment and Azure. Furthermore, the on-premise gateway provides a secure data transfer between on-premises data sources and Azure Services like Logic Apps.

Availability

BizTalk Group with two active nodes

Availability of BizTalk is determined by the number of nodes (BizTalk instances). The minimum is two active nodes.

Security

Firewall, Express Route

Securing on-premise endpoints

Scale

BizTalk Group with multiple active nodes

BizTalk Group can exist on several machines (BizTalk instances)(2)

Backup & Recovery

BizTalk Tracking Database, Blob Storage

With Feature Pack 2 you have the ability to backup to Azure Blob Storage – there is a job available for it.

Operations

BizTalk Administration Console

Depending on your operational requirements you can also leverage third-party products like BizTalk360.

Monitoring

Operation Management Suite (OMS)

Depending on your monitoring needs you can leverage the Azure capabilities or Third party tools like Serverless360, and Atomic Scope.

Archiving

Event Hubs, Azure SQL Data Warehouse

With Feature Pack 2 you can offload tracking data to event hubs and subsequently migrate that data to an Azure SQL Data Warehouse.

  1. With BizTalk Server Feature Pack 3 you will have access to the Office365 adapters.
  2. To scale out to multiple BizTalk machines, you will need the BizTalk Enterprise Edition.

Go to the Cloud full stop

The last option is to transition to the cloud full stop and refactor to cloud-native solutions (or serverless) and decommission your server infrastructure at the end. This choice will be an excellent fit if the strategy is to transform the business towards being entirely cloud-based. For startups it will be a natural choice as the startup costs are as low as procurements of servers, setting up an exchange and so on is not necessary. The Microsoft Cloud and the Azure provide enough capabilities to get you started. Furthermore, you can benefit what the cloud, Azure, has to offer regarding availability, scale, and platform.

The refactoring of your current solutions into cloud-native ones can mean a hefty investment depending on the number of solutions and size of the server infrastructure. However, if you leverage Microsoft Cloud (Exchange, Dynamics, Office365), you can achieve quite some cost savings. However, migration can still be a challenge, and the devil is in the details!

Looking at integration capabilities in Azure you have the option of leveraging Logic Apps, Service Bus, Event Grid, and API Management (integration services). The capabilities, these services offer can be enough for your integration needs.

The table below shows how the BizTalk capabilities map to the Azure services.

 

BizTalk Server

Azure

Capability

Message Routing

BizTalk MessageBox

Service Bus, Event Grid

Topics and subscriptions, Event-driven

Mapping

BizTalk Mapper, XSLT, C#

Integration Account(1)

Upload map to integration account (XSLT, Liquid(2))

Pipelines

BizTalk Pipeline, and pipeline components

Logic App

Logic Apps calling Azure Functions for pre- or post processing

Adapters

Out-of-the-box adapters (application and protocol)

Logic App

Connector(3)

Orchestration

BizTalk Orchestration

Logic App

Workflow definition

Rules

BizTalk Rule Engine (BRE)

Functions

Custom code(4)

BAM

BizTalk Activity Monitoring

Logic App, OMS

Tracked-properties, OMS portal

API (REST, SOAP)

Bindings in adapters (WCF)

API Management

OpenAPI Swagger, WSDL

Operations

BizTalk Admin Console, BizTalk360

Azure Portal, OMS, ARM-templates, PowerShell, CLI

Managing Azure Services

Monitoring

BizTalk Health Monitor, BizTalk360

Azure Monitor, Application Insights, Serverless360

Monitoring capability in Azure or Third Party

Archiving

BizTalk databases (Archive)

Azure Storage, Application Insights, Log Analysis

Logging & Tracking

  1. To build XML-schemas and maps, you will need the Microsoft Azure Logic Apps Enterprise Integration Tools for Visual Studio 2015: https://www.microsoft.com/en-us/download/details.aspx?id=53016 to apply them in a Logic App.
  2. For JSON mapping you can build Liquid templates, which do not depend on Visual Studio. You need to develop these maps yourself and upload them to an integration account.
  3. There is an option available to create your custom connector in case there is no Microsoft managed connector available.
  4. Currently, there is no service in Azure available similar to the BizTalk Rule Engine. To apply the rules, you will have to code them in an Azure Function. Moreover, you can call an Azure Function from a Logic App.

Wrap up

In this blog post, we tried to explain the options you and your organization have when it comes to migrating to the cloud potentially. It depends predominantly on your situation and chosen strategy for integration towards the future. Fortunately, there is a roadmap for BizTalk; a new version is in the works, and the BizTalk 2016 version is very suitable to support hybrid integration solutions. Unless your strategy is not going full stop to the cloud BizTalk Server is a safe bet for supporting your integration solutions. Furthermore, there is still an active partner eco-system around the product and a vibrant community.

If your companies strategy is moving to cloud and adopt serverless architecture for your solutions than we suggest reading the following blog articles:

Serverless360 is a one platform tool to operate, manage and monitor Azure Serverless components. It provides efficient tooling that is not and likely to be not available in Azure Portal. Try Serverless360 free for 30 days!

Serverless360-Free-Trial

Author: Steef-Jan Wiggers

Steef-Jan Wiggers is Azure Technology Consultant at Codit, is all in on Microsoft Azure, IoT, Integration, and Data Science. He has almost 20 years’ experience in a wide variety of scenarios such as custom .NET solution development, overseeing large enterprise integrations, designing and building API's and cloud solutions, managing projects, experimenting with data, SQL Server database administration, and consulting. Steef-Jan loves challenges in the Microsoft playing field combining it with his domain knowledge in energy, utility, banking, insurance, healthcare, agriculture, (local) government, bio-sciences, retail, travel, and logistics. He is very active in the community as a blogger, book author, InfoQ editor, and global public speaker. For these efforts, Microsoft has recognised him as a Microsoft MVP for the past nine years. Steef-Jan's can be found on twitter at @SteefJan.