Being able to quickly exploit new channels and devices, partnerships and third-party development through integrations is a huge competitive advantage. Building and maintaining APIs for your key systems and processes is a good way to achieve this, but it’s not straightforward. How can an ESB help you?
What is an ESB?
An Enterprise Service Bus (ESB) is a system primarily designed to integrate applications with a set of legacy services. These services may communicate in a variety of ways. To integrate each application directly with the services it requires is time-consuming and inflexible. And upgrading legacy services themselves to assist with the integration may be difficult. They may be owned by a third-party, poorly documented or tightly integrated with other systems. This can make their modification impossible, risky or uneconomical.
An ESB acts as a friendly broker for these legacy services. It typically exposes their operations through some common standard protocol, for example SOAP/XML. It usually assigns each message it receives a type, and stores it in a common format on a queue. Messages are then processed according to their type, and the ESB handles the calls to each required service in order to complete the processing.
An ESB wraps each legacy service in a simple and risk-free way. It abstracts away complexity and takes care of any idiosyncrasies or bugs in the services themselves. This pattern avoids the effort of multiple point-to-point integrations, and new applications can quickly and easily use the simplified common protocol. Furthermore an ESB can present an interface that represents logical business processes rather than the underlying services, making the system as a whole easier to work with.
The classic case for an ESB is to help new digital applications like a website to connect with legacy business systems. For example in the banking sector, complex financial products are often stored in old mainframe systems that are increasingly difficult to modify. Customers now demand access to these online or through a mobile app. An ESB helps bridge that divide.
I don’t have any legacy services. Why do I need an ESB?
Businesses frequently begin life now in an interconnected digital world – they may be built on a web application that already integrates with many services. Software in general is becoming much more about integration rather than data processing: many languages now have standard libraries to handle networking and remote calls, and open-sourcing means fewer software products are under total third-party control. The trend toward microservices means more services are exposed over common protocols such as HTTP/JSON with well-defined APIs.
Relatively new businesses can have applications and services that are much easier to integrate. So surely they don’t need ESBs anymore?
This is some truth in this. Many of the original integration challenges that produced middleware and ESBs are becoming less relevant. But it is also true that we now demand more from our integrations. Customers expect their applications to work on multiple devices, connect to other services or interact with social networks. Businesses expect to pivot on a penny: to change service, supplier or strategy almost overnight. Being able to adapt your systems and processes quickly to these new opportunities is a competitive advantage. And for tech teams, this invariably means connecting with or exposing new APIs in a rapid and reusable way.
But while software now may lend itself more easily to integration, such ready connectivity has some downsides that need management. For example:
- As the widespread internet outages on 21 October 2016 showed, your WiFi-enabled central heating system can now be exploited for a hacker’s attack. Proper management of connectivity is becoming increasingly important.
- Your startup’s website may be a single monolithic application that grew rapidly during the early stages without much planning. The complex web of dependencies within it, along with vestigial code from concepts that fell by the wayside, aren’t something you necessarily want to expose to other systems forever.
- Microservices that were originally built for internal use may not cover some aspects of an API very well, such as discoverability, security, authentication, versioning or rate-limiting.
- And don’t be fooled. Without careful management, today’s shiny new services can easily become tomorrow’s legacy burden.
Integration is still not trivial, it just has different challenges. On their websites, some ESB vendors like MuleSoft and TIBCO are even hiding away the term “ESB” down in the footer or a third-level menu item. It’s an effort to move on from the problems that ESBs were originally designed to solve. Now it’s more about good API management.
So what can an ESB do for my APIs?
A recent Gartner report on Full Life Cycle API Management provides a comparison of some vendors offering API management products. This is its diagram:
You can split the companies broadly into two groups. The first group are specialists in API management. They’re generally newer outfits whose API product is their business. They seem to have clever names like Apigee and Apiary. Other examples are Mashape’s Kong and even Amazon’s API Gateway. The second group comprises more established players such as Oracle, SAP and IBM who are building on their integration and ESB products.
The report finds that for pure API management, the specialist firms are competing on at least an equal footing. And if Google’s recent acquisition of Apigee is anything to go by, then they are leading the way.
But for a rapidly changing or growing business, API management alone is not enough. Where an ESB product has the edge is its array of tools for building and managing integrations. Coupled with good API management, this can make it a very powerful package. The Gartner report puts MuleSoft as the top ESB vendor for API management. So what are they doing so well?
MuleSoft’s approach is described as “API-led connectivity”. This includes a suggested hierarchy of APIs and services (which of course MuleESB supports):
This hierarchy is designed to be:
- efficient by reusing the lowest-level system layer APIs
- logical and flexible through the process layer APIs
- valuable for your applications through tailored experience layer APIs.
With so many APIs required, it’s no surprise perhaps that MuleSoft scores so highly for its API management.
But what this also shows is that they are heavily pushing a new use for ESB technology. Connecting these API layers requires good integration capability. And ESBs are perfect for that.
It shows that an ESB can be much more than just a single middleware layer translating between your services and applications. Used well, it can facilitate a rapidly evolving architecture. It can allow you to quickly refactor your services and their APIs as business needs change.
Integrations are no longer something you set up once for a digital transformation project. They are now a vital part of the constant evolution of a business, its architecture and APIs. A good ESB today is an integration and API management platform that not only connects your services, but helps you manage and adapt to this continuous change. To survive, ESBs must move to offering an Integration Platform as a Service (iPaaS). And the most successful products will come from the vendors that recognise this early.
Andy Worsley 