4 January 2015

Why REST is not a silver bullet for service integration

REST is sometimes described as the next “evolutionary step” in service integration. Given that the web is one of our more successful distributed systems, why not leverage the tools and techniques we already know when integrating services?

The microservice community advocates a service collaboration approach based around “smart end-points and dumb pipes”. This is in part a reaction to the enterprise service bus (ESB) approach where the communication mechanism provides a bunch of sophisticated features such as routing, orchestration and business logic. The problem with the ESB is that it often winds up becoming a vast well of complexity and repository for business logic that should belong in services.

This contrasts with the notion of the “dumb pipe” where the network protocols are just there to move data around. The “smart end-point” service takes responsibility for every other aspect of collaboration. REST is the “dump pipe” of choice for many as it allows autonomous services to collaborate freely using established tools and protocols. However, this approach does not take into account some of the shortcomings of REST and HTTP in particular.

REST creates temporal coupling…

One issue with REST is that it requires somebody to be actively listening to a request. If a service needs a response from the remote service before it can carry on with any further processing then it is closely coupled in a temporal sense.

Services that collaborate via REST are vulnerable to becoming “distributed ball of mud” where the overall system becomes entangled in a web of synchronous service calls. This is not so different from the over-complex, inter-connected monoliths that microservice architectures seek to overcome. The connections are just out-of-process calls to web services rather than in-process calls to components.

Careful service design can help to mitigate this. After all, if a service is truly autonomous then it should be able to discharge its responsibilities without having to talk to other services. An asynchronous approach based on event messaging would be a more effective means of eliminating temporal coupling. This will allow information to be broadcast without any expectations over who might be listening or responding, allowing for genuinely autonomous services.

…and location coupling too

If a service is based on a particular physical location or address then communication will break down if this location changes. REST inevitably involves a dependency on an external address which tends to create physically-coupled components held together by complex and fragile messes of configuration.

An architecture based on REST needs some form of broker in place to abstract away the physical addresses. Locations become logical destinations and addresses are effectively concealed from participating systems. This kind of service location is not without its problems and care should be taken to ensure that it does not become a bottleneck or security vulnerability.

REST comes in many different flavours

REST should help to reduce coupling between services by laying some widely-understood ground rules around resources, methods and hypermedia.

However, REST is a style rather than a standard and it is surprisingly hard to implement consistently across an organisation. Developing a REST API involves numerous subtle design decisions around areas such as resource design, method usage and status codes. It’s easy to get hung up on debates around what is and is not “RESTful”.

It is particularly difficult to achieve consistent services in the kind of distributed environment with light-touch governance which is popular among microservice advocates. The lack of any formal contract means that it’s easy to introduce subtle differences in style that, while still “RESTful”, require consumers to implement each service integration differently.

HTTP is pretty limited, even for a “dumb pipe”

All in all, HTTP is a pretty limited medium for service integration.  It is inherently fragile with little provision for fault tolerance. There is no support for server-initiated or peer-to-peer communications, nothing to help you manage transactions or long-running processes, no delivery guarantees and no provision for any buffering to help smooth out spikes in demand.

It’s left to the services themselves to find solutions to these problems. For example, fault tolerance and buffering can be provided by adding low-latency messaging behind the service end-point. This does place more of a burden on services to provide the scaffolding necessary to be “smart” end-point.

This kind of complexity is often handled more consistently by some kind of messaging-based middle-ware such as a service bus. This kind of “smart pipe” solves many of the problems associated with remote service integration centrally where REST merely shifts them to each individual service.

REST does have a place in service integration

REST does have a number of practical advantages. It’s neatly cross-platform, widely adopted (if not widely understood) and is a good choice for exposing APIs to external organisations. The request\response style of REST makes it a natural fit for query-based interfaces where fast, synchronous communication is called for.

By the same token, messaging is not a good choice for query-based interfaces. In this case, the fault tolerance and decoupling that it affords is a high price to pay for the extra complexity and latency.

There is a place for REST in a service-based architecture, particularly when you need an immediate response. It’s just a little too much of a “dumb pipe” to provide the only collaboration fabric for a large and complex set of decoupled, autonomous and collaborating services.

Filed under Architecture, Favourite posts, Integration, Messaging, Microservices, REST, SOA.