17 November 2019

There’s more to publishing external APIs than choosing an API management vendor

Publishing a public API involves many considerations, from billing and metering through to documentation and onboarding.

An API management platform can help you with some of these concerns, but not necessarily in the way that you’d like it to. You may find that many of these concerns are better addressed through a collection of solutions rather than any all-singing, all-dancing API management platform.

Accounts and licensing

You will need to decide how to organise access to your services. An account refers to the entity around which you organise access to API services, i.e. your customers, resellers and partners. Licensing on the other hand refers to how we manage an account’s right to use an API.

There are numerous ways in which you can grant licenses. They could be subscription-based, time-limited or based on some purchased credits. Alternatively, you can do away with the complexity of licensing and let any account access every API.

In larger organisations, licensing and accounts tend to become a systems integration issue. The authority store is more likely to be a CRM such as Salesforce rather than an API management platform. Provisioning becomes part of a wider process that involves sales, marketing, account management and support teams. The waters will be muddied further if you want to offer self-service provisioning or provide demo or trial accounts.

Billing and payment

Billing refers to the way pricing models are defined and applied to API usage. There are many different types of pricing that can be adopted, from tiered pricing where usage becomes cheaper with more use, to flat fees or one-off charges. You may want to charge overage fees for excessive usage. If you are operating in the enterprise space, you are likely to want client-specific plans and charges.

The more sophisticated your billing models, the more likely you are to need a dedicated billing platform such as Zuora or Chargify to calculate the value of billable usage. Many of these platforms also take care of the payment aspects of billing, though you’ll generally find this being regarded as a separate concern in larger organisations.

Usage-based pricing will always complicate an API infrastructure. Not only do you have to pipe API usage into a billing system, but you’ll need to decide what constitutes “billable usage”. This can become something of a rabbit warren as not all requests are treated equally, especially if they are errors or failures.

Authentication and authorization

Authentication is the mechanism for identifying who is making an API call, while authorization refers to understanding whether a user can access the system.

For better or worse, OAuth 2.0 is generally regarded as the gold standard here. It is quite a loose specification, requiring decisions for aspects such as how to validate access tokens, what scopes to support and what information to include in access tokens (if any). Implementations can often get bogged down in complexity, misunderstanding and unhelpful debate.

OAuth 2.0 is also an incomplete solution in that it is not really an authentication protocol. It does not provide a way of communicating the identity of the user to a client. An access token only provides delegated access rather than proof of identity. Many implementations adopt OpenID Connect as a way of filling this gap.

Despite the extra complexity that OAuth 2.0 brings, there are advantages to adopting a standards-driven approach. Your APIs will be more accessible to a wider range of development tools and integration platforms. A custom approach based on API keys may be easier to build, but you’re only transferring the implementation burden to your consumers.

Implementing OAuth 2.0 and OpenID Connect is typically regarded as a concern for API management as it allows consistent security protocols to be asserted across all APIs. Note that you will still need to provide a separate identity store to manage users and their credentials.

Entitlement

Once you have authenticated a request, you’ll need to determine whether it is entitled to use services. There’s more to this than checking permissions or granted OAuth scopes. An account might be suspended for non-payment or exceeding its quota. You might want a “kill switch” that allows you to immediately cut off access for misbehaving tokens or users.

You don’t want to weigh each request down with entitlement checks. High-scaling APIs in geographically distributed architectures often address this problem through distributed entitlement caches, where a fast and resilient data store is kept up to date by event-based messaging.

Fair usage

A different aspect of request validation concerns asserting any fair usage policy. You will need to protect your platform from clients who inadvertently become denial of service attackers by making too many requests.

Most API management platforms support asserting quotas and rate limits, but you will also need to consider how to react to problems. Do you cut off the user immediately or throttle them back? Is this really an account management issue that warrants a warning email or friendly call? Are there certain larger customers that would expect to be treated differently?

Metering

Metering refers to the process of tracking API usage. It’s easy enough to capture. You can do it from end points themselves and most API management platforms also support some form of metering. However, some thought needs to be put into what you want to capture about requests and responses. The more detail you capture, the more likely you are to come across security, privacy and capacity issues.

You’ll also need to consider a data infrastructure that can collate, aggregate and distribute data to different audiences. You may need to pipe usage information to a billing platform to calculate fees. Customers will expect to have access to usage reports. You’ll also want to enable ad hoc analysis of usage for internal stakeholders.

Usability and change

Usability is often a neglected aspect of API implementations, but it is key for encouraging adoption. You will want your APIs to have a consistent style and follow similar conventions. An API management platform can help here by allowing you to build a catalogue of public APIs that can be mapped onto application end points.

That said, consistent and reliable APIs do not come about by accident, particularly when multiple development teams are involved. A usable API must be curated. This requires a combination of people, process and tooling. Many organisations end up appointing an “API czar” who is responsible for asserting consistency and best practice across APIs.

Once an API is used by external organisations it immediately becomes much harder to change. A versioning strategy isn’t an answer to this as you can end up running multiple versions of an API in production or maintaining increasingly complex mappings. You will need a process for managing change in APIs to protect existing integrations.

Documentation

No self-respecting public API should be without comprehensive documentation, but this requires a lot of on-going investment. Tools like Swagger can help, though they tend not to be so good at describing context, explaining best practice or detailing processes that involve chains of API calls.

Developers aren’t always known for their ability to write meaningful documentation. If you really want good-quality technical documentation for an external audience you’re not going to get it from a development team. You need a technical author.

Portals and content management

You will need to provide a web interface to support features such as customer sign-up, account management and user provisioning. You’ll also need somewhere to store API documentation and sample code. If you want to enable self-service, you’ll need to enable provisioning and provide access to helpdesk and incident management.

API management platforms offer a portal, but they are often very limited. They tend to be geared around internal, enterprise use cases rather than external SaaS. That’s a polite way of saying that the user experience is generally terrible.

If you’re serious about building a developer ecosystem, then you’ll need to consider the UX aspects of the developer experience. You will need facilities to manage a large and growing body of content created by numerous editors. You may want to provide multi-lingual content or white label a solution for partners and resellers. This points towards a dedicated web content management solution rather than an anaemic API management portal.

Filed under API design, Architecture, Rants, SOA, Strategy.