API Lifecycle, Versioning, and Deprecation

Every API integration comes to life the same way. Someone reads the API docs for a service's endpoints, writes some code to get things connected, and boom, the integration is born!

There's only one problem. Every API integration ages slowly over time, and eventually has a "retirement date" lurking somewhere in the (hopefully) distant future.

API Geriatrics—or old age care—comes into play here. At some point, the site or service who owns the API endpoints may need to make "breaking changes" so they can provide a better end-user experience for their development projects. Eventually they'll release a new version, and may shut down the older API altogether.

As an API producer, how can you best plan for these necessary changes to your API and communicate the changes with your users? And as an API consumer, how can you mitigate your risks of the API you rely on being shut down or deeply changed?

After launching over 750 app integrations at Zapier, here are the best practices for API Geriatrics we've learned:

1. Know your reasons for creating a new version
2. Make sure your new version is also a better version
3. Provide plenty of time and assistance to existing consumer

We’ll go into each of these in detail below, including our suggestion for new API response headers that will help both humans and machines understand your API deprecation plans. We’ve been pleased with the response to the idea so far.

“API consumers don’t want to get caught off guard with APIs that stop working. These API deprecation headers are a proactive way to help avoid API downtime.” – John Musser, API Science

Read on to see what we’ve learned from watching hundreds of providers update, improve, and sometimes deprecate their APIs.

Why API Producers Make New API Versions

There are a number of technical reasons why API Producers decide to advance their API to a whole new version:

Changing the authentication type: Older APIs generally start out with Basic Authentication or API Keys. Today, many are gradually being updated to better/newer approaches like OAuth v2.

Finely granular permissions: The new API may use new/additional scopes, so that specific permissions can be requested/used by the endpoints, rather than giving "all access" to an integration.

Maintenance and performance: Sometimes an API may be re-written from scratch, because it has reached a point where it's bogged down by inefficient database calls, or becoming too unwieldy to maintain. At that point, it's difficult to fix bugs/issues, without causing "side-effects", that may break things for the current consumers.

Newer programming language: Another reason to re-write an API from scratch is that a different or newer programming language may be a better fit for handling the implementation of the endpoints.

Changing the data that's returned by the endpoints: There may be situations where an API is returning a list of "heavy objects" (like invoices with line-items). The newer implementation of the API might return a list of "shallow objects" instead, and then provide additional endpoints that must be used to fetch the deeper details of a given object.

How to Build a Better Version of Your API

If you're an API Producer, and are working on a brand new version of your endpoints, here are some helpful items to keep in mind:

Use REST and JSON: These have become the expected approach to web APIs over the last decade. Even early API proponents have dropped XML from new versions. RESTful APIs are easy for developers to try out, and JSON is a lightweight format that is also easily human-readable. You need a really good reason to use anything else.

Always provide date/time information using an ISO8601 format: like 1999-12-31T23:59:59Z. If you need to provide the date in a different or localized timestamp, provide that info within an additional field.

Provide a migration path: if your API is changing to a different authentication type (perhaps from Basic Auth to API Key, or API Key to Access Token), provide an endpoint that allows the consumers of your API to migrate existing users' auths to the newer authentication type. Some specific examples include Dropbox and Vimeo.

Maintain the same level of functionality: if your current API has special endpoints to perform specific tasks (like "update an invoice status"), make sure the new API can provide the same task (like using a PATCH request on the "invoice" endpoint, to update just the invoice status).

Provide an endpoint to subscribe individual Webhooks: the best way to notify API consumers about new or updated data is via a Webhook. Provide an endpoint that allows the consumer to specify a fine-grained item, for which they would like to receive notifications. Some real-world examples are MailChimp and GitHub.

Ensure that reading data is idempotent: when a consumer reads data from an endpoint, don't automatically keep track of a page/cursor state. Have the client provide some kind of cursor/page token across multiple calls. That way a client can "go back" and re-read information when needed.

How to Retire an Old API

After you have launched your new API, your team will need to determine when the old API/endpoints will be shutdown, and inform all of the consumers. Unfortunately, getting that message delivered may be difficult.

Announce the Old API's End Date

Here are the ideal steps:

• Announce the end-date on your website's blog, and email developers (if you have their contact info) to let them know about the upcoming end date.
• In your API docs, include a message/banner on all pages which mentions the end-date. This will help alert developers who may arrive there via a deep link from a search.
• Within your app itself, mention the upcoming API end-date and notify your customers. Zapier has had users contact our support team to let us know that some site/service is going to turn off their older API after seeing such a notification.

Ultimately, there are situations where your announcement won't be received, because the developer's contact info is no longer valid, or an integration was developed by a consultant/contractor who is no longer associated with the project.

The best solution is a consistent, machine-readable approach that can be recognized by all consumers of your API.

One approach that has been mentioned by Kin Lane, Clearbit, and others is specifying a response header called X-API-Warn. It's a great approach, but if providers use that "warning" for other purposes (besides deprecation announcements), then consumers may run into trouble assessing the severity of the warnings.

A more "fine-grained" approach is to have the deprecated API/endpoints include these headers within all responses, to pass along specific information about the API/endpoint's deprecation:

• X-API-Deprecation-Date: 2017-12-25T00:00:00Z
• X-API-Deprecation-Info: https://example.com/old-api-deprecation-info?utm=from-header

Even though the IETF discourages the use of "X-" prefixes, it's still commonly used for similar "API info" purposes, like communicating rate limits/quotas for services such as GitHub and Twitter.

"I like this format a lot. I've heard of similar ideas at conferences, but they never seem to take off. They usually attempted to be some sort of automated deprecation format, and that may have been the problem. A format that powers logs instead of automation seems much more practical." – Aaron Hedges, Vimeo

If your integration uses a client-library (rather than direct HTTP requests), it may be a bit more difficult to obtain/receive that header information. Eventually, it may be possible for the popular client/request libraries to provide a piece of middleware, which could raise an alert if/when these response headers are present.

Flicker the Lights, Then Pull the Plug

When the final day arrives for the API Producer, it's best to slowly or intermittently shut down the endpoints. If possible, use a "rolling blackout" approach, where your endpoints return an HTTP 410 "Gone" for a short period of time (like a few minutes) every two hours. This will help alert any remaining consumers about the upcoming shutdown.

After a few weeks of that process, any remaining consumers of your API are probably "zombie requests", and the old API can be shut down.

API Consumers: Updating Your API Integration

Do you have code that connects with an older API? Here are some some things to keep in mind when updating your code and integration:

• Determine whether it's possible to maintain the same level of functionality—the newer API may return data in a slightly different structure, and might handle queries/filtering differently.
• Investigate whether the new API has specific rate-limits which may affect your integration.
• Investigate whether any new OAuth scopes or permissions are required by the newer API.
• Investigate whether the new API requires the completion of an "app review" process before others can use your new integration.
• When deploying your new integration, see if it's possible to roll it out in parallel, and monitor for differences between the old and new. This is only feasible when fetching/reading info from the endpoints (and not writing data).
• Use a gradual process of migrating users from the old integration to the newer one, to identify whether any "edge cases" are present that weren't found during development and testing.

In the end, building a bunch of direct integrations is usually easier than maintaining all of those integrations. Sometimes, the best approach is integrating with Zapier, and letting them deal with the inevitable API deprecation and migration.

The Golden Years

Eventually, every API integration that your team develops will reach an "expiration date", where the provider decides to retire their old endpoints and requires you to manage the API geriatrics.

Hopefully, API producers and consumers can work together to provide a "standard machine-readable implementation". The headers X-API-Deprecation (mentioned above) makes this information available to everyone, and reduces the "unexpected surprise" when an API is deprecated. The team at Zapier currently watches for this response-header within the logs, so that our team can proactively assess what’s needed to migrate API versions.

We would love to hear more about your experiences with API geriatrics and deprecations (whether you're a API Producer, or an API Consumer)! Tweet us @Zapier or email us at partners@zapier.com, and let us know about situations you've encountered when updating an API or integration.