Client-Optimized Resource Representations in APIs

Create tiered representations of responses to optimize your API for mobile clients

One of the most important design principles for an API program is to embrace that APIs are products. It follows that successful APIs, much like products, must provide user-optimized experiences. Case in point: it is clearly inappropriate to send large response payloads to a mobile application. Your API may get away with it when responding to a request from an enterprise system communicating over a high-speed fiber-optic connection. Some clients may even want to get a rich response. But in general, assuming that the network is reliable and/or that bandwidth is infinite is a fallacy on the Web.

A Brute-Force Approach: Field Filtering The fact that mobile clients are not fans of large response payloads is hardly shocking news. API teams have historically allowed customization of API responses using a somewhat brute-force approach known as “field filtering” – allowing an API client to indicate a list of returned fields in the request. It can look something along the lines of:

  GET https://api.example.com/users/123?fields=firstname,lastname,age

We have also seen solutions where all default fields are allowed but a client has the ability to “ban” potentially expensive fields from the response:

  GET https://api.example.com/users/123?exclude=biography,resume

There Is a Better Way One of the main problems with field filtering is that it encourages tight coupling of a client with an API server. By allowing such a solution, we expect clients to maintain too granular an understanding of our API implementation. This can lead to unnecessary breaking changes and API versioning, which as – Mark Nottingham would have you know â€“ is evil.

Instead, what we can do is: create tiered representations of responses. Each tier of a resource representation can have a size-oriented or otherwise logical name, satisfying the need for tailoring output to a client group, without leading to extraneous coupling. As luck would have it: IETF already has an RFC that suits this purpose: RFC 7240 – Prefer Header for HTTP.

This is how you could create “minimal” (vs. “standard” or “full”) outputs:

  Get /users/123 HTTP/1.1
  Host: api.example.org
  Content-Type: application/json  
  Prefer: return=minimal
  Vary: Prefer,Accept,Accept-Encoding

  HTTP/1.1 200 OK
  Server: nginx/1.4.6 (Ubuntu)
  Date: Sat, 27 Jun 2015 11:03:32 GMT
  Content-Type: application/json; charset=utf-8
  Transfer-Encoding: chunked
  Connection: close
  Vary: Prefer,Accept,Accept-Encoding
  Preference-Applied: return=minimal
  Access-Control-Allow-Origin: *
  Access-Control-Allow-Headers: Content-Type

Please note that including “prefer” in the Vary header of the response is required per RFC 7240. It makes sure that some caching infrastructure won’t cache a wrong representation of the resource, since multiple representations will have the same URL, only differentiated by the Prefer header.

You can also see that the response acknowledged the preference by including Preference-Applied: return=minimal in the response.

The name of the preference batch doesn’t always have to be size-oriented. For instance, in case of news media and the blogosphere, a logical preference that might make sense, could be:

 Get /blog/1223 HTTP/1.1
 Host: api.example.org
 Content-Type: application/json  
 Prefer: return=teaser
 Vary: Prefer,Accept,Accept-Encoding

We’ve also seen elegant uses of the prefer header when clients ask the API server to transclude some of the data that would normally just be linked from the API response. This can be especially useful if the client would otherwise need to make too many (N+1?) additional requests to retrieve the same data. For instance:

Prefer: transclude=availibility_calendar
Vary: Prefer,Accept,Accept-Encoding

Word of caution: we need to make sure that prefer headers in our designs refer to logical batches and not individual fields. Just moving field filtering from the URL to the HTTP headers is not a solution, obviously.

TL;DR – In Conclusion Do not use field filtering to provide more sensibly-sized response payloads to mobile applications or other clients. This may create excessively tight coupling between the client and the server. Instead, use HTTP Prefer Header for HTTP – RFC 7240.

Acknowledgements I first learned the described technique (much like most things I know about APIs) from my dear friend and colleague Mike Amundsen. It builds upon thinking and work by amazing IETF contributors like James Snell and Mark Nottingham and possibly others in the Web community. I apologize if I accidentally missed anybody but please let me know if that is the case.

(This post was originally published on my personal blog.)