APIs are now the number one route to market for many businesses. They are driving mobile applications, web applications and other channels like IOT. The API is now the key to the customer experiences and organizations digital transformation. In times of greater demand, such as retail shopping events or health crisis around the world the API and its architectures can come under pressure from increased demand, in ways it was never designed for.
We will look at how APIs can be built and managed to deal with the scale needed to cope with increased demand. But first what can we do before the API is released to the users. Testing should not only give us the assurance that the API works, is secure and performant, but it can also give us the kind of statistics that help us understand how it should and will grow. What impact will the addition of new API servers or API gateways add to the performance as API demand grows. Testing should also be performed to closely match the users request patterns, solutions like BlazeMeter allow for functional and performance testing from the cloud so that API architectures can be tested from a number of locations and the results can really show how things like regional demand will affect the API. When testing is also integrated with Mock services it gives the architect a much clearer idea of where future results will benefit the API under increasing load. Mock services give organization a way to understand how the API will perform as various components are scaled up or down
Over the 20 years I have worked with APIs gateways, SOAP gateways and other security solutions I have seen one key advantage of testing the performance well in advance of increased demand. Often customers ask me how much load a gateway can handle. But that is rarely the question that is most important. How many TPS can your backend, LDAP or auditing server handle is often the question that is more important. Understanding the bottlenecks makes it easier to plan for growth. Good testing give us that insight.
So once you understand your APIs footprint and growth characteristics it’s time to help the API and its supporting technology handle the increased demand.
One of the most basic ways to increase the performance is through handling the traffic better that hit the API. Rate-limits and quotas start the process of protecting the API from extra calls and can slow down the traffic. We are managing the traffic not just dealing with it. Its better the API responds to some responses slowly than crashing because the server couldn’t handle the load. Likewise, it is often possible with an API gateway to cache API responses. Either caching the entire API response or part of that response can remove thousands of unwanted or unneeded API calls. Caching at the gateway level has often been a key part of scaling a gateway and now it still as important for non-transactional APIS.
Increased API load can also come from people misusing the API. This can be a determined misuse, accidental or through badly implemented code. Adding threat protection into and API gateway is always a good design pattern. Tailoring the threats for your API should also help, for example there is no need to protect against SQL injection if you don’t have a database in the API environment. But checking the maximum message size on the request and response it a good way of ensuring unwanted traffic is entering the API environment and no “extra” data us leaving in the responses.
Horizontal or vertical. When choosing and API technology you need to make sure all the components can scale well. API servers and API gateways need to support horizontal and vertical scaling. Adding extra power to the servers and gateway is vertical scaling and adding more and more instances or nodes is vertical. Vertical is often associated with clustering but many classical clustering solutions have moved on to support vertical scaling in other ways. Employing messaging solutions like Kafka could be one such approach. Each has their benefits and having a solution that supports both vertical and horizontal scaling is very important. Having a solution that can also work across on-premise, private clouds and public cloud is also very important. SAAS based solutions in the cloud are great, but having technology that can run and be charged for on a per instance basis can save organizations large amounts of money. Designing a solution that can scale without increasing you running costs needs to be a consideration. If your API is a success, then it will be called more. If that actually costs you money each time is it really a success! With pure SAAS solution this has been proven to be the case.
In the past API growth has been measured in servers and rack space. But now as more and more APIs are built in microservices architectures, in containers. Other considerations become important, being able to deploy an API, its server and gateways to a container environment or service mesh give organization more and more scope for dealing with the increased demand. Auto scaling in private clouds and public clouds is a really great way to deal with scale. At Layer7 we have been doing just that, watch our TechTalk on service mesh use cases for our API solutions.
Finally, I just wanted to look at how API design and implementation can also affect the scaling requirements. Graph QL was designed to help remove some of the very chatty application that Facebook has and as such is also a key element in building APIs that scale. Watch our TechTalk on how GraphQL can help ensure an API is not so badly designed it is the reason behind the unwanted or unneeded calls.