API Academy

Sponsored by Layer7

A judge 's gavel sitting on top of books next to a law book.

Using Consul as a Source of Truth in Microservices

By Balaji Radhakrishnan, PMP, CISSPPosted on

App Economy and Microservices

In today’s app economy, APIs are playing an essential role in helping customers to achieve their digital transformation goals. In this process, enterprises are trying to convert their monolithic applications into more granular and autonomous microservices to support their business objective and also speed to market.

Consul as Source of Truth

In a microservices environment, an increasing number of services can drive a correlated increase in policies governing those services. This can lead to duplication of common tasks for the sake of a small portion of policy related to each specific service. Deploying and scaling these services across various environments only adds to the complexity. 

In this article, we shall review an approach where we could minimize dependencies and expedite & automate the migration of these APIs by using Consul as a source of truth for these backend (micro)services.  The API Gateway is then used for policy enforcement that guarantees security, protection, throttling and transformation capabilities as needed for each and every incoming request. 

In this strategy, Consul can be used as

  • Service Discovery tool – where the application team can configure/register their (micro)services. As part of this registration, they could also leverage the KV store feature of consul to define:
    • Throttling parameters for rate limit and spread limit etc.,
    • Access Control – desired security protocol and other protection parameters as needed for managing their proxies and backend services.

As stated, in this approach an API gateway can leverage Consul for discovering services and once a service is discovered, it will apply the required throttling, security protocols, and protection needs via the combination of predefined policy templates and information retrieved from KV store (Consul). Relying on templates enables a more manageable number of policies that are then enriched with service-specific content within each deployment.

For example, we can leverage pre-defined policy templates for various API management requirements:

  • Throttling is applied via the predefined policy template but the values like “Rate Limitâ€, “Spread Limitâ€, and “Custom Data†for “Apply Rate Limit†assertions can be retrieved from Consul’s KV store during run time.
  • Similarly, Security is also applied via the predefined policy template with all possible security protocols, one of which can be associated dynamically via Consul’s KV store configuration during runtime for that specific service.

We could apply similar predefined policy templates to define a proxy and many of those API management options can be associated dynamically during runtime using KV store. Those options can be changed as needed without the need to configure, compile or migrate the policies. And most of these tasks of configuring these options are within Consul (Node, Service & KV store) and it is outside the APIM team (respective services team).

Developer plug-in and Ephemeral Form Factor

This strategy also eliminates the dependencies of generating or defining multiple proxies rather than a single proxy per app or one per technology domain as needed. This will also decrease the effort needed by the CI/CD team since the API management team can define the Ephemeral container image with all predefined policy templates for enforcement and to facilitate backend service integration. And the same image can be rolled across various regions as separate releases provided, we segregate the runtime environments separately as needed. This is where a Developer Plug-in could come handy and ease the overall API Lifecycle.

For more details on: