1. Introduction: Why API Versioning Matters

In the modern landscape of distributed systems and microservices, APIs have become the fundamental connective tissue that binds together applications, services, and entire organizations. Every mobile app, single-page application, IoT device, and third-party integration relies on well-defined API contracts to exchange data and trigger business logic. When these APIs evolve — as they inevitably must — the question of how to manage change without breaking existing consumers becomes one of the most critical architectural decisions a senior engineer or platform team will face.

API versioning is not merely a technical concern; it is a business strategy. Consider a scenario where a fintech company serves 200 partner integrations through its payment processing API. A breaking change to the response schema of the transaction endpoint could cascade failures across all 200 partners, potentially causing millions of dollars in lost revenue and irreparable damage to business relationships. Effective versioning ensures that existing consumers continue to operate seamlessly while new consumers can adopt improved API designs at their own pace.

The challenges of API versioning extend far beyond simply adding a version number to a URL path. Teams must contend with backward compatibility, deprecation timelines, schema evolution, client upgrade coordination, documentation synchronization, and monitoring of version adoption rates. At the same time, an API gateway must handle the routing, authentication, rate limiting, and transformation logic for every version of every API simultaneously. This creates a combinatorial explosion of complexity that requires careful, systematic design.

This guide provides a comprehensive, senior-level treatment of both API versioning strategies and API gateway management. We will explore the trade-offs between different versioning approaches, build a complete gateway architecture in C#, implement rate limiting algorithms, design circuit breaker patterns, and establish a full API lifecycle management process. By the end of this article, you will have the knowledge and code patterns necessary to design and implement a production-grade API versioning and gateway management system that can scale to serve millions of requests across dozens of API versions.

The Cost of Poor Versioning

Organizations that neglect proper versioning often find themselves in a state of "API debt" — a situation where the cost of making changes far exceeds the value of the changes themselves. Common symptoms include fragile deployment processes where a single API change requires coordinated updates across dozens of consumer teams, an inability to sunset legacy API versions because critical business processes depend on them, and a growing backlog of API change requests that cannot be fulfilled because the risk of breaking existing integrations is too high.

According to industry surveys, the average enterprise maintains between 15 and 30 active API versions simultaneously. Without proper tooling, automation, and governance, managing this many versions becomes an operational nightmare. Teams spend more time on version coordination than on building new features, and the quality of the developer experience degrades to the point where partner integrations take months instead of days.

The Role of the API Gateway

An API gateway serves as the single entry point for all API traffic, providing a centralized layer where cross-cutting concerns such as authentication, rate limiting, logging, and request transformation can be implemented consistently. When combined with effective versioning, the gateway becomes the orchestration layer that routes versioned requests to the appropriate backend services, applies version-specific transformations, and enforces deprecation policies.

The gateway pattern also enables a clean separation between external API contracts and internal service interfaces. Backend services can evolve their internal APIs independently, while the gateway manages the public-facing contracts and ensures backward compatibility. This separation is essential in large-scale systems where hundreds of internal services communicate with each other through various protocols and data formats.

Who This Guide Is For

This guide is designed for senior engineers, staff engineers, and platform architects who are responsible for designing or managing API platforms. It assumes familiarity with REST API design, C# programming, and distributed systems concepts. The code examples are written in C# and ASP.NET Core, but the architectural patterns and strategies are applicable to any technology stack.

2. API Versioning Strategies

Choosing the right versioning strategy is one of the most consequential decisions in API design. Each approach has distinct trade-offs in terms of discoverability, cacheability, routing simplicity, and client implementation effort. The four primary strategies — URL path versioning, query string versioning, header-based versioning, and content-type versioning — each serve different architectural contexts and organizational preferences. Understanding these strategies deeply, including their implementation details and operational implications, is essential for making an informed choice.

2.1 URL Path Versioning

URL path versioning is the most widely adopted approach, used by major platforms including Twitter, GitHub, Google, and Stripe. The version number is embedded directly in the URL path, making it immediately visible, easily cacheable, and straightforward to route through load balancers and gateways. The canonical format is /api/v1/resources or /api/v2/resources, though some organizations use a more granular approach with version segments per resource.

The primary advantage of URL path versioning is its explicit nature. Developers can see the version in every API call, making it trivial to understand which version they are using. This explicitness also simplifies gateway routing rules, as the version can be extracted from the URL path with a simple regex pattern. Browser-based testing is effortless — you can paste a versioned URL directly into a browser or Postman without configuring custom headers.

However, URL path versioning has notable downsides. From a REST purity perspective, the version number is not a resource identifier — it is metadata about the representation. This means the same logical resource lives at multiple URLs, which can confuse caching layers and violates the principle that a resource should have a single, canonical URI. Additionally, URL path versioning can lead to URL proliferation as versions accumulate, making API documentation and discovery more complex.

Here is a C# implementation of URL path versioning using ASP.NET Core's built-in support:

C#
// URL Path Versioning Configuration in Program.cs
builder.Services.AddApiVersioning(options =>
{
    options.DefaultApiVersion = new ApiVersion(1, 0);
    options.AssumeDefaultVersionWhenUnspecified = true;
    options.ReportApiVersions = true;
    options.ApiVersionReader = new UrlSegmentApiVersionReader();
})
.AddApiExplorer(options =>
{
    options.GroupNameFormat = "'v'VVV";
    options.SubstituteApiVersionInUrl = true;
});

// Versioned Controller Implementation
[ApiController]
[Route("api/v{version:apiVersion}/orders")]
[ApiVersion("1.0")]
public class OrdersV1Controller : ControllerBase
{
    private readonly IOrderServiceV1 _orderService;

    public OrdersV1Controller(IOrderServiceV1 orderService)
    {
        _orderService = orderService;
    }

    [HttpGet("{id}")]
    public async Task<ActionResult<OrderV1Response>> GetOrder(Guid id)
    {
        var order = await _orderService.GetByIdAsync(id);
        if (order == null) return NotFound();

        return Ok(new OrderV1Response
        {
            OrderId = order.Id,
            CustomerName = order.CustomerName,
            TotalAmount = order.TotalAmount,
            Status = order.Status.ToString(),
            CreatedAt = order.CreatedAt
        });
    }

    [HttpPost]
    public async Task<ActionResult<OrderV1Response>> CreateOrder(
        [FromBody] CreateOrderV1Request request)
    {
        var order = await _orderService.CreateAsync(new OrderDto
        {
            CustomerName = request.CustomerName,
            Items = request.Items.Select(i => new OrderItemDto
            {
                ProductId = i.ProductId,
                Quantity = i.Quantity
            }).ToList()
        });

        return CreatedAtAction(nameof(GetOrder),
            new { id = order.Id, version = "1.0" },
            MapToV1Response(order));
    }
}

[ApiController]
[Route("api/v{version:apiVersion}/orders")]
[ApiVersion("2.0")]
public class OrdersV2Controller : ControllerBase
{
    private readonly IOrderServiceV2 _orderService;

    public OrdersV2Controller(IOrderServiceV2 orderService)
    {
        _orderService = orderService;
    }

    [HttpGet("{id}")]
    public async Task<ActionResult<OrderV2Response>> GetOrder(Guid id)
    {
        var order = await _orderService.GetByIdAsync(id);
        if (order == null) return NotFound();

        return Ok(new OrderV2Response
        {
            Id = order.Id,
            Customer = new CustomerInfo
            {
                Id = order.CustomerId,
                Name = order.CustomerName,
                Email = order.CustomerEmail
            },
            LineItems = order.Items.Select(i => new LineItemResponse
            {
                ProductId = i.ProductId,
                ProductName = i.ProductName,
                UnitPrice = i.UnitPrice,
                Quantity = i.Quantity,
                Subtotal = i.UnitPrice * i.Quantity
            }).ToList(),
            Currency = order.Currency,
            TotalAmount = order.TotalAmount,
            Metadata = order.Metadata,
            CreatedAt = order.CreatedAt,
            UpdatedAt = order.UpdatedAt
        });
    }
}

2.2 Query String Versioning

Query string versioning places the version identifier as a query parameter, such as /api/orders?version=2. This approach keeps the base URL clean and resource-focused while still providing explicit version control. It is commonly used by APIs that prioritize URL simplicity and want to avoid URL path pollution.

The main advantage is that the resource URI remains consistent across versions — only the query parameter changes. This aligns better with REST principles since the version is treated as a parameter rather than a path segment. Query string versioning also integrates well with API testing tools, as the version can be easily toggled in query parameters.

The primary disadvantage is that query string parameters are often ignored by HTTP caching infrastructure. CDNs and reverse proxies may not distinguish between /api/orders?version=1 and /api/orders?version=2, leading to incorrect cache hits. This can be mitigated with proper cache key configuration, but it adds operational complexity.

C#
// Query String Versioning Configuration
builder.Services.AddApiVersioning(options =>
{
    options.DefaultApiVersion = new ApiVersion(1, 0);
    options.AssumeDefaultVersionWhenUnspecified = true;
    options.ApiVersionReader = new QueryStringApiVersionReader("api-version");
})
.AddApiExplorer(options =>
{
    options.GroupNameFormat = "'v'VVV";
});

// Custom middleware for query string version extraction
public class QueryStringVersionMiddleware
{
    private readonly RequestDelegate _next;
    private static readonly Dictionary<string, ApiVersion> VersionMap =
        new(StringComparer.OrdinalIgnoreCase)
    {
        ["1"] = new ApiVersion(1, 0),
        ["1.0"] = new ApiVersion(1, 0),
        ["2"] = new ApiVersion(2, 0),
        ["2.0"] = new ApiVersion(2, 0),
        ["3"] = new ApiVersion(3, 0),
        ["3.0"] = new ApiVersion(3, 0)
    };

    public QueryStringVersionMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        if (context.Request.Query.TryGetValue("api-version", out var versionValue))
        {
            if (VersionMap.TryGetValue(versionValue, out var version))
            {
                context.SetRequestedApiVersion(version);
            }
            else
            {
                context.Response.StatusCode = 400;
                await context.Response.WriteAsJsonAsync(new
                {
                    error = "Invalid API version",
                    supportedVersions = VersionMap.Keys.ToList()
                });
                return;
            }
        }
        await _next(context);
    }
}

2.3 Header-Based Versioning

Header-based versioning uses custom or standard HTTP headers to specify the desired API version. The most common approach uses a custom header like X-API-Version: 2, while a more standards-compliant approach uses the Accept header with a vendor media type such as Accept: application/vnd.myapi.v2+json. This strategy keeps URLs completely version-free, which is the purest REST approach.

The Accept header approach, sometimes called content negotiation versioning, is particularly elegant because it leverages existing HTTP content negotiation mechanisms. The version becomes part of the media type, and the server responds with the appropriate representation. This approach is favored by APIs that serve multiple representation formats (JSON, XML, Protocol Buffers) alongside multiple versions.

The main drawback of header-based versioning is reduced discoverability. Developers cannot see the version in URLs shared via chat, email, or documentation. Testing requires configuring custom headers, which adds friction during development. Additionally, some proxy servers and CDNs may strip custom headers, potentially causing version resolution failures in production.

C#
// Header-Based Versioning with Custom Header
builder.Services.AddApiVersioning(options =>
{
    options.DefaultApiVersion = new ApiVersion(1, 0);
    options.AssumeDefaultVersionWhenUnspecified = true;
    options.ApiVersionReader = new HeaderApiVersionReader("X-API-Version");
})
.AddApiExplorer(options =>
{
    options.GroupNameFormat = "'v'VVV";
});

// Accept Header Versioning with Vendor Media Type
public class AcceptHeaderVersionPolicy : IApiVersionReader
{
    public ApiVersion Read(HttpRequest request, ApiVersionModel model)
    {
        var acceptHeader = request.Headers.Accept.ToString();
        var match = Regex.Match(acceptHeader,
            @"application/vnd\.(\w+)\.v(\d+)(?:\.(\d+))?\+");

        if (match.Success)
        {
            var major = int.Parse(match.Groups[2].Value);
            var minor = match.Groups[3.Success ? 3 : 0].Value;
            return string.IsNullOrEmpty(minor)
                ? new ApiVersion(major, 0)
                : new ApiVersion(major, int.Parse(minor));
        }

        return ApiVersion.Default;
    }
}

// Comprehensive versioned controller using Accept header
[ApiController]
[Route("api/products")]
[ApiVersion("1.0")]
[ApiVersion("2.0")]
[ApiVersion("3.0")]
public class ProductsController : ControllerBase
{
    private readonly IProductService _service;
    private readonly IMapper _mapper;

    public ProductsController(IProductService service, IMapper mapper)
    {
        _service = service;
        _mapper = mapper;
    }

    [HttpGet("{id}")]
    public async Task<IActionResult> GetProduct(Guid id)
    {
        var product = await _service.GetByIdAsync(id);
        if (product == null) return NotFound();

        var apiVersion = HttpContext.GetRequestedApiVersion();

        return apiVersion.MajorVersion switch
        {
            1 => Ok(_mapper.Map<ProductV1Dto>(product)),
            2 => Ok(_mapper.Map<ProductV2Dto>(product)),
            3 => Ok(_mapper.Map<ProductV3Dto>(product)),
            _ => Ok(_mapper.Map<ProductV1Dto>(product))
        };
    }
}

2.4 Strategy Comparison

2.5 Hybrid and Multi-Reader Approaches

In practice, many production systems use a hybrid approach that supports multiple version readers simultaneously. This allows different consumers to use the versioning mechanism that best suits their technology stack. For example, browser-based clients might use URL path versioning for simplicity, while server-to-server integrations might prefer header-based versioning for cleaner URLs.

ASP.NET Core supports this through composite API version readers that check multiple sources in a configurable order. The gateway can also implement version translation, accepting one versioning format from external clients and converting it to another format for internal routing. This flexibility is essential when serving a diverse ecosystem of consumers with different capabilities and preferences.

When implementing a hybrid approach, it is critical to establish clear precedence rules. If a request includes version information in multiple places (e.g., both URL path and query string), the system must deterministically choose one. Ambiguity in version resolution leads to subtle bugs that are extremely difficult to diagnose in production, especially when different gateway nodes apply different precedence rules due to configuration drift.

3. Version Negotiation and Client Communication

Version negotiation is the process by which a client and server agree on which API version to use for a given request. Effective negotiation requires clear communication of supported versions, graceful handling of version mismatches, and well-defined policies for deprecation and sunset. Without robust negotiation mechanisms, clients will encounter confusing error messages, and the migration process from old versions to new ones will be chaotic and error-prone.

3.1 Version Discovery

Clients need a reliable way to discover which API versions are available and which ones are recommended. The most common approach is to expose version metadata through a dedicated discovery endpoint or through HTTP response headers on every API response. The API-Version response header can include the current version, while a Supported-Versions header can list all available versions along with their status (current, deprecated, sunset).

A well-designed version discovery endpoint provides comprehensive information about all available versions, their status, sunset dates, and migration guides. This endpoint serves as the single source of truth for version information and should be treated as a first-class API resource. It should itself be versioned (typically using the earliest supported version) to ensure it remains accessible even as other versions evolve.

3.2 Deprecation Communication

When a version is deprecated, clients must be informed through multiple channels simultaneously. At the HTTP protocol level, deprecated versions should return a Deprecation header and a Sunset header indicating the date after which the version will no longer be available. These headers should be present on every response from the deprecated version, ensuring that developers see deprecation notices during development, testing, and production monitoring.

C#
// Deprecation Middleware that adds headers to deprecated API versions
public class ApiDeprecationMiddleware
{
    private readonly RequestDelegate _next;
    private readonly ILogger<ApiDeprecationMiddleware> _logger;

    private static readonly Dictionary<string, DeprecationInfo> DeprecationMap = new()
    {
        ["1.0"] = new DeprecationInfo
        {
            DeprecatedDate = new DateTime(2025, 6, 1),
            SunsetDate = new DateTime(2026, 6, 1),
            ReplacementVersion = "2.0",
            MigrationGuideUrl = "https://docs.example.com/migration/v1-to-v2"
        },
        ["2.0"] = new DeprecationInfo
        {
            DeprecatedDate = new DateTime(2026, 1, 1),
            SunsetDate = new DateTime(2027, 1, 1),
            ReplacementVersion = "3.0",
            MigrationGuideUrl = "https://docs.example.com/migration/v2-to-v3"
        }
    };

    public ApiDeprecationMiddleware(
        RequestDelegate next,
        ILogger<ApiDeprecationMiddleware> logger)
    {
        _next = next;
        _logger = logger;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        var apiVersion = context.GetRequestedApiVersion()?.ToString();

        if (apiVersion != null &&
            DeprecationMap.TryGetValue(apiVersion, out var info))
        {
            context.Response.OnStarting(() =>
            {
                context.Response.Headers["Deprecation"] =
                    info.DeprecatedDate.ToString("yyyy-MM-dd");
                context.Response.Headers["Sunset"] =
                    info.SunsetDate.ToString("ddd, dd MMM yyyy HH:mm:ss") + " GMT";
                context.Response.Headers["Link"] =
                    $"; rel=""successor-version""; uri=""/api/v{info.ReplacementVersion}""";
                context.Response.Headers["X-Deprecation-Notice"] =
                    $"Version {apiVersion} is deprecated. " +
                    $"Please migrate to v{info.ReplacementVersion} " +
                    $"before {info.SunsetDate:yyyy-MM-dd}. " +
                    $"Migration guide: {info.MigrationGuideUrl}";

                _logger.LogWarning(
                    "Request to deprecated API version {Version} from {ClientIp}",
                    apiVersion, context.Connection.RemoteIpAddress);

                return Task.CompletedTask;
            });
        }

        await _next(context);
    }

    public class DeprecationInfo
    {
        public DateTime DeprecatedDate { get; set; }
        public DateTime SunsetDate { get; set; }
        public string ReplacementVersion { get; set; } = string.Empty;
        public string MigrationGuideUrl { get; set; } = string.Empty;
    }
}

3.3 Client Migration Strategies

Successful version migration requires a structured approach that minimizes disruption to existing clients while providing clear incentives and tools for upgrading. The most effective strategy combines automated notification, migration tooling, and a well-defined deprecation timeline. Organizations should aim for a minimum 12-month deprecation window for major versions and 6 months for minor versions, though the exact timeline depends on the client base and the severity of the changes.

Automated notification systems should contact API consumers through multiple channels: email notifications at regular intervals (30, 60, 90, and 120 days before sunset), in-app banners for developer portal users, and webhook notifications for organizations that register for deprecation alerts. Each notification should include the specific version being deprecated, the replacement version, a migration guide, and contact information for support.

Migration tooling significantly reduces the friction of version upgrades. This includes automated code generators that can transform client SDK code from one version to another, compatibility matrices that clearly document what changed between versions, and sandbox environments where clients can test against the new version without affecting production traffic. The investment in migration tooling pays for itself many times over in reduced support burden and faster adoption of new versions.

3.4 Version Compatibility Matrix

4. System Architecture Overview

The complete API versioning and gateway management system comprises several interconnected components that work together to provide a cohesive platform for managing the entire lifecycle of versioned APIs. This section presents the high-level architecture, explains the responsibilities of each component, and shows how they interact to handle a typical API request from a client through to the backend service response.

At its core, the architecture follows a layered design where the API gateway serves as the primary entry point for all external traffic. Behind the gateway, a version routing layer determines which backend service instance should handle each request based on the requested API version. A cross-cutting concerns layer provides authentication, authorization, rate limiting, logging, and monitoring across all versions. An API management plane provides the administrative interface for version lifecycle management, documentation, and analytics.

4.1 Component Responsibilities

The CDN and Edge Layer handles static content delivery, geographic routing, and initial DDoS protection. API responses can be cached at the edge for public, read-heavy endpoints, with cache keys that include the API version to prevent cross-version cache pollution. The WAF applies security rules that can be version-aware — for example, blocking deprecated version traffic from untrusted sources while allowing it from known partners.

The API Gateway Layer is the heart of the system. Each gateway node is a stateless, horizontally scalable service that handles the full request lifecycle: TLS termination, authentication, version resolution, rate limiting, request validation, routing, response transformation, and logging. The gateway nodes are identical and can be scaled independently based on traffic patterns. The version router component within the gateway is responsible for resolving the requested version, validating it against the list of supported versions, and selecting the appropriate backend service instance.

The Service Layer contains the versioned backend service instances. Each version of an API may be served by a different service instance, allowing independent deployment, scaling, and evolution of each version. In some architectures, multiple versions may be served by the same service instance with version-specific code paths, but dedicated instances per version provide stronger isolation and simpler deployment processes.

The Data Layer includes databases, caches, and message queues that support the backend services. Version-aware caching is critical here — the cache key must include the API version to ensure that responses from one version are not served to clients requesting a different version. Schema versioning in the database layer ensures that data can be read and written correctly by services running different API versions.

The Management Plane provides the administrative interface for the entire platform. This includes the admin portal for version lifecycle management, the documentation portal for API reference materials, the monitoring dashboard for real-time visibility into API health, and the analytics engine for tracking version adoption rates, performance metrics, and usage patterns.

4.2 Request Flow

A typical request through the system follows a well-defined path with clear checkpoints at each layer. Let us trace a request for GET /api/v2/orders/123 from a mobile application. First, the request travels through the CDN, which checks for a cached response with a key that includes the version identifier. If a cache hit occurs, the cached response is returned immediately. If the cache misses, the request proceeds to the WAF, which applies security rules such as IP filtering, request size limits, and bot detection.

Next, the load balancer selects an available gateway node based on health checks and connection counts. The gateway node terminates TLS, extracts the API version from the URL path (in this case, v2), and begins the authentication flow. The authenticator validates the bearer token, extracts the client identity and permissions, and attaches this information to the request context. The rate limiter then checks whether the client has exceeded their rate limit for the v2 API, applying appropriate throttling if necessary.

The version router resolves the backend service endpoint for v2 of the orders API. If the v2 API has been deprecated, the deprecation middleware adds the appropriate headers to the response. The request transformer may modify the request to match the expected format of the v2 backend service. Finally, the transformed request is forwarded to the backend service instance, and the response flows back through the gateway with any necessary response transformations applied.

4.3 Deployment Topology

5. API Gateway Core Architecture

The API gateway is the cornerstone of the entire versioning and management system. It must handle millions of requests per day with sub-millisecond added latency, support multiple versioning strategies simultaneously, and provide an extensible framework for cross-cutting concerns. This section dives deep into the gateway's internal architecture, request routing mechanisms, and load balancing strategies.

5.1 Gateway Pipeline

The gateway processes each request through a pipeline of middleware components, each responsible for a specific concern. This pipeline is similar to the ASP.NET Core middleware pipeline but is designed specifically for gateway-level operations. The order of middleware execution is critical — for example, authentication must happen before authorization, and rate limiting should occur early in the pipeline to reject excessive traffic before consuming resources on downstream components.

C#
// Complete API Gateway Configuration
public static class GatewayConfiguration
{
    public static IServiceCollection AddApiGateway(
        this IServiceCollection services,
        IConfiguration configuration)
    {
        services.Configure<GatewayOptions>(
            configuration.GetSection("Gateway"));

        // Core gateway services
        services.AddSingleton<IVersionResolver, VersionResolver>();
        services.AddSingleton<IVersionRegistry, VersionRegistry>();
        services.AddSingleton<IRouteTable, RouteTable>();
        services.AddSingleton<ILoadBalancer, RoundRobinLoadBalancer>();

        // Middleware services
        services.AddScoped<IAuthenticationHandler, JwtAuthHandler>();
        services.AddScoped<IAuthorizationHandler, ScopeAuthHandler>();
        services.AddSingleton<IRateLimiter, SlidingWindowRateLimiter>();
        services.AddSingleton<ICircuitBreaker, CircuitBreaker>();
        services.AddScoped<IRequestTransformer, RequestTransformer>();
        services.AddScoped<IResponseTransformer, ResponseTransformer>();

        // Health and monitoring
        services.AddSingleton<IGatewayMetrics, GatewayMetrics>();
        services.AddHealthChecks()
            .AddCheck<GatewayHealthCheck>("gateway")
            .AddRedis(configuration["Redis:ConnectionString"], name: "redis");

        return services;
    }

    public static IApplicationBuilder UseApiGateway(
        this IApplicationBuilder app)
    {
        // Pipeline order matters
        app.UseMiddleware<RequestLoggingMiddleware>();
        app.UseMiddleware<VersionResolutionMiddleware>();
        app.UseMiddleware<ApiDeprecationMiddleware>();
        app.UseMiddleware<AuthenticationMiddleware>();
        app.UseMiddleware<AuthorizationMiddleware>();
        app.UseMiddleware<RateLimitingMiddleware>();
        app.UseMiddleware<RequestTransformationMiddleware>();
        app.UseMiddleware<CircuitBreakerMiddleware>();
        app.UseMiddleware<RoutingMiddleware>();
        app.UseMiddleware<ResponseTransformationMiddleware>();
        app.UseMiddleware<MetricsCollectionMiddleware>();

        return app;
    }
}

// Route table that maps versioned paths to backend services
public class RouteTable : IRouteTable
{
    private readonly ConcurrentDictionary<string, RouteEntry> _routes = new();
    private readonly ILogger<RouteTable> _logger;

    public RouteTable(ILogger<RouteTable> logger)
    {
        _logger = logger;
    }

    public void RegisterRoute(string version, string pathPattern,
        ServiceEndpoint endpoint)
    {
        var key = $"{version}:{pathPattern}";
        _routes[key] = new RouteEntry
        {
            Version = version,
            PathPattern = pathPattern,
            Endpoint = endpoint,
            RegisteredAt = DateTime.UtcNow
        };
        _logger.LogInformation(
            "Registered route: v{Version} {Pattern} -> {Endpoint}",
            version, pathPattern, endpoint.Url);
    }

    public RouteEntry? Resolve(string requestedVersion, string path)
    {
        var candidates = _routes.Values
            .Where(r => r.Version == requestedVersion)
            .Where(r => MatchPattern(r.PathPattern, path))
            .OrderByDescending(r => r.Priority)
            .ToList();

        return candidates.FirstOrDefault();
    }

    private bool MatchPattern(string pattern, string path)
    {
        var patternParts = pattern.Split('/');
        var pathParts = path.Split('/');
        if (patternParts.Length != pathParts.Length) return false;
        return patternParts.Zip(pathParts).All((pair) =>
            pair.First.StartsWith('{') || pair.First == pair.Second);
    }
}

public record RouteEntry
{
    public string Version { get; init; } = string.Empty;
    public string PathPattern { get; init; } = string.Empty;
    public ServiceEndpoint Endpoint { get; init; } = null!;
    public int Priority { get; init; }
    public DateTime RegisteredAt { get; init; }
}

public record ServiceEndpoint
{
    public string Url { get; init; } = string.Empty;
    public int Port { get; init; }
    public string Protocol { get; init; } = "HTTP";
    public TimeSpan Timeout { get; init; } = TimeSpan.FromSeconds(30);
    public string[] HealthCheckPaths { get; init; } = Array.Empty<string>();
}

5.2 Version Resolution

The version resolver is responsible for determining the API version requested by the client. It supports multiple extraction strategies based on the configured versioning approach and applies fallback logic when the version is not explicitly specified. The resolver also validates that the requested version is currently available and returns appropriate error responses for unsupported or sunset versions.

5.3 Load Balancing

The load balancer distributes requests across backend service instances for a given API version. Unlike traditional load balancing, version-aware load balancing must consider the different resource requirements and traffic patterns of each version. For example, a deprecated v1 API may have fewer backend instances than a popular v2 API, and the load balancer must respect these capacity differences.

C#
// Version-Aware Load Balancer Implementation
public class VersionAwareLoadBalancer : ILoadBalancer
{
    private readonly ConcurrentDictionary<string, InstancePool> _pools = new();
    private readonly ILogger<VersionAwareLoadBalancer> _logger;

    public VersionAwareLoadBalancer(
        ILogger<VersionAwareLoadBalancer> logger)
    {
        _logger = logger;
    }

    public void RegisterInstance(string version, ServiceInstance instance)
    {
        var pool = _pools.GetOrAdd(version, _ => new InstancePool(version));
        pool.AddInstance(instance);
        _logger.LogInformation(
            "Registered instance for v{Version}: {Instance}",
            version, instance.Address);
    }

    public async Task<ServiceInstance?> GetInstanceAsync(
        string version, RequestContext context)
    {
        if (!_pools.TryGetValue(version, out var pool))
        {
            _logger.LogWarning(
                "No instances available for version {Version}", version);
            return null;
        }

        var healthyInstances = pool.GetHealthyInstances().ToList();
        if (!healthyInstances.Any())
        {
            _logger.LogError(
                "All instances for version {Version} are unhealthy", version);
            return null;
        }

        // Weighted least-connections algorithm
        var selected = healthyInstances
            .Select(i => new
            {
                Instance = i,
                Score = i.ActiveConnections / (double)i.Weight
            })
            .OrderBy(x => x.Score)
            .First()
            .Instance;

        Interlocked.Increment(ref selected.ActiveConnections);
        return selected;
    }

    public void ReleaseInstance(string version, ServiceInstance instance,
        TimeSpan responseTime, bool success)
    {
        Interlocked.Decrement(ref instance.ActiveConnections);
        instance.TotalRequests++;
        if (success) instance.SuccessfulRequests++;
        instance.UpdateResponseTime(responseTime);

        if (!success)
        {
            instance.ConsecutiveFailures++;
            if (instance.ConsecutiveFailures >= instance.FailureThreshold)
            {
                instance.State = CircuitState.Open;
                _logger.LogWarning(
                    "Circuit opened for instance {Instance}", instance.Address);
            }
        }
        else
        {
            instance.ConsecutiveFailures = 0;
            if (instance.State == CircuitState.HalfOpen)
                instance.State = CircuitState.Closed;
        }
    }
}

public class InstancePool
{
    private readonly string _version;
    private readonly ConcurrentBag<ServiceInstance> _instances = new();
    private readonly ReaderWriterLockSlim _lock = new();

    public InstancePool(string version) { _version = version; }

    public void AddInstance(ServiceInstance instance)
    {
        _lock.EnterWriteLock();
        try { _instances.Add(instance); }
        finally { _lock.ExitWriteLock(); }
    }

    public IEnumerable<ServiceInstance> GetHealthyInstances()
    {
        _lock.EnterReadLock();
        try
        {
            return _instances
                .Where(i => i.State != CircuitState.Open)
                .ToList();
        }
        finally { _lock.ExitReadLock(); }
    }
}

public class ServiceInstance
{
    public string Address { get; set; } = string.Empty;
    public int Port { get; set; }
    public int Weight { get; set; } = 1;
    public int ActiveConnections;
    public int TotalRequests;
    public long SuccessfulRequests;
    public int ConsecutiveFailures;
    public int FailureThreshold = 5;
    public CircuitState State = CircuitState.Closed;
    public double AvgResponseTimeMs;

    public void UpdateResponseTime(TimeSpan responseTime)
    {
        AvgResponseTimeMs = (AvgResponseTimeMs * 0.9) +
            (responseTime.TotalMilliseconds * 0.1);
    }
}

public enum CircuitState { Closed, HalfOpen, Open }

5.4 Gateway Performance Requirements

6. Rate Limiting and Throttling

Rate limiting is a critical gateway function that protects backend services from traffic spikes, abuse, and denial-of-service attacks while ensuring fair resource allocation across API consumers. A well-designed rate limiting system must support multiple algorithms, operate at the gateway level with minimal latency, handle distributed deployments correctly, and provide clear feedback to clients about their rate limit status.

6.1 Token Bucket Algorithm

The token bucket algorithm is one of the most widely used rate limiting mechanisms due to its simplicity, fairness, and ability to handle bursty traffic gracefully. The algorithm maintains a bucket that holds tokens, with a maximum capacity (burst size) and a refill rate (sustained rate). Each incoming request consumes one token; if the bucket is empty, the request is rejected or queued. Tokens are added to the bucket at the configured rate, up to the maximum capacity.

C#
// Token Bucket Rate Limiter with Distributed Support
public class TokenBucketRateLimiter : IRateLimiter
{
    private readonly IDatabase _redis;
    private readonly ILogger<TokenBucketRateLimiter> _logger;

    private const string LUA_SCRIPT = @"
        local key = KEYS[1]
        local capacity = tonumber(ARGV[1])
        local refill_rate = tonumber(ARGV[2])
        local now = tonumber(ARGV[3])
        local requested = tonumber(ARGV[4])

        local bucket = redis.call('HMGET', key, 'tokens', 'last_refill')
        local tokens = tonumber(bucket[1])
        local last_refill = tonumber(bucket[2])

        if tokens == nil then
            tokens = capacity
            last_refill = now
        end

        local elapsed = math.max(0, now - last_refill)
        local new_tokens = math.min(
            capacity, tokens + (elapsed * refill_rate))

        if new_tokens >= requested then
            new_tokens = new_tokens - requested
            redis.call('HMSET', key, 'tokens', new_tokens, 'last_refill', now)
            redis.call('EXPIRE', key, math.ceil(capacity / refill_rate) * 2)
            return {1, new_tokens, 0}
        else
            local wait_time = (requested - new_tokens) / refill_rate
            return {0, 0, math.ceil(wait_time)}
        end
    ";

    public TokenBucketRateLimiter(IConnectionMultiplexer redis,
        ILogger<TokenBucketRateLimiter> logger)
    {
        _redis = redis.GetDatabase();
        _logger = logger;
    }

    public async Task<RateLimitResult> CheckAsync(
        string clientId, RateLimitPolicy policy)
    {
        var key = $"ratelimit:{policy.Name}:{clientId}";
        var now = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds();

        var result = await _redis.ScriptEvaluateAsync(
            LUA_SCRIPT,
            new RedisKey[] { key },
            new RedisValue[]
            {
                policy.BurstCapacity,
                policy.RefillRatePerSecond,
                now,
                1
            });

        var values = (RedisValue[])result;
        var allowed = (long)values[0] == 1;
        var remaining = (long)values[1];
        var retryAfter = (long)values[2];

        return new RateLimitResult
        {
            IsAllowed = allowed,
            RemainingTokens = remaining,
            RetryAfter = retryAfter,
            Limit = policy.BurstCapacity,
            ResetAt = DateTime.UtcNow.AddSeconds(
                policy.BurstCapacity / policy.RefillRatePerSecond)
        };
    }
}

public class RateLimitPolicy
{
    public string Name { get; set; } = string.Empty;
    public int BurstCapacity { get; set; }
    public double RefillRatePerSecond { get; set; }
    public TimeSpan Window { get; set; }
    public RateLimitScope Scope { get; set; }
}

public enum RateLimitScope
{
    PerClient,
    PerClientPerEndpoint,
    PerClientPerVersion,
    Global
}

public class RateLimitResult
{
    public bool IsAllowed { get; set; }
    public long RemainingTokens { get; set; }
    public long RetryAfter { get; set; }
    public int Limit { get; set; }
    public DateTime ResetAt { get; set; }
}

6.2 Sliding Window Algorithm

The sliding window algorithm provides more accurate rate limiting than fixed-window approaches by considering request activity across a rolling time window. Unlike the fixed-window algorithm, which resets counters at predictable intervals creating a burst-at-the-boundary problem, the sliding window continuously evaluates the request rate based on the most recent N seconds of activity. This produces smoother limiting behavior and prevents the double-burst issue.

The sliding window can be implemented using two sub-approaches: the sliding window log and the sliding window counter. The sliding window log stores timestamps of every request within the window, providing perfect accuracy at the cost of higher memory usage. The sliding window counter uses weighted averages between the current and previous windows to approximate the true rate, trading a small amount of accuracy for significantly lower memory requirements. For most production systems, the counter approach provides sufficient accuracy with much better performance characteristics.

6.3 Multi-Tier Rate Limiting

Production API platforms typically implement rate limiting at multiple tiers to provide comprehensive protection. The first tier operates at the gateway level, applying global rate limits based on client identity. The second tier operates per-endpoint, ensuring that resource-intensive operations have tighter limits than lightweight operations. The third tier operates per-version, allowing different rate limit policies for different API versions.

6.4 Rate Limit Response Headers

Every API response should include rate limit headers to inform clients of their current status and help them implement adaptive request strategies. The standard headers include X-RateLimit-Limit (maximum requests allowed), X-RateLimit-Remaining (requests remaining in the current window), and X-RateLimit-Reset (UTC epoch time when the window resets). When a request is rejected, the response should include Retry-After (seconds until the client should retry) and return HTTP status 429.

C#
// Rate Limiting Middleware with Response Headers
public class RateLimitingMiddleware
{
    private readonly RequestDelegate _next;
    private readonly IRateLimiter _rateLimiter;
    private readonly ILogger<RateLimitingMiddleware> _logger;

    public RateLimitingMiddleware(
        RequestDelegate next,
        IRateLimiter rateLimiter,
        ILogger<RateLimitingMiddleware> logger)
    {
        _next = next;
        _rateLimiter = rateLimiter;
        _logger = logger;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        var clientId = context.User.FindFirst("client_id")?.Value
            ?? context.Connection.RemoteIpAddress?.ToString()
            ?? "anonymous";

        var apiVersion = context.GetRequestedApiVersion()?.ToString()
            ?? "1.0";

        var policy = ResolvePolicy(clientId, apiVersion, context.Request.Path);
        var result = await _rateLimiter.CheckAsync(clientId, policy);

        context.Response.OnStarting(() =>
        {
            context.Response.Headers["X-RateLimit-Limit"] =
                result.Limit.ToString();
            context.Response.Headers["X-RateLimit-Remaining"] =
                result.RemainingTokens.ToString();
            context.Response.Headers["X-RateLimit-Reset"] =
                new DateTimeOffset(result.ResetAt).ToUnixTimeSeconds()
                    .ToString();
            return Task.CompletedTask;
        });

        if (!result.IsAllowed)
        {
            _logger.LogWarning(
                "Rate limit exceeded for client {ClientId} on version {Version}",
                clientId, apiVersion);

            context.Response.StatusCode = 429;
            context.Response.Headers["Retry-After"] =
                result.RetryAfter.ToString();

            await context.Response.WriteAsJsonAsync(new
            {
                error = "Rate limit exceeded",
                message = $"You have exceeded the rate limit of {result.Limit} requests. Please retry after {result.RetryAfter} seconds.",
                retryAfter = result.RetryAfter,
                documentation = "https://docs.example.com/rate-limits"
            });
            return;
        }

        await _next(context);
    }

    private RateLimitPolicy ResolvePolicy(
        string clientId, string apiVersion, PathString path)
    {
        if (path.StartsWithSegments("/api/search"))
        {
            return new RateLimitPolicy
            {
                Name = "search-endpoint",
                BurstCapacity = 10,
                RefillRatePerSecond = 2,
                Scope = RateLimitScope.PerClientPerEndpoint
            };
        }

        if (apiVersion.StartsWith("1"))
        {
            return new RateLimitPolicy
            {
                Name = "legacy-v1",
                BurstCapacity = 50,
                RefillRatePerSecond = 10,
                Scope = RateLimitScope.PerClientPerVersion
            };
        }

        return new RateLimitPolicy
        {
            Name = "default",
            BurstCapacity = 100,
            RefillRatePerSecond = 20,
            Scope = RateLimitScope.PerClient
        };
    }
}

7. Authentication and Authorization at the Gateway

Centralizing authentication and authorization at the API gateway eliminates the need for each backend service to implement its own identity verification logic, ensuring consistent security enforcement across all API versions and services. The gateway validates client identity, enforces scope-based access control, and attaches standardized identity claims to the request context before forwarding it to backend services.

7.1 JWT Token Validation

JSON Web Tokens (JWT) are the most common authentication mechanism for API gateways due to their stateless nature and broad ecosystem support. The gateway validates the JWT signature against a public key or shared secret, checks token expiration, and extracts identity claims. For multi-version APIs, the token may contain version-specific scopes that determine which API versions and endpoints the client can access.

C#
// Gateway Authentication Handler with Version-Aware Scopes
public class GatewayAuthenticationHandler
{
    private readonly TokenValidationParameters _validationParams;
    private readonly ITokenRevocationStore _revocationStore;
    private readonly ILogger<GatewayAuthenticationHandler> _logger;

    public GatewayAuthenticationHandler(
        IConfiguration configuration,
        ITokenRevocationStore revocationStore,
        ILogger<GatewayAuthenticationHandler> logger)
    {
        _revocationStore = revocationStore;
        _logger = logger;

        _validationParams = new TokenValidationParameters
        {
            ValidateIssuer = true,
            ValidIssuer = configuration["Auth:Issuer"],
            ValidateAudience = true,
            ValidAudience = configuration["Auth:Audience"],
            ValidateIssuerSigningKey = true,
            IssuerSigningKey = new SymmetricSecurityKey(
                Encoding.UTF8.GetBytes(configuration["Auth:SigningKey"]!)),
            ValidateLifetime = true,
            ClockSkew = TimeSpan.FromSeconds(30),
            RequireExpirationTime = true
        };
    }

    public async Task<AuthenticationResult> AuthenticateAsync(HttpContext context)
    {
        var token = ExtractBearerToken(context.Request);
        if (token == null)
            return AuthenticationResult.Fail("No bearer token provided");

        if (await _revocationStore.IsRevokedAsync(token))
        {
            _logger.LogWarning("Revoked token used from {Ip}",
                context.Connection.RemoteIpAddress);
            return AuthenticationResult.Fail("Token has been revoked");
        }

        var handler = new JwtSecurityTokenHandler();
        try
        {
            var principal = handler.ValidateToken(
                token, _validationParams, out var validatedToken);

            var jwtToken = (JwtSecurityToken)validatedToken;
            var identity = new ApiClientIdentity
            {
                ClientId = jwtToken.Claims.First(c => c.Type == "sub").Value,
                ClientName = jwtToken.Claims.FirstOrDefault(c => c.Type == "name")?.Value,
                Scopes = jwtToken.Claims
                    .Where(c => c.Type == "scope")
                    .Select(c => c.Value).ToList(),
                Roles = jwtToken.Claims
                    .Where(c => c.Type == "role")
                    .Select(c => c.Value).ToList(),
                TokenExpiration = jwtToken.ValidTo,
                RateLimitTier = jwtToken.Claims
                    .FirstOrDefault(c => c.Type == "rate_limit_tier")?.Value ?? "standard"
            };

            return AuthenticationResult.Success(identity);
        }
        catch (SecurityTokenExpiredException)
        {
            return AuthenticationResult.Fail("Token has expired");
        }
        catch (SecurityTokenException ex)
        {
            _logger.LogWarning("Token validation failed: {Error}", ex.Message);
            return AuthenticationResult.Fail("Invalid token");
        }
    }

    private string? ExtractBearerToken(HttpRequest request)
    {
        var authHeader = request.Headers.Authorization.ToString();
        if (string.IsNullOrEmpty(authHeader) ||
            !authHeader.StartsWith("Bearer ", StringComparison.OrdinalIgnoreCase))
            return null;
        return authHeader["Bearer ".Length..].Trim();
    }
}

// Authorization middleware that checks version-specific permissions
public class VersionAuthorizationMiddleware
{
    private readonly RequestDelegate _next;

    private static readonly Dictionary<string, HashSet<string>>
        VersionScopeRequirements = new()
    {
        ["1.0"] = new() { "api:read", "api:v1:access" },
        ["2.0"] = new() { "api:read", "api:v2:access" },
        ["3.0"] = new() { "api:read", "api:v3:access" }
    };

    public VersionAuthorizationMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        var identity = context.Items["ClientIdentity"] as ApiClientIdentity;
        if (identity == null)
        {
            context.Response.StatusCode = 401;
            await context.Response.WriteAsJsonAsync(new { error = "Authentication required" });
            return;
        }

        var version = context.GetRequestedApiVersion()?.ToString() ?? "1.0";

        if (VersionScopeRequirements.TryGetValue(version, out var requiredScopes))
        {
            var hasAllScopes = requiredScopes.All(scope => identity.Scopes.Contains(scope));
            if (!hasAllScopes)
            {
                var missingScopes = requiredScopes.Where(s => !identity.Scopes.Contains(s)).ToList();
                context.Response.StatusCode = 403;
                await context.Response.WriteAsJsonAsync(new
                {
                    error = "Insufficient permissions",
                    message = $"Missing required scopes: {string.Join(", ", missingScopes)}",
                    requiredScopes = requiredScopes,
                    currentScopes = identity.Scopes
                });
                return;
            }
        }

        context.Items["AuthorizedIdentity"] = identity;
        await _next(context);
    }
}

7.2 OAuth 2.0 Flows for API Gateways

The gateway supports multiple OAuth 2.0 flows depending on the client type. Service-to-service communication uses the client credentials flow. Single-page applications use the authorization code flow with PKCE. Mobile applications use the authorization code flow with PKCE and custom redirect URIs. The gateway validates tokens from all these flows using the same JWT validation pipeline, but the scope requirements may differ based on the flow type.

8. Request/Response Transformation

Request and response transformation at the gateway is essential for maintaining backward compatibility, normalizing data formats across versions, and adapting external API contracts to internal service interfaces. As APIs evolve, the gateway acts as an intelligent translation layer that can reshape data structures, rename fields, convert serialization formats, and enrich responses with additional data — all without requiring changes to backend services or client applications.

8.1 Schema Transformation Engine

The transformation engine maps between different API version schemas using a declarative configuration format. This avoids hardcoding transformation logic for every version pair and allows transformation rules to be updated independently of the service code. The engine supports field renaming, type conversion, nesting changes, default value injection, field removal, and conditional transformations based on request context.

C#
// Declarative Transformation Configuration
public class TransformationRule
{
    public string SourceVersion { get; set; } = string.Empty;
    public string TargetVersion { get; set; } = string.Empty;
    public string Endpoint { get; set; } = string.Empty;
    public List<FieldMapping> RequestMappings { get; set; } = new();
    public List<FieldMapping> ResponseMappings { get; set; } = new();
    public List<TransformAction> Actions { get; set; } = new();
}

public class FieldMapping
{
    public string SourcePath { get; set; } = string.Empty;
    public string TargetPath { get; set; } = string.Empty;
    public string? DefaultValue { get; set; }
    public Func<object?, object?>? Transform { get; set; }
    public bool Required { get; set; } = true;
}

// Transformation engine that processes rules
public class TransformationEngine
{
    private readonly List<TransformationRule> _rules;
    private readonly ILogger<TransformationEngine> _logger;

    public TransformationEngine(
        IEnumerable<TransformationRule> rules,
        ILogger<TransformationEngine> logger)
    {
        _rules = rules.ToList();
        _logger = logger;
    }

    public async Task<JsonElement> TransformRequestAsync(
        JsonElement request, string sourceVersion,
        string targetVersion, string endpoint)
    {
        var rule = _rules.FirstOrDefault(r =>
            r.SourceVersion == sourceVersion &&
            r.TargetVersion == targetVersion &&
            r.Endpoint == endpoint);

        if (rule == null) return request;

        var result = new Dictionary<string, object?>();

        foreach (var mapping in rule.RequestMappings)
        {
            var sourceValue = ExtractValue(request, mapping.SourcePath);
            if (sourceValue == null && mapping.Required)
                throw new TransformationException(
                    $"Required field {mapping.SourcePath} is missing");

            var transformedValue = mapping.Transform != null
                ? mapping.Transform(sourceValue)
                : sourceValue ?? mapping.DefaultValue;

            SetValue(result, mapping.TargetPath, transformedValue);
        }

        foreach (var action in rule.Actions.Where(a => a.Phase == TransformPhase.Request))
            await action.ExecuteAsync(result);

        return JsonSerializer.SerializeToElement(result);
    }

    public async Task<JsonElement> TransformResponseAsync(
        JsonElement response, string sourceVersion,
        string targetVersion, string endpoint)
    {
        var rule = _rules.FirstOrDefault(r =>
            r.SourceVersion == targetVersion &&
            r.TargetVersion == sourceVersion &&
            r.Endpoint == endpoint);

        if (rule == null) return response;

        var result = new Dictionary<string, object?>();

        foreach (var mapping in rule.ResponseMappings)
        {
            var sourceValue = ExtractValue(response, mapping.SourcePath);
            var transformedValue = mapping.Transform != null
                ? mapping.Transform(sourceValue)
                : sourceValue ?? mapping.DefaultValue;
            SetValue(result, mapping.TargetPath, transformedValue);
        }

        foreach (var action in rule.Actions.Where(a => a.Phase == TransformPhase.Response))
            await action.ExecuteAsync(result);

        return JsonSerializer.SerializeToElement(result);
    }

    private JsonElement ExtractValue(JsonElement element, string path)
    {
        var parts = path.Split('.');
        var current = element;
        foreach (var part in parts)
        {
            if (current.ValueKind == JsonValueKind.Object &&
                current.TryGetProperty(part, out var next))
                current = next;
            else
                return default;
        }
        return current;
    }

    private void SetValue(Dictionary<string, object?> dict, string path, object? value)
    {
        var parts = path.Split('.');
        var current = dict;
        for (int i = 0; i < parts.Length - 1; i++)
        {
            if (!current.ContainsKey(parts[i]) ||
                current[parts[i]] is not Dictionary<string, object?> nested)
            {
                nested = new Dictionary<string, object?>();
                current[parts[i]] = nested;
            }
            current = nested;
        }
        current[parts[^1]] = value;
    }
}

8.2 Format Conversion

Modern API gateways must support multiple serialization formats to serve diverse client ecosystems. The transformation layer handles conversion between JSON, XML, Protocol Buffers, and other formats. For gRPC services behind the gateway, the transformation layer can translate between JSON REST requests and Protocol Buffer messages, enabling REST clients to interact with gRPC backends transparently.

8.3 Response Enrichment

The gateway can enrich responses from backend services with additional data that should not be duplicated across services. Common enrichment operations include adding hypermedia links (HATEOAS), inserting version metadata, appending rate limit information, and adding debugging headers in development environments. Enrichment is performed at the gateway level to keep backend services focused on business logic while ensuring consistent metadata across all API versions.

9. Circuit Breaker and Resilience Patterns

In a microservices architecture, the API gateway sits at the boundary between external consumers and the internal service mesh. When backend services experience failures, timeouts, or degraded performance, the gateway must detect these conditions quickly and take protective action to prevent cascading failures across the entire system. The circuit breaker pattern, combined with complementary resilience patterns such as bulkheads, retries, and timeouts, provides the foundation for building a gateway that degrades gracefully under stress rather than failing catastrophically.

9.1 Circuit Breaker Implementation

The circuit breaker monitors failures for each backend service instance and transitions between three states: Closed (normal operation), Open (failing fast), and Half-Open (testing recovery). When the failure rate exceeds a configured threshold within a time window, the circuit opens and all subsequent requests are immediately rejected without contacting the backend. After a configurable reset timeout, the circuit transitions to half-open state and allows a limited number of probe requests through.

C#
// Production Circuit Breaker with Per-Version State
public class GatewayCircuitBreaker : ICircuitBreaker
{
    private readonly ConcurrentDictionary<string, CircuitState> _circuits = new();
    private readonly ConcurrentDictionary<string, CircuitMetrics> _metrics = new();
    private readonly CircuitBreakerOptions _options;
    private readonly ILogger<GatewayCircuitBreaker> _logger;
    private readonly ISystemClock _clock;

    public GatewayCircuitBreaker(
        IOptions<CircuitBreakerOptions> options,
        ILogger<GatewayCircuitBreaker> logger,
        ISystemClock clock)
    {
        _options = options.Value;
        _logger = logger;
        _clock = clock;
    }

    public CircuitState GetState(string serviceKey)
    {
        return _circuits.GetOrAdd(serviceKey, _ => CircuitState.Closed);
    }

    public async Task<CircuitExecutionResult> ExecuteAsync(
        string serviceKey,
        Func<Task<HttpResponseMessage>> action,
        Func<Task<HttpResponseMessage>> fallback)
    {
        var state = GetState(serviceKey);

        if (state == CircuitState.Open)
        {
            var metrics = _metrics.GetOrAdd(serviceKey, _ => new CircuitMetrics());
            if (_clock.UtcNow - metrics.LastFailureTime >
                TimeSpan.FromSeconds(_options.ResetTimeoutSeconds))
            {
                _circuits[serviceKey] = CircuitState.HalfOpen;
                _logger.LogInformation("Circuit half-opened for {ServiceKey}", serviceKey);
                state = CircuitState.HalfOpen;
            }
            else
            {
                return new CircuitExecutionResult
                {
                    Success = false,
                    CircuitOpened = true,
                    Response = await fallback(),
                    FallbackUsed = true
                };
            }
        }

        try
        {
            var response = await action();
            OnSuccess(serviceKey);
            return new CircuitExecutionResult
            {
                Success = response.IsSuccessStatusCode,
                Response = response,
                FallbackUsed = false
            };
        }
        catch (Exception ex)
        {
            OnFailure(serviceKey);
            _logger.LogWarning(ex, "Circuit breaker recorded failure for {ServiceKey}", serviceKey);
            return new CircuitExecutionResult
            {
                Success = false,
                Response = await fallback(),
                FallbackUsed = true,
                Exception = ex
            };
        }
    }

    private void OnSuccess(string serviceKey)
    {
        var metrics = _metrics.GetOrAdd(serviceKey, _ => new CircuitMetrics());
        lock (metrics)
        {
            metrics.ConsecutiveFailures = 0;
            metrics.SuccessfulRequests++;
            if (_circuits[serviceKey] == CircuitState.HalfOpen)
            {
                metrics.HalfOpenSuccesses++;
                if (metrics.HalfOpenSuccesses >= _options.HalfOpenMaxProbes)
                {
                    _circuits[serviceKey] = CircuitState.Closed;
                    metrics.HalfOpenSuccesses = 0;
                    _logger.LogInformation("Circuit closed for {ServiceKey}", serviceKey);
                }
            }
        }
    }

    private void OnFailure(string serviceKey)
    {
        var metrics = _metrics.GetOrAdd(serviceKey, _ => new CircuitMetrics());
        lock (metrics)
        {
            metrics.ConsecutiveFailures++;
            metrics.LastFailureTime = _clock.UtcNow;
            metrics.TotalFailures++;

            if (_circuits[serviceKey] == CircuitState.HalfOpen)
            {
                _circuits[serviceKey] = CircuitState.Open;
                _logger.LogWarning("Circuit reopened for {ServiceKey} during half-open probe", serviceKey);
                return;
            }

            var windowStart = _clock.UtcNow.AddSeconds(-_options.FailureWindowSeconds);
            if (metrics.RecentFailures.Count(f => f > windowStart) >= _options.FailureThreshold)
            {
                _circuits[serviceKey] = CircuitState.Open;
                _logger.LogWarning(
                    "Circuit opened for {ServiceKey}: {Failures} failures in {Window}s",
                    serviceKey, metrics.RecentFailures.Count(f => f > windowStart),
                    _options.FailureWindowSeconds);
            }

            metrics.RecentFailures.Add(_clock.UtcNow);
        }
    }
}

public class CircuitBreakerOptions
{
    public int FailureThreshold { get; set; } = 5;
    public int FailureWindowSeconds { get; set; } = 60;
    public int ResetTimeoutSeconds { get; set; } = 30;
    public int HalfOpenMaxProbes { get; set; } = 3;
}

public class CircuitMetrics
{
    public int ConsecutiveFailures;
    public long SuccessfulRequests;
    public long TotalFailures;
    public DateTime LastFailureTime;
    public int HalfOpenSuccesses;
    public List<DateTime> RecentFailures { get; } = new();
}

public class CircuitExecutionResult
{
    public bool Success { get; set; }
    public bool CircuitOpened { get; set; }
    public HttpResponseMessage Response { get; set; } = null!;
    public bool FallbackUsed { get; set; }
    public Exception? Exception { get; set; }
}

9.2 Retry Policy with Exponential Backoff

Transient failures — momentary network blips, brief service restarts, or temporary overload conditions — should be handled with automatic retries before the circuit breaker is triggered. The retry policy defines the maximum number of retry attempts, the delay between attempts (using exponential backoff with jitter to prevent thundering herd problems), and the conditions under which a retry should be attempted. Not all errors are retryable — HTTP 400 Bad Request and HTTP 401 Unauthorized should not be retried, while HTTP 503 Service Unavailable and timeout errors are good candidates.

9.3 Bulkhead Isolation

The bulkhead pattern limits the number of concurrent requests to each backend service, preventing a slow or failing service from consuming all gateway resources. Each API version has its own bulkhead with configurable limits on maximum concurrent connections, queue depth, and timeout. When a bulkhead is full, additional requests are immediately rejected or queued, preserving resources for other services.

10. API Lifecycle Management

API lifecycle management encompasses the entire journey of an API from initial design through deployment, adoption, evolution, deprecation, and eventual retirement. A well-managed lifecycle ensures that APIs remain healthy, secure, and aligned with business needs while providing a predictable, transparent process for both API providers and consumers. Without lifecycle management, organizations accumulate API debt that slows innovation, increases security risk, and frustrates development teams.

10.1 Lifecycle Stages

Each API version progresses through a series of defined stages, each with specific criteria for advancement and specific responsibilities for the API team. The stages are not merely administrative checkpoints — they represent real operational states that affect how the API is deployed, monitored, and communicated to consumers.

The Design stage is where the API contract is defined through OpenAPI specifications, reviewed by stakeholders, and validated against architectural guidelines. During this stage, the team gathers requirements from potential consumers, reviews the API design against existing versions for consistency, and establishes the migration path from the previous version.

The Alpha stage involves internal development and testing. The API is deployed to a staging environment accessible only to the API team and selected internal consumers. During this stage, the team validates the implementation against the specification, performs load testing, and identifies any design issues. The alpha stage typically lasts 2-4 weeks.

The Beta stage opens the API to a limited set of external consumers for testing and feedback. Beta consumers are typically partner organizations that have agreed to provide feedback and tolerate potential breaking changes. This stage validates the API design against real-world usage patterns.

The General Availability (GA) stage marks the official release. At this point, the API is fully supported, documented, and available to all consumers. Rate limits, SLAs, and support commitments are in effect.

The Deprecated stage begins when a newer version has been released. The deprecated version continues to function normally but receives no new features. Deprecation notices are added to all responses, and consumers are actively encouraged to migrate.

The Retired stage is the final state where the API version is no longer available. All requests return HTTP 410 Gone.

C#
// API Version Lifecycle State Machine
public class ApiVersionLifecycle
{
    private static readonly Dictionary<LifecycleState, HashSet<LifecycleState>>
        AllowedTransitions = new()
    {
        [LifecycleState.Design] = new()
            { LifecycleState.Alpha, LifecycleState.Cancelled },
        [LifecycleState.Alpha] = new()
            { LifecycleState.Beta, LifecycleState.Design, LifecycleState.Cancelled },
        [LifecycleState.Beta] = new()
            { LifecycleState.GeneralAvailability, LifecycleState.Alpha, LifecycleState.Cancelled },
        [LifecycleState.GeneralAvailability] = new()
            { LifecycleState.Deprecated },
        [LifecycleState.Deprecated] = new()
            { LifecycleState.Retired, LifecycleState.GeneralAvailability },
        [LifecycleState.Retired] = new(),
        [LifecycleState.Cancelled] = new()
    };

    public async Task<TransitionResult> TransitionAsync(
        string apiVersion, LifecycleState targetState,
        IVersionRepository repository)
    {
        var current = await repository.GetVersionStateAsync(apiVersion);

        if (!AllowedTransitions.TryGetValue(current, out var allowed) ||
            !allowed.Contains(targetState))
        {
            return TransitionResult.Fail(
                $"Cannot transition from {current} to {targetState}");
        }

        var validation = await ValidateTransitionAsync(
            apiVersion, current, targetState, repository);

        if (!validation.IsValid)
            return TransitionResult.Fail(validation.Errors);

        await repository.UpdateVersionStateAsync(apiVersion, targetState);
        await OnTransitionAsync(apiVersion, current, targetState);
        return TransitionResult.Success();
    }

    private async Task OnTransitionAsync(
        string apiVersion, LifecycleState from, LifecycleState to)
    {
        switch (to)
        {
            case LifecycleState.Deprecated:
                await NotifyDeprecationAsync(apiVersion);
                await UpdateGatewayRoutingAsync(apiVersion, to);
                break;
            case LifecycleState.Retired:
                await RemoveFromGatewayAsync(apiVersion);
                await ArchiveDocumentationAsync(apiVersion);
                await NotifyRetirementAsync(apiVersion);
                break;
            case LifecycleState.GeneralAvailability:
                await EnableInGatewayAsync(apiVersion);
                await PublishDocumentationAsync(apiVersion);
                await NotifyReleaseAsync(apiVersion);
                break;
        }
    }
}

public enum LifecycleState
{
    Design, Alpha, Beta, GeneralAvailability,
    Deprecated, Retired, Cancelled
}

10.2 Version Sunset Policy

11. Version Coexistence and Migration Strategy

Version coexistence is the period during which multiple API versions are simultaneously active and serving production traffic. This is the most operationally complex phase of the API lifecycle, as the team must maintain backward compatibility for older versions while developing new features for the latest version. A well-defined coexistence strategy ensures that this period is manageable, predictable, and aligned with both business goals and consumer needs.

11.1 Shared Database Schema

One of the most challenging aspects of version coexistence is managing the database schema when different API versions expect different data structures. The recommended approach uses a forward-compatible schema design where new columns are added (never removed) and existing columns are never renamed or retyped. Older API versions read only the columns they understand, while newer versions read additional columns. This expand-and-contract pattern ensures that schema changes do not break older API versions.

11.2 Feature Parity Tracking

When maintaining multiple versions, it is critical to track which features are available in which versions. Features should be categorized as available, enhanced, deprecated, or unavailable. This tracking informs consumer migration decisions and helps the team understand the migration effort required.

C#
// Feature Parity Configuration for Multi-Version Support
public class FeatureParityConfig
{
    public List<VersionFeatureSet> Versions { get; set; } = new();
}

public class VersionFeatureSet
{
    public string Version { get; set; } = string.Empty;
    public LifecycleState State { get; set; }
    public List<FeatureEntry> Features { get; set; } = new();
    public ServiceEndpoints Endpoints { get; set; } = new();
}

public class FeatureEntry
{
    public string Name { get; set; } = string.Empty;
    public FeatureStatus Status { get; set; }
    public string? ReplacementInVersion { get; set; }
    public DateTime? DeprecatedSince { get; set; }
    public DateTime? SunsetDate { get; set; }
    public string? MigrationNotes { get; set; }
}

public enum FeatureStatus
{
    Available, Enhanced, Deprecated, Unavailable
}

// Check feature availability for a specific version
public class FeatureCheckMiddleware
{
    private readonly RequestDelegate _next;
    private readonly FeatureParityConfig _config;

    public FeatureCheckMiddleware(
        RequestDelegate next, IOptions<FeatureParityConfig> config)
    {
        _next = next;
        _config = config.Value;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        var version = context.GetRequestedApiVersion()?.ToString();
        var feature = context.Request.Headers["X-Feature-Name"].FirstOrDefault();

        if (version != null && feature != null)
        {
            var versionFeatures = _config.Versions
                .FirstOrDefault(v => v.Version == version);
            var featureEntry = versionFeatures?.Features
                .FirstOrDefault(f => f.Name == feature);

            if (featureEntry == null || featureEntry.Status == FeatureStatus.Unavailable)
            {
                context.Response.StatusCode = 404;
                await context.Response.WriteAsJsonAsync(new
                {
                    error = "Feature not available",
                    feature = feature,
                    apiVersion = version,
                    suggestion = featureEntry?.ReplacementInVersion != null
                        ? $"This feature is available in version {featureEntry.ReplacementInVersion}"
                        : "This feature is not available in this version"
                });
                return;
            }

            if (featureEntry.Status == FeatureStatus.Deprecated)
            {
                context.Response.Headers["X-Feature-Deprecated"] = "true";
                context.Response.Headers["X-Feature-Sunset"] =
                    featureEntry.SunsetDate?.ToString("yyyy-MM-dd") ?? "unknown";
            }
        }

        await _next(context);
    }
}

11.3 Migration Window Management

12. OpenAPI/Swagger Specification Management

The OpenAPI Specification (formerly Swagger) is the industry standard for describing REST APIs. Managing OpenAPI specs across multiple API versions requires a structured approach that ensures specifications remain accurate, consistent, and synchronized with the actual implementation. A well-managed spec repository serves as the single source of truth for API contracts, enabling automated code generation, testing, documentation, and client SDK production.

12.1 Specification Structure

Each API version should have its own complete OpenAPI specification file, stored alongside the version's implementation code. The specifications should follow a consistent structure with shared components that are versioned independently. This modular approach allows common definitions to be reused across versions while maintaining version-specific customizations.

YAML
# OpenAPI Specification for Multi-Version API
openapi: "3.1.0"
info:
  title: "Order Management API"
  version: "3.0.0"
  description: |
    The Order Management API provides endpoints for creating,
    retrieving, updating, and managing customer orders.
    This is version 3.0 of the API.
  contact:
    name: API Support
    email: api-support@ayodhyya.com
  x-api-lifecycle:
    state: general-availability
    release-date: "2026-01-15"
    deprecated-versions: ["1.0", "2.0"]
    sunset-dates:
      "1.0": "2026-06-01"
      "2.0": "2027-01-15"

servers:
  - url: https://api.ayodhyya.com/api/v3
    description: Production
  - url: https://sandbox-api.ayodhyya.com/api/v3
    description: Sandbox

paths:
  /orders:
    get:
      operationId: listOrders
      summary: List all orders
      tags: [Orders]
      parameters:
        - $ref: "#/components/parameters/CursorPagination"
        - $ref: "#/components/parameters/OrderFilter"
      responses:
        "200":
          description: Paginated list of orders
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/OrderListResponse"
        "401":
          $ref: "#/components/responses/Unauthorized"
        "429":
          $ref: "#/components/responses/RateLimited"

    post:
      operationId: createOrder
      summary: Create a new order
      tags: [Orders]
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: "#/components/schemas/CreateOrderRequest"
      responses:
        "201":
          description: Order created successfully
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/OrderResponse"

components:
  schemas:
    OrderResponse:
      type: object
      required: [id, customer, lineItems, status, createdAt]
      properties:
        id:
          type: string
          format: uuid
        customer:
          $ref: "#/components/schemas/CustomerInfo"
        lineItems:
          type: array
          items:
            $ref: "#/components/schemas/LineItem"
        status:
          type: string
          enum: [pending, confirmed, shipped, delivered, cancelled]
        currency:
          type: string
          example: "USD"
        totalAmount:
          type: number
          format: decimal
        metadata:
          type: object
          additionalProperties: true
        createdAt:
          type: string
          format: date-time
        updatedAt:
          type: string
          format: date-time

12.2 Specification Validation Pipeline

12.3 Multi-Version Spec Repository

The spec repository organizes OpenAPI files in a directory structure that mirrors the versioning strategy. Each version has its own directory with its own OpenAPI file, and a shared directory contains common component definitions that can be imported via $ref pointers. The build pipeline validates all specs, generates client SDKs, and publishes documentation for each version independently.

13. API Documentation Portal

A comprehensive API documentation portal is the primary interface between the API team and external consumers. It must provide version-aware documentation that allows developers to browse, search, and test API endpoints across all available versions. The portal should include interactive API explorers, code samples in multiple languages, migration guides between versions, and real-time status information about API health and deprecation timelines.

13.1 Portal Architecture

The documentation portal is a single-page application that consumes the OpenAPI specifications for all API versions and renders interactive documentation. It supports version switching, side-by-side comparison of endpoints across versions, and contextual migration guides. The portal also integrates with the analytics system to show real-time adoption metrics and with the lifecycle management system to display deprecation status and sunset timelines.

13.2 Version-Aware Documentation Features

The version switcher allows developers to select a specific version and view only the documentation relevant to that version. Deprecated versions are clearly marked with visual indicators (red banners, strikethrough text) and include links to migration guides. The side-by-side comparison view shows the differences between two versions of the same endpoint, highlighting added, removed, and changed fields. The migration wizard provides a step-by-step guide for upgrading from one version to another, including code examples, testing checklists, and estimated effort.

The API playground enables developers to make live API calls directly from the documentation. The playground respects version routing, authentication, and rate limiting, providing a realistic testing environment. Request and response examples are generated from the OpenAPI specification and are validated against the actual API responses to ensure accuracy. The playground also supports environment switching between production, staging, and sandbox environments.

13.3 Documentation Quality Metrics

14. Analytics and Monitoring at the Gateway

Comprehensive analytics and monitoring at the API gateway provide the visibility needed to make informed decisions about version lifecycle, capacity planning, performance optimization, and security. The gateway generates rich telemetry data on every request, including version-specific metrics, latency distributions, error rates, client behavior patterns, and resource utilization.

14.1 Metrics Collection Architecture

The metrics collection system uses a pipeline architecture where gateway nodes emit metrics to a local aggregator, which batches and forwards them to a time-series database for storage and analysis. The system supports both push-based collection (gateway pushes metrics to a collector) and pull-based collection (Prometheus scrapes metrics from gateway endpoints). High-frequency metrics such as request counts and latency histograms are aggregated locally before transmission to reduce network overhead.

14.2 Key Metrics to Track

C#
// Gateway Metrics Collection with Per-Version Tracking
public class GatewayMetricsCollector
{
    private readonly IMetrics _metrics;

    public GatewayMetricsCollector(IMetrics metrics)
    {
        _metrics = metrics;
    }

    public void RecordRequest(
        string version, string endpoint, string method,
        int statusCode, TimeSpan duration, string clientId,
        long requestSize, long responseSize)
    {
        _metrics.increment("gateway.requests.total",
            tags: new Dictionary<string, string>
            {
                ["version"] = version,
                ["endpoint"] = endpoint,
                ["method"] = method,
                ["status"] = statusCode.ToString()
            });

        _metrics.histogram("gateway.requests.duration_ms",
            duration.TotalMilliseconds,
            tags: new Dictionary<string, string>
            {
                ["version"] = version,
                ["endpoint"] = endpoint
            });

        if (statusCode >= 400)
        {
            _metrics.increment("gateway.errors.total",
                tags: new Dictionary<string, string>
                {
                    ["version"] = version,
                    ["status"] = statusCode.ToString(),
                    ["endpoint"] = endpoint
                });
        }

        _metrics.histogram("gateway.request.size_bytes", requestSize,
            tags: new Dictionary<string, string> { ["version"] = version });

        _metrics.histogram("gateway.response.size_bytes", responseSize,
            tags: new Dictionary<string, string> { ["version"] = version });

        _metrics.increment("gateway.client.requests",
            tags: new Dictionary<string, string>
            {
                ["client"] = clientId,
                ["version"] = version
            });
    }

    public void RecordCircuitBreakerEvent(
        string serviceKey, string eventType, CircuitState newState)
    {
        _metrics.increment("gateway.circuit_breaker.events",
            tags: new Dictionary<string, string>
            {
                ["service"] = serviceKey,
                ["event"] = eventType,
                ["state"] = newState.ToString()
            });
    }

    public void RecordRateLimitEvent(
        string clientId, string policyName, bool allowed)
    {
        _metrics.increment("gateway.rate_limit.events",
            tags: new Dictionary<string, string>
            {
                ["client"] = clientId,
                ["policy"] = policyName,
                ["allowed"] = allowed.ToString()
            });
    }
}

14.3 Alerting Rules

A well-configured alerting system is essential for maintaining the health of the API gateway. Alerts should be tiered by severity (critical, warning, info) and should include enough context for the on-call engineer to understand the issue and begin investigation immediately. The alerting rules should be version-aware, allowing different thresholds for different API versions. For example, a deprecated version may have relaxed alerting thresholds since reduced traffic is expected.

15. Multi-Protocol Support (REST, gRPC, GraphQL, WebSocket)

Modern API platforms must support multiple protocols to serve diverse client ecosystems effectively. REST remains the dominant protocol for web APIs, but gRPC offers superior performance for service-to-service communication, GraphQL provides flexible query capabilities for complex data requirements, and WebSocket enables real-time bidirectional communication. A well-designed API gateway handles all these protocols through a unified management layer while preserving the unique advantages of each protocol.

15.1 Protocol Comparison

15.2 Protocol-Aware Gateway Routing

The gateway must detect the protocol of each incoming request and route it through the appropriate handler pipeline. REST requests use the standard HTTP middleware pipeline. gRPC requests are detected by their content-type (application/grpc) and routed through a specialized gRPC handler. GraphQL requests are identified by the application/graphql content-type or the presence of a query parameter. WebSocket upgrade requests are detected by the Upgrade: websocket header and handled through a long-lived connection handler.

C#
// Multi-Protocol Gateway Configuration
public static class MultiProtocolGateway
{
    public static IServiceCollection AddMultiProtocolGateway(
        this IServiceCollection services, IConfiguration config)
    {
        // REST API versioning
        services.AddApiVersioning(options =>
        {
            options.ApiVersionReader = new UrlSegmentApiVersionReader();
            options.DefaultApiVersion = new ApiVersion(1, 0);
        })
        .AddApiExplorer();

        // gRPC services
        services.AddGrpcClient<OrderService.OrderServiceClient>(options =>
        {
            options.Address = new Uri(config["Grpc:OrderService"]!);
        });

        // GraphQL
        services.AddGraphQLServer()
            .AddQueryType<QueryType>()
            .AddMutationType<MutationType>()
            .AddSubscriptionType<SubscriptionType>()
            .AddType<OrderResponseType>();

        // WebSocket support
        services.Configure<WebSocketOptions>(options =>
        {
            options.KeepAliveInterval = TimeSpan.FromSeconds(30);
            options.AllowedOrigins.Add("https://app.ayodhyya.com");
        });

        return services;
    }

    public static IApplicationBuilder UseMultiProtocolGateway(
        this IApplicationBuilder app)
    {
        // Protocol detection middleware
        app.Use(async (context, next) =>
        {
            var contentType = context.Request.ContentType ?? "";

            if (contentType.Contains("application/grpc"))
            {
                context.Items["Protocol"] = "gRPC";
                await context.RequestServices
                    .GetRequiredService<GrpcProtocolHandler>()
                    .HandleAsync(context);
            }
            else if (contentType.Contains("application/graphql") ||
                     context.Request.Query.ContainsKey("query"))
            {
                context.Items["Protocol"] = "GraphQL";
                await app.ApplicationServices
                    .GetRequiredService<GraphQLMiddleware>()
                    .InvokeAsync(context);
            }
            else if (context.Request.Headers.Upgrade.Contains("websocket"))
            {
                context.Items["Protocol"] = "WebSocket";
                await next();
            }
            else
            {
                context.Items["Protocol"] = "REST";
                await next();
            }
        });

        app.UseWebSockets();
        app.UseRouting();
        app.MapGraphQL();
        app.MapGrpcService<OrderGrpcService>();
        app.MapControllers();

        return app;
    }
}

15.3 GraphQL Schema Versioning

GraphQL handles versioning differently from REST. Instead of explicit version numbers, GraphQL uses schema evolution with deprecation directives. Fields and types are marked as @deprecated with a reason string, and new fields are added alongside deprecated ones. Clients gradually migrate from deprecated fields to new ones, and the schema is periodically cleaned up by removing fields that have zero usage. This approach eliminates the need for parallel API versions but requires sophisticated usage analytics to track field-level adoption.

15.4 gRPC Service Versioning

gRPC handles versioning through Protocol Buffer package names and service definitions. Each version of a gRPC service is defined as a separate service within its own package namespace. The gateway maps incoming gRPC calls to the appropriate versioned service based on the fully-qualified service name. Backward compatibility is managed through protobuf field numbering rules — new fields are added with new field numbers, and deprecated fields are reserved but never reused.

16. Plugin and Middleware Architecture

An extensible plugin and middleware architecture is essential for maintaining a gateway that can adapt to changing requirements without core code modifications. Plugins encapsulate cross-cutting concerns such as custom authentication schemes, request validation rules, transformation logic, and monitoring integrations. The middleware pipeline processes each request through an ordered sequence of plugins, where each plugin can inspect, modify, or reject the request before passing it to the next middleware in the chain.

16.1 Plugin Lifecycle

Plugins follow a defined lifecycle from registration through execution to cleanup. During registration, plugins declare their dependencies, configuration requirements, and execution order constraints. During execution, plugins receive a request context that includes the API version, client identity, and shared state dictionary. Plugins can modify the request, attach data to the context for downstream plugins, or short-circuit the pipeline by returning a response directly.

C#
// Plugin Interface and Base Implementation
public interface IGatewayPlugin
{
    string Name { get; }
    int Order { get; }
    Task<PluginResult> OnRequestAsync(GatewayContext context);
    Task OnResponseAsync(GatewayContext context, HttpResponseMessage response);
    Task OnErrorAsync(GatewayContext context, Exception exception);
}

public abstract class GatewayPluginBase : IGatewayPlugin
{
    public abstract string Name { get; }
    public abstract int Order { get; }

    public virtual Task<PluginResult> OnRequestAsync(GatewayContext context)
    {
        return Task.FromResult(PluginResult.Continue());
    }

    public virtual Task OnResponseAsync(GatewayContext context, HttpResponseMessage response)
    {
        return Task.CompletedTask;
    }

    public virtual Task OnErrorAsync(GatewayContext context, Exception exception)
    {
        return Task.CompletedTask;
    }
}

// Example: Custom logging plugin
public class RequestLoggingPlugin : GatewayPluginBase
{
    public override string Name => "RequestLogging";
    public override int Order => 10;

    private readonly ILogger<RequestLoggingPlugin> _logger;

    public RequestLoggingPlugin(ILogger<RequestLoggingPlugin> logger)
    {
        _logger = logger;
    }

    public override async Task<PluginResult> OnRequestAsync(GatewayContext context)
    {
        context.Items["RequestTimestamp"] = Stopwatch.StartNew();
        context.Items["RequestId"] = Guid.NewGuid().ToString();

        _logger.LogInformation(
            "Incoming request {RequestId}: {Method} {Path} v{Version} from {Client}",
            context.Items["RequestId"],
            context.Request.Method,
            context.Request.Path,
            context.ApiVersion,
            context.ClientIdentity?.ClientId ?? "anonymous");

        return PluginResult.Continue();
    }

    public override async Task OnResponseAsync(
        GatewayContext context, HttpResponseMessage response)
    {
        if (context.Items["RequestTimestamp"] is Stopwatch sw)
        {
            sw.Stop();
            _logger.LogInformation(
                "Request {RequestId} completed: {StatusCode} in {ElapsedMs}ms",
                context.Items["RequestId"],
                (int)response.StatusCode,
                sw.ElapsedMilliseconds);
        }
    }
}

// Plugin manager that orchestrates execution
public class PluginManager
{
    private readonly IEnumerable<IGatewayPlugin> _plugins;
    private readonly ILogger<PluginManager> _logger;

    public PluginManager(
        IEnumerable<IGatewayPlugin> plugins,
        ILogger<PluginManager> logger)
    {
        _plugins = plugins.OrderBy(p => p.Order).ToList();
        _logger = logger;
    }

    public async Task<PluginPipelineResult> ExecuteRequestPhaseAsync(
        GatewayContext context)
    {
        foreach (var plugin in _plugins)
        {
            try
            {
                var result = await plugin.OnRequestAsync(context);
                if (result.Action == PluginAction.Reject)
                {
                    return PluginPipelineResult.ShortCircuited(
                        plugin.Name, result.RejectionResponse);
                }
            }
            catch (Exception ex)
            {
                _logger.LogError(ex,
                    "Plugin {PluginName} failed during request phase", plugin.Name);
                await plugin.OnErrorAsync(context, ex);
            }
        }

        return PluginPipelineResult.Continue();
    }

    public async Task ExecuteResponsePhaseAsync(
        GatewayContext context, HttpResponseMessage response)
    {
        foreach (var plugin in _plugins.Reverse())
        {
            try
            {
                await plugin.OnResponseAsync(context, response);
            }
            catch (Exception ex)
            {
                _logger.LogError(ex,
                    "Plugin {PluginName} failed during response phase", plugin.Name);
            }
        }
    }
}

public class GatewayContext
{
    public HttpRequest Request { get; set; } = null!;
    public string ApiVersion { get; set; } = string.Empty;
    public ApiClientIdentity? ClientIdentity { get; set; }
    public Dictionary<string, object> Items { get; } = new();
    public IServiceProvider ServiceProvider { get; set; } = null!;
}

public enum PluginAction { Continue, Reject }
public record PluginResult(PluginAction Action, HttpResponseMessage? RejectionResponse);
public record PluginPipelineResult(bool Continued, string? ShortCircuitedBy, HttpResponseMessage? Response)
{
    public static PluginPipelineResult Continue() =>
        new(true, null, null);
    public static PluginPipelineResult ShortCircuited(string by, HttpResponseMessage response) =>
        new(false, by, response);
}

16.2 Built-in Plugin Library

17. Performance Optimization (Caching, Connection Pooling)

Performance optimization at the API gateway is critical because every millisecond of gateway overhead is added to every API request. A gateway handling 50,000 requests per second with even 5ms of added latency translates to 250 seconds of cumulative added latency per second across all requests. Optimization strategies must address caching, connection management, memory allocation, serialization overhead, and network utilization to achieve the sub-millisecond latency targets expected of a production gateway.

17.1 Multi-Level Caching Strategy

The caching strategy operates at multiple levels to maximize cache hit rates while ensuring data freshness. The L1 cache is an in-memory cache within each gateway node, using a high-performance in-memory store for hot data with TTLs measured in seconds. The L2 cache is a shared Redis cluster that provides cross-node cache consistency with TTLs measured in minutes. The L3 cache is the CDN edge cache for public, read-heavy endpoints with TTLs measured in hours.

C#
// Multi-Level Cache for API Gateway Responses
public class GatewayCacheManager
{
    private readonly IMemoryCache _l1Cache;
    private readonly IDatabase _l2Redis;
    private readonly ILogger<GatewayCacheManager> _logger;
    private readonly GatewayCacheOptions _options;

    public GatewayCacheManager(
        IMemoryCache l1Cache,
        IConnectionMultiplexer redis,
        ILogger<GatewayCacheManager> logger,
        IOptions<GatewayCacheOptions> options)
    {
        _l1Cache = l1Cache;
        _l2Redis = redis.GetDatabase();
        _logger = logger;
        _options = options.Value;
    }

    public async Task<CachedResponse?> GetAsync(
        string cacheKey, string apiVersion)
    {
        // L1: Check in-memory cache first
        if (_l1Cache.TryGetValue<CachedResponse>(
            $"v{apiVersion}:{cacheKey}", out var l1Result))
        {
            _logger.LogDebug("L1 cache hit for {Key}", cacheKey);
            return l1Result;
        }

        // L2: Check Redis cache
        var l2Data = await _l2Redis.StringGetAsync(
            $"cache:v{apiVersion}:{cacheKey}");
        if (l2Data.HasValue)
        {
            var cached = JsonSerializer.Deserialize<CachedResponse>(l2Data!);
            // Populate L1 cache with shorter TTL
            _l1Cache.Set(
                $"v{apiVersion}:{cacheKey}",
                cached,
                TimeSpan.FromSeconds(_options.L1TtlSeconds));
            _logger.LogDebug("L2 cache hit for {Key}", cacheKey);
            return cached;
        }

        _logger.LogDebug("Cache miss for {Key}", cacheKey);
        return null;
    }

    public async Task SetAsync(
        string cacheKey, string apiVersion,
        CachedResponse response, TimeSpan? ttl = null)
    {
        var effectiveTtl = ttl ?? TimeSpan.FromSeconds(_options.L2TtlSeconds);
        var serialized = JsonSerializer.Serialize(response);

        // Set in both cache levels
        _l1Cache.Set(
            $"v{apiVersion}:{cacheKey}",
            response,
            TimeSpan.FromSeconds(Math.Min(
                _options.L1TtlSeconds, effectiveTtl.TotalSeconds)));

        await _l2Redis.StringSetAsync(
            $"cache:v{apiVersion}:{cacheKey}",
            serialized,
            effectiveTtl);

        // Track cache key for invalidation
        await _l2Redis.SetAddAsync(
            $"cache:keys:v{apiVersion}",
            cacheKey);
    }

    public async Task InvalidateVersionAsync(string apiVersion)
    {
        var keys = await _l2Redis.SetMembersAsync($"cache:keys:v{apiVersion}");
        if (keys.Length > 0)
        {
            var redisKeys = keys.Select(k =>
                (RedisKey)$"cache:v{apiVersion}:{k}").ToArray();
            await _l2Redis.KeyDeleteAsync(redisKeys);
            await _l2Redis.KeyDeleteAsync($"cache:keys:v{apiVersion}");
            _logger.LogInformation(
                "Invalidated {Count} cache entries for version {Version}",
                keys.Length, apiVersion);
        }
    }
}

public class CachedResponse
{
    public int StatusCode { get; set; }
    public Dictionary<string, string> Headers { get; set; } = new();
    public byte[] Body { get; set; } = Array.Empty<byte>();
    public DateTime CachedAt { get; set; }
    public string ApiVersion { get; set; } = string.Empty;
}

public class GatewayCacheOptions
{
    public bool Enabled { get; set; } = true;
    public int L1TtlSeconds { get; set; } = 5;
    public int L2TtlSeconds { get; set; } = 60;
    public long MaxL1SizeBytes { get; set; } = 100 * 1024 * 1024;
    public string[] CacheableMethods { get; set; } = { "GET", "HEAD" };
    public string[] CacheableStatusCodes { get; set; } = { "200", "301", "404" };
}

17.2 Connection Pooling

Connection pooling is essential for maintaining high throughput to backend services. The gateway maintains a pool of HTTP/2 connections to each backend service instance, with configurable pool sizes, connection timeouts, and health monitoring. HTTP/2 multiplexing allows multiple requests to share a single TCP connection, reducing connection establishment overhead and improving throughput. The connection pool also supports connection warm-up, where connections are pre-established before the first request arrives, eliminating cold-start latency.

17.3 Memory Optimization

The gateway minimizes memory allocations through several techniques: using Span<T> and Memory<T> for zero-copy buffer operations, employing pooled buffers from ArrayPool<T> for temporary byte arrays, using source generators for JSON serialization to avoid reflection overhead, and implementing object pooling for frequently created objects like request contexts and response objects. These optimizations collectively reduce GC pressure and improve throughput under sustained high load.

18. Interview Q&A

The following questions cover the key concepts discussed in this guide and are commonly asked in senior-level system design interviews. Each answer highlights the key trade-offs and decision points that interviewers are looking for.

Q1: When would you choose URL path versioning over header-based versioning?

A: URL path versioning is preferred when discoverability, simplicity, and browser testability are priorities. It is the best choice for public-facing APIs where third-party developers need to quickly understand and test the API, as the version is visible in every URL. Header-based versioning is preferred when REST purity and clean resource URIs are priorities, or when the API serves multiple representation formats alongside multiple versions. The practical recommendation is: use URL path versioning for public APIs with broad developer audiences, and header-based versioning for internal APIs or APIs serving sophisticated consumers who can manage custom headers.

Q2: How do you handle a breaking change in a field type within an existing API version?

A: A breaking change to an existing version should be avoided whenever possible. The recommended approach is to introduce the breaking change in a new version while keeping the old version functional. If the change must be applied to an existing version, use the gateway transformation layer to support both the old and new field types simultaneously during a transition period. The gateway can detect which format the client expects (based on a request header or client identification) and transform the response accordingly. This approach provides a graceful migration path without forcing all clients to upgrade simultaneously.

Q3: Explain the token bucket vs sliding window rate limiting trade-offs.

A: The token bucket algorithm allows controlled bursts up to the bucket capacity while maintaining a steady sustained rate. It is simpler to implement, requires minimal state (just tokens and last-refill-time), and handles bursty traffic patterns naturally. The sliding window algorithm provides more consistent rate limiting over time by considering request activity within a rolling window, eliminating the burst-at-boundary problem of fixed windows. The sliding window counter variant approximates this with just two counters per client. Choose token bucket when bursty traffic is expected and acceptable; choose sliding window when consistent, predictable rate limiting is required (e.g., billing APIs with strict per-second guarantees).

Q4: How would you design the gateway to handle a backend service outage without impacting other services?

A: This requires three complementary patterns: circuit breakers that detect failures and fail fast for the affected service, bulkhead isolation that limits concurrent connections to prevent resource exhaustion, and fallback mechanisms that provide degraded responses. The circuit breaker monitors failure rates per service and opens when thresholds are exceeded, immediately returning a fallback response (cached data, default response, or meaningful error). Bulkheads allocate a fixed number of concurrent connections per service version, preventing one slow service from consuming all gateway resources. Fallbacks can include serving stale cache data, returning a 503 with Retry-After header, or routing to a simplified read-only replica.

Q5: How do you manage OpenAPI specifications across 5+ active API versions without drift?

A: Preventing spec drift requires automated validation in the CI/CD pipeline. Each version has its own OpenAPI spec file that is validated against the actual implementation through automated contract testing. The pipeline uses tools like Prism to mock the spec and run integration tests against it, and Schemathesis to generate fuzz tests from the spec. A breaking change detection tool (like oasdiff) runs on every PR to flag breaking changes that would affect existing consumers. The spec is treated as code — it lives in version control, goes through code review, and is deployed as part of the release process. Shared components are extracted into separate files and imported via $ref to maintain consistency across versions.

Q6: What metrics would you prioritize when monitoring a multi-version API gateway?

A: The highest priority metrics are: (1) Version adoption rate — the percentage of traffic per version, which drives deprecation decisions; (2) Error rate per version — to detect version-specific regressions; (3) Latency per version and endpoint — to identify performance degradation in specific versions; (4) Rate limit utilization — to understand traffic patterns and adjust limits; (5) Circuit breaker state changes — to detect backend health issues; (6) Cache hit rate per version — to optimize caching strategies; (7) Client-specific metrics — to identify which partners are lagging in migration; (8) Deprecation progress — the migration curve from deprecated to current versions over time.

Q7: How do you implement zero-downtime deployment for a new API version?

A: Zero-downtime deployment of a new version follows a blue-green pattern at the gateway level. The new version is deployed alongside the existing version, initially with no traffic routed to it. The gateway's version registry is updated to include the new version, and a small percentage of traffic (canary) is gradually shifted. Health checks verify the new version's stability before increasing traffic. If issues are detected, traffic is immediately shifted back to the existing version. The key infrastructure requirements are: independent service deployments per version, gateway routing updates that can be applied atomically, health checks that validate both basic connectivity and business logic, and automated rollback mechanisms triggered by error rate or latency thresholds.

Q8: Explain the expand-and-contract database migration pattern for multi-version APIs.

A: The expand-and-contract pattern manages schema evolution across API versions in three phases. In the expand phase, new columns are added to the database table to support the new version's data requirements. Both old and new versions can operate simultaneously because old versions ignore the new columns and new versions read both old and new columns. In the migrate phase, data is backfilled into the new columns, and consumers gradually migrate to the new version. In the contract phase, after all consumers have migrated and the old version is retired, the old columns are removed. This pattern ensures that schema changes never break running services because the expand phase is always backward-compatible.

Q9: How do you handle authentication token scoping across multiple API versions?

A: Token scoping across versions is implemented through version-specific claims in the JWT. Each token includes a scope claim that lists the API versions and endpoints the client is authorized to access (e.g., api:v1:read api:v2:read api:v2:write). The gateway's authorization middleware extracts these scopes and validates them against the requested API version. This allows fine-grained control over which clients can access which versions. For example, a new partner might only have access to the latest version, while an existing partner retains access to both old and new versions during their migration period. Scope updates can be propagated through token refresh without requiring re-authentication.

Q10: What is the recommended approach for handling API versioning in a GraphQL API?

A: GraphQL handles versioning fundamentally differently from REST. Instead of explicit version numbers, GraphQL uses schema evolution. New fields are added to the schema without removing or breaking existing fields. Deprecated fields are marked with the @deprecated directive, which causes them to appear in the schema's deprecation documentation but remain functional. Clients are responsible for migrating from deprecated fields to new ones at their own pace. After a sufficient deprecation period, fields with zero usage (verified through schema analytics) can be removed. This approach eliminates version proliferation but requires robust field-level usage analytics and strong discipline around backward-compatible schema changes.