1. Why System Design Matters

System design is the discipline of defining the architecture, components, modules, interfaces, and data flow of a system to satisfy specified requirements. It is not about writing code. It is about deciding what code to write, how the pieces fit together, and what happens when things go wrong. Every production system you have ever used, from Netflix to Stripe to your banking app, was shaped by system design decisions made before a single line of code was written.

For junior engineers, system design feels abstract. You focus on making your function work correctly. But as you grow into a senior role, your impact shifts from writing correct functions to making correct architectural decisions. A single wrong architectural choice, like choosing the wrong database or failing to account for a bottleneck, can cost months of rework or cause catastrophic production outages. System design knowledge is the highest-leverage skill an engineer can develop because architectural decisions compound over time.

The Four Pillars of System Design

Every system design discussion revolves around four core properties. Understanding these properties and their relationships forms the foundation for every other concept in this guide.

The System Design Mindset

The difference between a junior and senior engineer is not knowledge. It is the ability to ask the right questions before proposing solutions. When presented with a new system requirement, a senior engineer asks: What is the expected read-to-write ratio? What are the latency requirements? How much data will we store? What happens when this component fails? Can we recover from data loss? These questions shape the design before any architecture diagram is drawn.

This guide is organized around the questions you should ask, the concepts that answer them, and the trade-offs you will face. Each section builds on the previous ones. By the end, you will have a complete mental framework for approaching any system design problem, whether in an interview or in production.

2. Scalability: Vertical vs Horizontal

Scalability is the ability of a system to handle increased load by adding resources. There are two fundamental approaches: vertical scaling (scaling up) and horizontal scaling (scaling out). Understanding the trade-offs between them is essential because the wrong choice can leave you either over-provisioned or unable to grow.

Vertical Scaling (Scale Up)

Vertical scaling means adding more power to a single machine: faster CPUs, more RAM, faster disks, or better network interfaces. It is the simplest approach because it requires no code changes. You deploy the same application on a bigger machine and it handles more load. For early-stage products with low traffic, vertical scaling is often the right choice. It avoids the complexity of distributed systems entirely.

The limitation is hard physical bounds. The largest cloud instances today offer 448 vCPUs and 24 TB of RAM. That is substantial, but it has limits. When your single database server cannot hold any more data in memory, or your application server cannot handle more concurrent connections, vertical scaling hits a ceiling. More importantly, a single machine is a single point of failure. When it goes down, everything goes down.

Horizontal Scaling (Scale Out)

Horizontal scaling means adding more machines to distribute the load. Instead of one powerful server, you run ten modest servers behind a load balancer. Each server handles a fraction of the total traffic. Horizontal scaling is theoretically unlimited; you can keep adding machines until you run out of money. But it introduces significant complexity: you need load balancers, session management, distributed data storage, and strategies for handling partial failures.

When to Choose Which

C# Example: Capacity Planning Calculator

C#
public class CapacityPlanner
{
    private readonly double _peakRps;
    private readonly double _singleServerCapacity;
    private readonly double _safetyMargin;

    public CapacityPlanner(double peakRps, double singleServerCapacity, double safetyMargin = 1.3)
    {
        _peakRps = peakRps;
        _singleServerCapacity = singleServerCapacity;
        _safetyMargin = safetyMargin;
    }

    public int ServersNeeded()
    {
        return (int)Math.Ceiling((_peakRps * _safetyMargin) / _singleServerCapacity);
    }

    public double CostPerServer(double monthlyCostPerServer)
    {
        return ServersNeeded() * monthlyCostPerServer;
    }

    public static void Main()
    {
        var planner = new CapacityPlanner(peakRps: 10000, singleServerCapacity: 500);
        Console.WriteLine($"Servers needed: {planner.ServersNeeded()}");
        Console.WriteLine($"Monthly cost: ${planner.CostPerServer(200):F2}");
    }
}

The Shared-Nothing Architecture

The gold standard for horizontal scalability is the shared-nothing architecture. In this pattern, each node operates independently with its own CPU, memory, and storage. There is no shared resource between nodes. This eliminates contention and allows linear scalability because adding a node adds capacity without competing for shared resources. Amazon DynamoDB, Google Bigtable, and Apache Cassandra all follow this principle. The trade-off is that data replication and consistency become your problem to solve.

3. High Availability and Uptime

Availability measures the percentage of time a system is operational and accessible to users. It is the most visible non-functional requirement because downtime is directly felt by users and directly measured by business metrics. A system with 99.9% availability still has 8.76 hours of downtime per year. For a high-traffic e-commerce platform, each hour of downtime can cost hundreds of thousands of dollars in lost revenue.

The Nines of Availability

Each additional nine costs exponentially more. Going from three nines to four nines might double your infrastructure cost. Going from four nines to five nines might quadruple it. Most web applications target three to four nines. Only mission-critical systems like financial exchanges, telecommunications infrastructure, and medical systems justify five nines or more.

SLAs, SLOs, and SLIs

These three terms form the vocabulary of availability management and are frequently confused:

• SLI (Service Level Indicator): A quantitative metric measuring service performance. Examples: request latency, error rate, throughput. The SLI is what you measure.
• SLO (Service Level Objective): A target value for an SLI. Example: "99.9% of requests complete within 200ms." The SLO is what you aim for internally.
• SLA (Service Level Agreement): A contractual commitment to customers about service performance, including consequences if the target is missed. The SLA is what you promise externally.

Building for High Availability

High availability is achieved through redundancy: running multiple copies of each component across different failure domains. The key strategies are eliminating single points of failure, using health checks and automatic failover, and designing for graceful degradation during partial outages.

C# Example: Health Check Implementation

C#
public class HealthCheckService
{
    private readonly Dictionary<string, Func<Task<bool>>> _checks = new();
    private readonly TimeSpan _timeout = TimeSpan.FromSeconds(5);

    public void RegisterCheck(string name, Func<Task<bool>> check)
    {
        _checks[name] = check;
    }

    public async Task<Dictionary<string, bool>> RunAllChecksAsync()
    {
        var results = new Dictionary<string, bool>();

        foreach (var check in _checks)
        {
            try
            {
                using var cts = new CancellationTokenSource(_timeout);
                var task = check.Value();
                if (await Task.WhenAny(task, Task.Delay(_timeout, cts.Token)) == task)
                    results[check.Key] = await task;
                else
                    results[check.Key] = false; // Timeout = unhealthy
            }
            catch
            {
                results[check.Key] = false;
            }
        }
        return results;
    }

    public async Task<bool> IsHealthyAsync()
    {
        var results = await RunAllChecksAsync();
        return results.Values.All(v => v);
    }
}

// Usage in ASP.NET Core
// app.MapHealthChecks("/health");
// builder.Services.AddHealthChecks()
//     .AddCheck<DatabaseHealthCheck>("database")
//     .AddCheck<RedisHealthCheck>("redis")
//     .AddCheck<ExternalApiHealthCheck>("payment-gateway");

Failover Strategies

Failover is the process of switching to a redundant system when the primary fails. Active-passive failover keeps a standby system ready but idle; when the primary fails, traffic switches to the standby. This wastes half your resources but is simpler to implement. Active-active failover runs both systems simultaneously, handling traffic in parallel. This is more efficient and provides zero-downtime failover, but requires careful coordination for stateful components like databases.

The critical insight is that failover must be automatic. Manual failover requires a human to detect the problem, diagnose it, and trigger the switch. In the middle of the night, this can take thirty minutes or more. Automatic failover, using health checks and load balancer configuration, can switch in seconds. Every production system should have automatic failover at every layer.

4. Reliability and Fault Tolerance

Reliability is the probability that a system performs its intended function without failure for a given period of time. It is not the same as availability. A system can be available but returning corrupted data, which makes it unreliable. Reliability requires data integrity, error detection, and graceful degradation under stress. In distributed systems, the fundamental assumption is that failures are not exceptional. They are inevitable.

The Failure Spectrum

Resilience Patterns

Circuit Breaker

A circuit breaker monitors calls to an external dependency. When failures exceed a threshold, the circuit opens and all subsequent calls fail immediately without attempting the network call. This prevents cascading failures and gives the failing service time to recover. After a timeout, the circuit enters a half-open state where a limited number of requests are allowed through to test recovery.

Retry with Exponential Backoff and Jitter

When a transient failure occurs, retrying after a short delay often succeeds. But without backoff, retries create a thundering herd effect where thousands of clients retry simultaneously, overwhelming the recovering service. Exponential backoff doubles the delay between each retry. Adding jitter (random delay) prevents synchronization of retries across clients.

Bulkhead Pattern

Named after ship bulkheads that prevent a hull breach from flooding the entire ship, the bulkhead pattern isolates components so that a failure in one does not cascade to others. In practice, this means using separate thread pools, connection pools, or even separate service instances for different dependencies. If the payment service is slow, it should not exhaust the thread pool used for the recommendation service.

Graceful Degradation

When a non-critical component fails, the system should continue operating with reduced functionality rather than failing entirely. For example, if the recommendation engine is down, the e-commerce site should still show product listings and allow purchases. It just will not show personalized recommendations. This requires explicitly categorizing features as critical vs non-critical and designing fallback behavior for each.

C# Example: Circuit Breaker Implementation

C#
public enum CircuitState { Closed, Open, HalfOpen }

public class CircuitBreaker
{
    private CircuitState _state = CircuitState.Closed;
    private int _failureCount;
    private DateTime _lastFailureTime;
    private readonly int _failureThreshold;
    private readonly TimeSpan _openDuration;

    public CircuitBreaker(int failureThreshold = 5, TimeSpan? openDuration = null)
    {
        _failureThreshold = failureThreshold;
        _openDuration = openDuration ?? TimeSpan.FromSeconds(30);
    }

    public async Task<T> ExecuteAsync<T>(Func<Task<T>> action)
    {
        if (_state == CircuitState.Open)
        {
            if (DateTime.UtcNow - _lastFailureTime > _openDuration)
                _state = CircuitState.HalfOpen;
            else
                throw new CircuitBreakerOpenException("Circuit is open");
        }

        try
        {
            var result = await action();

            if (_state == CircuitState.HalfOpen)
            {
                _state = CircuitState.Closed;
                _failureCount = 0;
            }

            return result;
        }
        catch (Exception)
        {
            _failureCount++;
            _lastFailureTime = DateTime.UtcNow;

            if (_failureCount >= _failureThreshold)
                _state = CircuitState.Open;

            throw;
        }
    }
}

// Usage
var breaker = new CircuitBreaker(failureThreshold: 3, openDuration: TimeSpan.FromSeconds(15));
try
{
    var result = await breaker.ExecuteAsync(async () =>
    {
        using var client = new HttpClient();
        return await client.GetStringAsync("https://api.external-service.com/data");
    });
}
catch (CircuitBreakerOpenException)
{
    // Return cached or default data
    return GetCachedFallback();
}

5. CAP Theorem and Consistency Models

The CAP theorem, proposed by Eric Brewer in 2000 and proven by Gilbert and Lynch in 2002, states that a distributed data store can provide at most two of three guarantees: Consistency, Availability, and Partition Tolerance. This theorem is the single most important theoretical framework in distributed systems because it forces you to make an explicit choice about what your system sacrifices.

Understanding the Three Properties

• Consistency (C): Every read receives the most recent write or an error. All nodes see the same data at the same time. This is the strongest form of consistency, sometimes called linearizability.
• Availability (A): Every request receives a non-error response, without the guarantee that it contains the most recent write. The system is always responsive.
• Partition Tolerance (P): The system continues to operate despite network partitions (messages lost or delayed between nodes). Network partitions are inevitable in distributed systems.

Since network partitions are a physical reality that cannot be prevented, you must choose between consistency and availability when a partition occurs. This gives you two practical choices:

CP Systems (Consistent + Partition-Tolerant)

When a network partition occurs, CP systems reject writes or reads to maintain consistency. They prioritize correctness over uptime. Examples include relational databases with synchronous replication (PostgreSQL, MySQL with InnoDB), distributed coordination services (ZooKeeper, etcd), and Google Spanner. These systems guarantee that if you read data, it is the most recent write. The trade-off is that during a partition, some requests will fail.

AP Systems (Available + Partition-Tolerant)

When a network partition occurs, AP systems continue accepting reads and writes but may return stale data. They prioritize uptime over immediate consistency. Examples include Cassandra, DynamoDB, CouchDB, and DNS. These systems guarantee that you always get a response, but the response might be slightly outdated. Eventual consistency means that once the partition heals, all replicas will converge to the same state, but there is a window where different replicas have different data.

C# Example: Consistency Levels in a Distributed Store

C#
public enum ConsistencyLevel
{
    One,          // Response from one replica
    Quorum,       // Response from (N/2 + 1) replicas
    All,          // Response from all replicas
    LocalQuorum   // Response from quorum in local datacenter
}

public class DistributedStore
{
    private readonly List<IReplica> _replicas;
    private readonly int _replicaCount;

    public DistributedStore(List<IReplica> replicas)
    {
        _replicas = replicas;
        _replicaCount = replicas.Count;
    }

    public async Task<T?> ReadAsync<T>(string key, ConsistencyLevel level)
    {
        var requiredResponses = level switch
        {
            ConsistencyLevel.One => 1,
            ConsistencyLevel.Quorum => (_replicaCount / 2) + 1,
            ConsistencyLevel.All => _replicaCount,
            ConsistencyLevel.LocalQuorum => (_replicaCount / 2) + 1,
            _ => 1
        };

        var tasks = _replicas.Select(r => r.GetAsync<T>(key));
        var results = await Task.WhenAll(tasks);
        var validResponses = results.Where(r => r != null).ToList();

        if (validResponses.Count < requiredResponses)
            throw new ConsistencyException(
                $"Required {requiredResponses} responses, got {validResponses.Count}");

        // Return the most recent version (highest timestamp)
        return validResponses
            .OrderByDescending(r => r!.Timestamp)
            .First()!.Value;
    }

    public async Task WriteAsync<T>(string key, T value, ConsistencyLevel level)
    {
        var requiredAcks = level switch
        {
            ConsistencyLevel.One => 1,
            ConsistencyLevel.Quorum => (_replicaCount / 2) + 1,
            ConsistencyLevel.All => _replicaCount,
            ConsistencyLevel.LocalQuorum => (_replicaCount / 2) + 1,
            _ => 1
        };

        var tasks = _replicas.Select(r => r.PutAsync(key, value));
        var acks = (await Task.WhenAll(tasks)).Count(success => success);

        if (acks < requiredAcks)
            throw new WriteConsistencyException(
                $"Required {requiredAcks} acks, got {acks}");
    }
}

Multi-Version Concurrency Control (MVCC)

MVCC is a technique used by many databases (PostgreSQL, MySQL InnoDB, CockroachDB) to provide consistent reads without locking. Each write creates a new version of the data with a timestamp. Readers see a consistent snapshot of the database at a specific point in time without blocking writers. This allows high throughput for both reads and writes simultaneously. The trade-off is increased storage (old versions must be retained until no reader needs them) and more complex garbage collection.

6. Load Balancing and Traffic Distribution

When you have multiple servers handling requests, you need a load balancer to distribute traffic across them. Load balancing is not just about spreading traffic evenly. It is about routing each request to the server best equipped to handle it, while detecting and removing unhealthy servers from the pool.

Load Balancer Layers

Load Balancing Algorithms

The algorithm determines which server receives each request. There is no single best algorithm. The right choice depends on your traffic patterns, server heterogeneity, and session requirements.

• Round Robin: Requests are distributed in sequence: Server 1, Server 2, Server 3, Server 1, Server 2, and so on. Simple, fair, and effective for homogeneous servers with similar request costs.
• Weighted Round Robin: Servers receive requests proportional to their configured weight. A server with weight 3 gets three times as many requests as one with weight 1. Useful when servers have different capacities.
• Least Connections: Each request goes to the server with the fewest active connections. This naturally adapts to varying request processing times. If one server is handling slow requests, it receives fewer new requests.
• IP Hash: A hash of the client IP determines which server receives the request. This provides session affinity, ensuring the same client always hits the same server. Useful for applications that store session state locally.
• Least Response Time: Requests go to the server with the lowest average response time. This requires the load balancer to track response times, adding overhead but improving user experience.

C# Example: Load Balancer with Health Checks

C#
public class LoadBalancer
{
    private readonly List<ServerInstance> _servers = new();
    private int _roundRobinIndex = 0;
    private readonly Timer _healthCheckTimer;

    public LoadBalancer(TimeSpan healthCheckInterval)
    {
        _healthCheckTimer = new Timer(
            async _ => await CheckHealthAsync(),
            null,
            TimeSpan.Zero,
            healthCheckInterval);
    }

    public void AddServer(string address, int weight = 1)
    {
        _servers.Add(new ServerInstance { Address = address, Weight = weight, IsHealthy = true });
    }

    public ServerInstance? NextServer()
    {
        var healthyServers = _servers.Where(s => s.IsHealthy).ToList();
        if (!healthyServers.Any()) return null;

        // Round Robin
        var server = healthyServers[_roundRobinIndex % healthyServers.Count];
        _roundRobinIndex++;
        return server;
    }

    public ServerInstance? LeastConnections()
    {
        return _servers
            .Where(s => s.IsHealthy)
            .OrderBy(s => s.ActiveConnections)
            .FirstOrDefault();
    }

    private async Task CheckHealthAsync()
    {
        using var client = new HttpClient { Timeout = TimeSpan.FromSeconds(3) };
        foreach (var server in _servers)
        {
            try
            {
                var response = await client.GetAsync($"{server.Address}/health");
                server.IsHealthy = response.IsSuccessStatusCode;
            }
            catch
            {
                server.IsHealthy = false;
            }
        }
    }
}

public class ServerInstance
{
    public string Address { get; set; } = "";
    public int Weight { get; set; } = 1;
    public bool IsHealthy { get; set; } = true;
    public int ActiveConnections { get; set; }
}

Global Load Balancing

For systems serving users across multiple continents, a global load balancer distributes traffic to the nearest or least-loaded region. DNS-based load balancing (AWS Route 53, Google Cloud DNS) routes users to the closest region based on their IP geolocation. Anycast routing (used by Cloudflare and CDN providers) advertises the same IP address from multiple locations, and BGP routing naturally directs each request to the nearest data center. The challenge with DNS-based load balancing is propagation delay; DNS changes can take minutes to hours to propagate globally.

Session Affinity

When an application stores session state on the server (user login sessions, shopping cart state in memory), requests from the same user must consistently route to the same server. This is called session affinity or sticky sessions. The load balancer uses cookies or IP hashing to maintain affinity. However, session affinity reduces load balancing effectiveness and complicates scaling and failover. The better approach is to externalize session state to a shared store like Redis, making your application servers stateless and any server capable of handling any request.

7. Caching Strategies

Caching is storing frequently accessed data in a fast, temporary storage layer to reduce the need to compute or fetch it from a slower source. It is the single most effective performance optimization in most systems. A well-placed cache can reduce response times from hundreds of milliseconds to single-digit milliseconds and reduce database load by orders of magnitude.

Where to Cache

Cache Invalidation Strategies

Cache invalidation is updating or removing cached data when the underlying data changes. Phil Karlton famously said, "There are only two hard things in Computer Science: cache invalidation and naming things." The three main strategies are:

• Write-Through: Data is written to the cache and the database simultaneously. The cache is always up-to-date, but write latency increases because both writes must complete. Best for data where consistency is critical.
• Write-Behind (Write-Back): Data is written to the cache first, then asynchronously flushed to the database. Writes are fast, but there is a window where the cache and database are inconsistent. If the cache crashes before flushing, data is lost. Best for write-heavy workloads that can tolerate brief inconsistency.
• Cache-Aside (Lazy Loading): The application checks the cache first. On a cache miss, it reads from the database and populates the cache. Writes go directly to the database and invalidate the cache. This is the most common pattern because it is simple and the cache only contains data that has been actually requested.

Cache Stampede Problem

When a popular cache entry expires, many concurrent requests may simultaneously detect the miss and attempt to rebuild the cache entry. This is called a cache stampede or thundering herd. All those requests hit the database simultaneously, potentially causing an outage. Solutions include: using locks so only one request rebuilds the cache while others wait, implementing probabilistic early expiration (refresh the cache before it expires with some probability), and using stale-while-revalidate (return stale data while asynchronously refreshing the cache).

C# Example: Cache-Aside Pattern with Redis

C#
public class CacheAsideService<T>
{
    private readonly IDatabase _redis;
    private readonly TimeSpan _defaultTtl;
    private readonly SemaphoreSlim _lock = new(1, 1);

    public CacheAsideService(IConnectionMultiplexer redis, TimeSpan? defaultTtl = null)
    {
        _redis = redis.GetDatabase();
        _defaultTtl = defaultTtl ?? TimeSpan.FromMinutes(5);
    }

    public async Task<T?> GetAsync(string key, Func<Task<T?>> loadFromSource)
    {
        // Try cache first
        var cached = await _redis.StringGetAsync(key);
        if (cached.HasValue)
            return JsonSerializer.Deserialize<T>(cached!);

        // Cache miss - load from source with lock to prevent stampede
        await _lock.WaitAsync();
        try
        {
            // Double-check after acquiring lock
            cached = await _redis.StringGetAsync(key);
            if (cached.HasValue)
                return JsonSerializer.Deserialize<T>(cached!);

            var data = await loadFromSource();
            if (data != null)
            {
                var serialized = JsonSerializer.Serialize(data);
                await _redis.StringSetAsync(key, serialized, _defaultTtl);
            }
            return data;
        }
        finally
        {
            _lock.Release();
        }
    }

    public async Task InvalidateAsync(string key)
    {
        await _redis.KeyDeleteAsync(key);
    }

    public async Task<T?> GetOrSetAsync(string key, Func<Task<T?>> factory, TimeSpan? ttl = null)
    {
        var cached = await _redis.StringGetAsync(key);
        if (cached.HasValue)
            return JsonSerializer.Deserialize<T>(cached!);

        var data = await factory();
        if (data != null)
            await _redis.StringSetAsync(key, JsonSerializer.Serialize(data), ttl ?? _defaultTtl);
        return data;
    }
}

// Usage
var cache = new CacheAsideService<Product>(redis, TimeSpan.FromMinutes(10));
var product = await cache.GetOrSetAsync(
    $"product:{productId}",
    async () => await _dbContext.Products.FindAsync(productId),
    TimeSpan.FromMinutes(30));

8. Database Design and Data Partitioning

The database is often the most critical component of any system. Choosing the right database technology, schema design, and partitioning strategy determines whether your system can scale to millions of users or collapses under a few thousand. The database landscape has expanded dramatically from the days of "just use MySQL" to include relational databases, document stores, key-value stores, column-family stores, graph databases, time-series databases, and more.

SQL vs NoSQL Decision Matrix

Data Partitioning (Sharding)

When a single database server cannot hold all your data or handle all your queries, you partition the data across multiple servers. This is called sharding. Each shard holds a subset of the total data, and each shard runs on its own database server. The challenge is choosing a shard key that distributes data evenly and supports your most common queries without requiring cross-shard operations.

Sharding Strategies

• Range-Based Sharding: Data is partitioned by ranges of the shard key. For example, users 1-1M go to Shard A, users 1M-2M go to Shard B. Simple to implement but can create hotspots if access patterns are skewed.
• Hash-Based Sharding: A hash function applied to the shard key determines the shard. Provides even distribution but makes range queries impossible across shards.
• Directory-Based Sharding: A lookup table maps each shard key to its shard. Flexible and allows dynamic rebalancing, but the directory itself becomes a potential bottleneck and single point of failure.

Read Replicas and CQRS

Most systems read data far more often than they write it. A common optimization is to create read replicas: copies of the database that receive asynchronous replication from the primary. All writes go to the primary, and reads are distributed across replicas. This scales read throughput without affecting write performance. The trade-off is replication lag; replicas may serve slightly stale data.

Command Query Responsibility Segregation (CQRS) formalizes this pattern by separating the read model from the write model into different services or databases. The write side uses a normalized, transactional database optimized for consistency. The read side uses a denormalized, denormalized view optimized for query performance, often powered by a search engine like Elasticsearch or a cache like Redis. This separation allows each side to be scaled and optimized independently.

C# Example: Database Connection Pool and Repository

C#
public interface IRepository<T> where T : class
{
    Task<T?> GetByIdAsync(int id);
    Task<IEnumerable<T>> GetAllAsync(int page, int pageSize);
    Task<T> CreateAsync(T entity);
    Task UpdateAsync(T entity);
    Task DeleteAsync(int id);
}

public class EfCoreRepository<T> : IRepository<T> where T : class
{
    private readonly IDbContextFactory<AppDbContext> _contextFactory;

    public EfCoreRepository(IDbContextFactory<AppDbContext> contextFactory)
    {
        _contextFactory = contextFactory;
    }

    public async Task<T?> GetByIdAsync(int id)
    {
        await using var context = await _contextFactory.CreateDbContextAsync();
        return await context.Set<T>().FindAsync(id);
    }

    public async Task<IEnumerable<T>> GetAllAsync(int page, int pageSize)
    {
        await using var context = await _contextFactory.CreateDbContextAsync();
        return await context.Set<T>()
            .Skip((page - 1) * pageSize)
            .Take(pageSize)
            .AsNoTracking()
            .ToListAsync();
    }

    public async Task<T> CreateAsync(T entity)
    {
        await using var context = await _contextFactory.CreateDbContextAsync();
        context.Set<T>().Add(entity);
        await context.SaveChangesAsync();
        return entity;
    }

    public async Task UpdateAsync(T entity)
    {
        await using var context = await _contextFactory.CreateDbContextAsync();
        context.Set<T>().Update(entity);
        await context.SaveChangesAsync();
    }

    public async Task DeleteAsync(int id)
    {
        await using var context = await _contextFactory.CreateDbContextAsync();
        var entity = await context.Set<T>().FindAsync(id);
        if (entity != null)
        {
            context.Set<T>().Remove(entity);
            await context.SaveChangesAsync();
        }
    }
}

// Register with connection pooling in DI
// services.AddDbContextFactory<AppDbContext>(options =>
//     options.UseNpgsql(connectionString,
//         npgsql => npgsql.EnableRetryOnFailure(3)),
//     lifetime: ServiceLifetime.Scoped);

Database Indexing Strategy

Proper indexing is the single most impactful database optimization. An index allows the database to find rows without scanning the entire table. However, every index slows down writes because it must be updated on each insert, update, and delete. The rule of thumb: index columns that appear in WHERE clauses, JOIN conditions, and ORDER BY clauses. For composite indexes, put the most selective column first. Monitor slow query logs to identify missing indexes, and use EXPLAIN ANALYZE to verify that indexes are being used as expected. Over-indexing is as dangerous as under-indexing because it bloats storage and degrades write performance.

9. Message Queues and Asynchronous Processing

Message queues decouple the production of work from its consumption. Instead of a service calling another service directly and waiting for a response, it publishes a message to a queue. The consuming service processes the message at its own pace. This decoupling provides temporal independence (the producer and consumer do not need to be available simultaneously), load leveling (the queue absorbs traffic spikes), and fault isolation (if the consumer is down, messages wait in the queue).

Message Queue Comparison

Message Processing Patterns

Point-to-Point (Task Queue)

One message is consumed by exactly one consumer. Ideal for distributing work across multiple worker instances. If you have a thousand image processing jobs, each worker pulls one job from the queue, processes it, and acknowledges completion. Adding more workers increases throughput linearly.

Publish-Subscribe (Fan-Out)

One message is delivered to all subscribers. When an order is placed, the order service publishes an "OrderPlaced" event. The notification service subscribes to send a confirmation email. The inventory service subscribes to reserve stock. The analytics service subscribes to record the transaction. Each service independently processes the event without affecting others.

Dead Letter Queue (DLQ)

Messages that fail processing after a configurable number of retries are moved to a dead letter queue. This prevents poison messages from blocking the main queue. Operators can inspect the DLQ, fix the underlying issue, and replay the messages. Every production queue should have a DLQ.

C# Example: Background Worker with Message Queue

C#
public class OrderMessage
{
    public string OrderId { get; set; } = "";
    public decimal Amount { get; set; }
    public string CustomerEmail { get; set; } = "";
    public DateTime Timestamp { get; set; }
}

public class OrderProcessingWorker : BackgroundService
{
    private readonly IConsumer<string, OrderMessage> _consumer;
    private readonly IServiceScopeFactory _scopeFactory;
    private readonly ILogger<OrderProcessingWorker> _logger;

    public OrderProcessingWorker(
        IConsumer<string, OrderMessage> consumer,
        IServiceScopeFactory scopeFactory,
        ILogger<OrderProcessingWorker> logger)
    {
        _consumer = consumer;
        _scopeFactory = scopeFactory;
        _logger = logger;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            try
            {
                var result = await _consumer.Consume(stoppingToken);

                using var scope = _scopeFactory.CreateScope();
                var processor = scope.ServiceProvider
                    .GetRequiredService<IOrderProcessor>();

                await processor.ProcessAsync(result.Message);

                _logger.LogInformation(
                    "Processed order {OrderId} successfully",
                    result.Message.OrderId);
            }
            catch (ConsumeException ex)
            {
                _logger.LogError(ex, "Failed to process message");
                // Message goes to retry queue, then DLQ after max retries
            }
        }
    }
}

// Dead Letter Queue handler
public class DeadLetterHandler
{
    public async Task HandleDeadLetterAsync(OrderMessage message, string reason)
    {
        _logger.LogWarning(
            "Message {OrderId} sent to DLQ. Reason: {Reason}",
            message.OrderId, reason);

        await _deadLetterStore.SaveAsync(new DeadLetterEntry
        {
            OriginalMessage = message,
            Reason = reason,
            FailedAt = DateTime.UtcNow,
            RetryCount = MaxRetries
        });

        await _alertService.NotifyOpsTeamAsync(
            $"Order {message.OrderId} requires manual review");
    }
}

10. Microservices vs Monolith Architecture

The monolith vs microservices debate is one of the most consequential architectural decisions in system design. A monolith is a single, unified application where all components run in a single process. Microservices decompose the application into small, independent services that communicate over a network. Neither is universally better. The right choice depends on your team size, domain complexity, and operational maturity.

Monolith Pros and Cons

Microservices Pros and Cons

The Modular Monolith: Best of Both Worlds?

A modular monolith organizes a single deployable application into well-defined, loosely-coupled modules with clear boundaries. Each module has its own internal domain model and data access. Modules communicate through well-defined interfaces, not direct database access. This gives you the simplicity of a monolith (single deployment, no network calls) with the modularity benefits of microservices (clear boundaries, independent development). Shopify and Basecamp have publicly shared their successful use of modular monoliths.

The key rule: if you cannot cleanly separate your domain into modules within a monolith, microservices will not magically fix that. Decomposition is a domain problem, not a technology problem. Get your domain boundaries right first, then decide whether to deploy them as modules in a monolith or as separate services.

When to Migrate from Monolith to Microservices

• When team size exceeds 8-10 engineers working in the same codebase
• When different components have different scaling requirements
• When you need technology diversity (different languages or frameworks per service)
• When deployment of one component should not require deploying everything
• When domain boundaries are well understood and stable

11. API Design and Versioning

APIs are the contracts between services, between the frontend and backend, and between your platform and external consumers. A well-designed API is intuitive, consistent, backward-compatible, and documented. A poorly designed API creates confusion, frustrates developers, and becomes a maintenance nightmare as it evolves.

REST API Design Principles

• Use nouns, not verbs: /users not /getUsers. The HTTP method (GET, POST, PUT, DELETE) already describes the action.
• Nest resources for relationships: /users/123/orders for orders belonging to user 123.
• Use query parameters for filtering: /products?category=electronics&sort=price
• Return appropriate HTTP status codes: 200 for success, 201 for created, 400 for bad request, 404 for not found, 500 for server error.
• Paginate list endpoints: /users?page=2&limit=50 or use cursor-based pagination for large datasets.
• Use HATEOAS links: Include links to related resources in responses so clients do not need to construct URLs.

API Versioning Strategies

C# Example: Versioned API Controller

C#
[ApiController]
[Route("api/v{version:apiVersion}/[controller]")]
[ApiVersion("1.0")]
[ApiVersion("2.0")]
public class UsersController : ControllerBase
{
    private readonly IUserService _userService;

    public UsersController(IUserService userService)
    {
        _userService = userService;
    }

    // V1: Returns flat user object
    [HttpGet("{id}")]
    [MapToApiVersion("1.0")]
    public async Task<ActionResult<UserV1Response>> GetUserV1(int id)
    {
        var user = await _userService.GetByIdAsync(id);
        if (user == null) return NotFound();

        return Ok(new UserV1Response
        {
            Id = user.Id,
            Name = user.FullName,
            Email = user.Email
        });
    }

    // V2: Returns nested user object with profile
    [HttpGet("{id}")]
    [MapToApiVersion("2.0")]
    public async Task<ActionResult<UserV2Response>> GetUserV2(int id)
    {
        var user = await _userService.GetByIdAsync(id);
        if (user == null) return NotFound();

        return Ok(new UserV2Response
        {
            Id = user.Id,
            Profile = new ProfileDto
            {
                FirstName = user.FirstName,
                LastName = user.LastName,
                Avatar = user.AvatarUrl
            },
            Email = user.Email,
            Preferences = user.Preferences
        });
    }

    // Deprecated V1 endpoint with sunset header
    [HttpGet("{id}")]
    [MapToApiVersion("1.0")]
    [Obsolete("Use V2 API instead. V1 will be removed on 2026-12-01.")]
    public async Task<ActionResult> GetUserV1WithSunset(int id)
    {
        Response.Headers.Add("Sunset", "Sat, 01 Dec 2026 00:00:00 GMT");
        Response.Headers.Add("Deprecation", "true");
        Response.Headers.Add("Link", $"/api/v2/users/{id}; rel=\"successor-version\"");
        return await GetUserV1(id);
    }
}

public class UserV1Response
{
    public int Id { get; set; }
    public string Name { get; set; } = "";
    public string Email { get; set; } = "";
}

public class UserV2Response
{
    public int Id { get; set; }
    public ProfileDto Profile { get; set; } = new();
    public string Email { get; set; } = "";
    public UserPreferences Preferences { get; set; } = new();
}

GraphQL: An Alternative to REST

GraphQL allows clients to request exactly the data they need in a single request. Instead of multiple REST endpoints returning fixed data structures, one GraphQL endpoint accepts a query that specifies the fields and relationships to return. This eliminates over-fetching (getting data you do not need) and under-fetching (needing multiple requests to get related data). The trade-off is added complexity on the server (query parsing, validation, resolvers) and potential for expensive queries if no query complexity limits are enforced. GraphQL is most beneficial when clients have diverse data needs or when network latency makes multiple REST requests expensive.

Rate Limiting and Throttling

Every public API must implement rate limiting to prevent abuse and ensure fair usage. Common strategies include fixed window (100 requests per minute, resetting at the minute boundary), sliding window (100 requests in any rolling 60-second period), and token bucket (each user gets tokens that replenish over time, and each request consumes one token). Rate limiting headers (X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset) inform clients of their current limits. When the limit is exceeded, return HTTP 429 Too Many Requests with a Retry-After header.

12. Security Fundamentals

Security is not a feature you add at the end. It is a property of the entire system that must be considered from the first design decision. A single security breach can destroy user trust, result in regulatory fines, and end careers. Every system design discussion should include security considerations: authentication, authorization, data protection, and defense in depth.

Authentication vs Authorization

OWASP Top 10 Threats

The OWASP Top 10 is the standard awareness document for web application security. Every system designer should know these threats and how to mitigate them:

1. Broken Access Control: Users acting beyond their permissions. Mitigate with server-side authorization checks on every request.
2. Cryptographic Failures: Sensitive data exposed through weak encryption. Use TLS everywhere, encrypt data at rest, never store passwords in plain text.
3. Injection: SQL, NoSQL, OS command injection. Use parameterized queries, input validation, and prepared statements.
4. Insecure Design: Missing security controls in the architecture. Threat model during design, not after deployment.
5. Security Misconfiguration: Default credentials, unnecessary features enabled. Automate configuration, remove defaults.

C# Example: JWT Authentication Middleware

C#
public class JwtTokenService
{
    private readonly JwtSettings _settings;

    public JwtTokenService(IOptions<JwtSettings> settings)
    {
        _settings = settings.Value;
    }

    public string GenerateAccessToken(User user, IList<string> roles)
    {
        var claims = new List<Claim>
        {
            new(ClaimTypes.NameIdentifier, user.Id.ToString()),
            new(ClaimTypes.Email, user.Email),
            new(JwtRegisteredClaimNames.Jti, Guid.NewGuid().ToString()),
            new(JwtRegisteredClaimNames.Iat,
                DateTimeOffset.UtcNow.ToUnixTimeSeconds().ToString(),
                ClaimValueTypes.Integer64)
        };

        claims.AddRange(roles.Select(role =>
            new Claim(ClaimTypes.Role, role)));

        var key = new SymmetricSecurityKey(
            Encoding.UTF8.GetBytes(_settings.SecretKey));
        var credentials = new SigningCredentials(
            key, SecurityAlgorithms.HmacSha256);

        var token = new JwtSecurityToken(
            issuer: _settings.Issuer,
            audience: _settings.Audience,
            claims: claims,
            expires: DateTime.UtcNow.AddMinutes(_settings.AccessTokenExpirationMinutes),
            signingCredentials: credentials);

        return new JwtSecurityTokenHandler().WriteToken(token);
    }

    public string GenerateRefreshToken()
    {
        var randomBytes = new byte[64];
        using var rng = RandomNumberGenerator.Create();
        rng.GetBytes(randomBytes);
        return Convert.ToBase64String(randomBytes);
    }

    public ClaimsPrincipal? ValidateToken(string token)
    {
        var handler = new JwtSecurityTokenHandler();
        var parameters = new TokenValidationParameters
        {
            ValidateIssuer = true,
            ValidateAudience = true,
            ValidateLifetime = true,
            ValidateIssuerSigningKey = true,
            ValidIssuer = _settings.Issuer,
            ValidAudience = _settings.Audience,
            IssuerSigningKey = new SymmetricSecurityKey(
                Encoding.UTF8.GetBytes(_settings.SecretKey)),
            ClockSkew = TimeSpan.FromMinutes(1)
        };

        return handler.ValidateToken(token, parameters, out _);
    }
}

Defense in Depth

Defense in depth means layering multiple security controls so that if one layer is breached, others still protect the system. The layers include: network security (firewalls, VPCs, network segmentation), transport security (TLS everywhere), application security (input validation, parameterized queries, CSRF protection), data security (encryption at rest, access controls, audit logging), and operational security (monitoring, alerting, incident response plans). No single layer is sufficient. Each layer provides an additional barrier that an attacker must overcome.

13. Observability: Logging, Monitoring, and Alerting

Observability is the ability to understand the internal state of a system by examining its external outputs. A system is observable when you can diagnose problems, understand performance bottlenecks, and predict failures without deploying new code or adding debug logging. The three pillars of observability are logs, metrics, and traces.

The Three Pillars of Observability

Key Metrics to Monitor (The RED Method)

• Rate: Requests per second for each service endpoint
• Errors: Error rate (percentage of requests that return 5xx or timeout)
• Duration: Latency distribution (p50, p95, p99 response times)

Additionally, monitor system-level metrics: CPU usage, memory usage, disk I/O, network I/O, and garbage collection statistics. Set alerts based on SLOs, not raw thresholds. An alert for "p99 latency exceeds 500ms for 5 minutes" is more meaningful than "CPU exceeds 80%" because it directly measures user impact.

Structured Logging

Structured logging outputs logs as JSON objects rather than free-form text. This makes logs machine-parseable, enabling efficient filtering, aggregation, and analysis. Every log entry should include: timestamp, log level, service name, request ID (for trace correlation), user ID (for debugging user-specific issues), and the message. Avoid logging sensitive data like passwords, tokens, or personal information.

C# Example: Structured Logging and Custom Middleware

C#
public class RequestLoggingMiddleware
{
    private readonly RequestDelegate _next;
    private readonly ILogger<RequestLoggingMiddleware> _logger;

    public RequestLoggingMiddleware(
        RequestDelegate next,
        ILogger<RequestLoggingMiddleware> logger)
    {
        _next = next;
        _logger = logger;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        var requestId = Guid.NewGuid().ToString();
        var stopwatch = Stopwatch.StartNew();

        context.TraceIdentifier = requestId;

        using (_logger.BeginScope(new Dictionary<string, object>
        {
            ["RequestId"] = requestId,
            ["Method"] = context.Request.Method,
            ["Path"] = context.Request.Path,
            ["UserAgent"] = context.Request.Headers.UserAgent.ToString(),
            ["RemoteIp"] = context.Connection.RemoteIpAddress?.ToString()
        }))
        {
            try
            {
                await _next(context);
                stopwatch.Stop();

                _logger.LogInformation(
                    "HTTP {Method} {Path} responded {StatusCode} in {ElapsedMs}ms",
                    context.Request.Method,
                    context.Request.Path,
                    context.Response.StatusCode,
                    stopwatch.ElapsedMilliseconds);
            }
            catch (Exception ex)
            {
                stopwatch.Stop();
                _logger.LogError(ex,
                    "HTTP {Method} {Path} failed after {ElapsedMs}ms",
                    context.Request.Method,
                    context.Request.Path,
                    stopwatch.ElapsedMilliseconds);
                throw;
            }
        }
    }
}

// Custom health metrics collector
public class ApplicationMetrics
{
    private readonly Counter<long> _requestCounter;
    private readonly Histogram<double> _requestDuration;
    private readonly Gauge<long> _activeConnections;

    public ApplicationMetrics(IMeterFactory meterFactory)
    {
        var meter = meterFactory.Create("MyApp");

        _requestCounter = meter.CreateCounter<long>(
            "app.requests.total",
            description: "Total number of requests");

        _requestDuration = meter.CreateHistogram<double>(
            "app.request.duration",
            unit: "ms",
            description: "Request duration in milliseconds");

        _activeConnections = meter.CreateGauge<long>(
            "app.connections.active",
            description: "Number of active connections");
    }

    public void RecordRequest(string method, string endpoint, int statusCode, double durationMs)
    {
        _requestCounter.Add(1,
            new KeyValuePair<string, object?>("method", method),
            new KeyValuePair<string, object?>("endpoint", endpoint),
            new KeyValuePair<string, object?>("status", statusCode));

        _requestDuration.Record(durationMs,
            new KeyValuePair<string, object?>("method", method),
            new KeyValuePair<string, object?>("endpoint", endpoint));
    }
}

Distributed Tracing

In a microservices architecture, a single user request may traverse dozens of services. When a request is slow, identifying which service is the bottleneck requires distributed tracing. Tools like Jaeger, Zipkin, and AWS X-Ray propagate a trace ID across service boundaries, recording timing information at each hop. This creates a waterfall visualization showing exactly where time is spent. Without distributed tracing, debugging latency in a microservices architecture is like trying to find a specific car in a traffic jam without traffic cameras.

14. Capacity Planning and Estimation

Capacity planning is estimating the resources your system needs to handle expected load. In system design interviews, back-of-the-envelope estimation demonstrates structured thinking. In production, it prevents both over-provisioning (wasting money) and under-provisioning (crashing under load). The key is starting from first principles: user count, request patterns, data volume, and growth rate.

The Estimation Framework

1. Start with users: How many daily active users (DAU)? What fraction are concurrent (online at the same time)?
2. Estimate requests per user: How many requests does each user make per day? What is the read-to-write ratio?
3. Compute peak QPS: Average QPS = (DAU * requests_per_user) / 86400. Peak QPS = Average QPS * 2-3 (peak factor).
4. Estimate data per request: How much data does each request read or write? What is the payload size?
5. Calculate storage: Daily storage = requests_per_day * data_per_request. Multiply by retention period for total storage.
6. Account for growth: Apply year-over-year growth rate to estimate capacity needed in 12-24 months.

Estimation Reference Table

C# Example: Capacity Estimation Calculator

C#
public class SystemEstimation
{
    public double DailyActiveUsers { get; set; }
    public double RequestsPerUserPerDay { get; set; }
    public double ReadWriteRatio { get; set; } // e.g. 10 for 10:1 read:write
    public double BytesPerRequest { get; set; }
    public double PeakMultiplier { get; set; } = 3.0;
    public double GrowthRateYearly { get; set; } = 0.5; // 50% growth

    public double AverageQps =>
        (DailyActiveUsers * RequestsPerUserPerDay) / 86400;

    public double PeakQps => AverageQps * PeakMultiplier;

    public double ReadQps => PeakQps * ReadWriteRatio / (ReadWriteRatio + 1);
    public double WriteQps => PeakQps / (ReadWriteRatio + 1);

    public double DailyStorageBytes =>
        DailyActiveUsers * RequestsPerUserPerDay * BytesPerRequest;

    public double YearlyStorageGb =>
        (DailyStorageBytes * 365 * (1 + GrowthRateYearly)) / (1024 * 1024 * 1024);

    public void PrintReport()
    {
        Console.WriteLine("=== System Capacity Estimation ===");
        Console.WriteLine($"Daily Active Users:      {DailyActiveUsers:N0}");
        Console.WriteLine($"Requests/User/Day:       {RequestsPerUserPerDay}");
        Console.WriteLine($"Average QPS:             {AverageQps:N0}");
        Console.WriteLine($"Peak QPS:                {PeakQps:N0}");
        Console.WriteLine($"  Read QPS:              {ReadQps:N0}");
        Console.WriteLine($"  Write QPS:             {WriteQps:N0}");
        Console.WriteLine($"Daily Storage:           {DailyStorageBytes / (1024*1024):N0} MB");
        Console.WriteLine($"Yearly Storage (w/ growth): {YearlyStorageGb:N0} GB");
    }
}

// Example: Design a URL shortener for 100M DAU
var estimation = new SystemEstimation
{
    DailyActiveUsers = 100_000_000,
    RequestsPerUserPerDay = 10,
    ReadWriteRatio = 10,  // 10 reads per 1 write (mostly URL redirections)
    BytesPerRequest = 500,
    GrowthRateYearly = 0.3
};
estimation.PrintReport();
// Peak QPS ~ 34,722
// Daily storage ~ 47 GB
// Yearly storage ~ 22 TB

AWS Service Capacity Quick Reference

15. Trade-offs and Architecture Decision Records

System design is the art of making trade-offs under uncertainty. There is no perfect architecture. Every decision involves sacrificing one quality to gain another. The skill is identifying which trade-offs matter for your specific context and documenting them clearly so future engineers understand why the choice was made.

Common Trade-offs in System Design

Architecture Decision Records (ADRs)

An Architecture Decision Record is a lightweight document that captures a single architectural decision, its context, and its consequences. ADRs prevent the "why did we do it this way?" problem that plagues every long-lived codebase. They are not design documents. They are brief, focused records of decisions that have been made.

C# Example: ADR Template as Code

C#
public class ArchitectureDecisionRecord
{
    public string Id { get; set; } = "";
    public string Title { get; set; } = "";
    public string Status { get; set; } = "Proposed"; // Proposed, Accepted, Deprecated, Superseded
    public DateTime Date { get; set; }
    public string Context { get; set; } = "";
    public List<string> OptionsConsidered { get; set; } = new();
    public string Decision { get; set; } = "";
    public List<string> Consequences { get; set; } = new();
    public string? SupersededBy { get; set; }

    public override string ToString()
    {
        return $"""
        # ADR {Id}: {Title}

        **Status:** {Status}
        **Date:** {Date:yyyy-MM-dd}

        ## Context
        {Context}

        ## Options Considered
        {string.Join("\n", OptionsConsidered.Select(o => $"- {o}"))}

        ## Decision
        {Decision}

        ## Consequences
        {string.Join("\n", Consequences.Select(c => $"- {c}"))}
        """;
    }
}

// Example ADR
var adr = new ArchitectureDecisionRecord
{
    Id = "ADR-007",
    Title = "Use event-driven architecture for order processing",
    Status = "Accepted",
    Date = DateTime.UtcNow,
    Context = """
        The order processing pipeline currently uses synchronous HTTP calls
        between services. Under peak load (Black Friday), the synchronous
        chain causes cascading timeouts. The payment service's 2-second
        average latency blocks the entire pipeline.
        """,
    OptionsConsidered = new()
    {
        "Option A: Add circuit breakers and retries to synchronous calls",
        "Option B: Migrate to event-driven architecture with Kafka",
        "Option C: Implement a hybrid approach with sync for critical paths"
    },
    Decision = """
        Adopt event-driven architecture using Apache Kafka for the order
        processing pipeline. Each step publishes events that downstream
        services consume independently.
        """,
    Consequences = new()
    {
        "+ Orders can be processed even when downstream services are slow",
        "+ Each service scales independently based on its own queue depth",
        "+ Better observability through event lineage tracking",
        "- Eventual consistency for order status (acceptable for this use case)",
        "- Increased operational complexity (Kafka cluster management)",
        "- Need to handle duplicate events (idempotent consumers)"
    }
};

Console.WriteLine(adr);

The Reversibility Principle

Prefer reversible decisions over irreversible ones. Choosing a logging library is easily reversible; choose quickly. Choosing a database is hard to reverse; invest more time. Jeff Bezos calls these "one-way doors" (irreversible, require deep analysis) and "two-way doors" (reversible, decide quickly and course-correct). Most architectural decisions are two-way doors that teams treat as one-way doors, wasting months of analysis on decisions that can be changed in days.

16. The System Design Interview Framework

System design interviews evaluate your ability to design complex systems under time constraints. The interview is not about finding the "right" answer. There is no right answer. It is about demonstrating structured thinking, identifying requirements, exploring trade-offs, and communicating your design clearly. The framework below provides a repeatable structure for any system design question.

The 4-Step Framework

1. Requirements Gathering (5 minutes): Clarify functional requirements (what the system does) and non-functional requirements (latency, availability, consistency, scale). Ask about expected user count, traffic patterns, and data retention. This step is the most important because designing the wrong system perfectly is worse than designing the right system imperfectly.
2. Back-of-the-Envelope Estimation (5 minutes): Estimate key numbers: QPS, storage, bandwidth. These estimates inform every subsequent decision. They tell you whether you need a single database or a sharded cluster, whether caching is essential or optional, and whether synchronous or asynchronous processing is appropriate.
3. High-Level Design (15 minutes): Draw the major components and how they interact. Start with the simplest design that meets the requirements. Identify the data model and API contracts. This is where you demonstrate breadth of knowledge across all the concepts in this guide.
4. Deep Dive (15 minutes): The interviewer will ask you to go deeper on specific components. This is where you demonstrate depth. Discuss database schema choices, caching strategies, message queue selection, and failure modes. This is where trade-off analysis matters most.

Common Interview Mistakes

• Jumping to design too quickly: Spend at least 5 minutes on requirements. Designing without clear requirements is the #1 reason candidates fail.
• Ignoring non-functional requirements: Every system has latency, availability, and consistency requirements. Ignoring them suggests you do not think about production.
• Not discussing trade-offs: Presenting only one option without alternatives suggests tunnel vision. Always mention at least two approaches and explain why you chose one.
• Over-engineering: Designing for Google-scale when the problem asks for a simpler solution wastes time and suggests you cannot make pragmatic decisions.
• Ignoring failure modes: What happens when a service goes down? When the database is overloaded? When a network partition occurs? Candidates who discuss failure modes stand out.

17. Real-World Architecture Patterns

Understanding common architecture patterns gives you a vocabulary for discussing solutions and a starting point for new designs. These patterns are battle-tested solutions to recurring problems in distributed systems.

CQRS + Event Sourcing

Command Query Responsibility Segregation (CQRS) separates read and write operations into different models. Event Sourcing stores every state change as an immutable event rather than the current state. Together, they create a system where the write side produces events, the read side consumes events to build optimized query models, and the full history of changes is preserved for audit and replay. This pattern is used by financial systems, collaboration tools (like Google Docs), and any system where the history of changes is as important as the current state.

Strangler Fig Pattern

The Strangler Fig pattern is the safest way to migrate from a monolith to microservices. Instead of rewriting the monolith from scratch (which almost always fails), you gradually replace specific functionality with new services. A proxy or gateway routes traffic to either the old monolith or the new service based on the request type. Over time, more functionality migrates to new services until the monolith is fully replaced. This pattern is named after strangler fig trees that grow around a host tree, eventually replacing it.

Sidecar Pattern

The Sidecar pattern deploys helper components alongside the main application in the same deployment unit (pod in Kubernetes). The sidecar handles cross-cutting concerns like logging, monitoring, TLS termination, and service mesh communication (Envoy, Istio). The main application focuses on business logic. This keeps the application code clean and allows infrastructure concerns to be updated independently.

CQRS + Event Sourcing C# Example

C#
// Domain Event
public abstract record DomainEvent(Guid AggregateId, DateTime Timestamp, int Version);

public record OrderPlacedEvent(
    Guid OrderId,
    Guid CustomerId,
    List<OrderItemDto> Items,
    DateTime Timestamp,
    int Version) : DomainEvent(OrderId, Timestamp, Version);

public record OrderShippedEvent(
    Guid OrderId,
    string TrackingNumber,
    DateTime Timestamp,
    int Version) : DomainEvent(OrderId, Timestamp, Version);

// Event Store
public class EventStore : IEventStore
{
    private readonly AppDbContext _context;

    public async Task AppendAsync<T>(T domainEvent) where T : DomainEvent
    {
        var entry = new EventEntry
        {
            Id = Guid.NewGuid(),
            AggregateId = domainEvent.AggregateId,
            EventType = typeof(T).Name,
            Data = JsonSerializer.Serialize(domainEvent),
            Version = domainEvent.Version,
            Timestamp = domainEvent.Timestamp
        };

        _context.Events.Add(entry);
        await _context.SaveChangesAsync();
    }

    public async Task<List<DomainEvent>> GetEventsAsync(Guid aggregateId)
    {
        return await _context.Events
            .Where(e => e.AggregateId == aggregateId)
            .OrderBy(e => e.Version)
            .Select(e => DeserializeEvent(e.EventType, e.Data))
            .ToListAsync();
    }

    public async Task RebuildAggregateAsync<T>(Guid aggregateId) where T : AggregateRoot, new()
    {
        var events = await GetEventsAsync(aggregateId);
        var aggregate = new T();
        foreach (var domainEvent in events)
            aggregate.Apply(domainEvent);
        // Save rebuilt state to read model
    }
}

// Read Model Projection
public class OrderProjection : INotificationHandler<OrderPlacedEvent>
{
    private readonly ReadDbContext _readDb;

    public async Task Handle(OrderPlacedEvent notification, CancellationToken ct)
    {
        var readModel = new OrderReadModel
        {
            OrderId = notification.OrderId,
            CustomerId = notification.CustomerId,
            ItemCount = notification.Items.Count,
            TotalAmount = notification.Items.Sum(i => i.Price * i.Quantity),
            Status = "Placed",
            PlacedAt = notification.Timestamp
        };

        _readDb.Orders.Add(readModel);
        await _readDb.SaveChangesAsync(ct);
    }
}

Event-Driven Microservices Pattern

In event-driven microservices, services communicate primarily through events rather than synchronous API calls. When a service performs an action, it publishes an event. Other services subscribe to events they care about and react accordingly. This creates loose coupling because producers do not know about consumers and vice versa. The trade-off is eventual consistency and the complexity of event choreography. For complex business processes, consider using a saga pattern (orchestration or choreography) to coordinate multi-step workflows across services.

Saga Pattern for Distributed Transactions

Distributed transactions across multiple services cannot use traditional two-phase commit (2PC) without severe performance and availability penalties. The saga pattern provides an alternative by breaking a distributed transaction into a sequence of local transactions, each with a compensating action. If any step fails, the compensating actions execute in reverse order to undo the completed steps. In orchestration sagas, a central coordinator directs the workflow. In choreography sagas, each service listens for events and decides its next action independently.

18. Interview Q&A: 20 Questions Every Engineer Should Answer

These questions cover the most frequently asked system design topics in senior-level interviews at top tech companies. Study each answer thoroughly and practice articulating the concepts clearly.

Q1: What is the difference between scalability and availability?

Scalability is the ability of a system to handle increased load by adding resources, either vertically (more powerful machines) or horizontally (more machines). Availability is the percentage of time a system is operational and accessible. A system can be scalable but not available if it crashes frequently despite handling high throughput. A system can be available but not scalable if it handles low traffic reliably but fails under growth. They are complementary but independent properties that must be engineered separately.

Q2: Explain the CAP theorem with real examples.

The CAP theorem states that a distributed system can provide at most two of Consistency, Availability, and Partition Tolerance. Since network partitions are inevitable, you choose between CP and AP. CP example: PostgreSQL with synchronous replication rejects writes during a partition to maintain consistency. AP example: Cassandra continues accepting writes during a partition but may return stale data. Most systems choose AP and use application-level consistency mechanisms (read-your-writes, quorum reads) to meet consistency requirements without sacrificing availability.

Q3: When would you use a message queue vs a direct API call?

Use a direct API call when you need an immediate synchronous response and the calling service can wait. Use a message queue when: the work is time-consuming and the caller should not wait, you need to buffer traffic spikes, the consumer may be temporarily unavailable, you want to fan out to multiple consumers, or you need guaranteed processing even if the producer crashes. The trade-off is increased complexity, eventual consistency, and the need for idempotent consumers.

Q4: How do you design a rate limiter?

Implement the token bucket algorithm: each user gets a bucket with a fixed number of tokens that replenishes at a constant rate. Each request consumes one token. If the bucket is empty, the request is rejected. Store the bucket state in Redis for distributed rate limiting. Use sliding window counters for more accurate counting than fixed windows. For distributed systems, consider using Redis with Lua scripts for atomic operations. Key design decisions: what to use as the rate limit key (user ID, API key, IP), the window size, the limit per window, and what happens when the limit is exceeded (reject with 429, queue for later processing).

Q5: How do you handle database scaling?

Scale reads with read replicas: create multiple copies of the database that receive asynchronous replication from the primary. Distribute read queries across replicas. Scale writes with sharding: partition data across multiple database servers, each holding a subset of the total data. Choose a shard key that distributes writes evenly and supports your query patterns. Use caching aggressively to reduce database load. Consider moving to a purpose-built database for specific workloads (Redis for sessions, Elasticsearch for search, time-series databases for metrics).

Q6: What is the difference between horizontal and vertical scaling?

Vertical scaling adds more resources (CPU, RAM, storage) to a single machine. It is simple but has physical limits and creates a single point of failure. Horizontal scaling adds more machines to distribute the load. It is theoretically unlimited but introduces distributed systems complexity (load balancing, data consistency, failure handling). Start vertical for simplicity, go horizontal when you hit limits or need high availability. Most production systems combine both: each server is vertically scaled to a reasonable size, and multiple copies run behind a load balancer.

Q7: Explain the trade-offs between SQL and NoSQL databases.

SQL databases (PostgreSQL, MySQL) provide strong consistency, ACID transactions, rich query capabilities with JOINs, and a mature ecosystem. They scale vertically and horizontally through read replicas. Best for complex queries, financial data, and applications requiring strict consistency. NoSQL databases (MongoDB, Cassandra, DynamoDB) provide flexible schemas, horizontal scaling with built-in sharding, higher write throughput, and simpler data models. Best for high-volume writes, evolving schemas, and globally distributed data. The trade-off is reduced consistency guarantees and limited query capabilities compared to SQL.

Q8: How do you implement caching effectively?

Use the cache-aside pattern: check the cache first, on miss load from database and populate the cache. Set TTLs on all cache entries to prevent stale data accumulation. Invalidate cache on writes. Protect against cache stampede with locks or probabilistic early expiration. Cache data that is frequently accessed, expensive to compute, and tolerant of brief staleness. Do not cache rapidly changing data or data that must be real-time. Use multiple cache layers (browser, CDN, application cache, database buffer pool) for maximum impact.

Q9: What is the difference between latency and throughput?

Latency is the time it takes to serve a single request, measured in milliseconds. Throughput is the number of requests the system can serve per second. They are related but independent. You can have low latency with low throughput (fast but few concurrent requests) or high latency with high throughput (slow but processing many requests concurrently). Optimizing one can hurt the other: batch processing increases throughput but individual request latency increases. The right balance depends on your use case: user-facing APIs prioritize low latency, while batch data pipelines prioritize throughput.

Q10: How do you design for fault tolerance?

Eliminate single points of failure with redundancy at every layer. Use circuit breakers to prevent cascading failures when a dependency is down. Implement retries with exponential backoff and jitter for transient failures. Use the bulkhead pattern to isolate failures. Design graceful degradation: when non-critical components fail, disable them and continue serving core functionality. Test failure modes through chaos engineering. Define and measure error budgets. Every component should have a failure mode that is tested and documented.

Q11: What is a service mesh and when do you need one?

A service mesh (Istio, Linkerd) provides infrastructure-level networking capabilities for microservices. It handles service-to-service communication, including load balancing, retries, circuit breaking, mTLS encryption, and observability. Each service gets a sidecar proxy that intercepts all network traffic. A service mesh is beneficial when you have many microservices (50+) and need consistent networking policies across all of them. For smaller systems, the overhead of managing a service mesh is not justified. Use simpler approaches like API gateways and library-based solutions.

Q12: How do you handle data consistency in microservices?

Use the Saga pattern for distributed transactions: break the transaction into local steps with compensating actions for rollback. For read consistency, implement read-your-writes consistency by routing reads through the same service that handled the write, or by using version vectors. For reporting and analytics, use event sourcing to build eventually consistent read models. For cross-service queries, use API composition (aggregate results from multiple services) or maintain denormalized read models that are updated asynchronously.

Q13: Explain the difference between synchronous and asynchronous communication.

In synchronous communication (REST, gRPC), the caller blocks until the response is received. It is simple, provides immediate feedback, and works well for short operations. In asynchronous communication (message queues, event streaming), the caller sends a message and continues without waiting for a response. The consumer processes the message independently. Asynchronous communication is better for long-running operations, decoupling services, handling traffic spikes, and building resilient systems. The trade-off is increased complexity and eventual consistency.

Q14: How do you design a URL shortener?

Use a base62 encoding of a unique integer (auto-increment or Snowflake ID generator) to create short URLs. Store the mapping in a database with the short code as the primary key for O(1) lookups. Cache popular URLs in Redis. Use a write-through cache strategy so the database and cache are always in sync for new URLs. For the redirect service, perform a GET on the short code, retrieve the long URL, and issue an HTTP 301 (permanent) or 302 (temporary) redirect. Handle custom aliases by checking for conflicts. Track analytics (click count, referrer, timestamp) asynchronously via a message queue.

Q15: What is database replication and why is it important?

Database replication copies data from a primary database to one or more replicas. It serves three purposes: scalability (distribute read queries across replicas), availability (if the primary fails, promote a replica to primary), and disaster recovery (replicas in different geographic regions survive regional outages). The trade-off is replication lag: replicas may serve stale data during high write periods. Synchronous replication provides strong consistency but increases write latency. Asynchronous replication provides better write performance but eventual consistency. Semi-synchronous replication provides a middle ground.

Q16: How do you handle long-running tasks in a web application?

Never run long tasks in the request-response cycle. Instead, accept the task asynchronously, return a task ID immediately, and process the task in the background. Use a message queue to decouple the HTTP handler from the background worker. The client can poll a status endpoint or use WebSockets/SSE for real-time updates. For tasks with user-visible progress, implement a progress tracker in the database and update it from the worker. Store task results in a cache or database with a TTL for later retrieval. This pattern keeps your API responsive and your users happy.

Q17: What are the trade-offs between REST and GraphQL?

REST is simpler, more cacheable (HTTP caching works natively), well-understood by all developers, and has excellent tooling. It works well when your data model is resource-oriented and clients need consistent data shapes. GraphQL allows clients to request exactly the data they need, eliminates over-fetching and under-fetching, and supports complex queries in a single request. It works well when clients have diverse data needs (mobile vs web vs third-party). The trade-off is GraphQL's complexity: query cost analysis, N+1 query problems, caching is harder, and the learning curve is steeper. For most applications, REST is sufficient. Choose GraphQL when client diversity and data flexibility are critical.

Q18: How do you approach capacity planning for a new system?

Start with user projections: how many users in 6 months, 12 months, 24 months. Estimate requests per user per day and the read-to-write ratio. Calculate peak QPS using a multiplier of 2-5x average. Estimate data size per request and multiply by daily volume for storage requirements. Add growth buffer (typically 50-100% above current estimates). Plan for peak scenarios, not average load. Use auto-scaling to handle variable traffic. Monitor actual usage against projections and adjust. The goal is not perfect accuracy but having a structured approach that prevents surprises.

Q19: What is the Strangler Fig pattern?

The Strangler Fig pattern is a migration strategy for replacing a monolith with microservices incrementally. A routing layer (reverse proxy or API gateway) intercepts all requests and routes them to either the legacy monolith or the new microservice based on the request path. New functionality is built as microservices. Existing functionality is gradually migrated, one bounded context at a time. The monolith shrinks over time until it can be decommissioned. This approach avoids the risk of a big-bang rewrite and allows the team to learn and adjust their microservices approach incrementally.

Q20: What are the most important system design principles for a senior engineer?

• Design for failure: Every component will fail. Design for graceful degradation, not perfect uptime.
• Keep it simple: Complexity is the enemy of reliability. Choose the simplest design that meets requirements.
• Make trade-offs explicit: Document why you chose one approach over another.
• Measure everything: You cannot improve what you do not measure. Instrument your systems from day one.
• Design for operability: A system that cannot be operated, monitored, and debugged in production is not production-ready.
• Iterate, don't over-engineer: Build for today's requirements and evolve as requirements change. Premature optimization is the root of all evil.
• Understand your data: Data modeling decisions are the most impactful architectural decisions. Choose them carefully.
• Automate everything: Manual processes do not scale. Automate testing, deployment, monitoring, and alerting.

Frequently Asked Questions

What is the difference between scalability and availability?

Scalability is the ability of a system to handle increased load by adding resources, either vertically (more power per machine) or horizontally (more machines). Availability is the percentage of time a system is operational and accessible. A system can be scalable but not available if it crashes frequently, or available but not scalable if it cannot handle growth. Both must be engineered independently.

What is the CAP theorem and why does it matter?

The CAP theorem states that a distributed system can provide at most two of three guarantees: Consistency (every read receives the most recent write), Availability (every request receives a response), and Partition Tolerance (the system continues operating despite network failures). Since network partitions are inevitable, you must choose between CP (consistent but may reject requests) and AP (available but may return stale data). Understanding this choice is fundamental to database and system selection.

How do I estimate system capacity in a system design interview?

Start with back-of-the-envelope calculations: identify daily active users, estimate requests per user per day, compute peak QPS (typically 2-3x average), estimate data per request, then calculate total storage and bandwidth. Round to powers of two. The exact numbers matter less than demonstrating a structured reasoning approach. Interviewers care about your process, not whether you get the exact number right.

When should I use microservices vs a monolith?

Start with a monolith. It is simpler to develop, test, deploy, and debug. Move to microservices when you have clear domain boundaries, team scaling needs (multiple teams working on different services), or specific scaling requirements for individual components. The overhead of distributed systems is only justified when the benefits outweigh the complexity.

What are the most important system design concepts for interviews?

The core concepts are: scalability (vertical vs horizontal), availability (nines, SLAs, SLOs), reliability (fault tolerance, redundancy), load balancing, caching strategies, database design (SQL vs NoSQL, sharding, replication), message queues, API design, and trade-off analysis. Understanding these deeply and knowing when to apply each is more important than memorizing specific architectures.

How do I handle data consistency in distributed systems?

Choose the right consistency model for your use case: strong consistency (synchronous replication, two-phase commit) for financial transactions, eventual consistency (async replication, event sourcing) for social feeds, and read-your-writes consistency (session stickiness or version vectors) for user-facing applications. Most real-world systems use a mix of consistency models across different components.

What is the difference between a load balancer and an API gateway?

A load balancer distributes incoming traffic across multiple server instances, typically operating at L4 (TCP) or L7 (HTTP). An API gateway sits in front of microservices and provides additional functionality: authentication, rate limiting, request routing, protocol translation, and response aggregation. In practice, the API gateway often includes load balancing capabilities, but the reverse is not true.

How do I design for fault tolerance?

Eliminate single points of failure by running multiple instances. Use circuit breakers to prevent cascading failures. Implement retries with exponential backoff and jitter. Design bulkheads to isolate failures. Plan for graceful degradation by disabling non-critical features under stress. Regularly test failure modes through chaos engineering practices. Document your failure modes and recovery procedures.