1. Why Caching Matters: The Performance Fundamentals

To understand why caching is so critical, we need to examine the fundamental economics of data access. Every piece of data your application needs lives somewhere, and that somewhere has a cost associated with retrieval. A CPU register access takes less than one nanosecond. An L1 cache hit takes about one nanosecond. An L2 cache hit takes around four nanoseconds. Main memory access takes roughly one hundred nanoseconds. A solid-state drive read takes about fifteen microseconds. A traditional hard disk seek takes around ten milliseconds. A network round-trip to a database server takes between one and ten milliseconds. And a cross-continent database query can take fifty milliseconds or more.

The difference between a cache hit and a cache miss is not a small optimization — it can be five or six orders of magnitude. When your application serves ten thousand requests per second and each request hits the database for a product listing that costs two milliseconds to compute, you are consuming twenty seconds of database time per second. That is a database running at maximum capacity just for one query type. But if you cache that product listing with a sixty-second TTL, you might serve nine thousand nine hundred requests from cache in zero point five milliseconds each, and only one hundred requests actually reach the database. Your database load drops by ninety-nine percent.

Beyond raw performance, caching provides three additional architectural benefits. First, it decouples read traffic from the database, allowing you to scale your read path independently of your storage layer. Second, it provides a natural buffer during traffic spikes — your cache absorbs the surge while the database sees a manageable increase. Third, it reduces infrastructure costs. A Redis cluster costing a few hundred dollars per month can eliminate the need for database replicas costing thousands. The return on investment for caching is almost always positive.

However, caching introduces trade-offs that you must understand and manage. The first is data freshness. When you cache data, you are serving a snapshot that may be out of date. Depending on your application, this may be perfectly acceptable (a news article does not change every second) or completely unacceptable (a bank balance must always be current). The second trade-off is complexity. A system without caching has one source of truth. A system with caching has at least two, and keeping them synchronized is a non-trivial engineering challenge. The third trade-off is operational overhead. Caches need monitoring, capacity planning, failure handling, and warm-up procedures.

Latency Comparison Table

The table above illustrates why caching at the application level is so transformative. A Redis cache hit at one millisecond versus a database query at fifteen milliseconds represents a fifteen-fold improvement. For a high-traffic API serving thousands of requests per second, that difference translates directly into infrastructure savings, user experience improvements, and the ability to handle growth without constant scaling.

2. Cache Hierarchy: L1, L2, L3, and the CDN Edge

A production caching strategy is never a single cache. It is a layered hierarchy, with each layer trading off latency for capacity and scope. Think of it as concentric rings of defense around your database. The closer the cache is to the requesting code, the faster it is but the smaller it is. The further from the code, the larger it is but the higher the latency of access.

L1 — In-Process Memory Cache: This is the fastest possible cache: data stored in your application's own process memory. In .NET, you would use IMemoryCache. This cache has zero network overhead and sub-microsecond access times. However, it is limited to the memory of a single process instance. If you have ten application servers, each has its own L1 cache that is not shared. L1 is best for data that is extremely hot (accessed thousands of times per second) and relatively small. Good candidates include configuration data, feature flags, and session data that the same server handles repeatedly due to sticky sessions.

L2 — Distributed In-Memory Cache: This is a shared cache accessed over the network, typically Redis or Memcached. It is shared across all application instances, so a cache populated by one server is available to all others. Access takes one to two milliseconds (including network round-trip), which is still fifty to one hundred times faster than a database query. L2 is the workhorse of most production caching strategies. It handles the bulk of your caching needs: user sessions, product catalogs, API response caches, rate limiting counters, and leaderboard data.

L3 — CDN Edge Cache: The Content Delivery Network caches content at hundreds or thousands of edge servers located in data centers around the world, close to end users. CDN latency is typically one to ten milliseconds. CDN caching is primarily for static assets (images, CSS, JavaScript files) and increasingly for dynamic API responses that are the same for many users (public product pages, search results for common queries). The CDN has the largest capacity but is limited to cacheable HTTP responses.

Database Query Cache: Many databases have their own internal query cache. MySQL has the Query Cache (deprecated in 8.0), PostgreSQL relies on the OS page cache, and MongoDB has its WiredTiger cache. This layer is often overlooked but can be significant. When a query result is identical to a previous query, the database can return the cached result without parsing or executing the query plan.

C# Multi-Level Cache Implementation

C#
using Microsoft.Extensions.Caching.Memory;
using StackExchange.Redis;

public class MultiLevelCacheService
{
    private readonly IMemoryCache _l1Cache;
    private readonly IConnectionMultiplexer _redis;
    private readonly IDatabase _l2Cache;

    public MultiLevelCacheService(
        IMemoryCache l1Cache,
        IConnectionMultiplexer redis)
    {
        _l1Cache = l1Cache;
        _redis = redis;
        _l2Cache = redis.GetDatabase();
    }

    public async Task<T?> GetAsync<T>(string key)
    {
        // L1: In-process memory (sub-microsecond)
        if (_l1Cache.TryGetValue(key, out T? l1Value))
        {
            return l1Value;
        }

        // L2: Redis distributed cache (~1ms)
        var l2Value = await _l2Cache.StringGetAsync(key);
        if (l2Value.HasValue)
        {
            var deserialized = JsonSerializer.Deserialize<T>(l2Value);

            // Backfill L1 for next time
            _l1Cache.Set(key, deserialized,
                new MemoryCacheEntryOptions
                {
                    AbsoluteExpirationRelativeToNow =
                        TimeSpan.FromSeconds(5) // Short L1 TTL
                });

            return deserialized;
        }

        return default;
    }

    public async Task SetAsync<T>(
        string key, T value,
        TimeSpan? l2Ttl = null)
    {
        l2Ttl ??= TimeSpan.FromMinutes(5);

        var serialized = JsonSerializer.Serialize(value);

        // Write to L2 first (durable across restarts)
        await _l2Cache.StringSetAsync(key, serialized, l2Ttl);

        // Write to L1 with shorter TTL
        _l1Cache.Set(key, value,
            new MemoryCacheEntryOptions
            {
                AbsoluteExpirationRelativeToNow =
                    TimeSpan.FromSeconds(5)
            });
    }
}

The implementation above shows the canonical multi-level cache pattern. Every read first checks the fast L1 cache, then falls back to the slower L2 cache, and finally (not shown) would fall back to the database. On a cache hit from L2, the value is backfilled into L1 so the next read is fast. Notice that L1 has a much shorter TTL than L2. This is deliberate. L1 is a performance optimization for repeated reads of the same data within a short window. It does not need to hold data as long because L2 will serve as the longer-lived cache. When L1 expires the entry but L2 still has it, the next read simply backfills from L2 to L1 at negligible cost.

The TTL asymmetry between layers is a critical design decision. If L1 and L2 had the same TTL, you would get inconsistent behavior: sometimes L1 expires before L2, sometimes they expire together, and the backfill logic becomes complex. By making L1 TTL always shorter than L2 TTL, you guarantee that L1 is always a strict subset of L2, which makes the consistency model simple to reason about.

3. Cache-Aside (Lazy Loading) Strategy

Cache-aside, also known as lazy loading, is the most common caching pattern in web applications. The application code explicitly manages the cache: it checks the cache before accessing the database, and it explicitly populates or invalidates the cache on writes. The cache itself is passive — it does not know about the database, and the database does not know about the cache. This separation of concerns is one of the pattern's greatest strengths.

The read flow is straightforward. The application first queries the cache using the data key. If the key exists (a cache hit), the cached value is returned immediately without touching the database. If the key does not exist (a cache miss), the application queries the database, receives the result, stores it in the cache with a TTL, and then returns the result to the caller. Subsequent reads for the same key will hit the cache until the TTL expires.

The write flow is equally explicit. When data changes, the application writes the new value to the database and then either deletes or updates the corresponding cache entry. Deleting is generally preferred over updating because it is simpler and avoids race conditions where two concurrent writes could cause the cache to hold stale data. After deletion, the next read will miss the cache, query the database for the fresh value, and repopulate the cache.

C# Cache-Aside Implementation

C#
public class ProductRepository
{
    private readonly IDatabase _cache;
    private readonly IDbConnection _db;
    private readonly TimeSpan _cacheTtl = TimeSpan.FromMinutes(10);

    public async Task<Product?> GetProductAsync(int productId)
    {
        string cacheKey = $"product:{productId}";

        // Step 1: Check cache
        var cached = await _cache.StringGetAsync(cacheKey);
        if (cached.HasValue)
        {
            return JsonSerializer.Deserialize<Product>(cached);
        }

        // Step 2: Cache miss - query database
        var product = await _db.QuerySingleOrDefaultAsync<Product>(
            "SELECT id, name, price, description FROM products WHERE id = @Id",
            new { Id = productId });

        if (product != null)
        {
            // Step 3: Populate cache
            await _cache.StringSetAsync(
                cacheKey,
                JsonSerializer.Serialize(product),
                _cacheTtl);
        }

        return product;
    }

    public async Task UpdateProductAsync(Product product)
    {
        // Step 1: Update database
        await _db.ExecuteAsync(
            "UPDATE products SET name = @Name, price = @Price, " +
            "description = @Description WHERE id = @Id",
            product);

        // Step 2: Invalidate cache (delete, don't update)
        string cacheKey = $"product:{product.Id}";
        await _cache.KeyDeleteAsync(cacheKey);
    }
}

Cache-aside has several important properties that make it the default choice for most applications. First, it is resilient to cache failures. If the cache goes down, the application simply falls back to the database. No data is lost, and the system degrades gracefully (albeit with higher latency). Second, it avoids overwriting fresh cache data with stale data. Because the cache is only populated on reads, and writes explicitly delete the cached entry, you never accidentally cache a stale value after a concurrent write. Third, it only caches data that is actually requested. Unlike write-through caching, which caches everything written regardless of whether it is read, cache-aside only uses cache space for data that has been accessed at least once.

Cache-Aside with Stale-While-Revalidate

A common enhancement to cache-aside is stale-while-revalidate. Instead of serving nothing when a cache entry expires, you serve the stale value and asynchronously refresh the cache in the background. This means the user always gets a fast response — either fresh data or slightly stale data that is being refreshed behind the scenes.

C#
public class StaleWhileRevalidateCache<T>
{
    private readonly IDatabase _cache;
    private readonly ConcurrentDictionary<string, SemaphoreSlim>
        _refreshLocks = new();

    public async Task<T?> GetOrRefreshAsync(
        string key,
        Func<Task<T?>> factory,
        TimeSpan freshTtl,
        TimeSpan staleTtl)
    {
        var cached = await _cache.HashGetAllAsync(key);

        if (cached.Length > 0)
        {
            var expiresAt = (long)cached["expires_at"];
            var data = cached["data"];

            if (DateTimeOffset.UtcNow.ToUnixTimeSeconds() < expiresAt)
            {
                // Fresh data
                return JsonSerializer.Deserialize<T>(data);
            }

            // Stale data - serve it and refresh in background
            _ = RefreshInBackgroundAsync(
                key, factory, freshTtl, staleTtl);

            return JsonSerializer.Deserialize<T>(data);
        }

        // Complete cache miss - must wait for fetch
        return await RefreshInBackgroundAsync(
            key, factory, freshTtl, staleTtl);
    }

    private async Task<T?> RefreshInBackgroundAsync<T>(
        string key,
        Func<Task<T?>> factory,
        TimeSpan freshTtl,
        TimeSpan staleTtl)
    {
        var lockObj = _refreshLocks.GetOrAdd(
            key, _ => new SemaphoreSlim(1, 1));

        if (!await lockObj.WaitAsync(0))
        {
            // Another refresh is in progress, wait briefly
            await Task.Delay(100);
            return await GetOrRefreshAsync(
                key, factory, freshTtl, staleTtl);
        }

        try
        {
            var result = await factory();
            if (result != null)
            {
                var now = DateTimeOffset.UtcNow.ToUnixTimeSeconds();
                var entries = new HashEntry[]
                {
                    new("data", JsonSerializer.Serialize(result)),
                    new("expires_at", now + staleTtl.TotalSeconds)
                };
                await _cache.HashSetAsync(key, entries);
                await _cache.KeyExpireAsync(key, staleTtl);
            }
            return result;
        }
        finally
        {
            lockObj.Release();
            _refreshLocks.TryRemove(key, out _);
        }
    }
}

4. Write-Through and Write-Behind Caching

While cache-aside handles reads efficiently, write-heavy systems often need caching strategies that optimize the write path as well. Write-through and write-behind (also called write-back) are two patterns that involve the cache in the write operation, rather than treating it as a passive store that only gets populated on reads.

Write-Through: In write-through caching, every write goes to both the cache and the database. The application writes to the cache first, and the cache synchronously writes to the database. The write is acknowledged to the application only after both writes complete. This guarantees that the cache is always consistent with the database — every write is reflected in both stores before the caller receives confirmation. The trade-off is write latency: every write pays the cost of both a cache write and a database write, which is slower than writing to either one alone.

Write-Behind (Write-Back): In write-behind caching, the application writes to the cache, and the cache acknowledges the write immediately. The cache then asynchronously persists the write to the database in the background. This dramatically reduces write latency because the application does not wait for the database. However, it introduces a risk: if the cache crashes before the asynchronous write completes, data is lost. Write-behind is only appropriate for data where occasional loss is acceptable, or where you can accept the trade-off of potentially losing recent writes in exchange for much higher write throughput.

C# Write-Through and Write-Behind Implementation

C#
public class WriteThroughCache<T>
{
    private readonly IDatabase _cache;
    private readonly IDbConnection _db;
    private readonly Channel<CacheWriteOperation> _writeBehindChannel;

    public WriteThroughCache(IDatabase cache, IDbConnection db)
    {
        _cache = cache;
        _db = db;
        _writeBehindChannel = Channel.CreateBounded<CacheWriteOperation>(
            new BoundedChannelOptions(10000)
            {
                FullMode = BoundedChannelFullMode.Wait,
                SingleReader = true
            });

        // Background consumer for write-behind
        _ = ConsumeWriteBehindOperations();
    }

    // Write-Through: synchronous dual write
    public async Task WriteThroughAsync<T>(
        string key, T value, TimeSpan ttl)
    {
        var serialized = JsonSerializer.Serialize(value);

        // Write to cache and database atomically
        using var transaction = _db.BeginTransaction();
        try
        {
            await _db.ExecuteAsync(
                "INSERT INTO cache_store (key, value) " +
                "VALUES (@Key, @Value) " +
                "ON CONFLICT (key) DO UPDATE SET value = @Value",
                new { Key = key, Value = serialized },
                transaction);

            await _cache.StringSetAsync(key, serialized, ttl);
            transaction.Commit();
        }
        catch
        {
            transaction.Rollback();
            throw;
        }
    }

    // Write-Behind: write to cache, async to database
    public async Task WriteBehindAsync<T>(
        string key, T value, TimeSpan ttl)
    {
        var serialized = JsonSerializer.Serialize(value);

        // Write to cache immediately
        await _cache.StringSetAsync(key, serialized, ttl);

        // Queue database write for background processing
        await _writeBehindChannel.Writer.WriteAsync(
            new CacheWriteOperation
            {
                Key = key,
                Value = serialized,
                Timestamp = DateTime.UtcNow
            });
    }

    private async Task ConsumeWriteBehindOperations()
    {
        await foreach (var operation in
            _writeBehindChannel.Reader.ReadAllAsync())
        {
            try
            {
                await _db.ExecuteAsync(
                    "INSERT INTO cache_store (key, value, updated_at) " +
                    "VALUES (@Key, @Value, @Timestamp) " +
                    "ON CONFLICT (key) DO UPDATE SET " +
                    "value = @Value, updated_at = @Timestamp",
                    operation);
            }
            catch (Exception ex)
            {
                // Log and potentially requeue
                // Data loss risk here is the key trade-off
            }
        }
    }
}

public class CacheWriteOperation
{
    public string Key { get; set; } = "";
    public string Value { get; set; } = "";
    public DateTime Timestamp { get; set; }
}

5. Read-Through and Read-Behind Caching

Read-through and read-behind are less commonly discussed patterns that complement cache-aside. In read-through caching, the cache itself is responsible for loading data from the database on a miss, rather than having the application code do it. The application only talks to the cache, and the cache knows how to fetch from the database. This encapsulates the database access logic inside the cache layer, simplifying application code.

Read-behind (also called cache-behind) is the inverse of read-through. When data is read from the cache and a miss occurs, the data is fetched from the database and cached, but also asynchronously written to a lower-latency cache layer. In essence, the cache acts as a buffer that asynchronously populates faster storage tiers. This is useful in scenarios where you want to pre-warm a fast cache from a slower data source based on actual access patterns.

C# Read-Through Cache Implementation

C#
public class ReadThroughCache<T>
{
    private readonly IDatabase _cache;
    private readonly Func<string, Task<T?>> _dataSource;
    private readonly TimeSpan _ttl;

    public ReadThroughCache(
        IDatabase cache,
        Func<string, Task<T?>> dataSource,
        TimeSpan ttl)
    {
        _cache = cache;
        _dataSource = dataSource;
        _ttl = ttl;
    }

    public async Task<T?> GetAsync(string key)
    {
        var cached = await _cache.StringGetAsync(key);
        if (cached.HasValue)
        {
            return JsonSerializer.Deserialize<T>(cached);
        }

        // Cache miss - let the data source fetch
        var data = await _dataSource(key);
        if (data != null)
        {
            await _cache.StringSetAsync(
                key,
                JsonSerializer.Serialize(data),
                _ttl);
        }

        return data;
    }
}

// Usage: the cache encapsulates the data source
var productCache = new ReadThroughCache<Product>(
    redis.GetDatabase(),
    async key =>
    {
        var id = int.Parse(key.Split(':')[1]);
        return await db.QuerySingleOrDefaultAsync<Product>(
            "SELECT * FROM products WHERE id = @Id",
            new { Id = id });
    },
    TimeSpan.FromMinutes(10));

// Application code is simple and clean
var product = await productCache.GetAsync("product:42");

The primary advantage of read-through caching is separation of concerns. The application code does not need to know whether data comes from the cache or the database. It simply asks the cache for data, and the cache handles the rest. This makes the application code simpler and more testable, since you can mock the cache interface without worrying about the database access logic.

The downside is that the cache layer becomes more complex. It needs to know how to connect to the database, handle connection pooling, manage errors, and deal with serialization. In practice, most teams prefer cache-aside because it keeps the cache simple and the application code explicit about what is happening. However, read-through caching is a good choice when you have a standardized data access layer that many different application services need to use.

6. Cache Invalidation: The Hardest Problem

Cache invalidation is the process of removing or updating cached data when the underlying source data changes. It is notoriously difficult because you must solve the fundamental question: how does the cache know when data has changed? In a single-server application, this is manageable. In a distributed system with multiple services, databases, and cache nodes, it becomes one of the hardest problems in software engineering.

There are four primary invalidation strategies, each with distinct trade-offs. The choice depends on your consistency requirements, the characteristics of your data, and the complexity your team can manage.

Strategy 1: TTL-Based Expiration

The simplest invalidation strategy is Time-To-Live (TTL). Every cached entry is assigned an expiration time, after which the cache automatically removes it. The next read after expiration will miss the cache and fetch fresh data from the database. TTL-based invalidation is passive — the cache does not need to know about data changes. It works by accepting that data may be stale for up to the TTL duration.

TTL is the right choice for data that changes infrequently or where temporary staleness is acceptable. Product catalogs, blog posts, configuration data, and search results are good candidates. The key insight is that TTL provides an eventual consistency guarantee with a known maximum staleness window. If your TTL is sixty seconds, you can guarantee that the cache will never be more than sixty seconds stale.

Strategy 2: Event-Driven Invalidation

When data changes, an event is published (typically to a message queue like Kafka, RabbitMQ, or Redis Pub/Sub), and the cache consumer listens for these events and invalidates the relevant keys. This provides near-real-time consistency: the cache is invalidated within milliseconds of the data changing. The trade-off is architectural complexity: you need a message broker, consumer services, and you must handle event ordering, delivery guarantees, and consumer failures.

C#
// Publisher: emit event on data change
public class ProductService
{
    private readonly IMessageBus _messageBus;
    private readonly IDbConnection _db;

    public async Task UpdateProductAsync(Product product)
    {
        await _db.ExecuteAsync(
            "UPDATE products SET name = @Name, price = @Price " +
            "WHERE id = @Id", product);

        // Publish invalidation event
        await _messageBus.PublishAsync(new CacheInvalidationEvent
        {
            EntityType = "Product",
            EntityId = product.Id,
            Action = InvalidateAction.Updated,
            Timestamp = DateTime.UtcNow
        });
    }
}

// Subscriber: invalidate cache on event
public class CacheInvalidationConsumer
{
    private readonly IDatabase _cache;

    public async Task HandleAsync(CacheInvalidationEvent evt)
    {
        var key = $"{evt.EntityType.ToLower()}:{evt.EntityId}";
        await _cache.KeyDeleteAsync(key);

        // Also invalidate any list caches that
        // might contain this entity
        var listKeys = await _cache.SetMembersAsync(
            $"index:{evt.EntityType.ToLower()}");
        foreach (var listKey in listKeys)
        {
            await _cache.KeyDeleteAsync(listKey);
        }
    }
}

Strategy 3: Write-Through Invalidation

As covered in Section 4, write-through caching automatically keeps the cache in sync with the database because every write goes to both stores simultaneously. There is no separate invalidation step — the write itself ensures consistency. The trade-off is write latency and complexity, but the consistency guarantee is strong.

Strategy 4: Versioned Cache Keys

Instead of invalidating cache entries, you change the cache key itself. For example, instead of caching a product at product:42, you cache it at product:42:v3, where v3 is a version number stored in the database. On every read, the application first fetches the version number, then constructs the versioned key. When the product changes, you increment the version in the database. Old cache entries with previous versions expire naturally via TTL, and the new version number causes a cache miss that loads fresh data.

C#
public async Task<Product?> GetProductVersionedAsync(int productId)
{
    // Fetch version (this itself is cached aggressively)
    var version = await GetProductVersionAsync(productId);
    string cacheKey = $"product:{productId}:v{version}";

    var cached = await _cache.StringGetAsync(cacheKey);
    if (cached.HasValue)
    {
        return JsonSerializer.Deserialize<Product>(cached);
    }

    var product = await _db.QuerySingleOrDefaultAsync<Product>(
        "SELECT * FROM products WHERE id = @Id",
        new { Id = productId });

    await _cache.StringSetAsync(
        cacheKey,
        JsonSerializer.Serialize(product),
        TimeSpan.FromHours(24)); // Long TTL, version handles invalidation

    return product;
}

private async Task<long> GetProductVersionAsync(int productId)
{
    string versionKey = $"product:{productId}:version";
    var version = await _cache.StringGetAsync(versionKey);
    if (version.HasValue) return (long)version;

    var dbVersion = await _db.QuerySingleAsync<long>(
        "SELECT version FROM products WHERE id = @Id",
        new { Id = productId });

    await _cache.StringSetAsync(versionKey, dbVersion,
        TimeSpan.FromHours(1));
    return dbVersion;
}

7. TTL Strategies and Expiration Policies

Choosing the right TTL is one of the most impactful decisions in caching design. A TTL that is too short means frequent cache misses and high database load. A TTL that is too long means users see stale data. The optimal TTL depends on three factors: how frequently the data changes, how critical freshness is, and how expensive the database query is.

There is a useful framework for thinking about TTLs. For data that changes very rarely (configuration, reference data), use TTLs of hours or even days. For data that changes periodically (product listings, blog posts), use TTLs of five to sixty minutes. For data that changes frequently (stock prices, social media feeds), use TTLs of seconds or use no-cache with conditional requests. For data that must always be fresh (bank balances, inventory counts), do not cache at the database level — use in-memory computation or read-through with near-zero TTL.

Dynamic TTL Based on Access Patterns

C#
public class DynamicTtlService
{
    private readonly IDatabase _cache;

    public TimeSpan CalculateTtl(string key, string entityType)
    {
        // Base TTLs by entity type
        var baseTtl = entityType switch
        {
            "config" => TimeSpan.FromHours(24),
            "product" => TimeSpan.FromMinutes(10),
            "user_profile" => TimeSpan.FromMinutes(5),
            "search_results" => TimeSpan.FromSeconds(30),
            "stock_price" => TimeSpan.FromSeconds(5),
            _ => TimeSpan.FromMinutes(5)
        };

        // Add jitter to prevent synchronized expiration
        var jitter = TimeSpan.FromSeconds(
            Random.Shared.Next(0, (int)(baseTtl.TotalSeconds * 0.1)));

        return baseTtl + jitter;
    }

    // Adaptive TTL: shorten TTL for frequently updated entities
    public async Task<TimeSpan> GetAdaptiveTtlAsync(string key)
    {
        string updateCountKey = $"updates:{key}";
        var updateCount = await _cache.StringGetAsync(updateCountKey);
        var count = updateCount.HasValue ? (int)updateCount : 0;

        if (count > 100)
            return TimeSpan.FromSeconds(5);   // Very volatile
        if (count > 20)
            return TimeSpan.FromSeconds(30);  // Somewhat volatile
        return TimeSpan.FromMinutes(5);       // Stable
    }
}

TTL Comparison Matrix

The table above shows a practical guide. Notice how cache hit rate decreases as freshness requirements increase. This is the fundamental trade-off: you cannot have both maximum freshness and maximum cache efficiency. The art is finding the sweet spot for each data type in your system.

8. Redis: The Swiss Army Knife of Caching

Redis has become the de facto standard for distributed caching, and for good reason. It offers sub-millisecond read and write performance, rich data structures beyond simple key-value pairs, built-in replication and persistence, Lua scripting for atomic operations, pub/sub messaging, and cluster mode for horizontal scaling. Understanding Redis deeply is essential for any senior engineer working on high-performance systems.

Redis stores all data in memory, which is why it is fast. But unlike a simple in-memory dictionary, Redis provides durability options. RDB snapshots save the dataset to disk periodically. AOF (Append Only File) logs every write operation. You can configure Redis to use both, giving you a durable in-memory store that can survive restarts. For pure caching use cases where data loss is acceptable, you can disable persistence entirely to reduce disk I/O.

Redis data structures are what set it apart from Memcached and other caches. Strings handle simple caching. Hashes store objects efficiently (a user profile as a single key with multiple fields). Lists implement message queues and recent-items feeds. Sets handle membership testing and tag-based queries. Sorted sets power leaderboards and time-series data. Streams (added in Redis 5.0) provide an event sourcing mechanism similar to Kafka for simple use cases.

Redis Cache Patterns in C#

C#
using StackExchange.Redis;

public class RedisCachePatterns
{
    private readonly IConnectionMultiplexer _redis;
    private readonly IDatabase _db;

    public RedisCachePatterns(IConnectionMultiplexer redis)
    {
        _redis = redis;
        _db = redis.GetDatabase();
    }

    // Pattern 1: Atomic cache-or-compute with SETNX
    public async Task<T?> GetOrComputeAsync<T>(
        string key,
        Func<Task<T?>> compute,
        TimeSpan ttl)
    {
        // Try to get from cache
        var cached = await _db.StringGetAsync(key);
        if (cached.HasValue)
            return JsonSerializer.Deserialize<T>(cached);

        // Cache miss - use distributed lock
        string lockKey = $"lock:{key}";
        bool hasLock = await _db.StringSetAsync(
            lockKey, "1", TimeSpan.FromSeconds(10),
            When.NotExists);

        if (!hasLock)
        {
            // Another process is computing, wait and retry
            await Task.Delay(200);
            return await GetOrComputeAsync(key, compute, ttl);
        }

        try
        {
            // Double-check after acquiring lock
            cached = await _db.StringGetAsync(key);
            if (cached.HasValue)
                return JsonSerializer.Deserialize<T>(cached);

            // Compute and cache
            var result = await compute();
            if (result != null)
            {
                await _db.StringSetAsync(
                    key,
                    JsonSerializer.Serialize(result),
                    ttl);
            }
            return result;
        }
        finally
        {
            await _db.KeyDeleteAsync(lockKey);
        }
    }

    // Pattern 2: Rate limiter using sliding window
    public async Task<bool> IsRateLimitedAsync(
        string userId, int maxRequests, int windowSeconds)
    {
        string key = $"ratelimit:{userId}";
        long now = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds();
        long windowStart = now - (windowSeconds * 1000);

        var batch = _db.CreateBatch();
        var addTask = batch.SortedSetAddAsync(
            key,
            now.ToString(),
            now);
        var removeTask = batch.SortedSetRemoveRangeByScoreAsync(
            key, 0, windowStart);
        var countTask = batch.SortedSetLengthAsync(key);
        var expireTask = batch.KeyExpireAsync(
            key, TimeSpan.FromSeconds(windowSeconds));
        batch.Execute();

        await Task.WhenAll(
            addTask, removeTask, countTask, expireTask);

        return countTask.Result > maxRequests;
    }

    // Pattern 3: Cache warming with pipeline
    public async Task WarmCacheAsync(
        IEnumerable<Product> products)
    {
        var batch = _db.CreateBatch();
        var tasks = new List<Task>();

        foreach (var product in products)
        {
            string key = $"product:{product.Id}";
            var serialized = JsonSerializer.Serialize(product);
            tasks.Add(batch.StringSetAsync(
                key, serialized, TimeSpan.FromMinutes(30)));
        }

        batch.Execute();
        await Task.WhenAll(tasks);
    }
}

Redis Memory Management

Redis memory management is critical for production stability. Redis stores everything in memory, so you must set a maximum memory limit using the maxmemory configuration directive. When memory reaches this limit, Redis evicts entries according to the configured eviction policy. The most common policies are allkeys-lru (evict least recently used keys across all keyspaces), volatile-lru (evict LRU keys that have an expiration time set), allkeys-lfu (evict least frequently used, available in Redis 4.0+), and noeviction (return errors on writes when memory is full).

For most caching use cases, allkeys-lru is the right choice. It automatically evicts the least recently accessed keys when memory is full, which naturally keeps the hottest data in cache. If you have a mix of caching and durable data in the same Redis instance, use volatile-lru so that only keys with TTLs are evicted, while durable keys are preserved.

9. Memcached: When Simplicity Wins

Memcached is a high-performance, distributed memory object caching system. Unlike Redis, it is deliberately simple: it stores key-value pairs where both keys and values are strings. There is no persistence, no replication, no complex data structures, and no Lua scripting. This simplicity is both its limitation and its strength.

Memcached uses a multi-threaded architecture, which means it can utilize multiple CPU cores efficiently on a single node. Redis, by contrast, is single-threaded for command execution (though it uses I/O threads in recent versions). This means Memcached can achieve higher throughput on a single node for simple get/set operations. However, Redis's richer feature set and persistence make it the better choice for most modern applications.

The cases where Memcached excels are: simple caching where you need maximum throughput with minimal memory overhead, situations where data loss on restart is completely acceptable (the cache will be repopulated), and environments where the multi-threaded architecture provides a measurable performance advantage. Facebook famously uses Memcached extensively for its social graph cache, though they have added significant customizations.

Redis vs Memcached Comparison

10. CDN Caching for Static and Dynamic Content

A Content Delivery Network (CDN) caches content at edge servers distributed globally, physically close to end users. Popular CDNs include Cloudflare, CloudFront (AWS), Fastly, Akamai, and Azure CDN. CDN caching is the outermost layer of your cache hierarchy and is uniquely powerful because it reduces both latency and bandwidth costs — the CDN serves content from its own infrastructure rather than pulling it from your origin servers.

For static assets (images, CSS, JavaScript, fonts, downloadable files), CDN caching is a no-brainer. You should set aggressive caching headers with long max-age values and content-hashed filenames for cache busting. An immutable JavaScript bundle referenced as app.a3b4c5d6.js can be cached for a year because the filename changes whenever the content changes.

Dynamic CDN caching is where things get more interesting. Many API responses are the same for all users — a public product page, a blog post, a search results page for a common query. These responses can be cached at the CDN edge with a short TTL. The key is to set the right HTTP cache headers: Cache-Control: public, s-maxage=60 tells the CDN to cache the response for sixty seconds while instructing browsers not to cache it. You must also vary on headers that affect the response — for example, Vary: Accept-Language if you serve different content based on the user's language.

HTTP Caching Headers Reference

HTTP

Cache-Control: public, max-age=31536000, immutable
ETag: "a3b4c5d6e7f8"


Cache-Control: public, s-maxage=60, max-age=0
Vary: Accept-Language


Cache-Control: private, max-age=300


Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache

CDN Caching Decision Matrix

Cache busting is the technique of forcing the CDN to serve new content when static assets change. The most reliable method is content hashing: include a hash of the file content in the filename. When the file changes, the hash changes, and the URL changes, so the CDN treats it as a new resource. The old version remains cached and serves users who have not yet requested the new version, preventing broken deployments. Tools like Webpack, Vite, and esbuild generate content-hashed filenames automatically.

11. Database Query Cache and Materialized Views

Database-level caching is often overlooked in favor of application-level caching, but it can provide significant benefits. Most relational databases maintain an internal cache of query results and page data. PostgreSQL, for example, uses shared buffers (typically twenty-five percent of system memory) to cache data pages. When a query can be satisfied entirely from shared buffers without reading from disk, it is called a buffer cache hit. The goal is to keep your working set small enough to fit in these buffers.

Materialized views are a powerful database-level caching mechanism. A materialized view is a query result stored as a physical table, rather than a virtual view that is computed on every access. You define a materialized view with a SELECT query, and the database computes and stores the result. Subsequent queries against the materialized view are as fast as querying a regular table. The trade-off is that the materialized view becomes stale when the underlying data changes, so you must refresh it periodically or on demand.

SQL
-- Materialized view for expensive dashboard query
CREATE MATERIALIZED VIEW mv_dashboard_stats AS
SELECT
    p.category,
    COUNT(DISTINCT o.id) AS order_count,
    SUM(o.total_amount) AS revenue,
    AVG(o.total_amount) AS avg_order_value,
    MAX(o.created_at) AS last_order_at
FROM products p
JOIN order_items oi ON oi.product_id = p.id
JOIN orders o ON o.id = oi.order_id
WHERE o.created_at >= NOW() - INTERVAL '30 days'
GROUP BY p.category;

-- Refresh when data changes (or on a schedule)
REFRESH MATERIALIZED VIEW CONCURRENTLY mv_dashboard_stats;

-- Query the materialized view (fast, pre-computed)
SELECT * FROM mv_dashboard_stats
ORDER BY revenue DESC
LIMIT 10;

Query result caching at the application level is another strategy. You can cache the serialized result of expensive SQL queries in Redis, keyed by the query and its parameters. This is particularly effective for complex reports, analytics dashboards, and search results that involve multiple joins and aggregations. The caching layer sits between the application and the database, intercepting repetitive queries.

C#
public class CachedQueryService
{
    private readonly IDatabase _cache;
    private readonly IDbConnection _db;

    public async Task<List<OrderSummary>> GetOrderSummaryAsync(
        DateTime startDate, DateTime endDate)
    {
        // Build a deterministic cache key from query parameters
        string cacheKey = $"query:order_summary:" +
            $"{startDate:yyyyMMdd}:{endDate:yyyyMMdd}";

        var cached = await _cache.StringGetAsync(cacheKey);
        if (cached.HasValue)
        {
            return JsonSerializer.Deserialize<List<OrderSummary>>(
                cached);
        }

        // Expensive query
        var results = (await _db.QueryAsync<OrderSummary>(@"
            SELECT
                u.name AS UserName,
                COUNT(o.id) AS OrderCount,
                SUM(o.total) AS TotalSpent
            FROM users u
            JOIN orders o ON o.user_id = u.id
            WHERE o.created_at BETWEEN @Start AND @End
            GROUP BY u.name
            ORDER BY TotalSpent DESC",
            new { Start = startDate, End = endDate }))
            .ToList();

        await _cache.StringSetAsync(
            cacheKey,
            JsonSerializer.Serialize(results),
            TimeSpan.FromMinutes(15));

        return results;
    }
}

12. Cache Stampede, Thundering Herd, and Hot Key Problems

Cache stampede (also known as the thundering herd problem) is one of the most dangerous failure modes in cached systems. It occurs when a popular cached key expires or is evicted, and a large number of concurrent requests simultaneously attempt to rebuild it. Each request finds a cache miss and queries the database. The database, which was previously handling a tiny fraction of traffic (because the cache was absorbing most reads), suddenly receives hundreds or thousands of concurrent queries for the same data. This can overwhelm the database, causing cascading failures.

The hot key problem is related but distinct. A hot key is a cache key that receives a disproportionately high number of reads compared to other keys. Even if the key is cached, the sheer volume of requests hitting a single Redis node (or even a single Redis key) can cause performance degradation. Redis is fast, but a single key being read fifty thousand times per second still places significant load on the Redis instance.

C# Stampede Prevention: Distributed Lock with Singleflight

C#
using System.Collections.Concurrent;

public class StampedeProtection<T>
{
    private readonly IDatabase _cache;
    private readonly ConcurrentDictionary<string, SemaphoreSlim>
        _inflight = new();
    private readonly TimeSpan _lockTimeout = TimeSpan.FromSeconds(5);

    public StampedeProtection(IDatabase cache)
    {
        _cache = cache;
    }

    public async Task<T?> GetOrLoadAsync(
        string key,
        Func<Task<T?>> loader,
        TimeSpan ttl)
    {
        // Try cache first
        var cached = await _cache.StringGetAsync(key);
        if (cached.HasValue)
            return JsonSerializer.Deserialize<T>(cached);

        // Get or create the inflight semaphore for this key
        var semaphore = _inflight.GetOrAdd(
            key, _ => new SemaphoreSlim(1, 1));

        await semaphore.WaitAsync(_lockTimeout);
        try
        {
            // Double-check after acquiring lock
            cached = await _cache.StringGetAsync(key);
            if (cached.HasValue)
                return JsonSerializer.Deserialize<T>(cached);

            // Load from source
            var result = await loader();
            if (result != null)
            {
                // Add jitter to TTL to prevent
                // synchronized expiration
                var jitter = TimeSpan.FromSeconds(
                    Random.Shared.Next(0,
                        (int)(ttl.TotalSeconds * 0.2)));
                await _cache.StringSetAsync(
                    key,
                    JsonSerializer.Serialize(result),
                    ttl + jitter);
            }

            return result;
        }
        finally
        {
            semaphore.Release();
            _inflight.TryRemove(key, out _);
        }
    }
}

Probabilistic Early Expiration (XFetch)

XFetch is an elegant algorithm for preventing cache stampedes without distributed locks. Instead of waiting for a key to expire and then rebuilding it, XFetch probabilistically triggers a background refresh before expiration. When a read arrives and the remaining TTL is below a threshold, there is a probability that the read will trigger an async refresh while still returning the current (slightly stale) value. This spreads the rebuild effort over time rather than concentrating it at the moment of expiration.

C#
public class XFetchCache<T>
{
    private readonly IDatabase _cache;

    public async Task<T?> GetAsync(
        string key,
        Func<Task<T?>> loader,
        TimeSpan ttl,
        TimeSpan beta)
    {
        var all = await _cache.HashGetAllAsync(key);

        if (all.Length == 0)
        {
            return await LoadAndCacheAsync(key, loader, ttl);
        }

        var value = JsonSerializer.Deserialize<T>(
            (string)all["value"]);
        var startTime = (long)all["start_time"];
        var bExp = (bool)all["background_refresh"];

        long delta = (long)(DateTimeOffset.UtcNow
            .ToUnixTimeMilliseconds() - startTime);
        long ttlMs = (long)ttl.TotalMilliseconds;

        double x = XFetchCalculate(delta, ttlMs, beta);

        if (x <= 1.0 && !bExp)
        {
            // Background refresh triggered
            await _cache.HashSetAsync(key, new HashEntry[]
            {
                new("background_refresh", true)
            });

            _ = Task.Run(async () =>
            {
                var fresh = await loader();
                await _cache.HashSetAsync(key, new HashEntry[]
                {
                    new("value", JsonSerializer.Serialize(fresh)),
                    new("start_time",
                        DateTimeOffset.UtcNow
                            .ToUnixTimeMilliseconds()),
                    new("background_refresh", false)
                });
                await _cache.KeyExpireAsync(key, ttl);
            });
        }

        return value;
    }

    private double XFetchCalculate(
        long delta, long ttlMs, TimeSpan beta)
    {
        // XFetch formula
        // Returns probability threshold
        double b = beta.TotalMilliseconds;
        double x = (b / (b - 1)) * (1 - (delta / (double)ttlMs));
        return x;
    }

    private async Task<T?> LoadAndCacheAsync(
        string key,
        Func<Task<T?>> loader,
        TimeSpan ttl)
    {
        var value = await loader();
        await _cache.HashSetAsync(key, new HashEntry[]
        {
            new("value", JsonSerializer.Serialize(value)),
            new("start_time",
                DateTimeOffset.UtcNow.ToUnixTimeMilliseconds()),
            new("background_refresh", false)
        });
        await _cache.KeyExpireAsync(key, ttl);
        return value;
    }
}

Hot Key Solutions

The hot key problem requires different solutions than stampede prevention. When a single key receives excessive traffic, you can: replicate the value across multiple keys (e.g., hot_key:1, hot_key:2, hot_key:3) and randomly distribute reads across them, use local in-memory caching for the hot key (L1 cache) to avoid hitting Redis on every read, or use Redis read replicas to distribute read load. For extremely hot keys, the best solution is often to remove the key from Redis entirely and serve it from application-level memory.

C#
public class HotKeyReplication<T>
{
    private readonly IDatabase _cache;
    private const int ReplicationFactor = 5;

    public async Task<T?> GetHotKeyAsync(string key)
    {
        // Randomly select a replica to distribute load
        int replica = Random.Shared.Next(0, ReplicationFactor);
        string replicaKey = $"{key}:replica:{replica}";

        var cached = await _cache.StringGetAsync(replicaKey);
        if (cached.HasValue)
            return JsonSerializer.Deserialize<T>(cached);

        // All replicas missed, load from source
        return default;
    }

    public async Task SetHotKeyAsync<T>(
        string key, T value, TimeSpan ttl)
    {
        var serialized = JsonSerializer.Serialize(value);

        // Write to all replicas
        var tasks = new Task[ReplicationFactor];
        for (int i = 0; i < ReplicationFactor; i++)
        {
            string replicaKey = $"{key}:replica:{i}";
            tasks[i] = _cache.StringSetAsync(
                replicaKey, serialized, ttl);
        }
        await Task.WhenAll(tasks);
    }
}

13. Cache Penetration, Breakdown, and Avalanche

Beyond stampedes and hot keys, there are three other critical failure modes that every senior engineer must understand and prepare for. These are cache penetration, cache breakdown, and cache avalanche. While related, each has distinct causes and solutions.

Cache Penetration

Cache penetration occurs when queries are made for data that does not exist in either the cache or the database. An attacker or a bug could send requests for millions of non-existent keys, causing every request to bypass the cache and hit the database. This defeats the purpose of caching entirely.

The solution is to cache null results. When a database query returns no results, cache a sentinel value (like "NULL" or "NOT_FOUND") with a short TTL. This way, subsequent requests for the same non-existent key will hit the cache instead of the database. Additionally, use Bloom filters at the cache layer to probabilistically check whether a key exists before querying the database. A Bloom filter can quickly tell you that a key definitely does not exist (preventing the database query) or might exist (allowing the query with a small probability of false positives).

C#
public class PenetrationProtectedCache<T>
{
    private readonly IDatabase _cache;
    private readonly TimeSpan _nullTtl = TimeSpan.FromMinutes(2);

    private const string NULL_SENTINEL = "CACHE_NULL";

    public async Task<T?> GetAsync(
        string key,
        Func<Task<T?>> loader,
        TimeSpan ttl)
    {
        var cached = await _cache.StringGetAsync(key);

        if (cached == NULL_SENTINEL)
        {
            // We previously cached that this key doesn't exist
            return default;
        }

        if (cached.HasValue)
        {
            return JsonSerializer.Deserialize<T>(cached);
        }

        // Cache miss - check database
        var result = await loader();

        if (result == null)
        {
            // Cache the absence of data
            await _cache.StringSetAsync(
                key, NULL_SENTINEL, _nullTtl);
        }
        else
        {
            await _cache.StringSetAsync(
                key,
                JsonSerializer.Serialize(result),
                ttl);
        }

        return result;
    }
}

Cache Breakdown

Cache breakdown is essentially the same as cache stampede. A hot key expires, and all requests fall through to the database simultaneously. The solution is the same: distributed locks, probabilistic early expiration, or stale-while-revalidate. The term "breakdown" emphasizes the cascading nature of the failure — the cache "breaks down" for a specific key, causing the database to break down under the sudden load.

Cache Avalanche

Cache avalanche is a larger-scale version of cache breakdown. Instead of a single key expiring, a large number of keys expire simultaneously. This can happen when many keys are set with the same TTL, causing them all to expire at the same time. The result is the same as a cache breakdown but at a much larger scale: the database receives a massive surge of requests for many different keys.

The solution is to add jitter to TTLs. Never set all keys with the exact same TTL. Instead, randomize the TTL within a range. For example, if your target TTL is sixty seconds, set each key's TTL to between fifty and seventy seconds. This spreads the expiration over time, preventing synchronized mass expiration. The code example in Section 7 demonstrates this jitter approach.

Failure Modes Summary

14. Multi-Layer Caching Architecture in Practice

Production systems rarely rely on a single caching layer. A well-designed multi-layer caching architecture uses each layer for what it does best: CDN for global content distribution, application-level distributed cache for shared state, and in-process cache for ultra-low-latency access to hot data. The challenge is coordinating these layers and ensuring consistency across them.

Consider a typical e-commerce platform. Product images and static assets are cached at the CDN edge with long TTLs. Product catalog data (prices, descriptions, availability) is cached in Redis with medium TTLs and explicit invalidation when products are updated. The current user's shopping cart and session data are cached in Redis with shorter TTLs. Frequently accessed product pages, after the first load, are cached in the application's in-memory cache with very short TTLs for the duration of a single request lifecycle.

C# Complete Multi-Layer Architecture

C#
public class MultiLayerCachingArchitecture
{
    private readonly IMemoryCache _l1;
    private readonly IDatabase _l2Redis;
    private readonly HttpClient _cdnClient;
    private readonly IDbConnection _db;

    public async Task<ProductPage?> GetProductPageAsync(
        int productId, string language)
    {
        string cacheKey = $"page:{productId}:{language}";

        // Layer 1: In-process memory (~0.001ms)
        if (_l1.TryGetValue<ProductPage>(cacheKey, out var l1Hit))
            return l1Hit;

        // Layer 2: Redis (~1ms)
        var l2Hit = await _l2Redis.StringGetAsync(cacheKey);
        if (l2Hit.HasValue)
        {
            var page = JsonSerializer.Deserialize<ProductPage>(l2Hit);

            // Backfill L1
            _l1.Set(cacheKey, page,
                new MemoryCacheEntryOptions
                {
                    AbsoluteExpirationRelativeToNow =
                        TimeSpan.FromSeconds(3)
                });

            return page;
        }

        // Layer 3: Try CDN for public pages
        if (IsPublicPage(productId))
        {
            string cdnUrl = $"https://cdn.example.com/{cacheKey}";
            var cdnResponse = await _cdnClient.GetAsync(cdnUrl);
            if (cdnResponse.IsSuccessStatusCode)
            {
                var cdnContent = await cdnResponse
                    .Content.ReadAsStringAsync();
                var page = JsonSerializer
                    .Deserialize<ProductPage>(cdnContent);

                // Backfill Redis
                await _l2Redis.StringSetAsync(
                    cacheKey,
                    JsonSerializer.Serialize(page),
                    TimeSpan.FromMinutes(5));

                return page;
            }
        }

        // Layer 4: Database (~15ms)
        var dbPage = await BuildProductPageAsync(
            productId, language);

        // Populate all cache layers
        await PopulateAllLayersAsync(cacheKey, dbPage);

        return dbPage;
    }

    private async Task PopulateAllLayersAsync<T>(
        string key, T value)
    {
        var serialized = JsonSerializer.Serialize(value);

        // L2: Redis with medium TTL
        await _l2Redis.StringSetAsync(
            key, serialized, TimeSpan.FromMinutes(5));

        // L1: Memory with short TTL
        _l1.Set(key, value,
            new MemoryCacheEntryOptions
            {
                AbsoluteExpirationRelativeToNow =
                    TimeSpan.FromSeconds(3)
            });
    }

    private bool IsPublicPage(int productId) => true;
    private Task<ProductPage> BuildProductPageAsync(
        int id, string lang) => Task.FromResult(new ProductPage());
}

public class ProductPage
{
    public int Id { get; set; }
    public string Name { get; set; } = "";
    public decimal Price { get; set; }
    public string Description { get; set; } = "";
    public List<string> ImageUrls { get; set; } = new();
}

Cache Layer Coordination Diagram

15. Caching Anti-Patterns and Production Pitfalls

Even experienced engineers make caching mistakes. These anti-patterns are common in production systems and can cause data corruption, performance degradation, or outright outages. Understanding them is essential for senior engineers who are responsible for system reliability.

Anti-Pattern 1: Caching Without an Invalidation Strategy

The most dangerous anti-pattern is caching data without a clear plan for how and when it will be invalidated. If you cache data and never invalidate it, the cache will eventually become stale. If you rely solely on TTL and the TTL is too long, users see outdated information. Always have two invalidation mechanisms: TTL as a safety net and explicit invalidation for critical updates. Every cached key should have a documented maximum staleness window.

Anti-Pattern 2: Caching Everything

Not everything should be cached. Data that is read once and never again (log entries, audit trails), data that changes on every read (counters, view counts), and data that must always be fresh (financial balances, inventory counts) should not be cached. Caching everything wastes memory, increases complexity, and makes the system harder to reason about. Cache only data that is read frequently, expensive to compute, and tolerant of some staleness.

Anti-Pattern 3: Using Cache as Primary Storage

A cache is not a database. Caches can lose data on restart, eviction, or failure. Never store data in a cache that is not also persisted to a durable store. If you need fast reads and writes, use a database with caching in front of it, not a cache as the sole store. The exception is ephemeral data like sessions or rate limiting counters, which can be regenerated or are inherently temporary.

Anti-Pattern 4: Unbounded Cache Growth

Without a maximum size limit, a cache can grow until it exhausts available memory, causing the application to crash. Always configure a maximum memory limit for your cache (Redis maxmemory, .NET MemoryCacheOptions.SizeLimit). Monitor cache size and set alerts for high usage. Consider using SizeLimit with eviction in .NET's MemoryCache to prevent unbounded growth.

C#
// Anti-pattern: unbounded cache
var badCache = new MemoryCache(new MemoryCacheOptions());

// Correct: bounded cache with eviction
var goodCache = new MemoryCache(new MemoryCacheOptions
{
    SizeLimit = 10_000,
    CompactionPercentage = 0.25,
    ExpirationScanFrequency = TimeSpan.FromMinutes(1)
});

// Use SizeLimit when adding entries
goodCache.Set("key", value, new MemoryCacheEntryOptions
{
    Size = 1,
    AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(5)
});

Anti-Pattern 5: Not Monitoring Cache Hit Rates

A cache that is not monitored is a cache that is not working. The single most important cache metric is the hit rate: the percentage of reads that are served from cache. A hit rate above ninety percent is excellent. A hit rate below seventy percent suggests that your cache keys do not match your access patterns, your TTL is too short, or your cache is too small. A hit rate near zero means your caching is completely ineffective. Instrument every cache operation with metrics and dashboards.

Anti-Pattern 6: Synchronized TTLs

Setting the same TTL for a large batch of keys causes them all to expire simultaneously, creating a cache avalanche. This commonly happens when a cache is warmed with a batch import (all products loaded at once, all with a ten-minute TTL). After ten minutes, every key expires at the same time, and the database is hit with a massive wave of requests. Always add jitter to TTLs when warming cache or loading batch data.

16. Monitoring, Metrics, and Observability

You cannot manage what you cannot measure. Cache monitoring is not optional — it is essential for operating a reliable, performant system. The key metrics you must track fall into three categories: performance metrics (how fast is the cache), efficiency metrics (how well is the cache being used), and capacity metrics (is the cache running out of space).

Essential Cache Metrics

C# Cache Instrumentation with Metrics

C#
using System.Diagnostics.Metrics;

public class InstrumentedCacheService
{
    private readonly IDatabase _cache;
    private readonly Counter<long> _hits;
    private readonly Counter<long> _misses;
    private readonly Counter<long> _errors;
    private readonly Histogram<double> _latency;

    public InstrumentedCacheService(
        IDatabase cache, IMeterFactory meterFactory)
    {
        _cache = cache;
        var meter = meterFactory.Create("CacheService");

        _hits = meter.CreateCounter<long>(
            "cache.hits", "requests");
        _misses = meter.CreateCounter<long>(
            "cache.misses", "requests");
        _errors = meter.CreateCounter<long>(
            "cache.errors", "requests");
        _latency = meter.CreateHistogram<double>(
            "cache.operation.duration", "milliseconds");
    }

    public async Task<string?> GetAsync(string key)
    {
        var sw = Stopwatch.StartNew();
        try
        {
            var result = await _cache.StringGetAsync(key);
            sw.Stop();
            _latency.Record(sw.Elapsed.TotalMilliseconds);

            if (result.HasValue)
            {
                _hits.Add(1, new KeyValuePair<string, object?>(
                    "key_prefix", GetKeyPrefix(key)));
            }
            else
            {
                _misses.Add(1, new KeyValuePair<string, object?>(
                    "key_prefix", GetKeyPrefix(key)));
            }

            return result;
        }
        catch (Exception ex)
        {
            sw.Stop();
            _errors.Add(1, new KeyValuePair<string, object?>(
                "error_type", ex.GetType().Name));
            throw;
        }
    }

    private string GetKeyPrefix(string key)
    {
        var parts = key.Split(':');
        return parts.Length > 0 ? parts[0] : "unknown";
    }
}

The instrumentation above uses .NET's built-in System.Diagnostics.Metrics library, which integrates with OpenTelemetry, Prometheus, and most modern observability platforms. The key insight is to track metrics by key prefix (like "product:", "user:", "session:") so you can identify which types of data have low hit rates and need attention.

17. Caching in Microservices and Distributed Systems

Caching in microservices architectures introduces additional complexity compared to monolithic applications. Each microservice may have its own cache, its own data store, and its own caching requirements. When one service updates data that another service has cached, you need a mechanism to propagate the invalidation across service boundaries. The shared-nothing architecture of microservices means you cannot simply call a method in another service to invalidate a cache — you need a distributed communication mechanism.

There are three primary patterns for cross-service cache invalidation. The first is direct invalidation via HTTP or gRPC calls: when Service A updates data, it calls Service B's cache invalidation endpoint. This is simple but creates coupling between services. The second is event-driven invalidation: Service A publishes an update event, and Service B consumes it to invalidate its cache. This is more decoupled but requires a message broker. The third is shared cache with namespaced keys: all services share a Redis instance but use key prefixes (like service_a:user:42 vs service_b:user:42) to avoid key collisions. This is the simplest approach but requires careful key management.

Event-Driven Cross-Service Cache Invalidation

C#
// Service A: Publish event when data changes
public class OrderService
{
    private readonly IMessageBus _bus;

    public async Task PlaceOrderAsync(Order order)
    {
        await _db.ExecuteAsync("INSERT INTO orders ...", order);

        // Notify other services to invalidate their caches
        await _bus.PublishAsync(new DomainEvent
        {
            Type = "OrderPlaced",
            Data = new
            {
                order.Id,
                order.UserId,
                order.ProductIds,
                order.TotalAmount
            }
        });
    }
}

// Service B: Consume event and invalidate local cache
public class InventoryCacheInvalidator
{
    private readonly IDatabase _cache;
    private readonly IMessageBus _bus;

    public void StartListening()
    {
        _bus.Subscribe<DomainEvent>("OrderPlaced", async evt =>
        {
            var orderData = evt.Data.ToObject<OrderEventData>();

            // Invalidate inventory cache for ordered products
            foreach (var productId in orderData.ProductIds)
            {
                string key = $"inventory:{productId}";
                await _cache.KeyDeleteAsync(key);
            }

            // Also invalidate the user's cart
            string cartKey = $"cart:{orderData.UserId}";
            await _cache.KeyDeleteAsync(cartKey);
        });
    }
}

The event-driven pattern is the gold standard for cross-service cache invalidation in microservices. It decouples the producer from the consumers, allows multiple services to react to the same event, and provides natural audit logging of all cache invalidation events. The trade-off is that events may be delivered out of order or with a delay, so you must design for eventual consistency.

18. Event-Driven Cache Invalidation with Message Queues

Building on the previous section, let us dive deeper into event-driven cache invalidation as a standalone architectural pattern. This approach treats cache invalidation as a first-class concern, not an afterthought. Every data modification produces a domain event, and interested cache layers subscribe to these events to invalidate their local caches.

The key design decisions in event-driven invalidation are: event granularity, delivery guarantees, ordering guarantees, and failure handling. Event granularity determines whether you emit events for individual field changes or whole-entity changes. Delivery guarantees determine whether events are at-most-once, at-least-once, or exactly-once. Ordering guarantees determine whether consumers see events in the order they were produced. And failure handling determines what happens when a consumer cannot process an event.

C#
// Domain event definition
public record CacheInvalidationEvent
{
    public string EventType { get; init; } = "";
    public string EntityId { get; init; } = "";
    public string EntityType { get; init; } = "";
    public Dictionary<string, string> Metadata { get; init; } = new();
    public DateTime OccurredAt { get; init; } = DateTime.UtcNow;
    public int RetryCount { get; init; } = 0;
}

// Reliable event publisher with retry
public class ReliableEventPublisher
{
    private readonly IMessageBus _bus;
    private readonly IDatabase _deadLetter;

    public async Task PublishInvalidationAsync(
        string entityType, string entityId,
        string eventType = "updated")
    {
        var evt = new CacheInvalidationEvent
        {
            EventType = eventType,
            EntityId = entityId,
            EntityType = entityType
        };

        try
        {
            await _bus.PublishAsync(evt);
        }
        catch (Exception)
        {
            // Store in dead letter queue for retry
            await _deadLetter.ListLeftPushAsync(
                "dead_letter:cache_events",
                JsonSerializer.Serialize(evt));
        }
    }
}

// Reliable consumer with idempotency
public class CacheEventConsumer
{
    private readonly IDatabase _cache;
    private readonly IDatabase _processedEvents;

    public async Task HandleEventAsync(CacheInvalidationEvent evt)
    {
        // Idempotency: check if already processed
        string processedKey = $"processed_event:{evt.EventType}:" +
            $"{evt.EntityType}:{evt.EntityId}:{evt.OccurredAt:yyyyMMddHHmmss}";

        bool alreadyProcessed = await _processedEvents
            .StringGetAsync(processedKey);

        if (alreadyProcessed)
            return;

        // Process the invalidation
        string cacheKey = $"{evt.EntityType.ToLower()}:" +
            $"{evt.EntityId}";
        await _cache.KeyDeleteAsync(cacheKey);

        // Also invalidate list/index caches
        var indexKeys = await _cache.SetMembersAsync(
            $"index:{evt.EntityType.ToLower()}");
        foreach (var indexKey in indexKeys)
        {
            await _cache.KeyDeleteAsync(indexKey);
        }

        // Mark as processed (with TTL for cleanup)
        await _processedEvents.StringSetAsync(
            processedKey, "1", TimeSpan.FromHours(24));
    }
}

19. Interview Questions and Answers

Caching is one of the most frequently tested topics in system design interviews. The questions below cover the concepts, trade-offs, and patterns that interviewers expect senior engineers to know deeply. Practice explaining these answers out loud — interviewers value clear communication as much as correct answers.

Q1: Design a caching strategy for a social media news feed.

Answer: Use a multi-layer approach. The user's personalized feed is cached in Redis using a sorted set, with the feed key being feed:{user_id} and scores being post timestamps. Set a TTL of five to fifteen seconds for active users. For the "cold" feed (new users or users with infrequent activity), use a longer TTL of one to five minutes. Cache the rendered HTML of popular posts at the CDN edge. Use write-behind caching for like counts and view counts — these can tolerate brief staleness. Implement feed pre-computation: when a user follows someone new or when a popular account posts, proactively update the follower's feed cache rather than computing it on read. Use event-driven invalidation: when a post is deleted, publish an event to invalidate all feed caches that contain that post.

Q2: How would you handle cache invalidation when a product's price changes in an e-commerce system?

Answer: This is a case where immediate consistency matters because showing a wrong price can lead to legal and customer trust issues. The strategy is: first update the database atomically, then publish a cache invalidation event to a message queue. The cache consumer receives the event and deletes the product cache key. Any subsequent reads will miss the cache and fetch the fresh price from the database. Additionally, invalidate any aggregate caches that contain the price (like category pages showing product lists with prices, search results, and recommendation caches). Use versioned cache keys as a backup: if the event-driven invalidation fails (consumer down, message lost), the TTL will eventually expire the stale cache. For extremely critical price data, use write-through caching to guarantee the cache is always in sync with the database on every write.

Q3: Explain the difference between Redis and Memcached. When would you choose each?

Answer: Redis supports rich data structures (strings, hashes, lists, sets, sorted sets, streams), persistence, replication, Lua scripting, pub/sub, and cluster mode. Memcached is simpler: key-value only, no persistence, no complex data structures, but multi-threaded for higher single-node throughput. Choose Memcached when you need a simple, high-throughput cache for objects that can be lost on restart with acceptable consequences, and when your objects fit comfortably in Memcached's one-megabyte value limit. Choose Redis for everything else — which is almost every modern application. Redis's versatility means you can use it not just for caching but also for rate limiting, session storage, message queues, real-time analytics, and leaderboards. The operational overhead of Redis is slightly higher, but the feature set more than compensates.

Q4: What is cache stampede and how do you prevent it?

Answer: A cache stampede occurs when a popular cached key expires and many concurrent requests simultaneously try to rebuild it, all hitting the database at once. Prevention strategies include: distributed mutex locks using Redis SETNX (only one process rebuilds, others wait and retry), probabilistic early expiration (XFetch) which refreshes the cache before expiry based on a probability function, stale-while-revalidate which serves stale data while asynchronously refreshing the cache, and TTL jitter which prevents synchronized expiration of related keys. In .NET, you can use the SemaphoreSlim class or a distributed lock library like RedLock.net. The singleflight pattern (from Go, implementable in C# with ConcurrentDictionary of SemaphoreSlim) ensures only one goroutine/thread performs the computation for a given key at a time.

Q5: Design a caching strategy for a URL shortener service.

Answer: A URL shortener is cache-friendly because the access pattern is ideal: many reads, few writes, and reads for the same short URL are identical. Use Redis as the primary cache with the short code as the key and the long URL as the value. Set a very long TTL (thirty days or more) since URLs rarely change. Use cache-aside: on a redirect request, check Redis first; on miss, query the database and populate the cache. For the top one percent most popular short URLs, keep them in an L1 in-process cache to avoid even the Redis network round-trip. Use Redis cluster for horizontal scaling since a popular URL shortener may need to handle millions of redirects per second. Implement a write-through pattern for URL creation: when a new short URL is created, write to both the database and the cache simultaneously. Monitor the hit rate — for a URL shortener, it should be above ninety-five percent.

Q6: How do you warm up a cache after a deployment or Redis restart?

Answer: Cache warming is the process of proactively populating the cache before it receives production traffic. Strategies include: pre-computing and loading the most frequently accessed keys (top one percent of products, most active user sessions) during deployment before routing traffic, using a dedicated warm-up script that reads from the database and populates the cache in batches, gradually ramping traffic using a load balancer (start with one percent, monitor hit rate and database load, increase gradually), and implementing a background refresh that continuously populates the cache from the database. During warm-up, monitor the database to ensure it can handle the surge of reads. A good deployment pipeline will: stop routing traffic to the instance being deployed, clear the old cache, run the warm-up script, verify the hit rate is above the target threshold, and then gradually resume routing traffic.

Q7: You notice your Redis cache hit rate dropped from 95% to 60% overnight. How do you diagnose and fix it?

Answer: First, check if the Redis instance was restarted (which would clear all keys) or if there was a deployment that cleared the cache. Check Redis memory usage — if the cache is full and evicting aggressively, you may need to increase maxmemory or add more nodes. Check if someone changed the TTL configuration. Check if there was a traffic spike that overwhelmed the cache (many new unique keys diluting the hit rate). Check if the access patterns changed (new API endpoints, different query parameters causing different cache keys). Use redis-cli INFO stats to check evictions and expired keys. Check application logs for errors in cache serialization/deserialization. Use Redis MONITOR or SLOWLOG to identify slow or unusual commands. The fix depends on the root cause: if keys are expiring too fast, increase TTLs. If the cache is too small, increase memory. If the key patterns changed, update the cache key design. If it is a cold cache after restart, implement cache warming.

Q8: Explain the CAP theorem implications for distributed caching.

Answer: In a distributed cache like Redis Cluster, you must choose between consistency and availability during network partitions. Redis Cluster uses asynchronous replication, which means during a failover, some writes may be lost (the replica may not have received the most recent writes from the master). This is an AP (availability + partition tolerance) system. If you need strong consistency (CP), you need synchronous replication, which adds latency. The practical implication: for most caching use cases, AP is acceptable because the cache is not the source of truth — the database is. If a cache partition causes some reads to be stale, the eventual consistency model means the data will be correct once the partition heals or the TTL expires. However, if you use Redis as a primary store (not recommended), you must carefully consider these implications.

Q9: Design a rate limiter using Redis that handles distributed requests.

Answer: Use the sliding window log algorithm with Redis sorted sets. Each user's requests are stored as members of a sorted set keyed by ratelimit:{user_id}, with the score being the request timestamp in milliseconds. On each request: remove entries older than the window, count remaining entries, and if the count exceeds the limit, reject the request. To handle distributed requests across multiple application servers, all servers share the same Redis instance, so rate limiting is consistent regardless of which server handles the request. Use Redis transactions (MULTI/EXEC) or Lua scripts to ensure atomicity of the check-and-increment operation. For extremely high-throughput rate limiting, use the fixed window counter algorithm (simpler, uses INCR) or the token bucket algorithm (allows burst capacity). Set the sorted set TTL to the window size plus a small buffer for automatic cleanup.

Q10: How do you test caching behavior in a development environment?

Answer: Testing caching requires covering multiple scenarios: cache hit (data exists in cache), cache miss (data not in cache, must fetch from database), cache expiration (TTL has passed), cache invalidation (data changed, cache entry removed), cache failure (Redis is unavailable, graceful degradation to database). Use integration tests with a real Redis instance (Testcontainers is excellent for this) or an in-memory Redis mock (like BookSleeve or RedisMock). For unit tests, mock the cache interface. Test scenarios include: verifying that a database query is not made on cache hit, verifying that the cache is populated after a miss, verifying that an update invalidates the correct cache keys, verifying that the application functions correctly when Redis is down (circuit breaker tests), and verifying that TTLs are set correctly. Use deterministic test data and clear the cache before each test to ensure consistent results.

Q11: Compare write-through, write-behind, and cache-aside for a write-heavy workload.

Answer: For a write-heavy workload, write-behind (write-back) offers the best write performance because it only writes to the cache and acknowledges immediately, deferring the database write to a background process. This dramatically reduces write latency and database load. However, it risks data loss if the cache crashes before the background write completes. Write-through provides strong consistency at the cost of higher write latency (every write goes to both cache and database synchronously). Cache-aside does not optimize writes at all — it only invalidates the cache on write, meaning subsequent reads repopulate the cache. For a write-heavy workload where eventual consistency is acceptable and data loss of the last few seconds is tolerable (like analytics counters or activity feeds), write-behind is ideal. For write-heavy workloads where every write must be durable (like order processing), write-through is safer. For write-heavy workloads where reads also need to be fast, combine write-behind for the write path with cache-aside for the read path.

Q12: How do you handle cache consistency across multiple data centers?

Answer: Multi-datacenter caching is one of the hardest problems in distributed systems. Each data center should have its own local Redis cluster for low-latency access. For data that is the same across all data centers (product catalog, configuration), use a publish-subscribe mechanism to propagate invalidation events from the data center where the write occurred to all other data centers. For user-specific data that primarily lives in one data center (user sessions, user preferences), route requests for that user to their home data center. Use conflict resolution strategies for concurrent writes to the same key in different data centers (last-write-wins, vector clocks, or application-level merge). Consider eventual consistency with a maximum staleness window — use TTLs as a safety net. For critical data, use a global Redis cluster with replicas in each data center, accepting the higher latency for cross-datacenter replication. The key insight is that perfect consistency across data centers is expensive and usually unnecessary — design for eventual consistency with bounded staleness.