1. Introduction & Motivation

A distributed cache is an in-memory data store that sits between your application and your primary database, providing sub-millisecond read access to frequently accessed data. At its core, caching is about trading space for time — keeping a copy of data in fast memory so you don't have to fetch it from slow storage.

Every major tech company relies heavily on distributed caching. Facebook uses Memcached across thousands of servers to cache social graph data, session information, and query results. Twitter caches timelines and user profiles. Netflix caches content metadata and viewing history. Amazon caches product catalog data and shopping cart state. Without distributed caching, these systems would simply not be able to handle their request volumes.

The challenge of designing a distributed cache is not just about storing data in memory. It's about managing consistency across multiple nodes, handling failures gracefully, choosing the right eviction policy, preventing cache stampedes, and scaling to millions of operations per second. This article covers every aspect of distributed cache design, from basic data structures to production-grade implementations in C#.

2. Interview Context

Why Interviewers Ask This

• System thinking: Caching touches every layer of the stack — application, data, network, and operations.
• Trade-off analysis: Consistency vs performance, memory vs hit rate, complexity vs simplicity.
• Failure handling: What happens when the cache goes down? Cache stampedes? Thundering herd?
• Production awareness: Eviction policies, monitoring, cache warming, invalidation strategies.
• Scalability: How do you scale from one cache node to hundreds? Consistent hashing, replication.

Common Interview Framing

Interviewers often frame this as: "Design a caching layer for [X system]" or "How would you add caching to [existing system]?" The key is to show you understand not just how to cache, but when to cache, what to cache, and how to handle cache failures.

3. Functional Requirements

Core Operations

• GET(key): Retrieve a value by key. Return cache hit or miss.
• SET(key, value, ttl): Store a value with an optional time-to-live.
• DELETE(key): Remove a key from the cache.
• Batch GET/SET: Retrieve or store multiple keys in a single operation.
• TTL expiration: Automatically expire stale data after configured duration.
• Eviction: When cache is full, evict entries based on configured policy.

Advanced Features

• Cache-aside / Read-through / Write-through: Configurable cache integration patterns.
• Pub/Sub invalidation: Broadcast cache invalidation events across nodes.
• Cache warming: Proactive pre-population of hot keys.
• Statistics: Hit rate, miss rate, eviction count, memory usage per node.

4. Non-Functional Requirements

5. Requirement Prioritization

6. Capacity Estimation

Workload Assumptions

Memory Sizing

Network Bandwidth

Hardware Requirements

7. Caching Patterns

How your application interacts with the cache determines the overall system behavior. There are five fundamental patterns, each with different trade-offs for consistency, latency, and complexity.

Pattern 1: Cache-Aside (Lazy Loading)

The application manages the cache explicitly. On read, check cache first, then load from DB on miss. On write, update DB and invalidate cache. This is the most common pattern.

public class CacheAsideCache<T>
{
    private readonly IDistributedCache _cache;
    private readonly AppDbContext _db;
    private readonly ILogger<CacheAsideCache<T>> _logger;
    private readonly TimeSpan _defaultTtl = TimeSpan.FromMinutes(5);

    public async Task<T?> GetAsync(string key)
    {
        // Step 1: Try cache first
        var cached = await _cache.GetStringAsync(key);
        if (cached != null)
        {
            _logger.LogDebug("Cache hit for key: {Key}", key);
            return JsonSerializer.Deserialize<T>(cached);
        }

        // Step 2: Cache miss - load from database
        _logger.LogDebug("Cache miss for key: {Key}", key);
        var entity = await _db.Set<T>().FindAsync(key);
        if (entity != null)
        {
            // Step 3: Populate cache for next time
            await _cache.SetStringAsync(key,
                JsonSerializer.Serialize(entity),
                new DistributedCacheEntryOptions
                {
                    AbsoluteExpirationRelativeToNow = _defaultTtl
                });
        }

        return entity;
    }

    public async Task SetAsync(string key, T value, TimeSpan? ttl = null)
    {
        // Write to database first
        _db.Set<T>().Update(value);
        await _db.SaveChangesAsync();

        // Then invalidate or update cache
        if (value == null)
            await _cache.RemoveAsync(key);
        else
            await _cache.SetStringAsync(key,
                JsonSerializer.Serialize(value),
                new DistributedCacheEntryOptions
                {
                    AbsoluteExpirationRelativeToNow = ttl ?? _defaultTtl
                });
    }
}

Pros: Simple, only caches what's actually requested, no stale data on write. Cons: First request always misses (cold start), extra round-trip on miss.

Pattern 2: Write-Through

Writes go to both cache and database simultaneously. Cache is always up-to-date. Higher write latency but guaranteed consistency.

public class WriteThroughCache<T>
{
    private readonly IDistributedCache _cache;
    private readonly AppDbContext _db;

    public async Task SetAsync(string key, T value, TimeSpan ttl)
    {
        // Write to both simultaneously
        var cacheTask = _cache.SetStringAsync(key,
            JsonSerializer.Serialize(value),
            new DistributedCacheEntryOptions
            {
                AbsoluteExpirationRelativeToNow = ttl
            });

        var dbTask = _db.Set<T>().Upsert(value).ExecuteAsync();

        await Task.WhenAll(cacheTask, dbTask);
    }
}

Pros: Cache is always consistent, no cold misses. Cons: Higher write latency, cache may be polluted with data that's never read.

Pattern 3: Write-Behind (Write-Back)

Writes go to cache immediately, then asynchronously flush to database. Lowest write latency but risk of data loss if cache crashes before flush.

public class WriteBehindCache<T>
{
    private readonly IDistributedCache _cache;
    private readonly AppDbContext _db;
    private readonly Channel<CacheWriteOperation<T>> _writeQueue;
    private readonly ILogger<WriteBehindCache<T>> _logger;

    public WriteBehindCache()
    {
        _writeQueue = Channel.CreateBounded<CacheWriteOperation<T>>(10000);
        _ = ProcessWriteQueue();
    }

    public async Task SetAsync(string key, T value, TimeSpan ttl)
    {
        // Write to cache immediately (fast path)
        await _cache.SetStringAsync(key,
            JsonSerializer.Serialize(value),
            new DistributedCacheEntryOptions
            {
                AbsoluteExpirationRelativeToNow = ttl
            });

        // Queue database write (async)
        await _writeQueue.Writer.WriteAsync(new CacheWriteOperation<T>
        {
            Key = key,
            Value = value,
            Operation = WriteOperation.Update,
            Timestamp = DateTimeOffset.UtcNow
        });
    }

    private async Task ProcessWriteQueue()
    {
        await foreach (var operation in _writeQueue.Reader.ReadAllAsync())
        {
            try
            {
                switch (operation.Operation)
                {
                    case WriteOperation.Update:
                        _db.Set<T>().Update(operation.Value);
                        break;
                    case WriteOperation.Delete:
                        _db.Set<T>().Remove(_db.Set<T>().Find(operation.Key));
                        break;
                }
                await _db.SaveChangesAsync();
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "Failed to flush write for key {Key}", operation.Key);
                // Re-queue for retry (with backoff)
            }
        }
    }
}

Pros: Lowest write latency, batched database writes. Cons: Data loss risk if cache crashes, eventual consistency, complex implementation.

Pattern 4: Read-Through

The cache itself loads data from the database on miss. Application only talks to the cache. Cleaner abstraction but requires cache-aware loading logic.

public class ReadThroughCache<T>
{
    private readonly IDistributedCache _cache;
    private readonly Func<string, Task<T?>> _loader;
    private readonly TimeSpan _ttl;

    public ReadThroughCache(
        IDistributedCache cache,
        Func<string, Task<T?>> loader,
        TimeSpan ttl)
    {
        _cache = cache;
        _loader = loader;
        _ttl = ttl;
    }

    public async Task<T?> GetAsync(string key)
    {
        var cached = await _cache.GetStringAsync(key);
        if (cached != null)
            return JsonSerializer.Deserialize<T>(cached);

        // Cache handles the loading
        var value = await _loader(key);
        if (value != null)
        {
            await _cache.SetStringAsync(key,
                JsonSerializer.Serialize(value),
                new DistributedCacheEntryOptions
                {
                    AbsoluteExpirationRelativeToNow = _ttl
                });
        }
        return value;
    }
}

Pattern 5: Cache Invalidation via Pub/Sub

When data changes, broadcast invalidation events to all cache nodes. Ensures all nodes evict stale data simultaneously.

public class PubSubInvalidator
{
    private readonly IConnectionMultiplexer _redis;
    private readonly IDistributedCache _cache;

    public async Task InvalidateAsync(string key)
    {
        // Remove from local cache
        await _cache.RemoveAsync(key);

        // Broadcast to all other nodes
        var subscriber = _redis.GetSubscriber();
        await subscriber.PublishAsync(
            RedisChannel.Literal("cache:invalidate"),
            JsonSerializer.Serialize(new { Key = key, Timestamp = DateTimeOffset.UtcNow }));
    }

    public void SubscribeToInvalidations()
    {
        var subscriber = _redis.GetSubscriber();
        subscriber.Subscribe(RedisChannel.Literal("cache:invalidate"),
            async (channel, message) =>
            {
                var invalidation = JsonSerializer.Deserialize<InvalidationEvent>(message);
                await _cache.RemoveAsync(invalidation.Key);
            });
    }
}

Pattern Comparison

8. Eviction Policies

When the cache is full, we must decide which entries to evict. The eviction policy directly determines the cache hit rate, which is the single most important metric for cache effectiveness.

LRU (Least Recently Used)

Evict the entry that hasn't been accessed for the longest time. The most common policy, provides good hit rates for most workloads due to temporal locality.

public class LRUCache<TKey, TValue>
{
    private readonly int _capacity;
    private readonly Dictionary<TKey, LinkedListNode<CacheEntry<TValue>>> _map;
    private readonly LinkedList<CacheEntry<TValue>> _list;
    private readonly object _lock = new();

    public LRUCache(int capacity)
    {
        _capacity = capacity;
        _map = new Dictionary<TKey, LinkedListNode<CacheEntry<TValue>>>(capacity);
        _list = new LinkedList<CacheEntry<TValue>>();
    }

    public TValue? Get(TKey key)
    {
        lock (_lock)
        {
            if (_map.TryGetValue(key, out var node))
            {
                // Move to front (most recently used)
                _list.Remove(node);
                _list.AddFirst(node);
                return node.Value.Value;
            }
            return default;
        }
    }

    public void Put(TKey key, TValue value)
    {
        lock (_lock)
        {
            if (_map.TryGetValue(key, out var existing))
            {
                _list.Remove(existing);
                existing.Value.Value = value;
                _list.AddFirst(existing);
            }
            else
            {
                if (_map.Count >= _capacity)
                {
                    // Evict least recently used (tail)
                    var lru = _list.Last;
                    _list.RemoveLast();
                    _map.Remove(lru.Value.Key);
                }

                var entry = new CacheEntry<TValue> { Key = key, Value = value };
                var node = new LinkedListNode<CacheEntry<TValue>>(entry);
                _list.AddFirst(node);
                _map[key] = node;
            }
        }
    }

    public bool Remove(TKey key)
    {
        lock (_lock)
        {
            if (_map.TryGetValue(key, out var node))
            {
                _list.Remove(node);
                _map.Remove(key);
                return true;
            }
            return false;
        }
    }

    private class CacheEntry<T>
    {
        public TKey Key { get; set; }
        public T Value { get; set; }
    }
}

LFU (Least Frequently Used)

Evict the entry with the lowest access count. Better for workloads with skewed access patterns but requires maintaining counters.

public class LFUCache<TKey, TValue>
{
    private readonly int _capacity;
    private readonly Dictionary<TKey, (TValue Value, int Freq)> _cache;
    private readonly Dictionary<int, LinkedList<TKey>> _freqBuckets;
    private int _minFreq;

    public LFUCache(int capacity)
    {
        _capacity = capacity;
        _cache = new Dictionary<TKey, (TValue, int)>(capacity);
        _freqBuckets = new Dictionary<int, LinkedList<TKey>>();
        _minFreq = 0;
    }

    public TValue? Get(TKey key)
    {
        if (!_cache.TryGetValue(key, out var entry))
            return default;

        // Promote to higher frequency bucket
        _freqBuckets[entry.Freq].Remove(key);
        if (_freqBuckets[entry.Freq].Count == 0)
        {
            _freqBuckets.Remove(entry.Freq);
            if (_minFreq == entry.Freq) _minFreq++;
        }

        var newFreq = entry.Freq + 1;
        if (!_freqBuckets.ContainsKey(newFreq))
            _freqBuckets[newFreq] = new LinkedList<TKey>();
        _freqBuckets[newFreq].AddLast(key);
        _cache[key] = (entry.Value, newFreq);

        return entry.Value;
    }

    public void Put(TKey key, TValue value)
    {
        if (_cache.ContainsKey(key))
        {
            _cache[key] = (value, _cache[key].Freq);
            Get(key); // Promote frequency
            return;
        }

        if (_cache.Count >= _capacity)
        {
            // Evict from minimum frequency bucket
            var victim = _freqBuckets[_minFreq].First;
            _freqBuckets[_minFreq].RemoveFirst();
            if (_freqBuckets[_minFreq].Count == 0)
                _freqBuckets.Remove(_minFreq);
            _cache.Remove(victim.Value);
        }

        _minFreq = 0;
        if (!_freqBuckets.ContainsKey(0))
            _freqBuckets[0] = new LinkedList<TKey>();
        _freqBuckets[0].AddLast(key);
        _cache[key] = (value, 0);
    }
}

Other Policies

9. High-Level Architecture

Client-Side Routing

The cache client computes which node owns a given key using consistent hashing. There is no central coordinator — each client knows the cluster topology and routes directly to the correct node. This eliminates a single point of failure and reduces latency.

public class CacheClient
{
    private readonly ConsistentHashRing _hashRing;
    private readonly Dictionary<string, ICacheConnection> _connections;
    private readonly CacheSerializer _serializer;

    public async Task<CacheResult<T>> GetAsync<T>(string key)
    {
        // 1. Determine which node owns this key
        var nodeId = _hashRing.GetNode(key);
        var connection = _connections[nodeId];

        // 2. Send GET to the correct node
        var response = await connection.GetAsync(key);

        // 3. Deserialize and return
        if (response.Status == CacheStatus.Hit)
            return CacheResult<T>.Hit(_serializer.Deserialize<T>(response.Value));

        return CacheResult<T>.Miss();
    }

    public async Task SetAsync<T>(string key, T value, TimeSpan? ttl = null)
    {
        var nodeId = _hashRing.GetNode(key);
        var connection = _connections[nodeId];
        var serialized = _serializer.Serialize(value);

        await connection.SetAsync(key, serialized, ttl);
    }

    public async Task<List<CacheResult<T>>> BatchGetAsync<T>(IEnumerable<string> keys)
    {
        // Group keys by node for batch efficiency
        var grouped = keys.GroupBy(k => _hashRing.GetNode(k));

        var tasks = grouped.Select(async group =>
        {
            var connection = _connections[group.Key];
            var batchKeys = group.ToArray();
            var responses = await connection.BatchGetAsync(batchKeys);

            return batchKeys.Zip(responses, (key, resp) =>
                new CacheResult<T>
                {
                    Key = key,
                    Status = resp.Status == CacheStatus.Hit ? CacheStatus.Hit : CacheStatus.Miss,
                    Value = resp.Status == CacheStatus.Hit
                        ? _serializer.Deserialize<T>(resp.Value)
                        : default
                });
        });

        var results = await Task.WhenAll(tasks);
        return results.SelectMany(r => r).ToList();
    }
}

10. Production Architecture Diagram

Write Path with Cache Stampede Protection

Replication Flow

11. Component Deep Dive

Consistent Hash Ring

using System.Security.Cryptography;
using System.Text;

public class ConsistentHashRing
{
    private readonly SortedDictionary<uint, string> _ring = new();
    private readonly int _virtualNodes;
    private readonly List<string> _physicalNodes;

    public ConsistentHashRing(IEnumerable<string> nodes, int virtualNodes = 150)
    {
        _virtualNodes = virtualNodes;
        _physicalNodes = nodes.ToList();

        foreach (var node in _physicalNodes)
            AddNode(node);
    }

    private uint Hash(string key)
    {
        using var sha = SHA256.Create();
        var hash = sha.ComputeHash(Encoding.UTF8.GetBytes(key));
        return BitConverter.ToUInt32(hash, 0);
    }

    public void AddNode(string nodeId)
    {
        for (int i = 0; i < _virtualNodes; i++)
        {
            var virtualKey = $"{nodeId}:vn{i}";
            var hash = Hash(virtualKey);
            _ring[hash] = nodeId;
        }
    }

    public void RemoveNode(string nodeId)
    {
        for (int i = 0; i < _virtualNodes; i++)
        {
            var virtualKey = $"{nodeId}:vn{i}";
            var hash = Hash(virtualKey);
            _ring.Remove(hash);
        }
    }

    public string GetNode(string key)
    {
        if (_ring.Count == 0)
            throw new InvalidOperationException("Ring is empty");

        var hash = Hash(key);

        // Find the first node clockwise from the hash position
        foreach (var kv in _ring)
        {
            if (kv.Key >= hash)
                return kv.Value;
        }

        // Wrap around to the first node
        return _ring.First().Value;
    }

    public List<string> GetReplicas(string key, int count)
    {
        var replicas = new List<string>();
        var seen = new HashSet<string>();
        var hash = Hash(key);

        foreach (var kv in _ring)
        {
            if (kv.Key >= hash && seen.Add(kv.Value))
                replicas.Add(kv.Value);

            if (replicas.Count == count)
                break;
        }

        // Wrap around if needed
        if (replicas.Count < count)
        {
            foreach (var kv in _ring)
            {
                if (seen.Add(kv.Value))
                    replicas.Add(kv.Value);

                if (replicas.Count == count)
                    break;
            }
        }

        return replicas;
    }
}

LRU Cache with TTL and Memory Tracking

public class MemoryAwareLRUCache<TValue>
{
    private readonly int _maxMemoryBytes;
    private readonly Dictionary<string, CacheNode<TValue>> _map;
    private readonly LinkedList<CacheNode<TValue>> _lruList;
    private long _currentMemoryBytes;
    private readonly object _lock = new();

    public MemoryAwareLRUCache(int maxMemoryMB)
    {
        _maxMemoryBytes = maxMemoryMB * 1024 * 1024;
        _map = new Dictionary<string, CacheNode<TValue>>();
        _lruList = new LinkedList<CacheNode<TValue>>();
    }

    public (bool Hit, TValue? Value) Get(string key)
    {
        lock (_lock)
        {
            if (_map.TryGetValue(key, out var node))
            {
                // Check TTL
                if (node.ExpiresAt.HasValue && node.ExpiresAt < DateTimeOffset.UtcNow)
                {
                    RemoveNode(node);
                    return (false, default);
                }

                // Move to front
                _lruList.Remove(node);
                _lruList.AddFirst(node);
                node.LastAccessed = DateTimeOffset.UtcNow;
                return (true, node.Value);
            }
            return (false, default);
        }
    }

    public void Set(string key, TValue value, TimeSpan? ttl = null)
    {
        lock (_lock)
        {
            var entrySize = EstimateSize(key, value);

            if (_map.TryGetValue(key, out var existing))
            {
                _lruList.Remove(existing);
                _currentMemoryBytes -= existing.MemorySize;
                existing.Value = value;
                existing.ExpiresAt = ttl.HasValue
                    ? DateTimeOffset.UtcNow + ttl.Value
                    : null;
                existing.MemorySize = entrySize;
                _currentMemoryBytes += entrySize;
                _lruList.AddFirst(existing);
            }
            else
            {
                // Evict until we have enough space
                while (_currentMemoryBytes + entrySize > _maxMemoryBytes && _lruList.Count > 0)
                {
                    var lru = _lruList.Last;
                    RemoveNode(lru);
                }

                var node = new CacheNode<TValue>
                {
                    Key = key,
                    Value = value,
                    ExpiresAt = ttl.HasValue ? DateTimeOffset.UtcNow + ttl.Value : null,
                    MemorySize = entrySize,
                    CreatedAt = DateTimeOffset.UtcNow,
                    LastAccessed = DateTimeOffset.UtcNow
                };
                _lruList.AddFirst(node);
                _map[key] = node;
                _currentMemoryBytes += entrySize;
            }
        }
    }

    public bool Remove(string key)
    {
        lock (_lock)
        {
            if (_map.TryGetValue(key, out var node))
            {
                RemoveNode(node);
                return true;
            }
            return false;
        }
    }

    private void RemoveNode(CacheNode<TValue> node)
    {
        _lruList.Remove(node);
        _map.Remove(node.Key);
        _currentMemoryBytes -= node.MemorySize;
    }

    private long EstimateSize(string key, TValue value)
    {
        var keyBytes = Encoding.UTF8.GetByteCount(key);
        var valueBytes = JsonSerializer.SerializeToUtf8Bytes(value).Length;
        return keyBytes + valueBytes + 64; // 64 bytes overhead per entry
    }

    public CacheStats GetStats()
    {
        lock (_lock)
        {
            return new CacheStats
            {
                EntryCount = _map.Count,
                MemoryUsedBytes = _currentMemoryBytes,
                MemoryMaxBytes = _maxMemoryBytes,
                MemoryUtilization = (double)_currentMemoryBytes / _maxMemoryBytes
            };
        }
    }

    public void CleanupExpired()
    {
        lock (_lock)
        {
            var now = DateTimeOffset.UtcNow;
            var expired = _lruList.Where(n =>
                n.ExpiresAt.HasValue && n.ExpiresAt < now).ToList();

            foreach (var node in expired)
                RemoveNode(node);
        }
    }

    private class CacheNode<T>
    {
        public string Key { get; set; }
        public T Value { get; set; }
        public DateTimeOffset? ExpiresAt { get; set; }
        public long MemorySize { get; set; }
        public DateTimeOffset CreatedAt { get; set; }
        public DateTimeOffset LastAccessed { get; set; }
    }
}

public class CacheStats
{
    public int EntryCount { get; set; }
    public long MemoryUsedBytes { get; set; }
    public long MemoryMaxBytes { get; set; }
    public double MemoryUtilization { get; set; }
    public long Hits { get; set; }
    public long Misses { get; set; }
    public double HitRate => Hits + Misses > 0 ? (double)Hits / (Hits + Misses) : 0;
}

Cache Stampede Prevention

public class StampedeProtection
{
    private readonly IDistributedLockProvider _lockProvider;
    private readonly IDistributedCache _cache;

    public async Task<T?> GetOrLoadAsync<T>(
        string key,
        Func<Task<T?>> loader,
        TimeSpan ttl)
    {
        // Fast path: check cache
        var cached = await _cache.GetStringAsync(key);
        if (cached != null)
            return JsonSerializer.Deserialize<T>(cached);

        // Slow path: acquire distributed lock
        var lockKey = $"lock:{key}";
        await using var lockHandle = await _lockProvider.TryAcquireAsync(
            lockKey, TimeSpan.FromSeconds(5));

        if (lockHandle == null)
        {
            // Another thread is loading; wait and retry
            await Task.Delay(50);
            cached = await _cache.GetStringAsync(key);
            return cached != null ? JsonSerializer.Deserialize<T>(cached) : null;
        }

        // Double-check after acquiring lock
        cached = await _cache.GetStringAsync(key);
        if (cached != null)
            return JsonSerializer.Deserialize<T>(cached);

        // Load from database
        var value = await loader();
        if (value != null)
        {
            await _cache.SetStringAsync(key,
                JsonSerializer.Serialize(value),
                new DistributedCacheEntryOptions
                {
                    AbsoluteExpirationRelativeToNow = ttl
                });
        }

        return value;
    }
}

12. Data Modeling

Internal Data Structures

Memory Layout

public struct CacheEntryHeader
{
    public uint KeyLength;        // 4 bytes
    public uint ValueLength;      // 4 bytes
    public uint Flags;            // 4 bytes (compressed, serialized type, etc.)
    public long ExpirationTicks;  // 8 bytes (0 = no expiry)
    public long LastAccessTicks;  // 8 bytes
    public uint HashCode;         // 4 bytes (for fast comparison)
    public uint Checksum;         // 4 bytes (CRC32 for integrity)
    // Total header: 36 bytes
}

13. API Design

Memcached-Compatible Protocol

SET user:123 0 300 36
{"name":"Alice","email":"alice@co.com"}
STORED

GET user:123
VALUE user:123 0 36
{"name":"Alice","email":"alice@co.com"}
END

DELETE user:123
DELETED

FLUSH_ALL
OK

Redis-Compatible Protocol

*3\r\n$3\r\nSET\r\n$7\r\nuser:123\r\n$36\r\n{"name":"Alice"}\r\n
+OK\r\n

*2\r\n$3\r\nGET\r\n$7\r\nuser:123\r\n
$36\r\n{"name":"Alice"}\r\n

*3\r\n$6\r\nEXPIRE\r\n$7\r\nuser:123\r\n$3\r\n300\r\n
:1\r\n

C# Client API

public interface IDistributedCacheClient
{
    // Basic operations
    Task<CacheResult<T>> GetAsync<T>(string key);
    Task SetAsync<T>(string key, T value, CacheOptions? options = null);
    Task<bool> DeleteAsync(string key);
    Task<bool> ExistsAsync(string key);

    // Batch operations
    Task<Dictionary<string, CacheResult<T>>> BatchGetAsync<T>(IEnumerable<string> keys);
    Task BatchSetAsync<T>(Dictionary<string, T> entries, CacheOptions? options = null);

    // TTL management
    Task<bool> SetTtlAsync(string key, TimeSpan ttl);
    Task<TimeSpan?> GetTtlAsync(string key);

    // Statistics
    Task<CacheClusterStats> GetStatsAsync();

    // Management
    Task<int> FlushAsync();
    Task<Dictionary<string, CacheNodeStats>> GetNodeStatsAsync();
}

public class CacheOptions
{
    public TimeSpan? Ttl { get; set; }
    public CacheWriteBehavior WriteBehavior { get; set; } = CacheWriteBehavior.WriteThrough;
    public bool Compress { get; set; }
    public string? Tags { get; set; }
}

public enum CacheWriteBehavior
{
    WriteThrough,    // Write to cache + DB simultaneously
    WriteBehind,     // Write to cache, async flush to DB
    CacheOnly        // Write to cache only
}

14. Database Design

Cache Metadata Store (PostgreSQL)

CREATE TABLE cache_clusters (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    name VARCHAR(64) UNIQUE NOT NULL,
    max_memory_bytes BIGINT NOT NULL,
    eviction_policy VARCHAR(32) NOT NULL DEFAULT 'lru',
    replication_factor INT NOT NULL DEFAULT 2,
    created_at TIMESTAMPTZ DEFAULT NOW()
);

CREATE TABLE cache_nodes (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    cluster_id UUID REFERENCES cache_clusters(id),
    host VARCHAR(255) NOT NULL,
    port INT NOT NULL,
    status VARCHAR(16) NOT NULL DEFAULT 'active',
    memory_used_bytes BIGINT DEFAULT 0,
    memory_max_bytes BIGINT NOT NULL,
    last_heartbeat TIMESTAMPTZ,
    UNIQUE(cluster_id, host, port)
);

CREATE TABLE cache_partitions (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    cluster_id UUID REFERENCES cache_clusters(id),
    partition_index INT NOT NULL,
    primary_node_id UUID REFERENCES cache_nodes(id),
    replica_node_ids UUID[] NOT NULL,
    hash_range_start BIGINT NOT NULL,
    hash_range_end BIGINT NOT NULL
);

CREATE INDEX idx_nodes_cluster ON cache_nodes(cluster_id);
CREATE INDEX idx_partitions_cluster ON cache_partitions(cluster_id);

15. Read/Write Path

Complete GET Flow

Complete SET Flow

16. Cache Consistency

Consistency Challenges

• Stale reads: Cache contains outdated data after database update.
• Thundering herd: Many clients hit a cold key simultaneously after expiration.
• Cache-DB inconsistency: Write succeeds to DB but cache invalidation fails.
• Split-brain: Network partition causes different nodes to have different values.

Invalidation Strategies

public class CacheInvalidationService
{
    private readonly IDistributedCache _cache;
    private readonly ISubscriber _pubsub;
    private readonly AppDbContext _db;

    // Pattern 1: TTL-based (simplest)
    public async Task SetWithTTL(string key, object value, TimeSpan ttl)
    {
        await _cache.SetStringAsync(key,
            JsonSerializer.Serialize(value),
            new DistributedCacheEntryOptions
            {
                AbsoluteExpirationRelativeToNow = ttl
            });
    }

    // Pattern 2: Event-driven invalidation
    public async Task InvalidateOnWrite(string key, object newValue)
    {
        // Write to DB
        await _db.SaveChangesAsync();

        // Invalidate cache
        await _cache.RemoveAsync(key);

        // Broadcast to all nodes
        await _pubsub.PublishAsync("cache:invalidate",
            JsonSerializer.Serialize(new { Key = key }));
    }

    // Pattern 3: Version-based keys
    public async Task SetWithVersion(string baseKey, object value, int version)
    {
        var versionedKey = $"{baseKey}:v{version}";
        await _cache.SetStringAsync(versionedKey,
            JsonSerializer.Serialize(value),
            new DistributedCacheEntryOptions
            {
                AbsoluteExpirationRelativeToNow = TimeSpan.FromHours(1)
            });

        // Remove old versions (optional cleanup)
        await RemoveOldVersions(baseKey, version);
    }
}

Cache-DB Synchronization

public class CacheSyncService
{
    private readonly IDistributedCache _cache;
    private readonly AppDbContext _db;

    // Write-behind with reconciliation
    public async Task<T?> GetWithReconciliation<T>(string key, Func<Task<T?>> dbLoader)
    {
        var cached = await _cache.GetStringAsync(key);
        if (cached != null)
        {
            var cacheValue = JsonSerializer.Deserialize<T>(cached);

            // Async reconciliation (non-blocking)
            _ = Task.Run(async () =>
            {
                var dbValue = await dbLoader();
                if (dbValue != null && !Equals(cacheValue, dbValue))
                {
                    // DB has newer data - update cache
                    await _cache.SetStringAsync(key,
                        JsonSerializer.Serialize(dbValue),
                        new DistributedCacheEntryOptions
                        {
                            AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(5)
                        });
                }
            });

            return cacheValue;
        }

        // Cache miss - load from DB
        var value = await dbLoader();
        if (value != null)
        {
            await _cache.SetStringAsync(key,
                JsonSerializer.Serialize(value),
                new DistributedCacheEntryOptions
                {
                    AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(5)
                });
        }
        return value;
    }
}

17. Distributed Coordination

Cluster Membership with Gossip

public class CacheClusterMembership
{
    private readonly string _localNodeId;
    private readonly Dictionary<string, CacheNodeInfo> _nodes = new();
    private readonly Timer _gossipTimer;

    public CacheClusterMembership(string localNodeId, IEnumerable<string> seedNodes)
    {
        _localNodeId = localNodeId;

        foreach (var node in seedNodes)
            _nodes[node] = new CacheNodeInfo { Id = node, Status = NodeStatus.Unknown };

        _gossipTimer = new Timer(Gossip, null, TimeSpan.Zero, TimeSpan.FromSeconds(1));
    }

    private async void Gossip(object? state)
    {
        var random = new Random();
        var targets = _nodes.Values
            .Where(n => n.Id != _localNodeId && n.Status == NodeStatus.Alive)
            .OrderBy(_ => random.Next())
            .Take(3)
            .ToList();

        foreach (var target in targets)
        {
            try
            {
                // Send our view of the cluster
                var ourState = _nodes.Values.ToDictionary(
                    n => n.Id,
                    n => new { n.Status, n.LastSeen, n.MemoryUsed });

                // Receive their view and merge
                await SendGossip(target.Id, ourState);
            }
            catch (Exception)
            {
                target.FailCount++;
                if (target.FailCount > 5)
                    target.Status = NodeStatus.Dead;
            }
        }

        // Detect failures
        var now = DateTimeOffset.UtcNow;
        foreach (var node in _nodes.Values)
        {
            if (node.Status == NodeStatus.Alive && now - node.LastSeen > TimeSpan.FromSeconds(10))
                node.Status = NodeStatus.Suspect;
        }
    }

    public string GetNodeForKey(string key)
    {
        var aliveNodes = _nodes.Values.Where(n => n.Status == NodeStatus.Alive).ToList();
        var hashRing = new ConsistentHashRing(aliveNodes.Select(n => n.Id));
        return hashRing.GetNode(key);
    }
}

Partition Rebalancing

18. Multi-Level Caching

public class MultiLevelCache<T>
{
    private readonly IMemoryCache _l1Cache;   // In-process
    private readonly IDistributedCache _l2Cache; // Redis/Memcached
    private readonly TimeSpan _l1Ttl = TimeSpan.FromSeconds(30);
    private readonly TimeSpan _l2Ttl = TimeSpan.FromMinutes(5);

    public async Task<(bool Hit, T? Value)> GetAsync(string key)
    {
        // L1: In-process memory (fastest)
        if (_l1Cache.TryGetValue<T>(key, out var l1Value))
            return (true, l1Value);

        // L2: Distributed cache
        var l2Value = await _l2Cache.GetStringAsync(key);
        if (l2Value != null)
        {
            var deserialized = JsonSerializer.Deserialize<T>(l2Value);

            // Populate L1 for next time
            _l1Cache.Set(key, deserialized, _l1Ttl);
            return (true, deserialized);
        }

        return (false, default);
    }

    public async Task SetAsync(string key, T value)
    {
        // Write to both levels
        _l1Cache.Set(key, value, _l1Ttl);
        await _l2Cache.SetStringAsync(key,
            JsonSerializer.Serialize(value),
            new DistributedCacheEntryOptions
            {
                AbsoluteExpirationRelativeToNow = _l2Ttl
            });
    }

    public async Task InvalidateAsync(string key)
    {
        _l1Cache.Remove(key);
        await _l2Cache.RemoveAsync(key);
    }
}

19. Scalability

Scaling Strategies

Auto-Scaling Based on Hit Rate

public class CacheAutoScaler
{
    private readonly CacheClusterManager _cluster;
    private readonly IMetricsCollector _metrics;

    public async Task EvaluateScaling()
    {
        var stats = await _cluster.GetClusterStats();

        // Scale up if hit rate drops below threshold
        if (stats.AverageHitRate < 0.90 && stats.MemoryUtilization > 0.85)
        {
            var nodesToAdd = CalculateNodesNeeded(stats);
            await _cluster.AddNodes(nodesToAdd);
        }

        // Scale down if hit rate is high and memory is underutilized
        if (stats.AverageHitRate > 0.98 && stats.MemoryUtilization < 0.40)
        {
            var nodesToRemove = CalculateNodesToRemove(stats);
            await _cluster.RemoveNodes(nodesToRemove);
        }
    }

    private int CalculateNodesNeeded(ClusterStats stats)
    {
        var memoryDeficit = stats.MemoryUtilization - 0.75; // Target 75%
        var entriesPerNode = stats.TotalEntries / stats.NodeCount;
        return (int)Math.Ceiling(memoryDeficit * stats.NodeCount);
    }
}

20. Distributed Systems Design

Failure Modes

Hot Key Handling

public class HotKeyDetector
{
    private readonly ConcurrentDictionary<string, SlidingWindowCounter> _accessCounts = new();
    private readonly int _hotKeyThreshold = 1000; // requests per second

    public void RecordAccess(string key)
    {
        var counter = _accessCounts.GetOrAdd(key,
            _ => new SlidingWindowCounter(1000, TimeSpan.FromSeconds(10)));
        counter.Increment();
    }

    public bool IsHotKey(string key)
    {
        if (_accessCounts.TryGetValue(key, out var counter))
            return counter.CurrentCount > _hotKeyThreshold;
        return false;
    }

    public List<string> GetHotKeys()
    {
        return _accessCounts
            .Where(kv => kv.Value.CurrentCount > _hotKeyThreshold)
            .Select(kv => kv.Key)
            .ToList();
    }
}

public class HotKeyCache
{
    private readonly IMemoryCache _localCache;
    private readonly IDistributedCache _distributedCache;

    public async Task<T?> GetAsync<T>(string key, bool isHotKey)
    {
        if (isHotKey)
        {
            // Hot keys: serve from local memory (L1)
            if (_localCache.TryGetValue<T>(key, out var local))
                return local;

            // Local miss for hot key: load from distributed cache
            var value = await LoadFromDistributed<T>(key);
            _localCache.Set(key, value, TimeSpan.FromSeconds(5)); // Short TTL for hot keys
            return value;
        }

        // Normal keys: serve from distributed cache
        return await LoadFromDistributed<T>(key);
    }
}

21. Consistency Models

Consistency in Caching

22. Reliability

Reliability Architecture

Data Recovery

public class CacheRecoveryService
{
    // Rebuild cache from database after full outage
    public async Task WarmCacheFromDatabase(
        IDistributedCache cache,
        AppDbContext db,
        IEnumerable<string> hotKeys)
    {
        var batchSize = 100;
        var batches = hotKeys.Chunk(batchSize);

        foreach (var batch in batches)
        {
            var tasks = batch.Select(async key =>
            {
                var entity = await db.FindAsync(key);
                if (entity != null)
                {
                    await cache.SetStringAsync(key,
                        JsonSerializer.Serialize(entity),
                        new DistributedCacheEntryOptions
                        {
                            AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(30)
                        });
                }
            });

            await Task.WhenAll(tasks);
        }
    }

    // Snapshot-based recovery
    public async Task<CacheSnapshot> CreateSnapshot(IDistributedCache cache)
    {
        var snapshot = new CacheSnapshot
        {
            Timestamp = DateTimeOffset.UtcNow,
            Entries = new Dictionary<string, string>()
        };

        // Iterate all keys (in production, use SCAN command)
        var keys = await GetAllKeys(cache);
        foreach (var key in keys)
        {
            var value = await cache.GetStringAsync(key);
            if (value != null)
                snapshot.Entries[key] = value;
        }

        return snapshot;
    }
}

23. Security

• Authentication: Require auth tokens for cache cluster membership. Client connections authenticated via HMAC-signed tokens.
• Encryption in transit: TLS for all cache connections. Use mutual TLS (mTLS) for cluster-internal communication.
• Encryption at rest: Optional AES-256 encryption for cached values containing sensitive data.
• Access control: Per-key prefix ACLs. Application A can only access keys with prefix "app_a:".
• Audit logging: Log all SET/DELETE operations for compliance and debugging.

24. Observability

Key Metrics

Health Check Endpoint

[ApiController]
[Route("health")]
public class CacheHealthController : ControllerBase
{
    private readonly IDistributedCacheClient _cacheClient;
    private readonly CacheClusterManager _cluster;

    [HttpGet]
    public async Task<IActionResult> HealthCheck()
    {
        var checks = new Dictionary<string, object>();

        // Test basic GET/SET
        var testKey = $"health:{Guid.NewGuid()}";
        var testValue = "ping";

        var sw = Stopwatch.StartNew();
        await _cacheClient.SetAsync(testKey, testValue, TimeSpan.FromSeconds(10));
        var getResult = await _cacheClient.GetAsync<string>(testKey);
        sw.Stop();

        checks["connectivity"] = getResult.Status == CacheStatus.Hit ? "healthy" : "unhealthy";
        checks["latency_ms"] = sw.ElapsedMilliseconds;

        // Check cluster status
        var clusterStats = await _cluster.GetClusterStats();
        checks["nodes_active"] = clusterStats.ActiveNodes;
        checks["nodes_total"] = clusterStats.TotalNodes;
        checks["hit_rate"] = clusterStats.AverageHitRate;
        checks["memory_utilization"] = clusterStats.MemoryUtilization;

        var isHealthy = (string)checks["connectivity"] == "healthy"
            && clusterStats.ActiveNodes == clusterStats.TotalNodes;

        await _cacheClient.DeleteAsync(testKey);

        return isHealthy ? Ok(checks) : StatusCode(503, checks);
    }
}

25. High Availability

Multi-Region Cache

Failover Strategy

public class CacheFailoverManager
{
    private readonly CacheClusterManager _cluster;
    private readonly ILogger<CacheFailoverManager> _logger;

    public async Task<T?> GetWithFailover<T>(string key)
    {
        var retries = 3;
        for (int i = 0; i < retries; i++)
        {
            try
            {
                var result = await _cluster.GetAsync<T>(key);
                if (result.Status == CacheStatus.Hit)
                    return result.Value;

                return null; // Genuine cache miss
            }
            catch (CacheNodeUnavailableException ex)
            {
                _logger.LogWarning("Cache node {Node} unavailable, attempt {Attempt}",
                    ex.NodeId, i + 1);

                // Mark node as failed, reroute to next replica
                _cluster.MarkNodeFailed(ex.NodeId);
            }
        }

        // All retries failed - return null (caller should load from DB)
        _logger.LogError("All cache retries failed for key {Key}", key);
        return null;
    }
}

26. Performance

Latency Comparison

Performance Optimizations

• Connection pooling: Reuse TCP connections to cache nodes. Eliminates TCP handshake overhead.
• Pipelining: Send multiple commands in a single network round-trip. Reduces RTT overhead for batch operations.
• Compression: LZ4 for values > 1KB. Reduces network bandwidth and memory usage at ~5% CPU cost.
• Serialization: Use System.Text.Json or MessagePack. Avoid slow serializers like JSON.NET for hot paths.
• Local L1 cache: In-process memory for hot keys. Eliminates network entirely for the hottest 1% of keys.
• Binary protocol: Use Redis RESP or Memcached binary protocol instead of text protocol.

27. Cost Analysis

28. Failure Scenarios

Thundering Herd Prevention

public class ThunderingHerdProtection
{
    private readonly SemaphoreSlim _loadLock = new(1, 1);

    public async Task<T?> GetOrLoad<T>(string key, Func<Task<T?>> loader, TimeSpan ttl)
    {
        var cached = await _cache.GetStringAsync(key);
        if (cached != null)
            return JsonSerializer.Deserialize<T>(cached);

        // Only one thread loads, others wait
        await _loadLock.WaitAsync();
        try
        {
            // Double-check after acquiring lock
            cached = await _cache.GetStringAsync(key);
            if (cached != null)
                return JsonSerializer.Deserialize<T>(cached);

            var value = await loader();
            if (value != null)
            {
                // Add random jitter to TTL (±10%) to prevent synchronized expiry
                var jitteredTtl = ttl.Add(TimeSpan.FromSeconds(
                    Random.Shared.Next(-(int)(ttl.TotalSeconds * 0.1),
                                        (int)(ttl.TotalSeconds * 0.1))));

                await _cache.SetStringAsync(key,
                    JsonSerializer.Serialize(value),
                    new DistributedCacheEntryOptions
                    {
                        AbsoluteExpirationRelativeToNow = jitteredTtl
                    });
            }
            return value;
        }
        finally
        {
            _loadLock.Release();
        }
    }
}

29. Technology Choices (C#)

C# Cache Stack

NuGet Packages

<PackageReference Include="StackExchange.Redis" Version="2.7.*" />
<PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="8.0.*" />
<PackageReference Include="Microsoft.Extensions.Caching.StackExchangeRedis" Version="8.0.*" />
<PackageReference Include="MessagePack" Version="2.5.*" />
<PackageReference Include="DistributedLock.Redis" Version="1.2.*" />

ASP.NET Core Integration

// Program.cs
builder.Services.AddMemoryCache();  // L1
builder.Services.AddStackExchangeRedisCache(options =>
{
    options.Configuration = builder.Configuration["Redis:ConnectionString"];
    options.InstanceName = "cache_";
});

// Custom multi-level cache registration
builder.Services.AddSingleton<IDistributedCacheClient, MultiLevelCacheClient>();

// Usage in controller
[ApiController]
[Route("api/[controller]")]
public class ProductsController : ControllerBase
{
    private readonly IDistributedCacheClient _cache;
    private readonly ProductRepository _repo;

    public ProductsController(IDistributedCacheClient cache, ProductRepository repo)
    {
        _cache = cache;
        _repo = repo;
    }

    [HttpGet("{id}")]
    public async Task<ActionResult<Product>> GetProduct(int id)
    {
        var key = $"product:{id}";
        var result = await _cache.GetAsync<Product>(key);

        if (result.Status == CacheStatus.Hit)
            return Ok(result.Value);

        var product = await _repo.GetByIdAsync(id);
        if (product != null)
            await _cache.SetAsync(key, product, new CacheOptions { Ttl = TimeSpan.FromMinutes(5) });

        return product != null ? Ok(product) : NotFound();
    }
}

30. Alternatives & Trade-offs

31. Real-World Case Studies

Facebook's Memcached Infrastructure

• Scale: 500M+ Memcached servers, 28 billion keys, 100M+ ops/sec.
• Architecture: McRouter (proxy), mcrouter replication, region-level caches.
• Innovation: Warm cache from database after outage (slab-by-slab rebuild), regional failover.
• Lesson: Simple protocols scale to enormous size. Memcached's simplicity is a feature.

Twitter's Redis Cache

• Scale: 100TB+ across thousands of Redis instances.
• Workload: User timelines, session data, rate limiting counters.
• Optimization: Client-side caching for hot keys, pipeline batching.
• Lesson: Client-side intelligence reduces server load significantly.

Netflix's EVCache

• Scale: 100+ million items, multiple regions.
• Innovation: Custom Memcached fork with replication, regional routing, smart client.
• Lesson: Building on proven technology (Memcached) and adding production features (replication, monitoring) is often better than building from scratch.

32. Interview Follow-ups

33. Senior/Staff/Principal Discussion

Architectural Decisions

• Cache vs CDN vs application-level optimization: Caching is not always the answer. Sometimes a better database query or an application-level optimization is more effective than adding a cache layer.
• When NOT to cache: Rarely accessed data (low hit rate), rapidly changing data (high invalidation overhead), data that must be strongly consistent.
• Cache hierarchy design: The right number of cache levels depends on latency requirements and access patterns. Not every system needs three tiers.
• Cost vs performance: More cache memory = higher hit rate = better performance, but diminishing returns past 95% hit rate.

Operational Excellence

• Capacity planning: Monitor hit rate trends, memory utilization growth, and traffic patterns to plan cache expansions 3-6 months ahead.
• Chaos testing: Regularly kill cache nodes to validate failover behavior. Test full cache cluster failure to verify DB can handle the load.
• Runbooks: Document procedures for hot key mitigation, cache cluster rebuild, memory leak investigation, and thundering herd response.

34. Architecture Evolution

35. Key Takeaways

Interview Quick Reference

36. References

• Designing Data-Intensive Applications (Kleppmann, 2017) — Chapter on caching
• Scaling Memcache at Facebook (Nishtala et al., 2013) — Foundational paper on distributed caching
• An Analysis of Facebook Photo Caching (Qiu et al., 2016) — Real-world cache optimization
• Caffeine: A High Performance Cache Library — W-TinyLFU implementation
• Redis Documentation — Data structures, eviction policies, cluster mode
• Memcached Wiki — Architecture, slab allocation, distributed hashing
• NCache Documentation — .NET distributed cache
• StackExchange.Redis Documentation — C# Redis client best practices
• Achieving Consistent Performance at Scale with EVCache — Netflix caching architecture

37. Conclusion

A well-designed distributed cache is one of the highest-leverage investments in system architecture. It can reduce database load by 95%, improve response times by 100x, and enable systems to handle traffic that would otherwise require 10x more database infrastructure.

The key to effective caching is understanding your workload. A cache with a 95% hit rate serves 95% of requests in sub-millisecond time. But achieving that hit rate requires the right eviction policy (LRU for most cases), the right TTL (short enough for consistency, long enough for efficiency), and the right invalidation strategy (TTL + event-driven for most cases).

In your interview, start with the simplest design (cache-aside with LRU + TTL), then discuss how you'd handle failures (fail open, stampede protection) and scale (consistent hashing, multi-level caching). This progression demonstrates both practical knowledge and architectural depth.

38. Cache Warming and Preloading Strategies

Cold cache starts cause latency spikes that degrade user experience. Cache warming pre-populates the most frequently accessed keys before traffic arrives. A production cache warming system analyzes access patterns, prioritizes hot keys, and coordinates warming across cluster nodes to avoid thundering herd on the database during startup.

public class CacheWarmer
{
    private readonly IAccessPatternAnalyzer _analyzer;
    private readonly IDistributedCache _cache;
    private readonly IDatabase _database;

    public async Task WarmCacheAsync(WarmingPolicy policy)
    {
        var hotKeys = await _analyzer.GetHotKeysAsync(
            policy.TopPercentile,
            policy.MinAccessCount);

        var batchSize = 500;
        var semaphore = new Semaphore(policy.MaxConcurrency);

        var tasks = hotKeys.Batch(batchSize).Select(async batch =>
        {
            await semaphore.WaitAsync();
            try
            {
                var dbTasks = batch.Select(async key =>
                {
                    var value = await _database.GetAsync(key);
                    if (value != null)
                    {
                        await _cache.SetAsync(key, value,
                            new DistributedCacheEntryOptions
                            {
                                AbsoluteExpirationRelativeToNow =
                                    TimeSpan.FromMinutes(policy.TtlMinutes)
                            });
                    }
                });
                await Task.WhenAll(dbTasks);
            }
            finally
            {
                semaphore.Release();
            }
        });

        await Task.WhenAll(tasks);
    }
}

Warming Strategy Comparison