Table of Contents

1. Introduction — The Chat System Landscape
2. Functional & Non-Functional Requirements
3. Capacity Estimation & Back-of-Envelope
4. Data Model & Storage Schema
5. API Design
6. High-Level Architecture
7. WebSocket Connection Management
8. Message Delivery Pipeline
9. Presence & Online Status
10. Group Chat & Channels
11. Media & File Sharing
12. Message Search
13. Read Receipts & Typing Indicators
14. End-to-End Encryption
15. Offline Support & Sync
16. Scaling WebSocket Connections
17. Reliability & Delivery Guarantees
18. Push Notifications for Chat
19. Content Moderation
20. Monitoring & Observability
21. Case Studies — Production Systems
22. Cost Estimation
23. Edge Cases
24. Interview Q&A
25. Conclusion

1. Introduction — The Chat System Landscape

Real-time chat systems are among the most complex distributed systems in production today. WhatsApp handles 100 billion messages per day across 2 billion users. Slack processes 1.5 billion messages daily. Discord handles 4 billion messages per day across 150 million monthly active users. These systems must deliver messages with sub-second latency, handle millions of concurrent WebSocket connections, support group conversations of up to 200K members, and maintain message history across years of conversation.

Building a chat system requires mastering several distributed systems concepts: persistent WebSocket connections for real-time delivery, message queues for offline users, consistent hashing for connection routing, conflict resolution for concurrent edits, and end-to-end encryption for privacy. Unlike stateless HTTP APIs, chat systems are inherently stateful — the server must know which users are connected, which servers they are connected to, and how to route messages between them.

2. Functional & Non-Functional Requirements

Functional Requirements

Non-Functional Requirements

3. Capacity Estimation & Back-of-Envelope

Daily Volume Estimates

Connection Estimates

4. Data Model & Storage Schema

Entity Relationship

Cassandra Schema (Messages — Write-Optimized)

CQL
CREATE TABLE messages_by_conversation (
    conversation_id timeuuid,
    message_id timeuuid,
    sender_id bigint,
    content text,
    message_type text,       -- 'text', 'image', 'video', 'file', 'system'
    reply_to_id timeuuid,
    status text,             -- 'sent', 'delivered', 'read'
    created_at timestamp,
    PRIMARY KEY (conversation_id, message_id)
) WITH CLUSTERING ORDER BY (message_id DESC)
  AND default_time_to_live = 315360000  -- 10 years
  AND gc_grace_seconds = 864000;        -- 10 days

CREATE TABLE messages_by_user (
    user_id bigint,
    last_active_conversation_id timeuuid,
    message_id timeuuid,
    conversation_id timeuuid,
    content text,
    created_at timestamp,
    PRIMARY KEY (user_id, message_id)
) WITH CLUSTERING ORDER BY (message_id DESC);

-- For search: denormalized table indexed by content
CREATE TABLE messages_by_conversation_text (
    conversation_id timeuuid,
    message_id timeuuid,
    sender_id bigint,
    content text,
    created_at timestamp,
    PRIMARY KEY (conversation_id, message_id)
) WITH CLUSTERING ORDER BY (message_id DESC);
            

PostgreSQL Schema (Users & Conversations — Read-Optimized)

SQL
CREATE TABLE users (
    id BIGSERIAL PRIMARY KEY,
    username VARCHAR(50) UNIQUE NOT NULL,
    display_name VARCHAR(100) NOT NULL,
    email VARCHAR(255) UNIQUE NOT NULL,
    phone VARCHAR(20),
    avatar_url TEXT,
    status VARCHAR(20) DEFAULT 'offline',
    last_seen_at TIMESTAMP,
    created_at TIMESTAMP DEFAULT NOW()
);

CREATE TABLE conversations (
    id BIGSERIAL PRIMARY KEY,
    type VARCHAR(20) NOT NULL,      -- 'direct', 'group', 'channel'
    name VARCHAR(100),
    avatar_url TEXT,
    creator_id BIGINT REFERENCES users(id),
    max_members INTEGER DEFAULT 200,
    created_at TIMESTAMP DEFAULT NOW(),
    updated_at TIMESTAMP DEFAULT NOW()
);

CREATE TABLE conversation_members (
    conversation_id BIGINT REFERENCES conversations(id),
    user_id BIGINT REFERENCES users(id),
    role VARCHAR(20) DEFAULT 'member',  -- 'admin', 'member', 'readonly'
    joined_at TIMESTAMP DEFAULT NOW(),
    last_read_at TIMESTAMP,
    notification_setting VARCHAR(20) DEFAULT 'all',
    muted_until TIMESTAMP,
    PRIMARY KEY (conversation_id, user_id)
);

CREATE INDEX idx_members_user ON conversation_members(user_id);
CREATE INDEX idx_conversations_updated ON conversations(updated_at DESC);
            

Why Cassandra for Messages?

5. API Design

REST APIs

HTTP
// Create conversation
POST /api/v1/conversations
{
    "type": "group",
    "name": "Engineering Team",
    "member_ids": [101, 102, 103, 104, 105]
}

// Get conversation list
GET /api/v1/conversations?page_size=20&cursor=abc123

// Get conversation details
GET /api/v1/conversations/{conversation_id}

// Add member to group
POST /api/v1/conversations/{conversation_id}/members
{
    "user_id": 106,
    "role": "member"
}

// Remove member
DELETE /api/v1/conversations/{conversation_id}/members/{user_id}

// Get message history (paginated)
GET /api/v1/conversations/{conversation_id}/messages?page_size=50&before=message_id_123

// Search messages
GET /api/v1/search/messages?q=hello&conversation_id=123&from_date=2025-01-01

// Upload media
POST /api/v1/media/upload
Content-Type: multipart/form-data
// Returns: { "media_id": "abc", "url": "https://cdn.example.com/abc.jpg" }

// Send message via REST (fallback for clients without WebSocket)
POST /api/v1/conversations/{conversation_id}/messages
{
    "content": "Hello everyone!",
    "message_type": "text",
    "reply_to_id": "msg_123"
}
            

WebSocket Protocol

JSON
// Client → Server: Send message
{
    "type": "message_send",
    "id": "client_msg_abc123",    // Client-generated ID for dedup
    "conversation_id": "conv_123",
    "content": "Hello world!",
    "message_type": "text",
    "timestamp": 1705123456789
}

// Server → Client: New message received
{
    "type": "message_received",
    "message_id": "msg_xyz789",
    "conversation_id": "conv_123",
    "sender_id": 101,
    "sender_name": "Alice",
    "content": "Hello world!",
    "message_type": "text",
    "created_at": "2025-01-15T10:30:00Z"
}

// Server → Client: Message delivered confirmation
{
    "type": "message_delivered",
    "client_id": "client_msg_abc123",
    "message_id": "msg_xyz789",
    "server_timestamp": 1705123456800
}

// Client → Server: Typing indicator
{
    "type": "typing_start",
    "conversation_id": "conv_123"
}

// Server → Client: User typing
{
    "type": "user_typing",
    "conversation_id": "conv_123",
    "user_id": 101,
    "user_name": "Alice"
}

// Client → Server: Mark as read
{
    "type": "mark_read",
    "conversation_id": "conv_123",
    "last_message_id": "msg_xyz789"
}

// Server → Client: Presence update
{
    "type": "presence_update",
    "user_id": 102,
    "status": "online",
    "last_seen_at": "2025-01-15T10:30:00Z"
}
            

6. High-Level Architecture

Component Responsibilities

7. WebSocket Connection Management

Connection Server Implementation

C#
public class WebSocketConnectionHandler
{
    private readonly IConnectionMultiplexer _redis;
    private readonly IProducer<string, PresenceEvent> _presenceProducer;
    private readonly ConcurrentDictionary<string, WebSocket> _connections = new();

    public async Task HandleConnectionAsync(HttpContext context)
    {
        var userId = await AuthenticateAsync(context);
        if (userId == null)
        {
            context.Response.StatusCode = 401;
            return;
        }

        var ws = await context.WebSockets.AcceptWebSocketAsync();
        var connectionId = Guid.NewGuid().ToString();

        _connections[connectionId] = ws;

        // Store connection mapping in Redis
        var db = _redis.GetDatabase();
        var serverId = Environment.MachineName;

        await db.HashSetAsync("user_connections", userId,
            $"{serverId}:{connectionId}:{DateTimeOffset.UtcNow.ToUnixTimeMilliseconds()}");

        // Publish online presence
        await _presenceProducer.ProduceAsync("presence-events", new Message<string, PresenceEvent>
        {
            Key = userId,
            Value = new PresenceEvent
            {
                UserId = userId,
                Status = "online",
                ServerId = serverId,
                ConnectionId = connectionId,
                Timestamp = DateTimeOffset.UtcNow
            }
        });

        // Listen for incoming messages
        var buffer = new byte[4096];
        try
        {
            while (ws.State == WebSocketState.Open)
            {
                var result = await ws.ReceiveAsync(new ArraySegment<byte>(buffer), CancellationToken.None);

                if (result.MessageType == WebSocketMessageType.Close)
                {
                    await ws.CloseAsync(WebSocketCloseStatus.NormalClosure, "", CancellationToken.None);
                }
                else if (result.MessageType == WebSocketMessageType.Text)
                {
                    var message = Encoding.UTF8.GetString(buffer, 0, result.Count);
                    await HandleMessageAsync(userId, connectionId, message);
                }
            }
        }
        finally
        {
            _connections.TryRemove(connectionId, out _);
            await db.HashDeleteAsync("user_connections", userId);

            // Publish offline presence
            await _presenceProducer.ProduceAsync("presence-events", new Message<string, PresenceEvent>
            {
                Key = userId,
                Value = new PresenceEvent
                {
                    UserId = userId,
                    Status = "offline",
                    ServerId = serverId,
                    ConnectionId = connectionId,
                    Timestamp = DateTimeOffset.UtcNow
                }
            });
        }
    }
}
            

Connection Routing Strategies

Heartbeat and Timeout

C#
public class ConnectionHealthMonitor
{
    private readonly TimeSpan _heartbeatInterval = TimeSpan.FromSeconds(30);
    private readonly TimeSpan _timeoutInterval = TimeSpan.FromSeconds(90);

    public async Task MonitorConnectionsAsync(CancellationToken ct)
    {
        while (!ct.IsCancellationRequested)
        {
            foreach (var (connectionId, socket) in _connections)
            {
                if (socket.State != WebSocketState.Open)
                {
                    _connections.TryRemove(connectionId, out _);
                    continue;
                }

                var lastActivity = await GetLastActivityAsync(connectionId);
                if (DateTime.UtcNow - lastActivity > _timeoutInterval)
                {
                    // Connection stale � close it
                    await socket.CloseAsync(
                        WebSocketCloseStatus.NormalClosure,
                        "Heartbeat timeout",
                        CancellationToken.None);
                    _connections.TryRemove(connectionId, out _);
                    continue;
                }

                // Send ping
                var pingBytes = Encoding.UTF8.GetBytes("{\"type\":\"ping\"}");
                await socket.SendAsync(
                    new ArraySegment<byte>(pingBytes),
                    WebSocketMessageType.Text,
                    true,
                    CancellationToken.None);
            }

            await Task.Delay(_heartbeatInterval, ct);
        }
    }
}
            

8. Message Delivery Pipeline

Message Processing Flow

C#
public class ChatService
{
    private readonly IProducer<string, ChatMessage> _kafkaProducer;
    private readonly IMessageRepository _messageRepo;
    private readonly IConnectionRouter _connectionRouter;
    private readonly IPresenceService _presenceService;

    public async Task<MessageResult> ProcessMessageAsync(ChatMessage message)
    {
        // 1. Assign server-side ID and timestamp
        message.Id = GenerateTimeUUID();
        message.ServerTimestamp = DateTime.UtcNow;
        message.Status = "sent";

        // 2. Publish to Kafka for async processing
        await _kafkaProducer.ProduceAsync("chat-messages",
            new Message<string, ChatMessage>
            {
                Key = message.ConversationId,
                Value = message
            });

        // 3. Return acknowledgment to sender immediately
        return new MessageResult
        {
            ClientId = message.ClientMessageId,
            ServerMessageId = message.Id,
            Status = "sent",
            ServerTimestamp = message.ServerTimestamp
        };
    }

    public async Task OnMessageConsumedAsync(ChatMessage message)
    {
        // 1. Persist to Cassandra
        await _messageRepo.SaveAsync(message);

        // 2. Get all conversation members
        var members = await _conversationRepo.GetMembersAsync(message.ConversationId);

        // 3. Deliver to online members
        foreach (var memberId in members.Where(m => m != message.SenderId))
        {
            var isOnline = await _presenceService.IsOnlineAsync(memberId);

            if (isOnline)
            {
                // Find the WebSocket server this user is connected to
                var serverInfo = await _connectionRouter.FindServerAsync(memberId);

                if (serverInfo != null)
                {
                    // Route via gRPC to the correct WS server
                    await _connectionRouter.RouteToServerAsync(serverInfo, message);
                }
            }
            else
            {
                // Queue for push notification
                await QueuePushNotificationAsync(memberId, message);
            }
        }
    }
}
            

Message Ordering

9. Presence & Online Status

Redis Presence Implementation

C#
public class PresenceService
{
    private readonly IConnectionMultiplexer _redis;
    private readonly TimeSpan _onlineTtl = TimeSpan.FromMinutes(5);
    private readonly TimeSpan _statusTtl = TimeSpan.FromDays(7);

    public async Task SetOnlineAsync(string userId)
    {
        var db = _redis.GetDatabase();

        // Set online status with TTL (auto-expires if heartbeat stops)
        await db.StringSetAsync($"presence:{userId}:status", "online", _onlineTtl);

        // Update last seen timestamp
        await db.StringSetAsync($"presence:{userId}:last_seen",
            DateTimeOffset.UtcNow.ToUnixTimeMilliseconds().ToString(),
            _statusTtl);

        // Add to online set for efficient bulk queries
        await db.SetAddAsync("online_users", userId);
    }

    public async Task SetOfflineAsync(string userId)
    {
        var db = _redis.GetDatabase();

        await db.StringSetAsync($"presence:{userId}:status", "offline", _statusTtl);
        await db.StringSetAsync($"presence:{userId}:last_seen",
            DateTimeOffset.UtcNow.ToUnixTimeMilliseconds().ToString(),
            _statusTtl);
        await db.SetRemoveAsync("online_users", userId);
    }

    public async Task<PresenceInfo> GetPresenceAsync(string userId)
    {
        var db = _redis.GetDatabase();

        var status = await db.StringGetAsync($"presence:{userId}:status");
        var lastSeen = await db.StringGetAsync($"presence:{userId}:last_seen");

        return new PresenceInfo
        {
            Status = status.HasValue ? status.ToString() : "offline",
            LastSeenAt = lastSeen.HasValue
                ? DateTimeOffset.FromUnixTimeMilliseconds(long.Parse(lastSeen))
                : null
        };
    }

    public async Task<Dictionary<string, PresenceInfo>> GetBulkPresenceAsync(string[] userIds)
    {
        var db = _redis.GetDatabase();
        var results = new Dictionary<string, PresenceInfo>();

        // Pipeline all Redis calls
        var tasks = userIds.Select(async userId =>
        {
            var info = await GetPresenceAsync(userId);
            lock (results) { results[userId] = info; }
        });

        await Task.WhenAll(tasks);
        return results;
    }

    // Heartbeat to keep online status fresh
    public async Task HeartbeatAsync(string userId)
    {
        var db = _redis.GetDatabase();
        await db.StringSetAsync($"presence:{userId}:status", "online", _onlineTtl);
    }
}
            

Presence Strategies Comparison

10. Group Chat & Channels

Group Size Considerations

Large Group Delivery Strategy

C#
public class GroupMessageDelivery
{
    public async Task DeliverToGroupAsync(ChatMessage message, Conversation conversation)
    {
        var memberCount = await _conversationRepo.GetMemberCountAsync(conversation.Id);

        if (memberCount <= 100)
        {
            // Small group: fan-out to all members directly
            await FanOutToMembersAsync(message, conversation);
        }
        else if (memberCount <= 10000)
        {
            // Medium group: fan-out with batching
            await FanOutWithBatchingAsync(message, conversation);
        }
        else
        {
            // Large channel: topic-based delivery
            await TopicBasedDeliveryAsync(message, conversation);
        }
    }

    private async Task TopicBasedDeliveryAsync(ChatMessage message, Conversation conversation)
    {
        // For large channels, use a pub/sub topic model
        // Members subscribe to topics, only receive messages for subscribed topics

        var topic = $"conversation:{conversation.Id}";

        // Publish to Kafka topic � only online subscribers consume
        await _kafkaProducer.ProduceAsync("large-group-messages",
            new Message<string, ChatMessage>
            {
                Key = conversation.Id.ToString(),
                Value = message,
                Headers = new Headers
                {
                    { "topic", Encoding.UTF8.GetBytes(topic) },
                    { "conversation_type", Encoding.UTF8.GetBytes("channel") }
                }
            });
    }
}
            

11. Media & File Sharing

Media Processing Pipeline

C#
public class MediaService
{
    private readonly IS3Client _s3;
    private readonly IImageProcessor _imageProcessor;
    private readonly IVideoProcessor _videoProcessor;

    public async Task<MediaUploadResult> UploadAsync(MediaUploadRequest request)
    {
        // Generate unique media ID
        var mediaId = Guid.NewGuid().ToString();

        // Store original in S3
        var originalKey = $"media/{mediaId}/original/{request.FileName}";
        await _s3.PutObjectAsync(new PutObjectRequest
        {
            BucketName = _config.MediaBucket,
            Key = originalKey,
            InputStream = request.FileStream,
            ContentType = request.ContentType
        });

        // Process based on media type
        var variants = new List<MediaVariant>();

        if (request.ContentType.StartsWith("image/"))
        {
            variants = await ProcessImageAsync(mediaId, request);
        }
        else if (request.ContentType.StartsWith("video/"))
        {
            variants = await ProcessVideoAsync(mediaId, request);
        }

        // Store metadata
        var metadata = new MediaMetadata
        {
            Id = mediaId,
            OriginalKey = originalKey,
            FileName = request.FileName,
            ContentType = request.ContentType,
            FileSize = request.FileStream.Length,
            Variants = variants,
            UploadedBy = request.UserId,
            UploadedAt = DateTime.UtcNow
        };

        await _mediaRepo.SaveAsync(metadata);

        return new MediaUploadResult
        {
            MediaId = mediaId,
            Url = GetCdnUrl(originalKey),
            Variants = variants.Select(v => new MediaVariantResult
            {
                Url = GetCdnUrl(v.Key),
                Width = v.Width,
                Height = v.Height,
                FileSize = v.FileSize
            }).ToList()
        };
    }

    private async Task<List<MediaVariant>> ProcessImageAsync(string mediaId, MediaUploadRequest request)
    {
        var variants = new List<MediaVariant>();

        var sizes = new[] { (150, 150, "thumbnail"), (800, 600, "medium") };

        foreach (var (width, height, label) in sizes)
        {
            var resized = await _imageProcessor.ResizeAsync(request.FileStream, width, height);
            var key = $"media/{mediaId}/{label}/{label}.webp";

            await _s3.PutObjectAsync(new PutObjectRequest
            {
                BucketName = _config.MediaBucket,
                Key = key,
                InputStream = resized,
                ContentType = "image/webp"
            });

            variants.Add(new MediaVariant
            {
                Key = key,
                Width = width,
                Height = height,
                Label = label
            });
        }

        return variants;
    }
}
            

12. Message Search

Search Architecture

Elasticsearch Index Mapping

JSON
{
    "mappings": {
        "properties": {
            "message_id": { "type": "keyword" },
            "conversation_id": { "type": "keyword" },
            "sender_id": { "type": "long" },
            "content": {
                "type": "text",
                "analyzer": "standard",
                "fields": {
                    "keyword": { "type": "keyword" },
                    "autocomplete": {
                        "type": "text",
                        "analyzer": "autocomplete_analyzer"
                    }
                }
            },
            "message_type": { "type": "keyword" },
            "created_at": { "type": "date" },
            "members": { "type": "long" }
        }
    },
    "settings": {
        "analysis": {
            "analyzer": {
                "autocomplete_analyzer": {
                    "type": "custom",
                    "tokenizer": "autocomplete_tokenizer",
                    "filter": ["lowercase"]
                }
            },
            "tokenizer": {
                "autocomplete_tokenizer": {
                    "type": "edge_ngram",
                    "min_gram": 2,
                    "max_gram": 10,
                    "token_chars": ["letter", "digit"]
                }
            }
        }
    }
}
            

Search API

HTTP
// Search messages
GET /api/v1/search/messages
{
    "query": "meeting tomorrow",
    "conversation_id": "conv_123",     // optional: scope to conversation
    "sender_id": 101,                 // optional: filter by sender
    "from_date": "2025-01-01",
    "to_date": "2025-01-31",
    "message_type": "text",
    "page_size": 20,
    "cursor": "search_cursor_abc"
}

// Response
{
    "results": [
        {
            "message_id": "msg_xyz789",
            "conversation_id": "conv_123",
            "conversation_name": "Engineering Team",
            "sender_name": "Alice",
            "content": "Let's schedule a meeting tomorrow at 3pm",
            "highlight": "...meeting tomorrow at 3pm...",
            "created_at": "2025-01-15T10:30:00Z"
        }
    ],
    "total_hits": 42,
    "cursor": "search_cursor_def"
}
            

13. Read Receipts & Typing Indicators

Read Receipt State Machine

C#
public class ReadReceiptService
{
    private readonly IConnectionMultiplexer _redis;

    public async Task UpdateLastReadAsync(string userId, string conversationId, string messageId)
    {
        var db = _redis.GetDatabase();

        // Update last read position for this user in this conversation
        await db.HashSetAsync($"read_positions:{conversationId}", userId, messageId);

        // Publish read event for real-time delivery to other members
        await _kafkaProducer.ProduceAsync("read-receipts",
            new Message<string, ReadReceiptEvent>
            {
                Key = conversationId,
                Value = new ReadReceiptEvent
                {
                    ConversationId = conversationId,
                    UserId = userId,
                    LastReadMessageId = messageId,
                    Timestamp = DateTime.UtcNow
                }
            });
    }

    public async Task<Dictionary<string, string>> GetReadPositionsAsync(string conversationId)
    {
        var db = _redis.GetDatabase();
        var entries = await db.HashGetAllAsync($"read_positions:{conversationId}");

        return entries.ToDictionary(
            e => e.Name.ToString(),
            e => e.Value.ToString());
    }

    // Calculate how many unread messages each member has
    public async Task<Dictionary<string, int>> GetUnreadCountsAsync(
        string conversationId, string latestMessageId)
    {
        var positions = await GetReadPositionsAsync(conversationId);
        var result = new Dictionary<string, int>();

        // Get all message IDs up to latest
        var messages = await _messageRepo.GetMessageIdsAsync(conversationId, latestMessageId);

        foreach (var (userId, lastReadId) in positions)
        {
            var unreadIndex = messages.FindIndex(m => m == lastReadId);
            result[userId] = unreadIndex >= 0 ? messages.Count - unreadIndex - 1 : messages.Count;
        }

        return result;
    }
}
            

Typing Indicator Implementation

C#
public class TypingIndicatorService
{
    private readonly IConnectionMultiplexer _redis;
    private readonly TimeSpan _typingTtl = TimeSpan.FromSeconds(5);

    public async Task UserStartedTypingAsync(string userId, string conversationId)
    {
        var db = _redis.GetDatabase();

        // Store typing status with TTL (auto-expires if user stops)
        await db.StringSetAsync(
            $"typing:{conversationId}:{userId}",
            "1",
            _typingTtl);

        // Publish typing event
        await _publisher.PublishAsync($"typing:{conversationId}",
            JsonSerializer.Serialize(new TypingEvent
            {
                ConversationId = conversationId,
                UserId = userId,
                IsTyping = true
            }));
    }

    public async Task UserStoppedTypingAsync(string userId, string conversationId)
    {
        var db = _redis.GetDatabase();
        await db.KeyDeleteAsync($"typing:{conversationId}:{userId}");

        await _publisher.PublishAsync($"typing:{conversationId}",
            JsonSerializer.Serialize(new TypingEvent
            {
                ConversationId = conversationId,
                UserId = userId,
                IsTyping = false
            }));
    }

    public async Task<List<string>> GetTypingUsersAsync(string conversationId)
    {
        var db = _redis.GetDatabase();
        var pattern = $"typing:{conversationId}:*";
        var keys = new List<string>();

        var server = _redis.GetServer(_redis.GetEndPoints().First());
        foreach (var key in server.Keys(pattern: pattern))
        {
            var userId = key.ToString().Split(':').Last();
            keys.Add(userId);
        }

        return keys;
    }
}
            

14. End-to-End Encryption

Signal Protocol Overview

E2E Encryption Architecture

C#
public class EndToEndEncryptionService
{
    // Generate and store pre-key bundles for offline key exchange
    public async Task<PreKeyBundle> GeneratePreKeyBundleAsync(string userId)
    {
        var identityKeyPair = KeyHelper.GenerateIdentityKeyPair();
        var signedPreKey = KeyHelper.GenerateSignedPreKey(identityKeyPair, 1);
        var oneTimePreKeys = KeyHelper.GeneratePreKeys(100);

        // Upload public keys to server
        await _keyStore.StoreIdentityKeyAsync(userId, identityKeyPair.PublicKey);
        await _keyStore.StoreSignedPreKeyAsync(userId, signedPreKey);
        await _keyStore.StoreOneTimePreKeysAsync(userId, oneTimePreKeys.Select(k => k.PublicKey));

        return new PreKeyBundle
        {
            UserId = userId,
            IdentityKey = identityKeyPair.PublicKey,
            SignedPreKey = signedPreKey.PublicKey,
            SignedPreKeySignature = signedPreKey.Signature,
            OneTimePreKey = oneTimePreKeys[0].PublicKey,
            PreKeyId = oneTimePreKeys[0].Id
        };
    }

    // Encrypt message before sending (client-side)
    public EncryptedMessage EncryptMessage(
        string plaintext, string senderIdentityKey, string recipientPreKeyBundle)
    {
        // Perform X3DH key agreement
        var sharedSecret = SignalProtocol.PerformX3DH(
            senderIdentityKey,
            recipientPreKeyBundle.IdentityKey,
            recipientPreKeyBundle.SignedPreKey,
            recipientPreKeyBundle.OneTimePreKey);

        // Derive message key from ratchet
        var messageKey = _ratchet.GetMessageKey(sharedSecret);

        // Encrypt with AES-256-GCM
        var encrypted = AES256GCM.Encrypt(
            Encoding.UTF8.GetBytes(plaintext),
            messageKey,
            out var nonce,
            out var authTag);

        return new EncryptedMessage
        {
            Ciphertext = encrypted,
            Nonce = nonce,
            AuthTag = authTag,
            RatchetPublic = _ratchet.GetPublicRatchetKey(),
            PreviousChainLength = _ratchet.PreviousChainLength,
            MessageNumber = _ratchet.MessageNumber
        };
    }
}
            

15. Offline Support & Message Sync

Offline Message Queue

C#
public class OfflineMessageQueue
{
    private readonly IConnectionMultiplexer _redis;

    // When a message is sent to an offline user, queue it
    public async Task EnqueueForOfflineUserAsync(string userId, ChatMessage message)
    {
        var db = _redis.GetDatabase();

        var entry = new OfflineMessageEntry
        {
            MessageId = message.Id,
            ConversationId = message.ConversationId,
            SenderId = message.SenderId,
            Content = message.Content,
            CreatedAt = message.CreatedAt,
            QueuedAt = DateTime.UtcNow
        };

        await db.ListLeftPushAsync(
            $"offline_queue:{userId}",
            JsonSerializer.Serialize(entry));

        // Set TTL (7 days for offline messages)
        await db.KeyExpireAsync($"offline_queue:{userId}", TimeSpan.FromDays(7));
    }

    // When user comes online, drain the queue
    public async Task<List<ChatMessage>> DrainQueueAsync(string userId, int batchSize = 100)
    {
        var db = _redis.GetDatabase();
        var messages = new List<ChatMessage>();

        for (int i = 0; i < batchSize; i++)
        {
            var entry = await db.ListRightPopAsync($"offline_queue:{userId}");
            if (entry.IsNullOrEmpty) break;

            var offlineMsg = JsonSerializer.Deserialize<OfflineMessageEntry>(entry.ToString());
            messages.Add(new ChatMessage
            {
                Id = offlineMsg.MessageId,
                ConversationId = offlineMsg.ConversationId,
                SenderId = offlineMsg.SenderId,
                Content = offlineMsg.Content,
                CreatedAt = offlineMsg.CreatedAt
            });
        }

        return messages;
    }
}
            

Client Sync Strategy

16. Scaling WebSocket Connections

Horizontal Scaling Pattern

Scaling Numbers

Consistent Hashing for Routing

C#
public class ConsistentHashRouter
{
    private readonly SortedDictionary<uint, string> _ring = new();
    private readonly int _virtualNodes = 150;

    public void AddServer(string serverId)
    {
        for (int i = 0; i < _virtualNodes; i++)
        {
            var hash = ComputeHash($"{serverId}:vn{i}");
            _ring[hash] = serverId;
        }
    }

    public void RemoveServer(string serverId)
    {
        for (int i = 0; i < _virtualNodes; i++)
        {
            var hash = ComputeHash($"{serverId}:vn{i}");
            _ring.Remove(hash);
        }
    }

    public string GetServer(string userId)
    {
        if (_ring.Count == 0) throw new InvalidOperationException("No servers available");

        var hash = ComputeHash(userId);

        // Find the first server clockwise from the hash
        var server = _ring.FirstOrDefault(kvp => kvp.Key >= hash);
        if (server.Value == null)
        {
            server = _ring.First(); // Wrap around
        }

        return server.Value;
    }

    // Find the server responsible for a given user
    public async Task RouteMessageAsync(string recipientUserId, ChatMessage message)
    {
        var targetServer = GetServer(recipientUserId);

        // Send via internal gRPC to the target server
        var channel = _grpcChannels.GetOrAdd(targetServer,
            id => new Channel(id, 5000, ChannelCredentials.Insecure));

        var client = new WebSocketRouter.WebSocketRouterClient(channel);
        await client.DeliverMessageAsync(new DeliveryRequest
        {
            UserId = recipientUserId,
            Message = message
        });
    }
}
            

17. Reliability & Delivery Guarantees

Message Delivery States

At-Least-Once Delivery

C#
public class ReliableMessageDelivery
{
    private readonly IConnectionMultiplexer _redis;

    public async Task<DeliveryResult> DeliverWithAckAsync(ChatMessage message, string recipientUserId)
    {
        var deliveryId = Guid.NewGuid().ToString();
        var db = _redis.GetDatabase();

        // Store pending delivery
        await db.HashSetAsync($"pending_deliveries:{recipientUserId}",
            deliveryId,
            JsonSerializer.Serialize(new PendingDelivery
            {
                MessageId = message.Id,
                DeliveryId = deliveryId,
                Attempts = 0,
                CreatedAt = DateTime.UtcNow
            }));

        // Send to recipient
        var sent = await _connectionRouter.SendToUserAsync(recipientUserId, message);

        if (!sent)
        {
            // Recipient not connected � queue for later delivery
            await _offlineQueue.EnqueueForOfflineUserAsync(recipientUserId, message);
            await db.HashDeleteAsync($"pending_deliveries:{recipientUserId}", deliveryId);
            return DeliveryResult.QueuedForLaterDelivery;
        }

        // Wait for ACK with timeout
        var ackReceived = await WaitForAckAsync(deliveryId, TimeSpan.FromSeconds(30));

        await db.HashDeleteAsync($"pending_deliveries:{recipientUserId}", deliveryId);

        return ackReceived ? DeliveryResult.Delivered : DeliveryResult.Timeout;
    }

    // Retry failed deliveries
    public async Task RetryFailedDeliveriesAsync()
    {
        var db = _redis.GetDatabase();
        var servers = _connectionRouter.GetAllServers();

        foreach (var serverId in servers)
        {
            var pendingKeys = await db.HashGetAllAsync($"pending_deliveries:*");

            foreach (var entry in pendingKeys)
            {
                var delivery = JsonSerializer.Deserialize<PendingDelivery>(entry.Value);

                if (delivery.Attempts < 3 && DateTime.UtcNow - delivery.CreatedAt > TimeSpan.FromSeconds(30))
                {
                    delivery.Attempts++;
                    // Retry delivery
                    await db.HashSetAsync(
                        $"pending_deliveries:{entry.Name}",
                        delivery.DeliveryId,
                        JsonSerializer.Serialize(delivery));
                }
                else if (delivery.Attempts >= 3)
                {
                    // Move to dead letter queue
                    await MoveToDeadLetterAsync(delivery);
                }
            }
        }
    }
}
            

Message Ordering Guarantee

18. Push Notifications for Chat

C#
public class ChatNotificationService
{
    public async Task MaybeSendPushAsync(ChatMessage message, string recipientId)
    {
        // Check if recipient is online
        var isOnline = await _presenceService.IsOnlineAsync(recipientId);

        if (isOnline) return; // Don't send push if online

        // Check notification preferences
        var prefs = await _preferenceService.GetAsync(recipientId);
        if (!prefs.PushEnabled) return;

        // Check if conversation is muted
        var memberInfo = await _conversationRepo.GetMemberAsync(
            message.ConversationId, recipientId);
        if (memberInfo.NotificationSetting == "muted") return;

        // Generate notification content
        var content = await GenerateNotificationContentAsync(message, recipientId);

        // Send push notification
        await _notificationService.SendAsync(new NotificationRequest
        {
            UserId = recipientId,
            Channel = "push",
            Priority = message.MentionedUsers?.Contains(recipientId) == true ? "high" : "normal",
            TemplateName = "chat_message",
            Variables = new Dictionary<string, object>
            {
                ["sender_name"] = content.SenderName,
                ["message_preview"] = content.Preview,
                ["conversation_name"] = content.ConversationName,
                ["conversation_id"] = message.ConversationId
            }
        });
    }
}
            

19. Content Moderation

Moderation Pipeline

Moderation Rules

20. Monitoring & Observability

Key Metrics Dashboard

Alerting Rules

21. Case Studies � Production Systems

WhatsApp Architecture

Discord Architecture

Slack Architecture

22. Cost Estimation

Monthly Cost (500M DAU, 20B messages/day)

23. Edge Cases & Corner Cases

24. Interview Q&A

25. Conclusion

Designing a real-time chat system requires balancing three fundamental tensions: real-time delivery vs scalability, message reliability vs latency, and privacy vs functionality. The architecture choices you make � WebSocket servers for persistence, Kafka for message bus, Cassandra for storage, Redis for state � directly determine which trade-offs you can make.

The key insights for your next interview:

• Stateful connections require careful routing: Unlike stateless HTTP, you must know which server each user is connected to. Redis-based connection registry with consistent hashing is the industry standard.
• Message ordering is per-conversation, not global: Each conversation is an independent stream, ordered within itself. This simplifies the architecture significantly.
• Offline support is not optional: The offline message queue and sync protocol are essential for mobile users with unreliable connections.
• Group chat scales differently than 1-on-1: Large groups require subscription-based delivery, not fan-out. This is a critical distinction for interviewers.
• Monitoring is everything: Message delivery latency, connection churn, and consumer lag are the metrics that tell you when things are going wrong.

The chat system design question is one of the richest system design problems because it touches nearly every aspect of distributed systems: persistent connections, message queues, state management, conflict resolution, encryption, and real-time delivery. Mastering it demonstrates deep understanding of distributed systems principles.