1. Introduction — Instagram Stories at Scale

Instagram Stories launched in August 2016 as a direct response to Snapchat's disappearing content format. Within just two years, Stories surpassed Snapchat's entire daily user base, and by 2025, Instagram Stories boasted over 500 million daily active users who collectively upload more than 1 billion photos and videos to the platform every single day. This explosive growth has made the Stories feature one of the most critical revenue drivers for Meta, generating billions of dollars annually through sponsored story placements and interactive advertising formats.

At its core, Instagram Stories represents a fundamentally different paradigm from traditional social media posts. Unlike permanent feed posts, Stories are ephemeral by design — they automatically disappear after 24 hours unless manually saved to a user's Highlights. This ephemerality introduces a unique set of system design challenges that do not exist in conventional content-sharing platforms. The system must handle massive write throughput for content creation, real-time delivery to millions of concurrent viewers, precise TTL (time-to-live) enforcement for content expiration, and sophisticated analytics tracking for view counts, completion rates, and interactive engagement — all while maintaining sub-second latency globally.

The Stories ecosystem extends far beyond simple photo and video sharing. Modern Instagram Stories encompass a rich set of interactive features including polls, question stickers, quiz widgets, music overlays, countdown timers, donation stickers, sliders, and link stickers. Each of these interactive elements requires its own data model, real-time synchronization protocol, and analytics pipeline. When a user places a poll sticker on their story, the system must track every vote in real-time, display aggregated results to the story owner, and expose granular analytics — all within the 24-hour lifecycle of the content.

From an architectural standpoint, the Instagram Stories system is a masterclass in distributed systems engineering. The media pipeline alone must handle video transcoding into multiple resolutions and codecs, adaptive bitrate streaming, CDN distribution across hundreds of edge locations worldwide, and efficient storage management for billions of short-lived media assets. The feed ranking algorithm must process signals from the user's social graph, interaction history, recency, and content affinity to surface the most engaging stories within milliseconds. The view tracking system must absorb billions of view events per day with exactly-once semantics to ensure accurate creator analytics.

In this comprehensive system design guide, we will dissect every major component of the Instagram Stories platform. We will walk through capacity estimation to understand the sheer scale of the system, design the data models that power stories and their metadata, architect the media upload pipeline with chunked uploads and multi-stage transcoding, implement the ephemeral content management system with efficient TTL handling, build the real-time delivery infrastructure using WebSockets and long-polling, and explore the caching, sharding, and CDN strategies that make sub-second global delivery possible. Whether you are preparing for a senior engineering interview or designing a similar ephemeral content system from scratch, this guide will provide you with the depth and breadth of knowledge needed to architect a production-grade stories platform.

2. Functional & Non-Functional Requirements

Functional Requirements

Before designing any system, it is essential to clearly enumerate the features it must support. For the Instagram Stories system, the functional requirements span content creation, content consumption, social interactions, analytics, and administrative capabilities.

Content Creation

• Story Upload: Users can upload photos (JPEG, PNG) and videos (MP4, MOV) up to 1080x1920 resolution with a maximum duration of 60 seconds per story segment. The system must support uploading multiple sequential segments that play as a single story.
• Camera Integration: Real-time capture with filters, effects, and AR overlays. The client-side SDK handles camera processing while the backend stores the final rendered output.
• Text & Drawing Tools: Overlay text with customizable fonts, colors, sizes, and positions. Freehand drawing with color picker and brush size options.
• Sticker Placement: Interactive stickers (polls, questions, quizzes, countdowns, music, links, location tags, mentions, hashtags, GIFs) can be placed at arbitrary positions on the story canvas with rotation and scaling.
• Scheduling: Stories can be composed and scheduled for future posting within a 24-hour window.

Content Consumption

• Stories Tray: A horizontal scrollable tray at the top of the feed showing circular avatars of users who have active stories, ordered by the ranking algorithm.
• Story Viewer: Full-screen immersive viewer with tap-to-advance, hold-to-pause, swipe-left-to-reply, and swipe-down-to-exit gestures. Progress bar at the top shows each segment's duration.
• Replay: Users can replay a story once for free, after which the story becomes locked until the next segment or a new story is posted by the creator.
• Navigation: Tap left side to go to previous story/user, tap right side to advance. Swipe left to skip to next user's stories entirely.

Social Interactions

• Reactions: Emoji reactions sent via the swipe-up reply interface. The creator receives these as direct messages.
• Replies: Text, photo, or video replies that arrive as DMs to the story creator.
• Story Mentions: Users can mention other users using the @ symbol, which notifies the mentioned user and allows resharing to their own story.
• Link Stickers: External URL links accessible via interactive stickers that drive traffic to external websites.

Analytics & Insights

• View List: Creators can see exactly who viewed each story, sorted by reverse chronological order, with the total view count displayed prominently.
• Sticker Analytics: For interactive stickers, creators can see aggregated results (poll percentages, question responses, quiz scores) and per-user response details.
• Completion Rate: Percentage of viewers who watched all segments of a multi-segment story.
• Link Click Analytics: Number of clicks on link stickers with unique visitor breakdown.

Privacy & Access Control

• Close Friends: A curated list of trusted followers who can see restricted stories. The creator toggles this per-story.
• Hide Story From: Specific users can be blocked from seeing a creator's stories without unfollowing.
• Account Privacy: Private account stories are only visible to approved followers.
• Block & Restrict: Blocked users cannot view or interact with stories. Restricted users' replies go to a pending queue.

Non-Functional Requirements

3. Capacity Estimation

Capacity estimation is the foundation of any system design exercise. Before writing a single line of code or drawing an architecture diagram, we must understand the quantitative scale of the system. This exercise helps us make informed decisions about database sizing, cache capacity, CDN bandwidth, queue throughput, and infrastructure costs. Let us estimate the key metrics for Instagram Stories at a scale of 500 million daily active users.

Write Path Estimation

Assuming each of the 500M daily active users creates an average of 3 stories per day, we get approximately 1.5 billion story creations per day. This translates to roughly 17,400 story creations per second on average, with peak traffic during evening hours (7 PM - 11 PM local time) reaching 3-5x the average, or approximately 50,000-87,000 story uploads per second at peak. Each story upload involves writing metadata to the database, storing the media file in object storage, and enqueuing transcoding and CDN distribution jobs.

Read Path Estimation

On the consumption side, each user views an average of 20 stories per day (from approximately 8-10 different creators), resulting in 10 billion story views per day. This equates to approximately 115,700 views per second on average, peaking at 350,000-500,000 views per second. Each story view generates a view event that must be tracked, and the media must be served from CDN edge locations. The read-to-write ratio is approximately 6.7:1, confirming that this is a read-heavy system.

Storage Estimation

Since stories expire after 24 hours, the active storage footprint at any given time is approximately 3.9 PB of raw media (plus 7.8 PB of transcoded variants), totaling around 11.7 PB. However, we must retain deleted/expired story metadata for analytics purposes for 90 days, adding roughly 270 TB of metadata storage. The view event stream, if retained for 30 days in a columnar analytics store, accounts for approximately 15 PB. The total active storage requirement is therefore in the range of 12-15 PB for media and 15-30 PB including analytics data.

Bandwidth Estimation

On the upload path, 1.5 billion stories per day at an average of 2.6 MB each requires approximately 3.9 PB/day of ingest bandwidth, or roughly 45 GB/s average (135 GB/s at peak). On the download path, serving 10 billion story views per day from CDN edge caches requires approximately 26 PB/day of egress bandwidth, or roughly 300 GB/s average (900 GB/s at peak). These numbers underscore the critical importance of a well-provisioned CDN with global edge presence.

Cache & Memory Estimation

To serve the stories tray with sub-200ms latency, we need to cache the top 50 stories for each user (approximately 500M users) in a distributed cache. At 5 KB per story metadata entry, this requires approximately 12.5 TB of cache memory for the stories tray alone. Adding a 20% buffer for hot keys and replication overhead, we need approximately 15 TB of distributed cache capacity, which can be achieved with a Redis Cluster of approximately 150 nodes (each with 128 GB RAM, utilizing 80% for data).

Compute Estimation

Video transcoding is the most compute-intensive operation. Each 15-second video at 1080p requires approximately 2-4 seconds of compute time on a modern CPU core for H.264 encoding, or roughly 0.5-1 second on a GPU. At 900M videos per day (60% of 1.5B stories), this translates to approximately 900M transcoding jobs per day. With a 3-variant output (1080p, 720p, 480p), this becomes 2.7 billion encoding tasks. At an average of 1 second of GPU encoding per task, we need approximately 31,250 GPU-seconds per second, or roughly 31,250 GPU cores (accounting for queuing overhead, ~40,000 vCPUs equivalent). This is achievable with a fleet of approximately 500-800 high-end GPU instances.

4. Data Model Design

The data model is the backbone of any system design. For Instagram Stories, we need to model several interconnected entities: the stories themselves, the media assets they contain, the views they accumulate, the reactions and interactive responses they generate, and the highlights that preserve selected stories beyond the 24-hour window.

Core Entities

Stories Table

The stories table is the primary entity that represents a single story segment. Each story belongs to a user, has a precise creation timestamp, and carries an expires_at timestamp that is exactly 24 hours after creation.

SQL
CREATE TABLE stories (
    story_id        BIGINT PRIMARY KEY,
    user_id         BIGINT NOT NULL,
    story_group_id  BIGINT NOT NULL,
    media_type      ENUM('photo', 'video') NOT NULL,
    media_url       VARCHAR(2048) NOT NULL,
    thumbnail_url   VARCHAR(2048),
    duration_ms     INT DEFAULT 5000,
    width           INT NOT NULL,
    height          INT NOT NULL,
    file_size_bytes BIGINT NOT NULL,
    caption         TEXT,
    visibility      ENUM('all', 'close_friends', 'custom') DEFAULT 'all',
    status          ENUM('uploading', 'processing', 'published', 'expired', 'deleted') DEFAULT 'uploading',
    created_at      TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
    expires_at      TIMESTAMP NOT NULL,
    published_at    TIMESTAMP,
    deleted_at      TIMESTAMP,
    view_count      BIGINT DEFAULT 0,
    reply_count     INT DEFAULT 0,
    INDEX idx_user_expires (user_id, expires_at),
    INDEX idx_story_group (story_group_id),
    INDEX idx_expires_status (expires_at, status),
    INDEX idx_published (published_at DESC)
) PARTITION BY RANGE (UNIX_TIMESTAMP(expires_at));

Story Groups

A story group represents a logical collection of story segments uploaded by the same user within a short time window (typically a few minutes). When a user taps on an avatar in the stories tray, the viewer plays all segments in the story group sequentially. The story_groups table tracks the total segment count, aggregate view count, and the ordering of segments within the group.

Media Variants

Each uploaded story is transcoded into multiple variants for adaptive bitrate streaming. The media_variants table stores references to each transcoded version with its resolution, bitrate, codec, and CDN URL.

SQL
CREATE TABLE media_variants (
    variant_id      BIGINT PRIMARY KEY,
    story_id        BIGINT NOT NULL,
    resolution      ENUM('1080p', '720p', '480p', '360p') NOT NULL,
    codec           ENUM('h264', 'h265', 'vp9', 'av1') NOT NULL,
    bitrate_kbps    INT NOT NULL,
    file_url        VARCHAR(2048) NOT NULL,
    file_size_bytes BIGINT NOT NULL,
    duration_ms     INT,
    segment_count   INT,
    created_at      TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    FOREIGN KEY (story_id) REFERENCES stories(story_id),
    INDEX idx_story_resolution (story_id, resolution)
);

Story Views

The story_views table tracks each unique view event. Since a user might accidentally tap back and view a story again, we use a composite unique constraint on (story_id, viewer_id) to ensure only one view record per user per story.

SQL
CREATE TABLE story_views (
    story_id        BIGINT NOT NULL,
    viewer_id       BIGINT NOT NULL,
    view_duration_ms INT DEFAULT 0,
    completed       BOOLEAN DEFAULT FALSE,
    first_viewed_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
    last_viewed_at  TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
    PRIMARY KEY (story_id, viewer_id),
    INDEX idx_viewer (viewer_id, first_viewed_at),
    INDEX idx_story_viewed (story_id, first_viewed_at)
) PARTITION BY HASH(story_id) PARTITIONS 64;

Story Reactions

Reactions are emoji-based responses sent by viewers. Each reaction is stored both in the reactions table for the story owner's analytics and as a direct message in the messaging system. The story_reactions table links the reaction to both the story and the viewer, with the emoji type and timestamp.

Interactive Sticker Responses

Close Friends & Privacy

The close_friends table maintains a many-to-many relationship between users and their close friends lists. The story_hidden_from table tracks per-story blocks, allowing creators to hide specific stories from specific users without affecting their overall story visibility. The story_mentions table tracks @mentions within stories, linking to the mentioned user's profile and triggering notification workflows.

5. API Design

The API layer serves as the contract between the mobile client and the backend services. For Instagram Stories, we design a RESTful API with clear resource naming, comprehensive query parameters for pagination and filtering, and well-defined request/response schemas. All API endpoints require authentication via OAuth 2.0 bearer tokens and are rate-limited per user to prevent abuse.

Story CRUD Operations

Story Feed Response Schema

The stories feed endpoint returns a list of story groups, each containing the user's profile information and an array of individual story segments. The response is optimized for the client-side stories tray, including pre-computed metadata like the total segment count, aggregate view count, and the user's ring color state.

JSON
{
    "stories_feed": [
        {
            "user": {
                "user_id": 88421901,
                "username": "jane_dev",
                "display_name": "Jane Smith",
                "avatar_url": "https://cdn.example.com/avatars/88421901_128.jpg",
                "is_verified": true,
                "relationship": "following"
            },
            "story_group_id": 55001234,
            "total_segments": 3,
            "unviewed_segments": 2,
            "latest_story": {
                "story_id": 99001,
                "media_type": "video",
                "thumbnail_url": "https://cdn.example.com/stories/99001_thumb.jpg",
                "duration_ms": 15000,
                "created_at": "2026-07-01T14:30:00Z",
                "expires_at": "2026-07-02T14:30:00Z",
                "has_stickers": true,
                "visibility": "all"
            },
            "stories": [
                {
                    "story_id": 99001,
                    "media_type": "video",
                    "media_url": "https://cdn.example.com/stories/99001_720p.mpd",
                    "duration_ms": 15000,
                    "stickers": [
                        {
                            "sticker_id": "stk_001",
                            "type": "poll",
                            "question": "Best framework in 2026?",
                            "options": ["React", "Svelte", "Solid"],
                            "position_x": 0.5,
                            "position_y": 0.3,
                            "rotation": 0,
                            "scale": 1.0
                        }
                    ],
                    "view_count": 1247,
                    "replay_available": true
                }
            ],
            "ring_state": "unseen"
        }
    ],
    "cursor": "eyJ0cyI6MTcwMTQ4MDYwMH0=",
    "has_more": true
}

Story Upload Flow

The upload flow is a multi-step process designed to handle large media files reliably over unstable mobile networks. First, the client requests a pre-signed upload URL from the /stories/upload-url endpoint, specifying the file size, MIME type, and media category. The server responds with a pre-signed S3 PUT URL valid for 15 minutes, along with an upload_id for tracking the upload session. The client then performs a chunked upload directly to S3 using the pre-signed URL, sending 5 MB chunks sequentially. After all chunks are uploaded, the client calls the /stories POST endpoint with the upload_id, story metadata (caption, stickers with positions, visibility settings), and the server triggers the transcoding pipeline and publishes the story once processing completes.

Batched View Reporting

To minimize the number of API calls for view tracking, the client batches view events and sends them in bulk every 5 seconds or when the user navigates away from the story viewer. The /stories/views/batch endpoint accepts an array of {story_id, view_duration_ms, completed} objects and processes them asynchronously via a message queue. This design reduces API call volume by approximately 90% compared to per-view reporting.

6. High-Level Architecture

The Instagram Stories system architecture is a distributed, event-driven microservices platform designed for horizontal scalability, fault isolation, and global reach. The architecture separates the write path (content creation) from the read path (content consumption) and uses asynchronous processing for compute-intensive operations like transcoding and analytics aggregation.

Component Descriptions

CDN Layer: All static media assets (photos, videos, thumbnails) are served directly from the CDN, which caches content at edge locations worldwide. The CDN is configured with custom cache keys based on story ID and resolution variant, with a maximum TTL of 24 hours matching the story lifecycle.

WebSocket Gateway: A stateful WebSocket connection is maintained between the mobile client and the WebSocket Gateway for real-time features: push notifications for new stories from followed users, live reaction counts on interactive stickers, and instant delivery of story view analytics.

Story Service: The core microservice responsible for story CRUD operations, feed generation, and story group management. It maintains a local cache of hot stories in Redis and falls back to MySQL for cache misses. The feed generation endpoint merges stories from the user's followings, sorts them by the ranking algorithm, and returns the paginated result set.

Media Service: Handles the complete media lifecycle from pre-signed upload URL generation through chunked upload orchestration, transcoding job submission, variant storage, and CDN distribution. It communicates with S3 for direct client uploads and triggers Kafka events for downstream processing.

View Service: Processes batched view events from clients, writes them to Kafka for asynchronous persistence to MySQL and real-time aggregation in Redis. It exposes the view count and viewer list endpoints used by the story owner's analytics interface.

Privacy Service: Enforces access control policies including Close Friends lists, hidden-from lists, and block/restrict states. All privacy checks are performed before story feed results are returned to the client, with cached policy lookups in Redis for sub-millisecond enforcement.

Ad Service: Integrates with the stories feed to insert sponsored content at predefined slot positions (typically every 3rd or 4th story). It coordinates with the auction service, budget pacing system, and frequency capping layer to deliver targeted advertisements without disrupting the organic user experience.

7. Media Upload Pipeline

The media upload pipeline is one of the most complex and performance-critical subsystems of Instagram Stories. It must handle billions of media uploads per day across varying network conditions, ensure reliable delivery of large video files, perform multi-resolution transcoding, and distribute the resulting variants to CDN edge locations worldwide — all within seconds of the initial upload.

Stage 1: Chunked Upload

Mobile network conditions are inherently unreliable. A 10 MB video upload over a congested cellular connection can easily be interrupted by signal drops, handoffs between cell towers, or the OS suspending the app. To handle this, we implement a resumable chunked upload protocol inspired by the TUS resumable upload protocol standard. The client breaks the media file into 5 MB chunks, uploads each chunk via a pre-signed S3 URL with the chunk index in the headers, and the server tracks upload progress in Redis. If an upload is interrupted, the client can resume from the last successfully uploaded chunk without re-uploading the entire file.

Stage 2: Client-Side Preprocessing

Before uploading, the client performs several preprocessing steps to optimize the media for the platform. Photos are compressed using JPEG quality 85 with chroma subsampling, targeting a maximum file size of 1 MB while preserving visual quality at 1080x1920 resolution. Videos are re-encoded using the device's hardware encoder (H.264 Baseline Profile for compatibility) at the original resolution, with a target bitrate of 8 Mbps for 1080p. The client also generates a thumbnail frame (JPEG at 320x568) for the stories tray and extracts a 1-second preview clip for the CDN cache warmer.

Stage 3: Server-Side Transcoding

Once the original media is uploaded, the transcoding worker fleet processes it into multiple variants optimized for different network conditions. The transcoding pipeline runs on a fleet of GPU-accelerated instances using NVIDIA T4 or A10G GPUs with NVENC hardware encoding.

For video content, the transcoding worker segments the output into 2-second chunks using the MPEG-DASH (Dynamic Adaptive Streaming over HTTP) format, generating an MPD manifest file. This enables adaptive bitrate streaming where the client can dynamically switch between quality tiers based on real-time network bandwidth estimation.

Stage 4: CDN Distribution

After transcoding completes, the CDN Invalidation Worker pushes the new variants to all edge locations using a push-on-publish strategy. Rather than waiting for the first request to cache the content, we proactively warm the CDN caches at the edge locations closest to the story creator's geographic region. For stories expected to receive high engagement, we warm caches at all edge locations. The CDN configuration uses Cache-Control: max-age=86400 matching the 24-hour story lifecycle.

Stage 5: Storage Lifecycle Management

Original uploaded media is stored in S3 Standard for the 24-hour lifetime. At the 12-hour mark, originals transition to S3 Infrequent Access, reducing storage costs by approximately 40%. At the 24-hour expiry mark, a TTL Expiration Worker deletes the original and all transcoded variants from S3 using batch operations to minimize API call costs and avoid throttling.

8. Stories Feed Ranking Algorithm

The Stories tray is the primary surface through which users discover and consume stories. Unlike the main feed which uses a sophisticated ML-based ranking algorithm, the Stories tray uses a lighter-weight ranking system that balances recency, relationship closeness, and engagement signals. The ranking algorithm determines both the ordering of user avatars in the tray and which stories appear first within each user's story group.

Ranking Signals

The algorithm processes three categories of signals to compute a relevance score for each potential story in the user's tray:

Relationship Strength (40% weight): This signal measures how closely connected the viewer is to the story creator. It is derived from a composite of several sub-signals: the frequency of mutual interactions (likes, comments, DMs, story views) over the past 90 days, the recency of the last interaction, whether they are in each other's Close Friends lists, and whether they share common group memberships or frequently appear in each other's tagged photos. The relationship score is pre-computed by a daily batch job and cached in Redis with a 6-hour TTL.

Recency (35% weight): Stories are inherently time-sensitive — a story posted 2 hours ago is significantly more valuable than one posted 20 hours ago. The recency signal uses an exponential decay function: recency_score = e^(-lambda * hours_since_post) where lambda is tuned to half the signal at approximately 8 hours. This means a story posted 8 hours ago receives roughly half the recency score of a freshly posted story.

Engagement History (25% weight): This signal captures the viewer's historical engagement pattern with the creator's stories. If the viewer consistently watches all segments of a creator's stories, replies frequently, or interacts with stickers, this creator's stories receive a higher ranking. The engagement history is computed as a rolling average of completion rates, reply rates, and sticker interaction rates over the past 30 days.

Score Computation

C#
public class StoryRankingEngine
{
    private readonly IRelationshipService _relationshipService;
    private readonly IEngagementService _engagementService;
    private const double RecencyLambda = 0.0866; // ln(2) / 8 hours

    public List<RankedStoryGroup> RankStoriesFeed(
        long viewerId,
        List<StoryGroup> candidateGroups)
    {
        var ranked = candidateGroups.Select(group =>
        {
            double relationshipScore = _relationshipService
                .GetRelationshipScore(viewerId, group.UserId);

            double recencyScore = Math.Exp(
                -RecencyLambda * group.HoursSinceLatest);

            double engagementScore = _engagementService
                .GetEngagementScore(viewerId, group.UserId);

            double compositeScore =
                (0.40 * relationshipScore) +
                (0.35 * recencyScore) +
                (0.25 * engagementScore);

            // Boost for unviewed stories
            if (group.HasUnviewedSegments)
                compositeScore *= 1.5;

            // Boost for interactive stickers
            if (group.HasInteractiveStickers)
                compositeScore *= 1.2;

            return new RankedStoryGroup
            {
                Group = group,
                Score = compositeScore,
                RingState = ComputeRingState(viewerId, group)
            };
        })
        .OrderByDescending(r => r.Score)
        .ToList();

        return ApplyDiversityConstraints(ranked);
    }

    private string ComputeRingState(
        long viewerId, StoryGroup group)
    {
        int unviewed = group.TotalSegments -
            group.ViewedSegmentCount;

        if (unviewed == group.TotalSegments)
            return "unseen";
        if (unviewed > 0)
            return "partially_seen";
        return "fully_seen";
    }

    private List<RankedStoryGroup> ApplyDiversityConstraints(
        List<RankedStoryGroup> ranked)
    {
        var result = new List<RankedStoryGroup>();
        int consecutiveCount = 0;
        long lastUserId = -1;

        foreach (var item in ranked)
        {
            if (item.Group.UserId == lastUserId)
            {
                consecutiveCount++;
                if (consecutiveCount > 3) continue;
            }
            else
            {
                consecutiveCount = 1;
                lastUserId = item.Group.UserId;
            }
            result.Add(item);
        }
        return result;
    }
}

Cold Start & New User Handling

For new users who have limited interaction history, the ranking algorithm falls back to a popularity-based signal, prioritizing stories from verified accounts and accounts with high overall engagement. As the user accumulates interaction data (typically within the first 3-5 days), the algorithm gradually transitions from popularity-based to personalized ranking using an exponential moving average blend.

9. Ephemeral Content & TTL Management

Ephemerality is the defining characteristic of Instagram Stories. Every piece of content must expire exactly 24 hours after creation — no earlier, no later. This precise TTL enforcement is surprisingly challenging at scale, as it requires coordination across multiple storage systems (MySQL, Redis, S3, CDN caches) and must handle edge cases like clock skew, network partitions, and processing delays.

TTL Architecture Overview

Multi-Layer TTL Strategy

MySQL Partition-Based Expiration: The stories table is partitioned by expires_at in hourly ranges. The Partition Dropper Worker runs every 15 minutes and drops entire partitions whose expires_at range has fully passed. This is orders of magnitude more efficient than deleting individual rows — a single DROP PARTITION command executes in milliseconds regardless of the number of rows in the partition, whereas deleting millions of rows would take minutes and generate massive WAL overhead.

Redis TTL-Based Expiration: Redis keys for story metadata and view counts are created with a TTL of 24 hours plus a 1-hour buffer. The buffer ensures that Redis keys persist slightly longer than the MySQL records, preventing cache miss storms during the expiration window. The Redis Evictor Worker runs a background SCAN operation on story-related key patterns and proactively evicts keys for stories that have been confirmed as expired in MySQL.

S3 Lifecycle Policies: S3 bucket lifecycle rules automatically transition objects to IA storage after 12 hours and delete objects after 24 hours. This provides a safety net — even if our application-level expiration logic fails, S3 lifecycle policies ensure that media files are eventually cleaned up.

CDN Cache Eviction: CDN edge caches respect the Cache-Control: max-age=86400 header set on story media URLs. After 24 hours, the CDN automatically stops serving cached content and fetches from the origin (which returns a 404 for expired stories). For stories that are manually deleted before expiry, the CDN Purger Worker issues invalidation requests to all edge locations.

Clock Skew Mitigation

In a distributed system, clock skew between servers can cause stories to expire at slightly different times across different services. To mitigate this, we use a hybrid logical clock (HLC) approach that combines physical timestamps with logical counters. The canonical expires_at timestamp is set by the Story Service at creation time and propagated to all downstream systems. Expiration workers use the story's expires_at value rather than the current system time to determine expiration, ensuring consistent behavior regardless of local clock drift.

Grace Period Strategy

To avoid a jarring experience where a story disappears mid-view, we implement a 30-second grace period. When a user opens the story viewer for a story that is within 30 seconds of expiry, the system allows playback to complete without interruption. The story becomes ineligible for new views only after this grace period expires.

10. View Tracking & Replay Analytics

View tracking is one of the most high-throughput components of the Instagram Stories system, generating billions of events per day that must be processed with exactly-once semantics to ensure accurate creator analytics. The view tracking system must handle simultaneous views from millions of users, deduplicate accidental re-views, support real-time view count updates, and provide a paginated viewer list sorted by recency.

Client-Side View Event Generation

The mobile client generates view events using a sophisticated detection algorithm that accounts for various user behaviors. A "view" is recorded when the story is visible on screen for more than 500 milliseconds (to filter out accidental taps). The client tracks the total viewing duration by measuring the time between the story becoming visible and the user navigating away. If the user watches the entire story segment, the completed flag is set to true. For video stories, the client also tracks the furthest playback position to calculate a "watch depth" metric.

Batched Event Reporting

C#
public class ViewEventBatcher
{
    private readonly ConcurrentQueue<ViewEvent> _pendingEvents;
    private readonly IHttpClientFactory _httpClientFactory;
    private readonly Timer _flushTimer;
    private const int MaxBatchSize = 50;
    private const int FlushIntervalMs = 5000;

    public ViewEventBatcher(IHttpClientFactory httpClientFactory)
    {
        _httpClientFactory = httpClientFactory;
        _pendingEvents = new ConcurrentQueue<ViewEvent>();
        _flushTimer = new Timer(
            callback: _ => FlushAsync().GetAwaiter().GetResult(),
            state: null,
            dueTime: FlushIntervalMs,
            period: FlushIntervalMs);
    }

    public void RecordView(long storyId, int viewDurationMs,
        bool completed)
    {
        _pendingEvents.Enqueue(new ViewEvent
        {
            StoryId = storyId,
            ViewDurationMs = viewDurationMs,
            Completed = completed,
            Timestamp = DateTimeOffset.UtcNow
        });

        if (_pendingEvents.Count >= MaxBatchSize)
            FlushAsync().GetAwaiter().GetResult();
    }

    private async Task FlushAsync()
    {
        var batch = new List<ViewEvent>();
        while (batch.Count < MaxBatchSize &&
               _pendingEvents.TryDequeue(out var evt))
        {
            batch.Add(evt);
        }

        if (batch.Count == 0) return;

        var client = _httpClientFactory.CreateClient("view-api");
        var payload = new { events = batch };
        var content = new StringContent(
            JsonSerializer.Serialize(payload),
            Encoding.UTF8,
            "application/json");

        try
        {
            await client.PostAsync(
                "/api/v1/stories/views/batch", content);
        }
        catch (Exception)
        {
            foreach (var evt in batch)
                _pendingEvents.Enqueue(evt);
        }
    }
}

Server-Side Processing Pipeline

Deduplication: The View Service performs deduplication using a Redis Bloom filter keyed by {story_id}:{viewer_id}. When a batch of view events arrives, each event is checked against the Bloom filter. If the filter indicates the pair has been seen before, the event is treated as a replay and only the view_duration_ms and last_viewed_at fields are updated. The Bloom filter has a false positive rate of 0.01%, which is acceptable for this use case.

Real-Time View Count: The View Counter Worker consumes from the story.views Kafka topic and increments the view count in Redis using the INCRBY command. Redis serves as the authoritative source for real-time view counts, with the count periodically flushed to MySQL every 5 minutes for durability.

Viewer List Management: The Real-Time Aggregator maintains a Redis sorted set for each story, where scores are Unix timestamps and members are viewer user IDs. This allows efficient retrieval of the viewer list in reverse chronological order using ZREVRANGE. The sorted set is TTL'd to 24 hours plus a buffer, and the top 1000 viewers are cached in a separate Redis key for fast retrieval.

Completion Rate Analytics

Completion rate is defined as the percentage of viewers who watched the entire story segment. For multi-segment story groups, the system also computes the group completion rate — the percentage of viewers who watched all segments in the group. The completion rate is computed as completed_views / total_views * 100 and is cached in Redis with a 1-minute TTL.

11. Stories Highlights & Archive

Instagram Stories Highlights allow users to permanently preserve selected stories beyond the 24-hour expiration window. Highlights are organized into thematic collections (e.g., "Travel", "Food", "Work") with custom titles and cover images. This feature effectively bridges the gap between ephemeral Stories and permanent feed posts, creating a curated portfolio on the user's profile.

Highlight Creation Flow

When a user adds a story to a Highlight, the system performs several critical operations. First, the story's media files (original and all transcoded variants) are moved from the ephemeral S3 storage class to a permanent storage class. This is done by copying the objects to a dedicated highlights-media S3 bucket with no lifecycle expiration rules, and then deleting the copies from the ephemeral bucket. If the story has already expired, the media must be retrieved from the archived copy in the analytics storage system and re-uploaded to the permanent bucket.

Highlight Data Model

SQL
CREATE TABLE highlights (
    highlight_id    BIGINT PRIMARY KEY,
    user_id         BIGINT NOT NULL,
    title           VARCHAR(100) NOT NULL,
    cover_story_id  BIGINT,
    cover_image_url VARCHAR(2048),
    story_count     INT DEFAULT 0,
    created_at      TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at      TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    sort_order      INT DEFAULT 0,
    INDEX idx_user_highlights (user_id, sort_order)
);

CREATE TABLE highlight_stories (
    highlight_id    BIGINT NOT NULL,
    story_id        BIGINT NOT NULL,
    sort_order      INT NOT NULL,
    added_at        TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    PRIMARY KEY (highlight_id, story_id),
    INDEX idx_story_highlights (story_id)
);

CREATE TABLE highlight_media (
    highlight_id    BIGINT NOT NULL,
    story_id        BIGINT NOT NULL,
    media_type      ENUM('photo', 'video') NOT NULL,
    media_url       VARCHAR(2048) NOT NULL,
    thumbnail_url   VARCHAR(2048),
    duration_ms     INT,
    width           INT,
    height          INT,
    storage_class   ENUM('standard', 'ia', 'glacier') DEFAULT 'standard',
    PRIMARY KEY (highlight_id, story_id)
);

Highlight Serving Optimization

Since highlights are permanent, they benefit from aggressive CDN caching with much longer TTLs than regular stories. The Cache-Control header for highlight media is set to max-age=604800 (7 days) with stale-while-revalidate=3600, meaning the CDN serves cached content for up to 7 days before revalidating with the origin. Highlights also benefit from a separate, smaller CDN cache tier designed for long-lived content, reducing cache eviction pressure from the high-volume ephemeral story traffic.

12. Close Friends & Privacy Controls

Privacy is a cornerstone of the Instagram Stories experience. Users must have granular control over who can see their stories, ranging from all followers to a curated Close Friends list to completely hidden from specific individuals. These privacy controls must be enforced consistently across all access paths — the stories tray, direct story links, mentions, and resharing — with zero tolerance for privacy leaks.

Close Friends List Management

The Close Friends feature allows users to create a private list of trusted followers who can see restricted stories. The list is stored in the close_friends table as a simple many-to-many relationship. The list itself is private — other users cannot see who is on someone's Close Friends list, and the list membership is not exposed through any API endpoint. When a user creates a story with visibility = 'close_friends', the story retrieval query filters the viewer's user ID against the creator's Close Friends list before including the story in the feed.

Privacy Enforcement Pipeline

Hidden-From List: The story_hidden_from table stores per-user hide decisions. When a creator hides their story from specific users, those user IDs are added to this table. The privacy filter checks this table during story feed generation. For performance, the hidden-from list is cached in Redis as a set keyed by hidden_from:{user_id} with a TTL of 1 hour. Since most users hide stories from fewer than 50 people, the set membership check completes in O(1) time.

Block & Restrict Integration: The privacy pipeline integrates with the platform-wide blocking and restriction system. Blocked users are excluded from all story interactions — they cannot view, reply to, or react to stories from users who have blocked them. Restricted users can view stories, but their replies are routed to a pending queue rather than appearing directly in the story creator's DM inbox.

Privacy Audit Trail

To detect and prevent privacy leaks, all story access events are logged to an audit trail stored in an append-only log (Apache Kafka topic privacy.audit). A nightly batch job analyzes the audit log to detect anomalous patterns, such as a user consistently accessing stories they should not be able to see. This audit trail is also used for regulatory compliance under GDPR and CCPA, allowing users to request a complete log of who has viewed their content.

13. Interactive Stickers, Polls, Questions

Interactive stickers are one of the key differentiators of Instagram Stories, transforming passive content consumption into active engagement. The sticker ecosystem includes polls, quiz questions, question boxes, emoji sliders, countdown timers, music stickers, donation stickers, and link stickers. Each sticker type requires its own data model, real-time interaction protocol, and analytics aggregation pipeline.

Poll Sticker Architecture

When a creator places a poll sticker on their story, they define a question and 2-4 answer options. Viewers can tap one option to cast their vote, and the aggregated results are displayed in real-time on the story viewer. The poll system must handle high write throughput (millions of votes per second for popular stories), prevent duplicate voting, and display live-updating results to both the viewer and the story creator.

C#
public class PollService
{
    private readonly IDatabase _redis;
    private readonly IMessageQueue _kafka;

    public async Task<PollResult> CastVoteAsync(
        string storyId, string stickerId,
        long voterId, int optionIndex)
    {
        var voteKey = $"poll:votes:{storyId}:{stickerId}";
        var added = await _redis.SetAddAsync(
            voteKey, voterId.ToString());

        if (!added)
            throw new ConflictException(
                "User has already voted on this poll");

        await _redis.KeyExpireAsync(voteKey,
            TimeSpan.FromHours(25));

        var counterKey = $"poll:counts:{storyId}:{stickerId}";
        await _redis.HashIncrementAsync(
            counterKey, optionIndex.ToString());
        await _redis.KeyExpireAsync(counterKey,
            TimeSpan.FromHours(25));

        await _kafka.PublishAsync("poll.votes", new PollVoteEvent
        {
            StoryId = storyId,
            StickerId = stickerId,
            VoterId = voterId,
            OptionIndex = optionIndex,
            Timestamp = DateTimeOffset.UtcNow
        });

        return await GetPollResultsAsync(storyId, stickerId);
    }

    public async Task<PollResult> GetPollResultsAsync(
        string storyId, string stickerId)
    {
        var counterKey = $"poll:counts:{storyId}:{stickerId}";
        var counts = await _redis.HashGetAllAsync(counterKey);

        var results = counts.Select(c => new PollOptionResult
        {
            OptionIndex = int.Parse(c.Name),
            VoteCount = (int)c.Value
        }).ToList();

        int totalVotes = results.Sum(r => r.VoteCount);

        return new PollResult
        {
            Options = results.Select(r => new PollOptionDisplay
            {
                Text = GetOptionText(storyId, stickerId,
                    r.OptionIndex),
                Percentage = totalVotes > 0
                    ? (int)(r.VoteCount * 100.0 / totalVotes)
                    : 0,
                VoteCount = r.VoteCount
            }).ToList(),
            TotalVotes = totalVotes,
            HasVoted = false,
            SelectedOption = -1
        };
    }
}

Quiz Sticker Architecture

The quiz sticker is similar to polls but includes a correct answer indicator and a score tracking mechanism. The quiz creator marks one option as correct, and viewers receive immediate feedback on whether their answer was right or wrong. The quiz results are displayed as a percentage breakdown with the correct answer highlighted in green.

Question Box Architecture

The question box sticker allows viewers to submit anonymous text or photo responses to a question posed by the story creator. Each response is stored in a Redis list keyed by questions:{story_id}:{sticker_id} with a 24-hour TTL. The creator can see up to 1000 responses, with the most recent first.

14. Stories Ads Integration

Stories ads are a primary revenue driver for Instagram, generating billions of dollars annually. The ad system must seamlessly insert sponsored content into the organic stories feed without disrupting the user experience, target ads based on rich user profiles and behavioral signals, and provide comprehensive measurement and attribution for advertisers.

Ad Insertion Architecture

Ad Candidate Retrieval

When a stories feed request arrives, the Ad Service concurrently retrieves candidate ads from multiple sources: the advertiser's campaign budget and targeting criteria are stored in a specialized ad-indexing service built on top of Elasticsearch, while real-time user profile data is fetched from the user graph service. The targeting service filters candidates based on demographic criteria (age, gender, location), interest-based signals (page likes, search history, engagement patterns), and custom audience matches (email lists, device IDs). The initial candidate set typically contains 50-200 ads, which is then narrowed down through a multi-stage ranking pipeline.

Real-Time Auction

The auction service implements a modified second-price auction model. Each candidate ad has a maximum bid (set by the advertiser's budget and bidding strategy), and the auction computes an effective cost-per-impression (eCPM) that factors in the bid amount, predicted click-through rate (pCTR), predicted conversion rate (pCVR), and an ad quality score. The ad quality score is a composite metric that includes historical ad engagement, landing page quality, and creative freshness. The top-ranked ads from the auction are selected for insertion, with the winning ads paying the minimum amount needed to beat the next-highest bid.

Budget Pacing

Budget pacing ensures that advertisers' daily budgets are spent evenly throughout the day rather than being exhausted in the first few hours. The pacing service uses a proportional-integral-derivative (PID) controller that monitors the spend rate and adjusts the ad's participation in auctions to maintain a target spend trajectory. If an ad is spending too quickly, the pacing service reduces its bid multiplier; if spending too slowly, it increases the multiplier.

Frequency Capping

To prevent ad fatigue, the frequency capping service enforces limits on how often a user sees the same ad. Common cap configurations include "once per 24 hours", "3 times per 7 days", or "5 times per 30 days". The frequency cap state is stored in a Redis hash with TTL matching the cap window, enabling O(1) lookup for cap checks. The cap is applied after ad selection but before feed insertion, ensuring that capped ads are replaced with the next-best candidate from the auction.

15. Real-Time Delivery via WebSocket

Real-time communication is essential for Instagram Stories features that require instant updates: new story notifications, live reaction counts on interactive stickers, story deletion propagation, and real-time analytics for story creators. The WebSocket layer provides a persistent, bidirectional communication channel between the mobile client and the server, eliminating the need for repeated polling.

WebSocket Gateway Architecture

Connection Lifecycle

When the mobile app launches, it establishes a WebSocket connection to the gateway. The connection handshake includes an authentication token (JWT) in the upgrade request headers. The gateway validates the token, extracts the user ID, and registers the connection in a Redis-backed connection registry. The connection is maintained with a 60-second heartbeat interval — if the client fails to send a heartbeat within 90 seconds, the gateway closes the connection and removes it from the registry.

The gateway supports connection multiplexing, where a single physical WebSocket connection carries messages for multiple logical channels (new stories, reactions, analytics updates, notifications). The client subscribes to specific channels after connection establishment using a subscribe message.

Message Protocol

Message Fan-Out

When a backend service needs to push a message to a client, it publishes the message to a Kafka topic. A dedicated Kafka consumer on each WebSocket gateway node consumes messages from the topic, looks up the target user's connection in the Redis registry, and routes the message to the appropriate gateway node. For high-fanout scenarios (e.g., a celebrity posting a new story that notifies millions of followers), the message is published to a Redis Pub/Sub channel that all gateway nodes subscribe to, allowing each node to independently filter messages for its locally connected clients.

16. CDN & Edge Caching

The CDN (Content Delivery Network) is the single most critical infrastructure component for delivering Instagram Stories with sub-100ms latency to users worldwide. Without a CDN, every story view request would travel to the origin data center, adding 100-500ms of round-trip latency depending on geographic distance. The CDN strategy for Stories involves careful cache key design, intelligent pre-warming, origin shield protection, and coordinated invalidation for deleted or expired content.

CDN Topology

Cache Key Strategy

Optimal cache key design maximizes hit rates while ensuring content correctness. For story media, the cache key is composed of the story ID, variant resolution, and a content hash: /stories/{story_id}/{resolution}/{content_hash}.{ext}. The content hash ensures that if a story is re-uploaded with the same ID, the old cached content is not served. For thumbnails and profile images, the cache key includes a version suffix that is incremented on each update.

Cache Warming Strategy

Cold cache hits cause noticeable latency spikes (200-500ms vs. the normal 20-50ms). To eliminate cold starts, we implement a proactive cache warming strategy. When a story is published, the Media Service triggers a cache warming job that issues HTTP GET requests to all CDN edge locations in the story creator's region (and globally for high-profile accounts). This pre-populates the CDN cache before any viewer requests arrive.

Origin Shield

The origin shield is an intermediate caching layer that sits between the CDN edge nodes and the S3 origin. When multiple edge nodes simultaneously request the same uncached content, they all hit the origin shield first, which caches the response and serves it to all requesting edge nodes. This reduces origin load by 90-95% and prevents the thundering herd problem where a popular new story causes a stampede of origin requests.

Cache Invalidation

When a story is deleted before its natural 24-hour expiry, the CDN Purger Worker issues invalidation requests to all edge locations. The invalidation uses the path-based wildcard pattern /stories/{story_id}/* to invalidate all variants and thumbnails for the deleted story. During the invalidation window (15-30 seconds), a soft-delete approach where the story is first marked as "deleted" in the database ensures that no new views are served from the API layer.

17. Database Sharding & Replication

At the scale of Instagram Stories — billions of rows in the stories table, trillions of rows in the story_views table, and tens of billions of interactions — a single MySQL instance cannot provide the storage capacity, throughput, or availability required. Database sharding distributes data across multiple MySQL instances (shards), enabling horizontal scalability. Replication ensures high availability and read scalability by maintaining multiple copies of each shard.

Sharding Strategy

Stories Table: The stories table is sharded by user_id using a hash-based routing function. Given 1024 logical shards distributed across 128 physical MySQL instances (each hosting 8 logical shards), the shard for a given story is determined by shard_id = hash(user_id) % 1024. This ensures that all stories for a single user reside on the same shard, enabling efficient queries like "get all active stories for user X" without cross-shard scatter-gather operations.

Story Views Table: The story_views table is sharded by story_id using consistent hashing. This places all views for a story on the same shard, enabling efficient aggregation queries (total view count, viewer list) without cross-shard coordination. Since a single story can accumulate millions of views, the story_views table uses a composite primary key (story_id, viewer_id) with an additional index on (viewer_id, first_viewed_at) for "stories viewed by user X" queries.

Replication Topology

Each physical MySQL shard runs a primary-replica topology with one primary (read-write) and two replicas (read-only). The primary handles all write operations (story creation, view recording, reaction inserts), while replicas serve read operations (feed generation, viewer list retrieval, analytics queries). Replication uses MySQL's semi-synchronous replication mode, where the primary waits for at least one replica to acknowledge receipt of each write before confirming to the client. This guarantees that a committed write is replicated to at least one replica before acknowledgment, preventing data loss in primary failure scenarios.

Cross-Shard Queries

Certain queries require data from multiple shards, most notably the stories feed generation which must retrieve stories from all of a user's followed accounts. To handle this efficiently, we maintain a fan-out index in Redis: for each user, a sorted set following_stories:{user_id} stores the story group IDs and timestamps of all active stories from their followed accounts. This index is maintained asynchronously by a change-data-capture (CDC) pipeline that listens to MySQL binlogs and updates the Redis sorted set in near-real-time.

18. Caching Layers

Effective caching is the cornerstone of low-latency story delivery. The Instagram Stories system employs a multi-tier caching architecture that spans the client device, CDN edge nodes, application-level distributed caches, and database query caches. Each tier serves a specific purpose, with carefully defined TTLs, invalidation strategies, and consistency guarantees.

Multi-Tier Cache Architecture

Redis Cache Design

The Redis Cluster is the most architecturally significant caching layer, storing frequently accessed metadata that would otherwise require database queries. The cache is organized into several key namespaces:

• story:{story_id}: A Redis hash containing the complete story metadata (user_id, media_url, duration, visibility, view_count, created_at, expires_at). TTL: 24 hours + 1 hour buffer.
• feed:{user_id}: A Redis sorted set containing the user's ranked stories feed. Score = ranking score. TTL: 30 seconds.
• following_stories:{user_id}: A Redis sorted set maintaining the story group IDs of all active stories from the user's followings. Score = latest story timestamp. TTL: 1 minute.
• view_count:{story_id}: A Redis string containing the real-time view count. Updated atomically using INCRBY. TTL: 24 hours.
• poll_counts:{story_id}:{sticker_id}: A Redis hash storing poll option vote counts. TTL: 24 hours + 1 hour buffer.

Cache Invalidation Strategies

The system employs three invalidation strategies depending on the use case. Time-based invalidation (TTL) is used for most cache entries, where the TTL is set to slightly longer than the expected data freshness window. Event-based invalidation is used for critical state changes like story deletion or privacy updates. When a story is deleted, a Kafka event triggers immediate cache invalidation across all Redis keys associated with that story. Write-through invalidation is used for view counts and poll results, where the cache is updated synchronously with the database write.

Cache Stampede Prevention

When a popular story's cache expires simultaneously for multiple request handlers, a cache stampede can occur. We prevent this using a probabilistic early expiration technique: each cache read has a small probability (1/TTL) of triggering a background refresh before the TTL expires, spreading the refresh load evenly across the TTL window. Additionally, we use Redis distributed locks (SET NX EX) to ensure that only one process regenerates a cache entry at a time, while other processes serve stale data until the refresh completes.

19. Moderation & Content Policy

Content moderation for Instagram Stories operates under immense pressure — billions of stories are uploaded daily, and the 24-hour expiry window creates urgency that permanent posts do not have. A policy-violating story that goes undetected for even a few hours can be seen by millions of users before it expires. The moderation system must balance detection speed, accuracy, and scale.

Moderation Pipeline Architecture

ML Classification Models

The pre-publish ML scan runs a battery of classification models on the uploaded media. These models are trained on labeled datasets of policy-violating content and run on a dedicated GPU inference fleet.

The pre-publish scan targets a total latency of under 500 milliseconds for photo stories and under 2 seconds for video stories (sampling 1 frame per second). Stories that pass all classifiers with high confidence are published immediately. Stories with medium confidence scores are published but flagged for post-publish review, while high-confidence violations are blocked before publication and the creator is notified with a specific policy citation and appeal option.

Human Review Queue

Human reviewers are organized into specialized teams (nudity, violence, hate speech, misinformation) with different expertise areas. The review queue is prioritized based on the story's reach (view count), the creator's previous violation history, and the severity of the detected content. Stories with more than 10,000 views are escalated to priority review within 15 minutes. The average review time for non-priority items is 2 hours, which is acceptable given the 24-hour story lifecycle.

20. Multi-Region Design

Instagram Stories serves a global user base spanning every continent. To deliver sub-100ms latency worldwide, the system must be deployed across multiple geographic regions with active-active replication. This means that every region can independently serve both read and write requests, with data synchronized across regions asynchronously.

Multi-Region Architecture

Write Routing

Write requests (story creation, view recording, reactions) are routed to the user's home region, determined by the user's profile region setting (typically set during account creation based on the initial IP geolocation). This ensures that all writes for a given user always go to the same primary region, avoiding write-write conflicts. Read requests are served from the nearest region using anycast DNS or geo-routed load balancing.

Cross-Region Replication

Story data is replicated across regions asynchronously using a combination of MySQL replication (for structured data) and S3 Cross-Region Replication (for media files). The replication lag between regions is typically 100-500 milliseconds for MySQL and 15-60 minutes for S3 (depending on file size). This means that a story created in US-East may not be immediately visible to users routed to EU-West — but this is acceptable because the Stories tray is regenerated on each app open, and the replication lag is typically shorter than the time between app opens.

Failover Strategy

If a region becomes unavailable, the system automatically routes traffic to the next-nearest healthy region. Since all regions maintain a reasonably current copy of the data (within seconds for MySQL, within minutes for S3), failover results in minimal data loss. The failover is managed by a global traffic manager that monitors region health via heartbeat probes and updates DNS records within 30 seconds of detecting a failure.

21. Cost Estimation

Understanding the infrastructure cost of running Instagram Stories at scale is essential for capacity planning, pricing decisions, and ROI analysis. The cost breakdown spans compute (API servers, transcoding workers), storage (databases, object storage, caches), network (CDN egress, inter-region replication), and third-party services (ML inference, monitoring).

Monthly Cost Breakdown

Total Monthly Cost

The total estimated monthly infrastructure cost for the Instagram Stories system is approximately $7.6 million, with CDN egress being the single largest line item at $2.08 million (27% of total). Storage costs (S3 for ephemeral and highlight media) account for $1.675 million (22%). Compute costs (API servers, transcoding, WebSocket gateway) total $748,800 (10%). The engineering team cost of $2.5 million represents the people needed to build and operate the system. On a per-user basis, the infrastructure cost is approximately $0.015 per monthly active user per month, or roughly $0.18 per user per year.

22. Interview Q&A

The following questions and answers are designed to help senior and staff-level engineers prepare for system design interviews focused on ephemeral content platforms like Instagram Stories. Each answer includes architectural trade-offs and production-grade considerations.

Q1: How do you ensure stories expire exactly after 24 hours across all storage systems?

A: We use a multi-layer TTL strategy with defense in depth. At the MySQL layer, the stories table is partitioned by expires_at range, and a Partition Dropper Worker runs every 15 minutes to drop entire partitions whose time window has fully passed. At the Redis layer, all story-related keys are created with a TTL of 24 hours + 1 hour buffer. At the S3 layer, lifecycle policies automatically transition objects to IA after 12 hours and delete them after 24 hours. At the CDN layer, the Cache-Control: max-age=86400 header ensures edge caches serve content only within the 24-hour window. A 30-second grace period prevents stories from disappearing mid-view. All expiration decisions use the story's canonical expires_at timestamp rather than the current system time to mitigate clock skew.

Q2: How would you handle a celebrity posting a story that gets 50 million views in the first hour?

A: This is a classic hot-key problem. The view count would need to be incremented 50 million times in an hour (~14,000 increments/second). We handle this by distributing the view count across multiple Redis shards using a key-per-shard pattern: view_count:{story_id}:{shard_id} where shard_id ranges from 0-9. Each view event is assigned to a random shard, and the total count is computed as the sum of all shard counters. This distributes the write load across 10 Redis nodes, reducing per-node throughput to 1,400 increments/second. The viewer list uses a similar approach with sharded sorted sets. For the CDN, a celebrity's story is proactively warmed at all edge locations upon publication.

Q3: How do you prevent a user from viewing the same story twice and inflating the view count?

A: Deduplication happens at multiple levels. On the client side, the replay_available flag tracks whether the user has already viewed the story segment. On the server side, the View Service uses a Redis Bloom filter keyed by {story_id}:{viewer_id} to quickly check if a view has been recorded. If the Bloom filter indicates the pair has been seen, the view event is treated as a replay and only the last_viewed_at and view_duration_ms fields are updated. The story_views table uses a composite primary key (story_id, viewer_id) to enforce uniqueness at the database level.

Q4: Design the system for Close Friends stories. How do you efficiently filter stories for millions of followers?

A: The Close Friends list is stored in a close_friends table and cached in Redis as a set keyed by close_friends:{user_id} with a 1-hour TTL. When generating the stories feed for a viewer, the system first retrieves all candidate story groups from followed users, then applies the privacy filter. For each story with visibility = 'close_friends', the filter checks set membership using SISMEMBER close_friends:{creator_id} {viewer_id}, which is an O(1) operation. The Close Friends list is limited to 150 users, so the Redis set fits in a single hash slot and is highly cacheable.

Q5: How would you design the real-time poll results feature?

A: Poll results use Redis as the primary data store for both vote counting and deduplication. Each poll has two Redis keys: a SET for deduplication (poll:votes:{story_id}:{sticker_id}) and a HASH for vote counts (poll:counts:{story_id}:{sticker_id}). When a user votes, we first SADD their user ID to the votes SET — if the return value is 0 (already present), the vote is rejected. We then HINCRBY the corresponding option counter. Both operations are O(1) and atomic. For real-time display, the client polls the results endpoint every 5 seconds while viewing the poll story.

Q6: How do you handle the CDN cache when a user deletes their story before the 24-hour expiry?

A: We employ a two-phase approach. First, the story's status is immediately updated to deleted in MySQL and the Redis cache is invalidated. The API layer filters stories with status = 'deleted' before including them in feed responses. Second, the CDN Purger Worker issues path-based invalidation requests to all CDN edge locations using the pattern /stories/{story_id}/*. During the 15-30 second invalidation window, a soft-delete approach ensures that no new views are served from the API layer. For high-profile accounts, we additionally return 403 from the origin on any request for deleted story media.

Q7: Explain the data model for a multi-segment story. How do you handle partial views and segment ordering?

A: A story group contains multiple segments (up to 100), each stored as a separate row in the stories table with a shared story_group_id. The segment_index field defines the playback order (0-based). For partial views, the story_views table tracks per-segment completion: a view record is created for each segment the user watches. The viewer's current position within a story group is computed as max(segment_index) WHERE completed = true + 1. This allows the viewer to resume playback from where they left off if they close and reopen the story viewer.

Q8: How would you design the system to support scheduling stories for future publication?

A: Scheduled stories use a two-phase commit pattern. When the user schedules a story, the media is immediately uploaded and transcoded, and the story record is created with status = 'scheduled' and a scheduled_at timestamp. A Scheduled Story Worker runs every minute, querying for stories where scheduled_at <= NOW() AND status = 'scheduled'. When found, it updates the status to published, sets published_at, triggers CDN cache warming, and sends push notifications to followers. The scheduled story media is stored in a separate S3 prefix with a longer retention policy (7 days) to account for scheduling delays.

Q9: How do you handle the "stories tray" ranking when a user follows 2,000+ accounts, most of whom have active stories?

A: With 2,000 followed accounts, potentially all posting stories simultaneously, the tray cannot display all of them. The ranking algorithm (Section 8) computes a relevance score for each candidate and returns only the top 50-100 story groups. The score is a weighted combination of relationship strength (40%), recency (35%), and engagement history (25%). Pre-computed relationship and engagement scores are cached in Redis, so the ranking computation itself is purely in-memory and completes in under 10ms. The diversity constraint ensures no more than 3 consecutive stories from the same user appear in the tray.

Q10: Design a system to detect and prevent story view manipulation (bot farms inflating view counts).

A: We implement a multi-layered anti-fraud system. First, device fingerprinting identifies views from emulators, rooted devices, or devices with multiple accounts. Second, behavioral analysis detects patterns inconsistent with human behavior (e.g., viewing stories at exact 5-second intervals, viewing hundreds of stories in rapid succession, consistent geographic patterns). Third, a machine learning model trained on known bot patterns assigns a fraud probability score to each view event. Views with a fraud score above 0.8 are silently discarded without incrementing the view count. Views between 0.5-0.8 are flagged for manual review. The system maintains a blocklist of known bot IP ranges and device IDs, updated daily from threat intelligence feeds.

Q11: How would you implement "Add to Your Story" for resharing someone else's mention?

A: When a user is mentioned in a story, the mention creates a link in their DM inbox with an "Add to Your Story" action. When tapped, the client fetches the original story media via the /stories/{storyId}/reshare endpoint (which checks mention permissions). The original media is rendered as a sticker-like overlay on the resharing user's story canvas, with a "Shared from @{username}" attribution badge. The reshare creates a new story record in the database with a reshared_from_story_id foreign key linking to the original. The original creator receives a notification that their story was reshared, and reshare analytics are tracked separately from original story views.

Q12: What happens when the transcoding service is overloaded and stories are queued for processing?

A: When the transcoding queue depth exceeds a threshold (e.g., > 100,000 pending jobs), the system activates several degradation strategies. First, the publish flow switches to a "progressive publishing" mode where the original (untranscoded) media is immediately published with a low-resolution thumbnail, and transcoded variants are substituted as they become available. This ensures the story is visible within seconds of upload, even if high-quality variants take 30-60 seconds to process. Second, the transcoder prioritizes stories from high-engagement accounts (accounts with > 10,000 followers) to minimize the impact on popular content. Third, auto-scaling triggers additional GPU instances from a warm pool, typically bringing new capacity online within 2-3 minutes.

23. Full C# Implementation

Below is a complete C# implementation of the core Instagram Stories service, including story creation, feed generation, view tracking, TTL management, and highlight operations. This implementation follows clean architecture principles with dependency injection, repository patterns, and async/await throughout.

C#
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Caching.Distributed;
using Microsoft.Extensions.Logging;

namespace InstagramStories.Core.Models
{
    public enum MediaType { Photo, Video }
    public enum StoryVisibility { All, CloseFriends, Custom }
    public enum StoryStatus { Uploading, Processing, Published, Expired, Deleted }
    public enum RingState { Unseen, PartiallySeen, FullySeen }

    public class Story
    {
        public long StoryId { get; set; }
        public long UserId { get; set; }
        public long StoryGroupId { get; set; }
        public MediaType MediaType { get; set; }
        public string MediaUrl { get; set; } = string.Empty;
        public string ThumbnailUrl { get; set; } = string.Empty;
        public int DurationMs { get; set; } = 5000;
        public int Width { get; set; } = 1080;
        public int Height { get; set; } = 1920;
        public long FileSizeBytes { get; set; }
        public StoryVisibility Visibility { get; set; } = StoryVisibility.All;
        public StoryStatus Status { get; set; } = StoryStatus.Uploading;
        public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
        public DateTime ExpiresAt { get; set; }
        public DateTime? PublishedAt { get; set; }
        public long ViewCount { get; set; }
        public int SegmentIndex { get; set; }
        public bool HasStickers { get; set; }
        public long? ResharedFromStoryId { get; set; }
    }

    public class StoryGroup
    {
        public long StoryGroupId { get; set; }
        public long UserId { get; set; }
        public List<Story> Stories { get; set; } = new();
        public int TotalSegments => Stories.Count;
        public int ViewedSegmentCount { get; set; }
        public DateTime LatestStoryCreatedAt =>
            Stories.Max(s => s.CreatedAt);
        public double HoursSinceLatest =>
            (DateTime.UtcNow - LatestStoryCreatedAt).TotalHours;
        public bool HasUnviewedSegments =>
            ViewedSegmentCount < TotalSegments;
        public bool HasInteractiveStickers =>
            Stories.Any(s => s.HasStickers);
    }

    public class ViewEvent
    {
        public long StoryId { get; set; }
        public long ViewerId { get; set; }
        public int ViewDurationMs { get; set; }
        public bool Completed { get; set; }
        public DateTime Timestamp { get; set; } = DateTime.UtcNow;
    }

    public class RankedStoryGroup
    {
        public StoryGroup Group { get; set; } = null!;
        public double Score { get; set; }
        public RingState RingState { get; set; }
    }

    public class PollResult
    {
        public List<PollOptionDisplay> Options { get; set; } = new();
        public int TotalVotes { get; set; }
        public bool HasVoted { get; set; }
        public int SelectedOption { get; set; } = -1;
    }

    public class PollOptionDisplay
    {
        public string Text { get; set; } = string.Empty;
        public int Percentage { get; set; }
        public int VoteCount { get; set; }
    }
}

namespace InstagramStories.Core.Interfaces
{
    using Models;

    public interface IStoryRepository
    {
        Task<Story?> GetStoryByIdAsync(long storyId);
        Task<List<Story>> GetStoriesByUserAsync(long userId);
        Task<List<StoryGroup>> GetStoryGroupsForViewerAsync(
            long viewerId, int limit, string? cursor);
        Task<Story> CreateStoryAsync(Story story);
        Task UpdateStoryStatusAsync(
            long storyId, StoryStatus status);
        Task DeleteStoryAsync(long storyId);
        Task<int> GetViewCountAsync(long storyId);
        Task<List<long>> GetViewerIdsAsync(
            long storyId, int limit, int offset);
    }

    public interface IRedisCache
    {
        Task<T?> GetAsync<T>(string key);
        Task SetAsync<T>(string key, T value, TimeSpan ttl);
        Task<long> IncrementAsync(string key, long amount = 1);
        Task<bool> SetAddAsync(string key, string value);
        Task<bool> HashSetIncrementAsync(
            string key, string field, long amount = 1);
        Task<Dictionary<string, long>> HashGetAllAsync(
            string key);
        Task<bool> SortedSetAddAsync(
            string key, string member, double score);
        Task<List<string>> SortedSetRangeByRankDescendingAsync(
            string key, int start, int stop);
        Task<bool> KeyExpireAsync(string key, TimeSpan ttl);
        Task<bool> ExistsAsync(string key);
        Task<int> SetIntersectionLengthAsync(
            string key1, string key2);
    }

    public interface IKafkaProducer
    {
        Task PublishAsync<T>(string topic, T message);
    }

    public interface IBloomFilter
    {
        Task<bool> MightContainAsync(string key, string value);
        Task AddAsync(string key, string value);
    }

    public interface IRelationshipService
    {
        Task<double> GetRelationshipScoreAsync(
            long userId1, long userId2);
        Task<bool> IsCloseFriendAsync(
            long creatorId, long viewerId);
        Task<bool> IsBlockedAsync(
            long blockerId, long blockedId);
        Task<bool> IsFollowingAsync(
            long followerId, long followeeId);
    }
}

namespace InstagramStories.Core.Services
{
    using Models;
    using Interfaces;

    public class StoryService
    {
        private readonly IStoryRepository _storyRepo;
        private readonly IRedisCache _cache;
        private readonly IKafkaProducer _kafka;
        private readonly IRelationshipService _relationships;
        private readonly ILogger<StoryService> _logger;

        private const string StoryCacheKey = "story:{0}";
        private const string FeedCacheKey = "feed:{0}";
        private const string ViewCountKey = "view_count:{0}";
        private const string FollowingStoriesKey =
            "following_stories:{0}";
        private const int StoryTtlHours = 25;
        private const int FeedCacheTtlSeconds = 30;

        public StoryService(
            IStoryRepository storyRepo,
            IRedisCache cache,
            IKafkaProducer kafka,
            IRelationshipService relationships,
            ILogger<StoryService> logger)
        {
            _storyRepo = storyRepo;
            _cache = cache;
            _kafka = kafka;
            _relationships = relationships;
            _logger = logger;
        }

        public async Task<Story> CreateStoryAsync(
            long userId, long storyGroupId, MediaType mediaType,
            string mediaUrl, int durationMs, long fileSize,
            StoryVisibility visibility,
            List<StickerPlacement>? stickers = null)
        {
            var storyId = GenerateSnowflakeId();
            var story = new Story
            {
                StoryId = storyId,
                UserId = userId,
                StoryGroupId = storyGroupId,
                MediaType = mediaType,
                MediaUrl = mediaUrl,
                DurationMs = durationMs,
                FileSizeBytes = fileSize,
                Visibility = visibility,
                Status = StoryStatus.Processing,
                CreatedAt = DateTime.UtcNow,
                ExpiresAt = DateTime.UtcNow.AddHours(24),
                HasStickers = stickers?.Any() == true
            };

            await _storyRepo.CreateStoryAsync(story);

            await _kafka.PublishAsync("story.created",
                new StoryCreatedEvent
                {
                    StoryId = storyId,
                    UserId = userId,
                    MediaType = mediaType,
                    MediaUrl = mediaUrl,
                    CreatedAt = story.CreatedAt,
                    ExpiresAt = story.ExpiresAt
                });

            _logger.LogInformation(
                "Story {StoryId} created by user {UserId}",
                storyId, userId);

            return story;
        }

        public async Task<List<RankedStoryGroup>>
            GetStoriesFeedAsync(long viewerId,
                int limit = 50, string? cursor = null)
        {
            var cacheKey = string.Format(FeedCacheKey, viewerId);
            var cached = await _cache.GetAsync<
                List<RankedStoryGroup>>(cacheKey);
            if (cached != null && cursor == null)
                return cached.Take(limit).ToList();

            var candidateGroups = await _storyRepo
                .GetStoryGroupsForViewerAsync(
                    viewerId, limit * 3, cursor);

            var ranked = new List<RankedStoryGroup>();
            foreach (var group in candidateGroups)
            {
                if (!await MeetsPrivacyRequirementsAsync(
                    viewerId, group)) continue;

                var score = await ComputeRankingScoreAsync(
                    viewerId, group);
                var ringState = await ComputeRingStateAsync(
                    viewerId, group);

                ranked.Add(new RankedStoryGroup
                {
                    Group = group,
                    Score = score,
                    RingState = ringState
                });
            }

            var result = ranked
                .OrderByDescending(r => r.Score)
                .Take(limit)
                .ToList();

            await _cache.SetAsync(cacheKey, result,
                TimeSpan.FromSeconds(FeedCacheTtlSeconds));

            return result;
        }

        private async Task<double> ComputeRankingScoreAsync(
            long viewerId, StoryGroup group)
        {
            var relationshipScore = await _relationships
                .GetRelationshipScoreAsync(
                    viewerId, group.UserId);

            var recencyScore = Math.Exp(
                -0.0866 * group.HoursSinceLatest);

            var engagementScore =
                await ComputeEngagementScoreAsync(
                    viewerId, group.UserId);

            double composite =
                (0.40 * relationshipScore) +
                (0.35 * recencyScore) +
                (0.25 * engagementScore);

            if (group.HasUnviewedSegments)
                composite *= 1.5;

            if (group.HasInteractiveStickers)
                composite *= 1.2;

            return composite;
        }

        private async Task<double>
            ComputeEngagementScoreAsync(
                long viewerId, long creatorId)
        {
            var cacheKey =
                $"engagement:{viewerId}:{creatorId}";
            var cached = await _cache.GetAsync<
                double?>(cacheKey);
            if (cached.HasValue) return cached.Value;

            var score = new Random().NextDouble();
            await _cache.SetAsync(cacheKey, score,
                TimeSpan.FromHours(6));
            return score;
        }

        private async Task<RingState> ComputeRingStateAsync(
            long viewerId, StoryGroup group)
        {
            if (group.ViewedSegmentCount == 0)
                return RingState.Unseen;
            if (group.ViewedSegmentCount <
                group.TotalSegments)
                return RingState.PartiallySeen;
            return RingState.FullySeen;
        }

        private async Task<bool>
            MeetsPrivacyRequirementsAsync(
                long viewerId, StoryGroup group)
        {
            var creatorId = group.UserId;

            if (await _relationships.IsBlockedAsync(
                creatorId, viewerId))
                return false;

            if (!await _relationships.IsFollowingAsync(
                viewerId, creatorId))
                return false;

            var latestStory = group.Stories
                .OrderByDescending(s => s.CreatedAt)
                .First();

            if (latestStory.Visibility ==
                StoryVisibility.CloseFriends)
            {
                return await _relationships
                    .IsCloseFriendAsync(creatorId, viewerId);
            }

            return true;
        }

        private long GenerateSnowflakeId()
        {
            var timestamp = DateTimeOffset.UtcNow
                .ToUnixTimeMilliseconds();
            var random = new Random().Next(0, 4096);
            return (timestamp << 12) | (long)random;
        }
    }

    public class ViewTrackingService
    {
        private readonly IRedisCache _cache;
        private readonly IKafkaProducer _kafka;
        private readonly IBloomFilter _bloomFilter;
        private readonly ILogger<ViewTrackingService> _logger;

        public ViewTrackingService(
            IRedisCache cache,
            IKafkaProducer kafka,
            IBloomFilter bloomFilter,
            ILogger<ViewTrackingService> logger)
        {
            _cache = cache;
            _kafka = kafka;
            _bloomFilter = bloomFilter;
            _logger = logger;
        }

        public async Task<bool> RecordViewAsync(
            long storyId, long viewerId,
            int viewDurationMs, bool completed)
        {
            var bloomKey = $"views_seen:{storyId}";
            var wasAdded = await _bloomFilter
                .MightContainAsync(bloomKey,
                    viewerId.ToString());

            if (wasAdded)
            {
                _logger.LogDebug(
                    "Duplicate view for story {StoryId}" +
                    " from user {ViewerId}",
                    storyId, viewerId);
                return false;
            }

            await _bloomFilter.AddAsync(bloomKey,
                viewerId.ToString());

            var viewCountKey = string.Format(
                "view_count:{0}", storyId);
            await _cache.IncrementAsync(viewCountKey);

            var viewerListKey = $"viewers:{storyId}";
            await _cache.SortedSetAddAsync(
                viewerListKey,
                viewerId.ToString(),
                DateTimeOffset.UtcNow
                    .ToUnixTimeMilliseconds());

            await _kafka.PublishAsync("story.views",
                new ViewEvent
                {
                    StoryId = storyId,
                    ViewerId = viewerId,
                    ViewDurationMs = viewDurationMs,
                    Completed = completed,
                    Timestamp = DateTime.UtcNow
                });

            return true;
        }

        public async Task<int> GetViewCountAsync(
            long storyId)
        {
            var key = string.Format(
                "view_count:{0}", storyId);
            var count = await _cache.GetAsync<long>(key);
            return (int)count;
        }

        public async Task<List<long>> GetViewersAsync(
            long storyId, int limit = 100, int offset = 0)
        {
            var key = $"viewers:{storyId}";
            var members = await _cache
                .SortedSetRangeByRankDescendingAsync(
                    key, offset, offset + limit - 1);
            return members.Select(long.Parse).ToList();
        }
    }

    public class TTLExpirationService
    {
        private readonly IStoryRepository _storyRepo;
        private readonly IRedisCache _cache;
        private readonly IKafkaProducer _kafka;
        private readonly ILogger<TTLExpirationService> _logger;

        public TTLExpirationService(
            IStoryRepository storyRepo,
            IRedisCache cache,
            IKafkaProducer kafka,
            ILogger<TTLExpirationService> logger)
        {
            _storyRepo = storyRepo;
            _cache = cache;
            _kafka = kafka;
            _logger = logger;
        }

        public async Task ProcessExpiredStoriesAsync()
        {
            var expiredStories = await _storyRepo
                .GetExpiredStoriesAsync();

            foreach (var story in expiredStories)
            {
                await ExpireStoryAsync(story);
            }

            _logger.LogInformation(
                "Processed {Count} expired stories",
                expiredStories.Count);
        }

        private async Task ExpireStoryAsync(Story story)
        {
            await _storyRepo.UpdateStoryStatusAsync(
                story.StoryId, StoryStatus.Expired);

            var storyCacheKey = string.Format(
                "story:{0}", story.StoryId);
            await _cache.KeyExpireAsync(
                storyCacheKey, TimeSpan.FromSeconds(1));

            var viewCountKey = string.Format(
                "view_count:{0}", story.StoryId);
            await _cache.KeyExpireAsync(
                viewCountKey, TimeSpan.FromSeconds(1));

            await _kafka.PublishAsync("story.expired",
                new StoryExpiredEvent
                {
                    StoryId = story.StoryId,
                    UserId = story.UserId,
                    ExpiredAt = DateTime.UtcNow
                });

            _logger.LogDebug(
                "Story {StoryId} expired and cleaned up",
                story.StoryId);
        }
    }

    public class PollService
    {
        private readonly IRedisCache _cache;
        private readonly IKafkaProducer _kafka;

        public PollService(
            IRedisCache cache, IKafkaProducer kafka)
        {
            _cache = cache;
            _kafka = kafka;
        }

        public async Task<PollResult> CastVoteAsync(
            string storyId, string stickerId,
            long voterId, int optionIndex)
        {
            var voteKey =
                $"poll:votes:{storyId}:{stickerId}";
            var added = await _cache.SetAddAsync(
                voteKey, voterId.ToString());

            if (!added)
                throw new InvalidOperationException(
                    "User already voted on this poll");

            await _cache.KeyExpireAsync(voteKey,
                TimeSpan.FromHours(25));

            var counterKey =
                $"poll:counts:{storyId}:{stickerId}";
            await _cache.HashSetIncrementAsync(
                counterKey, optionIndex.ToString());
            await _cache.KeyExpireAsync(counterKey,
                TimeSpan.FromHours(25));

            await _kafka.PublishAsync("poll.votes",
                new PollVoteEvent
                {
                    StoryId = storyId,
                    StickerId = stickerId,
                    VoterId = voterId,
                    OptionIndex = optionIndex,
                    Timestamp = DateTimeOffset.UtcNow
                });

            return await GetResultsAsync(
                storyId, stickerId);
        }

        public async Task<PollResult> GetResultsAsync(
            string storyId, string stickerId)
        {
            var counterKey =
                $"poll:counts:{storyId}:{stickerId}";
            var counts = await _cache
                .HashGetAllAsync(counterKey);

            var totalVotes = counts.Values.Sum();
            var options = counts.Select(kvp =>
                new PollOptionDisplay
            {
                Text = $"Option {kvp.Key}",
                VoteCount = (int)kvp.Value,
                Percentage = totalVotes > 0
                    ? (int)(kvp.Value * 100.0 / totalVotes)
                    : 0
            }).ToList();

            return new PollResult
            {
                Options = options,
                TotalVotes = (int)totalVotes,
                HasVoted = false,
                SelectedOption = -1
            };
        }
    }

    public class HighlightService
    {
        private readonly IStoryRepository _storyRepo;
        private readonly IRedisCache _cache;
        private readonly IKafkaProducer _kafka;

        public HighlightService(
            IStoryRepository storyRepo,
            IRedisCache cache,
            IKafkaProducer kafka)
        {
            _storyRepo = storyRepo;
            _cache = cache;
            _kafka = kafka;
        }

        public async Task AddStoryToHighlightAsync(
            long userId, long highlightId, long storyId)
        {
            var story = await _storyRepo
                .GetStoryByIdAsync(storyId);
            if (story == null || story.UserId != userId)
                throw new UnauthorizedAccessException();

            await _kafka.PublishAsync(
                "highlight.story.added",
                new HighlightStoryEvent
                {
                    UserId = userId,
                    HighlightId = highlightId,
                    StoryId = storyId,
                    MediaUrl = story.MediaUrl,
                    Timestamp = DateTimeOffset.UtcNow
                });

            var cacheKey = $"highlight:{highlightId}";
            await _cache.KeyExpireAsync(cacheKey,
                TimeSpan.FromMinutes(5));
        }

        public async Task ReorderHighlightsAsync(
            long userId,
            List<(long HighlightId, int SortOrder)> order)
        {
            foreach (var (highlightId, sortOrder) in order)
            {
                await _kafka.PublishAsync(
                    "highlight.reordered",
                    new HighlightReorderEvent
                    {
                        UserId = userId,
                        HighlightId = highlightId,
                        SortOrder = sortOrder
                    });
            }
        }
    }
}

namespace InstagramStories.Infrastructure
{
    using Core.Interfaces;
    using Core.Models;
    using System.Collections.Concurrent;
    using System.Text.Json;

    public class ViewEventBatcher : IDisposable
    {
        private readonly ConcurrentQueue<ViewEvent>
            _pendingEvents;
        private readonly IHttpClientFactory _httpClientFactory;
        private readonly Timer _flushTimer;
        private readonly ConcurrentQueue<ViewEvent>
            _retryQueue;
        private const int MaxBatchSize = 50;
        private const int FlushIntervalMs = 5000;
        private const int MaxRetries = 3;

        public ViewEventBatcher(
            IHttpClientFactory httpClientFactory)
        {
            _httpClientFactory = httpClientFactory;
            _pendingEvents =
                new ConcurrentQueue<ViewEvent>();
            _retryQueue =
                new ConcurrentQueue<ViewEvent>();
            _flushTimer = new Timer(
                callback: _ =>
                    FlushAsync().GetAwaiter().GetResult(),
                state: null,
                dueTime: FlushIntervalMs,
                period: FlushIntervalMs);
        }

        public void RecordView(
            long storyId, long viewerId,
            int viewDurationMs, bool completed)
        {
            _pendingEvents.Enqueue(new ViewEvent
            {
                StoryId = storyId,
                ViewerId = viewerId,
                ViewDurationMs = viewDurationMs,
                Completed = completed,
                Timestamp = DateTimeOffset.UtcNow
            });

            if (_pendingEvents.Count >= MaxBatchSize)
                FlushAsync().GetAwaiter().GetResult();
        }

        private async Task FlushAsync()
        {
            var batch = new List<ViewEvent>();
            while (batch.Count < MaxBatchSize &&
                _pendingEvents.TryDequeue(out var evt))
            {
                batch.Add(evt);
            }

            while (_retryQueue.TryDequeue(out var retryEvt))
            {
                if (batch.Count < MaxBatchSize)
                    batch.Add(retryEvt);
            }

            if (batch.Count == 0) return;

            var client = _httpClientFactory
                .CreateClient("view-api");
            var payload = new { events = batch };
            var content = new StringContent(
                JsonSerializer.Serialize(payload),
                System.Text.Encoding.UTF8,
                "application/json");

            try
            {
                var response = await client.PostAsync(
                    "/api/v1/stories/views/batch",
                    content);
                response.EnsureSuccessStatusCode();
            }
            catch (Exception)
            {
                foreach (var evt in batch)
                    _retryQueue.Enqueue(evt);
            }
        }

        public void Dispose()
        {
            _flushTimer?.Dispose();
            FlushAsync().GetAwaiter().GetResult();
        }
    }
}

24. Conclusion

Designing an Instagram Stories system at billion-user scale is a profound engineering challenge that spans nearly every domain of distributed systems: real-time data processing, media pipelines, ephemeral content management, privacy enforcement, content moderation, global CDN distribution, and multi-region active-active replication. In this comprehensive guide, we have walked through every major component of the system, from the high-level architecture down to the database schema, API contracts, caching strategies, and C# implementation details.

The key takeaways from this design exercise are: ephemerality demands a multi-layer TTL strategy that coordinates across MySQL partitions, Redis key expiration, S3 lifecycle policies, and CDN cache headers — no single mechanism is sufficient. Read-heavy workloads benefit enormously from multi-tier caching, with the CDN absorbing 98.5% of media requests and Redis serving 95% of metadata reads from memory. Write-heavy workloads like view tracking require careful batching and deduplication, using Bloom filters, batched API calls, and Kafka-based async processing to achieve exactly-once semantics at scale. Interactive features like polls and questions are best served from Redis, which provides the atomic operations and sub-millisecond latency needed for real-time aggregation.

The cost analysis reveals that CDN egress is the single largest infrastructure expense at $2.08 million per month (27% of total), underscoring the importance of efficient caching and compression strategies. The engineering team investment of $2.5 million per month represents the largest single cost item, highlighting that the complexity of building and operating a system at this scale requires a deep bench of senior engineers with expertise across the full stack.

For system design interview preparation, the most critical concepts to master are: capacity estimation (understanding the numbers that drive architectural decisions), data modeling for ephemeral content (partition strategies, TTL enforcement), real-time processing pipelines (Kafka, Redis, batch processing), and privacy/access control at scale (Close Friends, hidden-from lists, block integration). The 12 interview questions and answers in Section 22 cover the most frequently asked scenarios and provide production-grade solutions that demonstrate senior-level thinking.

As ephemeral content continues to dominate social media engagement — with Stories format being adopted by virtually every major platform including TikTok, YouTube, LinkedIn, and Facebook — the architectural patterns and principles discussed in this guide remain highly relevant and transferable. The core challenges of ephemeral content (precise TTL management, real-time engagement tracking, privacy enforcement, and CDN optimization) are universal, and the solutions presented here can be adapted to platforms of any scale.

Whether you are building the next generation of ephemeral content features, preparing for a system design interview at a top-tier technology company, or simply deepening your understanding of distributed systems architecture, we hope this guide has provided you with actionable insights and practical knowledge. The Instagram Stories system is a testament to what modern distributed systems engineering can achieve — and the patterns we have explored here will continue to be foundational building blocks for the real-time, ephemeral, interactive experiences that define the future of social media.