1. Introduction & Why Video Streaming is Hard

Video streaming is one of the most bandwidth-intensive and technically challenging problems in modern system design. YouTube alone serves over 1 billion hours of video per day, Netflix accounts for approximately 15% of global internet bandwidth, and TikTok delivers billions of short-form videos to mobile users worldwide. The sheer scale of video data — a single 4K movie can be 50-100 GB in raw format — makes streaming fundamentally different from text-based or image-based applications.

The core challenge is that video content must be stored, processed, and delivered at massive scale while maintaining a smooth, buffer-free playback experience for users on varying network conditions. A user on a 5G connection in New York expects instant 4K playback, while a user on a 3G network in rural India needs a smooth 144p experience. The system must adapt to both without perceptible quality degradation or buffering.

The technical complexity spans multiple domains: video encoding (converting raw video into efficient formats), adaptive bitrate streaming (switching quality on-the-fly based on network conditions), content delivery (getting video bits close to users via CDN), content recommendation (keeping users engaged with relevant suggestions), and content moderation (ensuring platform safety at scale). Each of these domains is a deep technical challenge on its own; combining them into a cohesive platform is what makes video streaming one of the most complex system design problems.

The Scale of the Problem

Consider the numbers involved in a YouTube-scale platform. Over 500 hours of video are uploaded every minute, requiring an encoding pipeline that can process content faster than it arrives. The platform stores hundreds of petabytes of video data across multiple resolutions and formats. At peak hours, millions of concurrent viewers generate hundreds of terabits per second of outgoing traffic — more than many entire countries consume in internet bandwidth. The recommendation system must serve billions of personalized suggestions per day, and the content moderation system must review millions of videos for policy violations.

For system design interviews, video streaming questions test your ability to reason about bandwidth-intensive workloads, asynchronous processing pipelines, CDN architecture, and machine learning at scale. The interviewer wants to see that you understand the unique challenges of video data — large file sizes, CPU-intensive encoding, strict latency requirements for playback, and the trade-offs between quality and bandwidth.

2. Functional & Non-Functional Requirements

Functional Requirements

1. Video Upload: Users can upload videos (up to 12 hours, 128 GB). The system must support resumable uploads for large files.
2. Video Processing: Uploaded videos are automatically transcoded into multiple resolutions (144p to 8K), formats (H.264, H.265/HEVC, AV1), and container formats (MP4, WebM). Thumbnail extraction and metadata parsing happen in parallel.
3. Video Playback: Users can stream videos with adaptive bitrate switching. Playback must start within 2 seconds and buffer-free playback must be maintained.
4. Search & Discovery: Users can search videos by title, tags, description, and creator. The system provides personalized recommendations on the home feed.
5. Social Features: Users can like/dislike, comment, subscribe to channels, create playlists, and share videos.
6. Watch History: The system tracks watch progress and allows resuming playback across devices.
7. Content Moderation: Automated and human review for policy violations, copyright claims, and inappropriate content.
8. Monetization: Pre-roll, mid-roll, and overlay ads. Premium subscription tiers for ad-free viewing.
9. Live Streaming: Creators can broadcast live content to their subscribers with real-time chat.
10. Analytics: Creators can view detailed analytics about views, watch time, audience demographics, and revenue.

Non-Functional Requirements

3. Capacity Estimation & Back-of-Envelope

Video Storage

Assume 500 hours of video uploaded per minute. At an average bitrate of 5 Mbps for 1080p, each hour requires approximately 2.25 GB of raw encoded data. With three encoding resolutions (360p, 720p, 1080p) at an average of 1.5 GB per hour per resolution, the storage requirement per hour of uploaded video is approximately 4.5 GB.

• Upload rate: 500 hours/minute = 30,000 hours/hour = 720,000 hours/day
• Daily storage: 720,000 hours × 4.5 GB = 3.24 PB/day
• Annual storage: ~1,180 PB = ~1.18 EB (exabyte)
• With replication (3 copies): ~3.5 EB

Bandwidth

Assume 2 billion daily active users watching an average of 30 minutes per day. At an average playback bitrate of 3 Mbps for adaptive streaming:

• Concurrent viewers (peak): 500 million (assuming 25% peak concurrent)
• Peak bandwidth: 500M × 3 Mbps = 1.5 Pbps (petabits per second)
• Daily data transfer: 2B users × 30 min × 3 Mbps = 108 PB/day
• Monthly data transfer: ~3,240 PB

QPS Estimates

4. Data Model & Storage Schema

The data model must accommodate diverse data types: structured metadata, semi-structured analytics events, large binary video files, and real-time streaming data. No single database technology can efficiently serve all these access patterns, so a polyglot persistence approach is essential.

Video Entity (PostgreSQL)

SQL
CREATE TABLE videos (
    video_id        UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    channel_id      UUID NOT NULL REFERENCES channels(channel_id),
    title           VARCHAR(500) NOT NULL,
    description     TEXT,
    upload_status   VARCHAR(20) NOT NULL DEFAULT 'uploading',
        -- uploading, processing, published, failed, deleted
    processing_status VARCHAR(30),
        -- pending, transcoding, thumbnailing, indexing, complete
    visibility      VARCHAR(15) NOT NULL DEFAULT 'public',
        -- public, unlisted, private, restricted
    duration_seconds INTEGER,
    original_filename VARCHAR(500),
    original_format VARCHAR(20),
    original_resolution VARCHAR(10),
    file_size_bytes BIGINT,
    view_count      BIGINT DEFAULT 0,
    like_count      BIGINT DEFAULT 0,
    dislike_count   BIGINT DEFAULT 0,
    comment_count   INTEGER DEFAULT 0,
    tags            TEXT[],
    category_id     INTEGER,
    language        VARCHAR(10),
    captions_enabled BOOLEAN DEFAULT FALSE,
    created_at      TIMESTAMPTZ DEFAULT NOW(),
    published_at    TIMESTAMPTZ,
    updated_at      TIMESTAMPTZ DEFAULT NOW(),
    processed_at    TIMESTAMPTZ
);

CREATE INDEX idx_videos_channel ON videos(channel_id, published_at DESC);
CREATE INDEX idx_videos_status ON videos(processing_status) WHERE processing_status != 'complete';
CREATE INDEX idx_videos_published ON videos(published_at DESC) WHERE visibility = 'public';

Video Variant (Transcoded Versions)

SQL
CREATE TABLE video_variants (
    variant_id      UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    video_id        UUID NOT NULL REFERENCES videos(video_id),
    resolution      VARCHAR(10) NOT NULL,    -- 360p, 720p, 1080p, 1440p, 2160p
    codec           VARCHAR(20) NOT NULL,    -- h264, h265, av1
    container       VARCHAR(10) NOT NULL,    -- mp4, webm
    bitrate_kbps    INTEGER NOT NULL,
    file_size_bytes BIGINT NOT NULL,
    storage_path    VARCHAR(1000) NOT NULL,  -- s3://bucket/path
    checksum        VARCHAR(64),
    status          VARCHAR(15) DEFAULT 'pending',
    created_at      TIMESTAMPTZ DEFAULT NOW()
);

CREATE INDEX idx_variants_video ON video_variants(video_id);

User Watch History (Cassandra)

SQL
CREATE TABLE watch_history (
    user_id         UUID,
    video_id        UUID,
    watched_at      TIMESTAMP,
    watch_duration  INT,          -- seconds watched
    progress_pct    FLOAT,        -- percentage completed
    last_position   INT,          -- resume position in seconds
    completed       BOOLEAN,
    device_type     TEXT,         -- mobile, desktop, tv, tablet
    PRIMARY KEY (user_id, watched_at, video_id)
) WITH CLUSTERING ORDER BY (watched_at DESC);

Channel Entity (PostgreSQL)

SQL
CREATE TABLE channels (
    channel_id      UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    owner_user_id   UUID NOT NULL REFERENCES users(user_id),
    name            VARCHAR(200) NOT NULL,
    handle          VARCHAR(100) UNIQUE NOT NULL,
    description     TEXT,
    avatar_url      VARCHAR(1000),
    banner_url      VARCHAR(1000),
    subscriber_count BIGINT DEFAULT 0,
    total_view_count BIGINT DEFAULT 0,
    total_video_count INTEGER DEFAULT 0,
    monetization_enabled BOOLEAN DEFAULT FALSE,
    created_at      TIMESTAMPTZ DEFAULT NOW()
);

Storage Strategy

5. High-Level Architecture Overview

The video streaming platform is composed of five major subsystems: the upload/encoding pipeline, the storage layer, the delivery system (CDN), the recommendation engine, and the metadata/search system. These subsystems communicate asynchronously through message queues (Kafka) to decouple upload from processing, processing from delivery, and user interactions from analytics.

Request Flow: Video Playback

When a user clicks on a video thumbnail, the following sequence occurs within 200 milliseconds:

1. The client requests video metadata from the API server (title, duration, available resolutions, thumbnail URLs). This is served from Redis cache with a 5-minute TTL.
2. The API server returns the manifest URL (HLS .m3u8 or DASH .mpd) pointing to the CDN origin.
3. The client fetches the manifest from the CDN edge server closest to the user. The manifest lists all available video variants with their resolution, bitrate, and CDN URLs.
4. The client's adaptive bitrate player measures available bandwidth and selects the highest quality variant that can be streamed without buffering. It begins downloading video segments (typically 2-10 seconds each) from the CDN.
5. As bandwidth fluctuates, the player dynamically switches between variants — upgrading to higher quality when bandwidth improves and downgrading when it degrades.
6. Meanwhile, the client sends a watch event to the analytics pipeline (via Kafka) to track the view, and the watch history service records the user's viewing progress.

Request Flow: Video Upload

Video upload is an asynchronous pipeline with multiple stages:

1. The client requests an upload session from the Upload Service, providing file metadata (size, duration, title). The service returns a unique upload ID and pre-signed S3 URLs for resumable chunk upload.
2. The client uploads the video file in 5 MB chunks directly to S3 using the pre-signed URLs. Each chunk is independently verifiable via checksum. If the connection drops, the client can resume from the last successful chunk.
3. Once all chunks are uploaded, the client signals upload completion. The Upload Service assembles the chunks, verifies the complete file checksum, and publishes a "video uploaded" event to Kafka.
4. The Encoding Orchestrator picks up the event and creates a processing pipeline: transcode to multiple resolutions, extract thumbnails, run AI content analysis, generate subtitles, and create HLS/DASH manifests.
5. Each encoding job reports progress back to the metadata service. The client (and channel page) polls for status or receives updates via WebSocket.
6. Once all processing stages complete, the video status changes to "published" and becomes searchable and viewable.

6. Video Upload Pipeline

Video upload is deceptively complex. Unlike uploading a profile picture, video files can be 10-128 GB, requiring resumable uploads that survive network interruptions, browser crashes, and device switches. The upload pipeline must handle massive file sizes while providing a smooth user experience with progress tracking and error recovery.

Resumable Chunked Upload

The upload system uses a resumable upload protocol inspired by Google's Resumable Upload Protocol and TUS (an open standard for resumable file uploads). The key insight is to never upload the entire file in a single HTTP request — instead, break the file into manageable chunks that can be individually verified and retried.

C#
public class ResumableUploadService
{
    private readonly IS3Client _s3Client;
    private readonly IUploadRepository _uploadRepo;
    private const long ChunkSizeBytes = 5 * 1024 * 1024; // 5 MB chunks

    public async Task<UploadSession> CreateUploadSessionAsync(
        UploadRequest request)
    {
        var sessionId = Guid.NewGuid();
        var totalChunks = (int)Math.Ceiling(
            request.FileSizeBytes / (double)ChunkSizeBytes);

        var session = new UploadSession
        {
            SessionId = sessionId,
            VideoId = Guid.NewGuid(),
            FileName = request.FileName,
            FileSizeBytes = request.FileSizeBytes,
            TotalChunks = totalChunks,
            UploadedChunks = new List<int>(),
            Status = UploadStatus.Created,
            CreatedAt = DateTime.UtcNow,
            ExpiresAt = DateTime.UtcNow.AddHours(24)
        };

        await _uploadRepo.CreateSessionAsync(session);

        // Generate pre-signed S3 URLs for each chunk
        var chunkUrls = new List<ChunkUploadUrl>();
        for (int i = 0; i < totalChunks; i++)
        {
            var chunkKey = $"uploads/{sessionId}/chunk_{i:D6}";
            var presignedUrl = await _s3Client.GeneratePresignedUrlAsync(
                chunkKey, TimeSpan.FromHours(24), HttpVerb.PUT);
            chunkUrls.Add(new ChunkUploadUrl
            {
                ChunkIndex = i,
                Url = presignedUrl,
                OffsetBytes = i * ChunkSizeBytes,
                SizeBytes = Math.Min(ChunkSizeBytes,
                    request.FileSizeBytes - i * ChunkSizeBytes)
            });
        }

        return new UploadSession
        {
            SessionId = sessionId,
            VideoId = session.VideoId,
            ChunkUrls = chunkUrls,
            ChunkSizeBytes = ChunkSizeBytes
        };
    }

    public async Task<ChunkUploadResult> UploadChunkAsync(
        Guid sessionId, int chunkIndex, Stream chunkData,
        string checksum)
    {
        var session = await _uploadRepo.GetSessionAsync(sessionId);

        if (session.Status == UploadStatus.Expired)
            throw new UploadSessionExpiredException();

        if (session.UploadedChunks.Contains(chunkIndex))
            return new ChunkUploadResult
            {
                ChunkIndex = chunkIndex,
                AlreadyUploaded = true
            };

        // Verify checksum of received chunk
        var computedChecksum = await ComputeSHA256Async(chunkData);
        if (computedChecksum != checksum)
            throw new ChecksumMismatchException(checksum, computedChecksum);

        // Upload chunk to S3
        chunkData.Position = 0;
        var chunkKey = $"uploads/{sessionId}/chunk_{chunkIndex:D6}";
        await _s3Client.PutObjectAsync(chunkKey, chunkData,
            "application/octet-stream");

        // Record chunk completion
        session.UploadedChunks.Add(chunkIndex);
        session.LastChunkUploadedAt = DateTime.UtcNow;

        if (session.UploadedChunks.Count == session.TotalChunks)
        {
            session.Status = UploadStatus Complete;
            await TriggerAssemblyAsync(session);
        }

        await _uploadRepo.UpdateSessionAsync(session);

        return new ChunkUploadResult
        {
            ChunkIndex = chunkIndex,
            AlreadyUploaded = false,
            UploadComplete = session.Status == UploadStatus.Complete,
            BytesUploaded = session.UploadedChunks.Count *
                ChunkSizeBytes
        };
    }

    private async Task TriggerAssemblyAsync(UploadSession session)
    {
        // Verify all chunks are present and compute full file checksum
        var objectKeys = Enumerable.Range(0, session.TotalChunks)
            .Select(i => $"uploads/{session.SessionId}/chunk_{i:D6}")
            .ToList();

        var assemblyResult = await _s3Client.AssembleMultipartAsync(
            $"videos/{session.VideoId}/original",
            objectKeys);

        // Verify complete file checksum matches
        if (assemblyResult.Checksum != session.ExpectedChecksum)
        {
            session.Status = UploadStatus.Failed;
            await _uploadRepo.UpdateSessionAsync(session);
            throw new FileAssemblyFailedException();
        }

        // Publish event to trigger encoding pipeline
        await _eventBus.PublishAsync(new VideoUploadedEvent
        {
            VideoId = session.VideoId,
            OriginalPath = $"videos/{session.VideoId}/original",
            FileSizeBytes = session.FileSizeBytes,
            Checksum = assemblyResult.Checksum,
            UploadedAt = DateTime.UtcNow
        });
    }
}

Upload Protocol Flow

Upload Optimization Strategies

Upload Resilience

The upload system must handle various failure scenarios gracefully. Network interruptions are the most common failure — a user on a mobile connection may lose connectivity for minutes or hours. The system handles this by storing each chunk independently in S3 and tracking which chunks have been uploaded. When the user reconnects, the client queries the upload session to identify missing chunks and resumes from there. Browser crashes are handled similarly — the upload session persists server-side with a 24-hour expiration, and the client can resume from any point. Device switches (e.g., starting upload on phone, finishing on desktop) are supported by making the upload session accessible via the user's account across devices.

For extremely large files (broadcast-quality content, 4K+ raw footage), the system supports a multi-stage upload: upload a low-resolution proxy for immediate viewing, then upload the full-resolution file in the background. This ensures creators can share content quickly while the high-quality version processes asynchronously.

7. Transcoding & Encoding System

Transcoding is the most computationally expensive operation in the entire platform. A single 10-minute 4K video requires approximately 2-4 hours of GPU time to transcode into all required formats and resolutions. At YouTube scale (500 hours uploaded per minute), the encoding cluster must provide thousands of GPU-hours per minute — a massive distributed computing challenge.

Encoding Pipeline Architecture

The encoding pipeline is a Directed Acyclic Graph (DAG) of processing stages, where each stage can run independently or in parallel. The orchestrator manages the execution of this DAG, handling retries, progress tracking, and error recovery.

Encoding Configuration Matrix

Encoding Orchestrator (C#)

C#
public class EncodingOrchestrator
{
    private readonly IJobQueue _jobQueue;
    private readonly IVideoMetadataRepo _metadataRepo;
    private readonly IEventBus _eventBus;

    public async Task<ProcessingPipeline> CreatePipelineAsync(
        VideoUploadedEvent evt)
    {
        var metadata = await ProbeVideoAsync(evt.OriginalPath);
        var variants = DetermineVariants(metadata);
        var pipeline = new ProcessingPipeline
        {
            VideoId = evt.VideoId,
            Stages = new List<ProcessingStage>()
        };

        // Stage 1: Pre-processing (audio normalization, deinterlacing)
        pipeline.Stages.Add(new ProcessingStage
        {
            StageType = StageType.PreProcess,
            Status = StageStatus.Pending,
            InputPath = evt.OriginalPath,
            EstimatedDuration = TimeSpan.FromMinutes(2)
        });

        // Stage 2: Parallel encoding jobs
        foreach (var variant in variants)
        {
            pipeline.Stages.Add(new ProcessingStage
            {
                StageType = StageType.Encode,
                Status = StageStatus.Pending,
                Parameters = new EncodingParams
                {
                    Resolution = variant.Resolution,
                    Codec = variant.Codec,
                    Bitrate = variant.Bitrate,
                    Profile = variant.Profile,
                    Preset = variant.Codec == CodecType.AV1
                        ? "slower" : "medium",
                    PixelFormat = "yuv420p",
                    Scoring = "psnr"
                },
                OutputPath = $"videos/{evt.VideoId}/{variant.Key}",
                EstimatedDuration = EstimateEncodingTime(
                    metadata.Duration, variant)
            });
        }

        // Stage 3: Packaging (HLS + DASH manifests)
        pipeline.Stages.Add(new ProcessingStage
        {
            StageType = StageType.Package,
            Status = StageStatus.Pending,
            DependsOn = pipeline.Stages
                .Where(s => s.StageType == StageType.Encode)
                .Select(s => s.StageId).ToList(),
            EstimatedDuration = TimeSpan.FromSeconds(30)
        });

        // Stage 4: Thumbnail extraction (parallel)
        pipeline.Stages.Add(new ProcessingStage
        {
            StageType = StageType.Thumbnail,
            Status = StageStatus.Pending,
            DependsOn = new List<Guid> { pipeline.Stages[0].StageId },
            EstimatedDuration = TimeSpan.FromSeconds(15)
        });

        // Stage 5: AI processing (parallel, independent)
        pipeline.Stages.Add(new ProcessingStage
        {
            StageType = StageType.AIClassify,
            Status = StageStatus.Pending,
            DependsOn = new List<Guid> { pipeline.Stages[0].StageId },
            EstimatedDuration = TimeSpan.FromMinutes(3)
        });

        pipeline.Stages.Add(new ProcessingStage
        {
            StageType = StageType.SpeechToText,
            Status = StageStatus.Pending,
            DependsOn = new List<Guid> { pipeline.Stages[0].StageId },
            EstimatedDuration = TimeSpan.FromMinutes(5)
        });

        // Total estimated: duration-dependent, typically 0.5x to 2x
        // real-time for 1080p H.264, 2-4x for 4K AV1
        pipeline.TotalEstimatedDuration = pipeline.Stages
            .Where(s => !s.DependsOn.Any())
            .Max(s => s.EstimatedDuration) // critical path
            + pipeline.Stages
            .Where(s => s.StageType == StageType.Package)
            .Max(s => s.EstimatedDuration);

        await _metadataRepo.SavePipelineAsync(pipeline);
        await EnqueueStagesAsync(pipeline);

        return pipeline;
    }

    private List<VideoVariant> DetermineVariants(VideoMetadata meta)
    {
        var variants = new List<VideoVariant>();
        var targetResolutions = GetTargetResolutions(meta.Resolution);

        foreach (var res in targetResolutions)
        {
            // Always produce H.264 for compatibility
            variants.Add(CreateVariant(res, CodecType.H264));

            // Produce H.265 for resolutions >= 1080p
            if (res.Height >= 1080)
                variants.Add(CreateVariant(res, CodecType.H265));

            // Produce AV1 for resolutions <= 1080p (encoding is slower
            // but bandwidth savings are highest at common resolutions)
            if (res.Height <= 1080 && meta.Duration <= 7200)
                variants.Add(CreateVariant(res, CodecType.AV1));
        }

        return variants;
    }
}

Encoding Cluster Sizing

To handle YouTube-scale upload volumes, the encoding cluster must be sized for peak throughput. A modern NVIDIA A100 GPU can transcode approximately 30-50 streams of 1080p H.264 simultaneously, or 10-15 streams of 4K HEVC. For AV1 encoding using the SVT-AV1 encoder, throughput is approximately 5-10 streams of 1080p per GPU.

At 500 hours uploaded per minute, with an average encoding ratio of 0.5x (10 minutes of video takes 5 minutes to encode for 1080p), the cluster needs approximately 2,500 GPU-minutes per minute of encoding capacity. This translates to roughly 80-100 high-end GPUs running continuously, with headroom for peak bursts. The actual number is higher because multiple variants are produced per upload (H.264, H.265, AV1 at different resolutions), multiplying the total compute requirement by 3-5x.

Encoding Optimization Techniques

• Two-pass encoding: First pass analyzes video complexity, second pass optimizes bitrate allocation. Improves quality by 15-20% at the same bitrate but doubles encoding time.
• Per-title encoding: Instead of fixed bitrate ladders, analyze each video's complexity and create custom bitrate/resolution pairs. A static talking-head video needs much less bitrate than an action sports video at the same resolution.
• Chunked encoding: Split the video into 10-second chunks and encode them in parallel across multiple GPUs. Reduces total encoding time by 10-50x depending on chunk size and GPU count.
• Hardware-accelerated decoding: Use GPU hardware decoders (NVDEC) to decode the source video, freeing GPU resources for encoding. This allows encoding to be the bottleneck rather than decode+encode.
• Encode once, package many: Produce a single high-quality intermediate format (ProRes, DNxHR) and generate all distribution formats from it. This avoids re-decoding the original for each variant.

8. Adaptive Bitrate Streaming (HLS/DASH)

Adaptive Bitrate Streaming (ABR) is the technology that enables smooth video playback across varying network conditions. Instead of serving a single fixed-quality video stream, ABR creates multiple quality versions of each video and dynamically switches between them based on real-time bandwidth measurements. This is what allows a user to start watching in 1080p and seamlessly downgrade to 480p when they enter a tunnel, then upgrade back to 1080p when the signal improves — all without any perceptible interruption.

How ABR Works

The fundamental mechanism is elegant: the video is encoded at multiple bitrates and resolutions, segmented into small chunks (typically 2-10 seconds each), and organized into a manifest file. The player downloads the manifest, measures available bandwidth, and requests the highest-quality chunk it can download before the next chunk is needed. If a chunk downloads faster than expected, the player requests a higher quality for the next segment. If it downloads slower, the player downgrades.

HLS vs DASH

ABR Algorithm Implementation

C#
public class AdaptiveBitratePlayer
{
    private readonly List<VideoVariant> _variants;
    private readonly BandwidthEstimator _bandwidthEstimator;
    private readonly PlaybackBuffer _buffer;

    private const int BufferTargetSeconds = 30;
    private const int BufferLowWatermark = 10;
    private const int BufferHighWatermark = 45;

    public VideoVariant SelectNextVariant()
    {
        var estimatedBandwidth = _bandwidthEstimator.CurrentEstimate;
        var bufferLevel = _buffer.CurrentLevelSeconds;

        // Sort variants by bitrate descending
        var candidates = _variants
            .OrderByDescending(v => v.BitrateKbps)
            .ToList();

        foreach (var variant in candidates)
        {
            // Can we sustain this bitrate?
            var safetyMargin = 0.7f; // 30% safety margin
            if (variant.BitrateKbps * safetyMargin <=
                estimatedBandwidth)
            {
                // Additional buffer-based adjustment
                if (bufferLevel < BufferLowWatermark &&
                    variant.BitrateKbps >=
                    _currentVariant.BitrateKbps)
                {
                    // Buffer is low — don't upgrade, maybe downgrade
                    if (_buffer.FillRate < _buffer.DrainRate)
                        return _currentVariant; // Stay at current
                }

                if (bufferLevel > BufferHighWatermark &&
                    variant.BitrateKbps >
                    _currentVariant.BitrateKbps)
                {
                    // Buffer is full — safe to upgrade
                    return variant;
                }

                // Normal case: select highest sustainable bitrate
                return variant;
            }
        }

        // All variants exceed available bandwidth — use lowest
        return candidates.Last();
    }

    public class BandwidthEstimator
    {
        private readonly Queue<BandwidthSample> _recentSamples = new();
        private const int MaxSamples = 20;
        private const int WindowSeconds = 60;

        public void RecordSample(long bytesDownloaded,
            TimeSpan downloadTime)
        {
            var kbps = (bytesDownloaded * 8.0 /
                downloadTime.TotalSeconds) / 1000;

            _recentSamples.Enqueue(new BandwidthSample
            {
                Kbps = kbps,
                Timestamp = DateTime.UtcNow
            });

            // Remove old samples outside the window
            while (_recentSamples.Count > 0 &&
                _recentSamples.Peek().Timestamp <
                DateTime.UtcNow.AddSeconds(-WindowSeconds))
            {
                _recentSamples.Dequeue();
            }
        }

        public double CurrentEstimate
        {
            get
            {
                if (!_recentSamples.Any()) return 0;

                // Use weighted moving average with recent bias
                var weights = _recentSamples.Select((s, i) =>
                    Math.Pow(2, i / (double)MaxSamples)).ToList();
                var weightedSum = _recentSamples.Zip(weights,
                    (s, w) => s.Kbps * w).Sum();
                return weightedSum / weights.Sum();
            }
        }
    }
}

HLS Manifest Example

HLS
#EXTM3U
#EXT-X-VERSION:7
#EXT-X-STREAM-INF:BANDWIDTH=6000000,
    RESOLUTION=1920x1080,CODECS="avc1.640028,mp4a.40.2",
    FRAME-RATE=30
1080p/master.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=3500000,
    RESOLUTION=1280x720,CODECS="avc1.64001f,mp4a.40.2",
    FRAME-RATE=30
720p/master.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1500000,
    RESOLUTION=854x480,CODECS="avc1.64001e,mp4a.40.2",
    FRAME-RATE=30
480p/master.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=500000,
    RESOLUTION=640x360,CODECS="avc1.64001e,mp4a.40.2",
    FRAME-RATE=30
360p/master.m3u8

9. Distributed Storage Architecture

Video storage at scale is fundamentally a distributed storage problem. A single video may be stored across dozens of geographic regions, replicated 3-5 times for durability, and organized into hot/warm/cold tiers based on access patterns. The storage architecture must balance three competing concerns: durability (never lose a video), availability (always serve a video), and cost (don't go bankrupt storing exabytes of data).

Storage Hierarchy

Content-Aware Storage Placement

Not all videos are accessed equally. A power-law distribution means that roughly 1% of videos account for 80% of views. The storage system exploits this by placing frequently accessed videos on fast storage (SSDs with high IOPS) and infrequently accessed videos on cheaper storage (HDDs or archive tiers). This tiering is automated based on real-time access patterns.

C#
public class StorageTieringService
{
    private readonly IAccessPatternAnalyzer _analyzer;
    private readonly IStorageTierRepository _tierRepo;

    public async Task RunTieringJobAsync()
    {
        var videos = await _tierRepo.GetAllActiveVideos();

        foreach (var video in videos)
        {
            var accessMetrics = await _analyzer
                .GetAccessMetricsAsync(video.VideoId,
                    TimeSpan.FromDays(7));

            var targetTier = DetermineTier(accessMetrics);

            if (targetTier != video.CurrentTier)
            {
                await TransitionTierAsync(video, targetTier);
            }
        }
    }

    private StorageTier DetermineTier(AccessMetrics metrics)
    {
        // Viral video (> 100K views/day) → Hot tier
        if (metrics.DailyViews > 100_000)
            return StorageTier.Hot;

        // Active video (1K-100K views/day) → Warm tier
        if (metrics.DailyViews > 1_000)
            return StorageTier.Warm;

        // Standard video (10-1K views/day) → Cold tier
        if (metrics.DailyViews > 10)
            return StorageTier.Cold;

        // Low-traffic video (< 10 views/day) → Archive
        return StorageTier.Archive;
    }

    private async Task TransitionTierAsync(
        CachedVideo video, StorageTier targetTier)
    {
        // For hot→warm: background copy, then redirect reads
        // For warm→cold: async copy, update metadata
        // For cold→hot: priority copy, wait for completion

        if (targetTier == StorageTier.Hot)
        {
            // URGENT: must copy before serving from hot tier
            await _storageClient.CopyToFastTierAsync(
                video.StoragePath, video.VideoId);
        }
        else
        {
            // Non-urgent: copy in background
            await _backgroundJobs.EnqueueAsync(
                new StorageTransitionJob
            {
                VideoId = video.VideoId,
                FromTier = video.CurrentTier,
                ToTier = targetTier,
                Priority = targetTier == StorageTier.Archive
                    ? JobPriority.Low : JobPriority.Normal
            });
        }

        video.CurrentTier = targetTier;
        await _tierRepo.UpdateAsync(video);
    }
}

Multi-Region Replication

For a global streaming platform, video content must be replicated across multiple geographic regions to ensure low-latency access and disaster recovery. The replication strategy uses a primary-secondary model for metadata (strong consistency within a region, eventual consistency across regions) and a multi-primary model for video content (any region can serve any video, with eventual consistency acceptable).

The replication pipeline uses an event-driven architecture: when a new video is uploaded and encoded, the completed variants are written to the primary region's object storage. A replication controller then asynchronously copies the files to secondary regions based on predicted demand. Popular videos are replicated eagerly to all regions; niche content is replicated lazily on first access from a remote region.

10. CDN & Video Delivery

The CDN (Content Delivery Network) is the single most critical component for video delivery quality. It determines whether a user in São Paulo gets buffer-free playback or watches a spinning loading icon. The CDN caches video segments at edge locations worldwide, so users download video from a server near them rather than from the origin data center thousands of miles away.

CDN Architecture for Video

Video CDNs use a hierarchical caching architecture with three tiers: edge servers (thousands of locations near users), mid-tier caches (regional aggregation points), and the origin shield (the authoritative source in the primary data center). This hierarchy reduces origin load by 99.9% — for every 1,000 requests for a popular video segment, only 1 request reaches the origin.

CDN Cache Strategy

Video segments are immutable — once encoded, a segment at a specific resolution and time offset never changes. This makes video content ideal for CDN caching because cache invalidation is never needed. The cacheability rules are straightforward:

• Video segments (.ts, .m4s): Cache for 1 year (immutable content)
• Manifest files (.m3u8, .mpd): Cache for 30-60 seconds (updated when encoding completes or ad insertion changes)
• Thumbnails: Cache for 24 hours (rarely change)
• API responses: Cache for 5-60 seconds depending on staleness tolerance
• Static assets (JS, CSS): Cache for 1 year with content-hash filenames

CDN Cost Optimization

CDN costs are the largest operational expense for a video streaming platform, typically 50-70% of total infrastructure costs. Optimization strategies include:

• Efficient encoding (AV1): 30% bitrate reduction at the same quality directly translates to 30% CDN cost reduction
• Multi-CDN strategy: Use 2-3 CDN providers and route traffic based on real-time cost and performance metrics. A single CDN provider creates vendor lock-in and leaves you vulnerable to regional outages.
• Tiered CDN: Use premium CDN for high-traffic videos (fast cache hit, low latency) and origin-pull CDN for long-tail content (lower cost, higher latency acceptable)
• Predictive pre-positioning: Pre-push upcoming video content to edge servers before the upload goes live (e.g., a scheduled premiere). This eliminates cold-start misses.
• Regional bandwidth balancing: If one CDN region is expensive, shift traffic to a cheaper region (even if slightly higher latency) for non-interactive use cases.

CDN Performance Monitoring

C#
public class CDNHealthMonitor
{
    private readonly MetricsCollector _metrics;

    public async Task<CDNHealthReport> CheckCDNHealthAsync(
        string cdnProvider, string region)
    {
        // Measure cache hit ratio
        var hitRatio = await _metrics.GetCacheHitRatioAsync(
            cdnProvider, region, TimeSpan.FromMinutes(5));

        // Measure time-to-first-byte (TTFB)
        var ttfbP50 = await _metrics.GetTTFTPercAsync(
            cdnProvider, region, 50);
        var ttfbP99 = await _metrics.GetTTFTPercAsync(
            cdnProvider, region, 99);

        // Measure error rate
        var errorRate = await _metrics.GetErrorRateAsync(
            cdnProvider, region, TimeSpan.FromMinutes(5));

        return new CDNHealthReport
        {
            CacheHitRatio = hitRatio,
            TTFB_P50 = ttfbP50,
            TTFB_P99 = ttfbP99,
            ErrorRate = errorRate,
            Healthy = hitRatio > 0.95 &&
                      ttfbP99 < 200 &&
                      errorRate < 0.01,
            // Auto-route traffic away from unhealthy CDN
            ShouldFailover = hitRatio < 0.80 ||
                            ttfbP99 > 500 ||
                            errorRate > 0.05
        };
    }
}

11. Content Recommendation Engine

The recommendation engine is responsible for keeping users engaged by suggesting relevant content. It is arguably the most important system for user retention — Netflix estimates that 80% of content watched on their platform comes from recommendations. The recommendation system must balance relevance (show users content they want), diversity (don't create filter bubbles), freshness (promote new content), and business objectives (promote premium content and creators).

Recommendation Pipeline Architecture

Candidate Generation

Candidate generation is the first stage of the recommendation pipeline, where millions of videos are filtered down to a few hundred potential candidates. This stage prioritizes recall (not missing good content) over precision (showing only the best content). Multiple candidate sources run in parallel:

• Collaborative Filtering: "Users who watched Video A also watched Videos B, C, D." Uses matrix factorization or neural collaborative filtering on the user-item interaction matrix. This is the strongest signal for content relevance but suffers from the cold-start problem (new videos and new users have no interaction history).
• Content-Based Filtering: "This video has similar tags, description, and visual features to videos you've watched." Uses TF-IDF on video metadata, CNN features from video frames, and audio fingerprinting. Works well for new videos without interaction history.
• Deep Personalization: A two-tower neural network that learns user preferences and video attributes in a shared embedding space. The user tower processes watch history, subscriptions, and demographic features. The video tower processes metadata, visual features, and engagement statistics. Cosine similarity between the two embeddings provides a personalized relevance score.
• Trending/Popular: Provides a baseline of popular content to ensure new users and users with sparse history still get quality recommendations. Useful for discovery but must be blended carefully to avoid a popularity bias.
• Subscription Feed: Recent uploads from channels the user has subscribed to. This is the highest-signal source for engaged users but provides limited discovery value.

Ranking Model (C# Feature Engineering)

C#
public class RecommendationRanker
{
    private readonly IFeatureStore _featureStore;
    private readonly IMLInferenceService _mlService;

    public async Task<List<RankedVideo>> RankCandidatesAsync(
        UserProfile user,
        List<VideoCandidate> candidates,
        RecommendationContext context)
    {
        var rankedResults = new List<RankedVideo>();

        foreach (var candidate in candidates)
        {
            var features = await BuildFeatureVectorAsync(
                user, candidate, context);
            var prediction = await _mlService.PredictAsync(
                "recommendation_v3", features);

            rankedResults.Add(new RankedVideo
            {
                Video = candidate.Video,
                Score = prediction.EngagementScore,
                // Blend multiple objectives
                FinalScore = BlendScores(
                    prediction.ClickProbability,
                    prediction.WatchTimePrediction,
                    prediction.CompletionProbability,
                    prediction.SatisfactionScore),
                Explanation = prediction.TopFeatures
            });
        }

        // Apply diversity re-ranking (MMR algorithm)
        var diverseResults = ApplyDiversity(rankedResults,
            lambda: 0.7f); // 70% relevance, 30% diversity

        return diverseResults.Take(50).ToList();
    }

    private async Task<Dictionary<string, float>>
        BuildFeatureVectorAsync(
            UserProfile user,
            VideoCandidate candidate,
            RecommendationContext context)
    {
        var features = new Dictionary<string, float>();

        // User features
        features["user_watch_history_size"] =
            user.WatchHistory.Count;
        features["user_avg_watch_time"] =
            user.AverageWatchTimeMinutes;
        features["user_subscription_count"] =
            user.Subscriptions.Count;
        features["user_device_type"] =
            EncodeDevice(context.DeviceType);
        features["user_time_of_day"] =
            NormalizeHour(context.RequestHour);
        features["user_day_of_week"] =
            context.RequestDayOfWeek;

        // Video features
        features["video_view_count"] =
            candidate.Video.ViewCount;
        features["video_like_ratio"] =
            candidate.Video.LikeRatio;
        features["video_duration_seconds"] =
            candidate.Video.DurationSeconds;
        features["video_age_hours"] =
            candidate.Video.Age.TotalHours;
        features["video_channel_subs"] =
            candidate.Video.Channel.SubscriberCount;
        features["video_ctr"] =
            candidate.Video.HistoricalCTR;
        features["video_avg_watch_pct"] =
            candidate.Video.AverageCompletionPct;

        // Cross features
        features["user_channel_subscribed"] =
            user.Subscriptions.Contains(candidate.Video.ChannelId)
                ? 1.0f : 0.0f;
        features["user_watched_similar"] =
            await _featureStore.HasSimilarWatchAsync(
                user.UserId, candidate.Video.Tags)
                ? 1.0f : 0.0f;

        // Context features
        features["is_weekend"] =
            context.IsWeekend ? 1.0f : 0.0f;
        features["is_prime_time"] =
            context.IsPrimeTime ? 1.0f : 0.0f;

        return features;
    }

    private float BlendScores(
        float clickProb, float watchTimePred,
        float completionProb, float satisfactionScore)
    {
        // Weighted blend optimizing for watch time
        // (YouTube's primary metric)
        return 0.25f * clickProb +
               0.40f * watchTimePred +
               0.20f * completionProb +
               0.15f * satisfactionScore;
    }
}

Real-Time Feature Serving

The recommendation model needs up-to-date features to make accurate predictions. User behavior changes in real-time — a user who just watched a cooking video is more likely to watch another cooking video, even if their long-term profile suggests tech content. The feature store (backed by Redis) provides sub-millisecond feature access for both user features (recent watch history, session context) and video features (recent engagement metrics, trending scores).

The feature pipeline operates on two timescales: batch features (computed nightly via Spark) capture long-term preferences like favorite categories and average watch time; streaming features (computed via Flink/Kafka) capture real-time signals like current session behavior and video trending velocity. The ranking model uses both types to make predictions that reflect both long-term preferences and immediate context.

12. Video Metadata & Search

Video search is more complex than web search because videos contain multiple modalities of information: text (title, description, tags), visual content (what appears in the video), audio content (speech, music), and engagement signals (views, likes, watch time). Effective video search must combine all these signals to provide relevant results.

Search Index Schema (Elasticsearch)

JSON
{
  "mappings": {
    "properties": {
      "video_id": { "type": "keyword" },
      "title": {
        "type": "text",
        "analyzer": "standard",
        "fields": {
          "autocomplete": {
            "type": "text",
            "analyzer": "edge_ngram",
            "search_analyzer": "standard"
          },
          "exact": { "type": "keyword" }
        }
      },
      "description": { "type": "text", "analyzer": "standard" },
      "tags": { "type": "keyword" },
      "category": { "type": "keyword" },
      "channel_name": {
        "type": "text",
        "fields": { "exact": { "type": "keyword" } }
      },
      "language": { "type": "keyword" },
      "duration_seconds": { "type": "integer_range" },
      "view_count": { "type": "long" },
      "like_count": { "type": "long" },
      "publish_date": { "type": "date" },
      "engagement_score": { "type": "float" },
      "transcript_text": {
        "type": "text",
        "analyzer": "standard"
      },
      "visual_embeddings": {
        "type": "dense_vector",
        "dims": 128,
        "index": true,
        "similarity": "cosine"
      }
    }
  }
}

Search Relevance Signals

Search Autocomplete

Search autocomplete (typeahead) is a critical UX feature that helps users find content faster. As the user types, the system suggests completions based on popular queries and the user's personal history. The autocomplete service is backed by a Trie data structure cached in Redis, with prefix matching that returns suggestions within 20ms.

C#
public class VideoSearchService
{
    private readonly IElasticsearchClient _es;
    private readonly ICacheService _cache;
    private readonly IRankingModel _rankingModel;

    public async Task<SearchResults> SearchAsync(
        string query, UserProfile user,
        SearchFilters filters, int page = 1)
    {
        var cacheKey = $"search:{ComputeHash(query, filters)}";
        var cached = await _cache.GetAsync<SearchResults>(
            cacheKey);
        if (cached != null && page == 1)
            return await PersonalizeResults(cached, user);

        var searchRequest = new SearchRequest
        {
            Query = new BoolQuery
            {
                Should = new List<Query>
                {
                    new MatchQuery("title", query)
                        { Boost = 3.0f },
                    new MatchQuery("description", query)
                        { Boost = 1.5f },
                    new MatchQuery("tags", query)
                        { Boost = 2.0f },
                    new MatchQuery("transcript_text", query)
                        { Boost = 1.0f },
                    new MatchPhraseQuery("title.autocomplete",
                        query) { Boost = 5.0f }
                },
                MinimumShouldMatch = 1
            },
            // Apply filters
            Filter = BuildFilters(filters),
            // Sort by relevance score * freshness decay
            Sort = new List<SortClause>
            {
                new ScoreSort { Order = SortOrder.Descending },
                new ScriptSort
                {
                    Script = "doc['publish_date'].value.millis",
                    Order = SortOrder.Descending
                }
            },
            Size = 20,
            From = (page - 1) * 20,
            Highlight = new HighlightConfig
            {
                Fields = new Dictionary<string, HighlightField>
                {
                    ["title"] = new() { NumberOfFragments = 0 },
                    ["description"] = new()
                        { NumberOfFragments = 3 }
                }
            }
        };

        var results = await _es.SearchAsync<VideoDocument>(
            searchRequest);

        var searchResults = new SearchResults
        {
            TotalHits = results.Total,
            Videos = results.Hits.Select(h => new SearchResult
            {
                Video = MapToVideo(h.Source),
                RelevanceScore = h.Score,
                Highlights = h.Highlight
            }).ToList()
        };

        // Cache hot queries
        if (results.Total > 1000)
            await _cache.SetAsync(cacheKey, searchResults,
                TimeSpan.FromMinutes(5));

        return await PersonalizeResults(searchResults, user);
    }
}

13. Watch History & Progress Tracking

Watch history tracking serves two purposes: enabling resume playback across devices (the user experience feature) and feeding the recommendation engine with engagement data (the ML feature). The system must handle extremely high write throughput (billions of watch events per day) while providing fast reads for the "Continue Watching" section on the home page.

Watch Event Processing

Every video playback session generates a continuous stream of watch events. The client sends heartbeats every 10-30 seconds reporting the current playback position, buffer level, and any quality switches. These events flow through Kafka into the watch history service, which maintains both a real-time view (current session position) and a durable view (historical watch data).

C#
public class WatchHistoryService
{
    private readonly ICassandraClient _cassandra;
    private readonly IRedisCache _redis;
    private readonly IKafkaProducer _eventProducer;

    public async Task RecordWatchEventAsync(WatchEvent evt)
    {
        // Update real-time position in Redis (fast reads)
        var redisKey = $"watch:{evt.UserId}:{evt.VideoId}";
        await _redis.HashSetAsync(redisKey, new Dictionary<string, string>
        {
            ["position"] = evt.CurrentPositionSeconds.ToString(),
            ["progress"] = evt.ProgressPercentage.ToString("F2"),
            ["duration"] = evt.DurationSeconds.ToString(),
            ["quality"] = evt.CurrentQuality,
            ["updated"] = DateTime.UtcNow.Ticks.ToString(),
            ["completed"] = evt.IsCompleted.ToString()
        });
        await _redis.KeyExpireAsync(redisKey,
            TimeSpan.FromDays(30));

        // Persist to Cassandra (durable history)
        await _cassandra.ExecuteAsync(
            @"INSERT INTO watch_history
              (user_id, video_id, watched_at, watch_duration,
               progress_pct, last_position, completed,
               device_type)
              VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
            evt.UserId, evt.VideoId, DateTime.UtcNow,
            evt.WatchDurationSeconds, evt.ProgressPercentage,
            evt.CurrentPositionSeconds, evt.IsCompleted,
            evt.DeviceType);

        // Publish to Kafka for analytics pipeline
        await _eventProducer.ProduceAsync("watch-events",
            new WatchAnalyticsEvent
            {
                UserId = evt.UserId,
                VideoId = evt.VideoId,
                WatchDuration = evt.WatchDurationSeconds,
                ProgressPct = evt.ProgressPercentage,
                Quality = evt.CurrentQuality,
                BufferStalls = evt.BufferStallCount,
                DeviceType = evt.DeviceType,
                Timestamp = DateTime.UtcNow
            });
    }

    public async Task<List<ContinueWatchingItem>>
        GetContinueWatchingAsync(Guid userId, int limit = 20)
    {
        // Get recently watched videos with saved positions
        var keys = await _redis.KeysAsync(
            $"watch:{userId}:*");

        var items = new List<ContinueWatchingItem>();
        foreach (var key in keys.Take(limit))
        {
            var data = await _redis.HashGetAllAsync(key);
            var progress = float.Parse(
                data["progress"].ToString());

            // Skip completed videos (> 90% watched)
            if (progress > 0.9f) continue;

            var videoId = Guid.Parse(
                key.ToString().Split(':')[2]);
            var video = await GetVideoMetadataAsync(videoId);

            items.Add(new ContinueWatchingItem
            {
                Video = video,
                PositionSeconds = int.Parse(
                    data["position"].ToString()),
                ProgressPercentage = progress,
                LastWatched = new DateTime(long.Parse(
                    data["updated"].ToString()))
            });
        }

        return items.OrderByDescending(i => i.LastWatched)
            .Take(limit).ToList();
    }
}

Progress Tracking Architecture

14. Comments, Likes & Social Features

Social features transform a video platform from a passive viewing experience into an interactive community. Comments, likes, subscriptions, and sharing are engagement signals that keep users on the platform and help the recommendation engine understand content quality.

Comments System Architecture

Comments must support nested replies (threaded discussions), real-time updates (new comments appear without refresh), moderation (flagging and removal), and scaling (popular videos can accumulate millions of comments). The system uses a hybrid approach: MongoDB for flexible document storage of comment threads, Redis for real-time comment feeds, and Kafka for moderation pipeline.

C#
public class CommentService
{
    private readonly IMongoCollection<Comment> _comments;
    private readonly IRedisCache _cache;
    private readonly IModerationService _moderation;

    public async Task<Comment> AddCommentAsync(
        Guid userId, Guid videoId, string text,
        Guid? parentCommentId = null)
    {
        // Validate content through moderation
        var moderationResult = await _moderation
            .CheckContentAsync(text, userId);
        if (moderationResult.RequiresReview)
        {
            // Publish for human review but show optimistically
        }

        var comment = new Comment
        {
            CommentId = Guid.NewGuid(),
            VideoId = videoId,
            UserId = userId,
            Text = text,
            ParentCommentId = parentCommentId,
            LikeCount = 0,
            ReplyCount = 0,
            Status = moderationResult.AutoApproved
                ? CommentStatus.Approved
                : CommentStatus.PendingReview,
            CreatedAt = DateTime.UtcNow
        };

        await _comments.InsertOneAsync(comment);

        // Update video comment count
        await UpdateCommentCount(videoId, 1);

        // Invalidate comment cache for this video
        await _cache.RemoveAsync(
            $"comments:{videoId}:top");
        await _cache.RemoveAsync(
            $"comments:{videoId}:recent");

        // Publish event for real-time notification
        if (parentCommentId.HasValue)
        {
            var parent = await _comments
                .Find(c => c.CommentId == parentCommentId)
                .FirstOrDefaultAsync();
            if (parent.UserId != userId)
            {
                await NotifyUserAsync(parent.UserId,
                    new ReplyNotification
                {
                    VideoId = videoId,
                    CommentId = comment.CommentId,
                    ReplyPreview = text.Substring(0,
                        Math.Min(100, text.Length))
                });
            }
        }

        return comment;
    }

    public async Task<CommentPage> GetCommentsAsync(
        Guid videoId, CommentSort sort, string? cursor,
        int pageSize = 20)
    {
        var cacheKey = $"comments:{videoId}:{sort}";
        var cached = await _cache.GetAsync<CommentPage>(
            cacheKey);
        if (cached != null && cursor == null)
            return cached;

        var filter = Builders<Comment>.Filter.And(
            Builders<Comment>.Filter.Eq(c => c.VideoId, videoId),
            Builders<Comment>.Filter.Eq(c => c.Status,
                CommentStatus.Approved));

        var sortDef = sort == CommentSort.Top
            ? Builders<Comment>.Sort.Descending(
                c => c.LikeCount)
            : Builders<Comment>.Sort.Descending(
                c => c.CreatedAt);

        var comments = await _comments.Find(filter)
            .Sort(sortDef)
            .Limit(pageSize + 1)
            .ToListAsync();

        var hasMore = comments.Count > pageSize;
        if (hasMore) comments.RemoveAt(comments.Count - 1);

        return new CommentPage
        {
            Comments = comments,
            HasMore = hasMore,
            NextCursor = hasMore
                ? comments.Last().CommentId.ToString()
                : null
        };
    }
}

Likes & Engagement Metrics

Likes are a high-frequency, low-consistency operation. At peak, a popular video may receive thousands of likes per second during a viral moment. The system uses an eventually consistent approach: likes are counted in Redis (fast increment/decrement) and periodically flushed to PostgreSQL for durable storage. The like count displayed to users may be slightly stale (1-5 seconds behind actual count), which is acceptable for the use case.

Subscription System

Subscriptions are stored as a directed graph: User A subscribes to Channel B. The subscription graph is stored in PostgreSQL with a composite index on (subscriber_id, channel_id) for fast lookup. The "subscription feed" is generated by querying the most recent videos from all subscribed channels, sorted by publish time. For users with many subscriptions (100+), the feed is pre-computed and cached in Redis, refreshed every 5 minutes via a background job.

15. Content Moderation & Copyright Detection

Content moderation at scale is a critical safety and legal requirement. A platform serving billions of videos must automatically detect and remove content that violates policies: hate speech, graphic violence, sexual content, misinformation, spam, and copyright infringement. The moderation system combines AI classification with human review to balance speed (catching violations quickly) and accuracy (avoiding false positives that harm creators).

Moderation Pipeline

AI Classification Categories

Copyright Detection (Content ID)

Content ID is YouTube's automated copyright detection system and one of the most sophisticated fingerprinting systems ever built. When a video is uploaded, it is compared against a database of copyrighted content submitted by content owners. The system uses three types of fingerprints: audio fingerprinting (perceptual hash of the audio waveform), visual fingerprinting (perceptual hash of key frames), and metadata matching (title, description, and tag matching against known copyrighted works).

The audio fingerprinting algorithm converts the audio waveform into a compact representation (approximately 10 KB per hour of audio) that is robust against quality degradation, speed changes, and minor audio modifications. The system compares the uploaded video's fingerprint against millions of reference fingerprints using approximate nearest-neighbor search in a high-dimensional embedding space. A match triggers an automatic copyright claim, which can result in the video being blocked, monetized by the content owner, or tracked for analytics — depending on the content owner's preference settings.

16. Live Streaming Architecture

Live streaming adds a fundamentally different challenge compared to on-demand video: there is no buffer between production and consumption. The encoder produces video frames that must be delivered to millions of viewers within 2-10 seconds. The system cannot retry a failed segment or wait for transcoding to complete — every frame is a one-shot opportunity.

The live streaming architecture uses RTMP for ingest from the creator's encoder to the platform's ingest servers, then converts to LL-HLS (Low-Latency HLS) or LL-DASH for CDN delivery. The key trade-off is latency vs. reliability: shorter segments (0.5-2s) reduce latency but increase the risk of buffering; longer segments (4-10s) are more reliable but add latency. Most platforms target 3-6 seconds of end-to-end latency for general live content, with sub-2-second latency for interactive use cases like live auctions or gaming.

Live vs. On-Demand Comparison

17. Monetization & Ad Serving

Monetization is the business engine that funds the entire platform. For ad-supported platforms, the ad serving system must match advertisers to viewers in real-time, deliver ads without buffering the video experience, and provide measurement and attribution for advertisers. For subscription platforms, the billing and entitlement system must handle recurring payments, free trials, and content gating.

Revenue Model Comparison

18. Reliability, Failure Modes & Disaster Recovery

Video streaming platforms must maintain high availability for playback while tolerating failures in non-critical subsystems. The guiding principle is graceful degradation: the core viewing experience must never be interrupted, even if search, recommendations, comments, or analytics are degraded.

Failure Mode Analysis

19. Cost Estimation & Infrastructure Sizing

Monthly Cost Breakdown (YouTube-Scale)

20. Interview Q&A Deep Dive

Q1: How would you handle a video going viral?

Answer: Viral videos stress every layer simultaneously. CDN handles this naturally — popular content has high cache hit ratios. The challenge is metadata: millions of view count updates overwhelm a single database. Buffer view counts in Redis (atomic INCRBY handles millions/sec) and flush to PostgreSQL in batches every 30 seconds. Detect viral velocity via Flink streaming analytics and update trending inputs within minutes.

Q2: How do you handle the cold-start problem for new videos?

Answer: Solutions: (1) Content-based features (tags, visual analysis, transcript) provide initial recommendations. (2) Subscriber notifications and subscription feeds. (3) Reserve 5-10% of recommendation slots for exploration. (4) Boost videos with positive initial engagement signals. Bootstrapping takes 1-6 hours.

Q3: How do you prevent content piracy?

Answer: Raise the cost of piracy: DRM encryption (Widevine, FairPlay), forensic watermarking to trace leaks, rate limiting to prevent scraping, session-based auth to prevent link sharing, Content ID to detect uploaded copies. Make legal streaming more convenient than piracy.

Q4: How would you handle 10 million concurrent live viewers?

Answer: 10M viewers x 3 Mbps = 30 Tbps. Multi-region CDN with all major providers, pre-position manifests before event, use longer segments for better cache hit ratios, live-to-VOD conversion for post-event caching. Degrade quality for new joiners rather than buffer for existing viewers.

Key Numbers to Remember

21. Adaptive Bitrate Streaming and Quality Adaptation

Adaptive bitrate streaming (ABR) is the cornerstone of modern video delivery. The client dynamically switches between quality levels based on network conditions, buffer health, and device capabilities. A well-tuned ABR algorithm minimizes buffering events while maximizing average video quality, directly impacting user engagement metrics like completion rate and session duration.

public class AdaptiveBitrateController
{
    private readonly INetworkMonitor _networkMonitor;
    private readonly IBufferManager _bufferManager;

    public QualityLevel SelectQuality(StreamSession session)
    {
        var bandwidth = _networkMonitor.GetCurrentBandwidth(session.UserId);
        var bufferLevel = _bufferManager.GetBufferLevel(session.SessionId);
        var avgBandwidth = _networkMonitor.GetAverageBandwidth(
            session.UserId, windowMinutes: 30);

        // Throughput-based: select highest quality below bandwidth
        var throughputQuality = GetMaxQualityForBandwidth(
            bandwidth * 0.8); // 20% safety margin

        // Buffer-based: conservative when buffer is low
        var bufferQuality = bufferLevel switch
        {
            < 10  => QualityLevel.Low360p,
            < 20  => QualityLevel.Medium480p,
            < 30  => QualityLevel.High720p,
            _     => QualityLevel.Ultra1080p
        };

        // Hybrid approach: take the minimum of both strategies
        var selected = (QualityLevel)Math.Min(
            (int)throughputQuality,
            (int)bufferQuality);

        // Never increase by more than one level at a time (smooth transitions)
        if ((int)selected > (int)session.CurrentQuality + 1)
        {
            selected = session.CurrentQuality + 1;
        }

        session.CurrentQuality = selected;
        return selected;
    }

    private QualityLevel GetMaxQualityForBandwidth(double bps)
    {
        return bps switch
        {
            > 5_000_000  => QualityLevel.Ultra1080p,
            > 2_500_000  => QualityLevel.High720p,
            > 1_000_000  => QualityLevel.Medium480p,
            > 500_000    => QualityLevel.Low360p,
            _            => QualityLevel.AudioOnly
        };
    }
}

Quality Level Specifications

Content Moderation and Upload Safety Pipeline

User-generated video content must be scanned for policy violations before becoming publicly viewable. The moderation pipeline combines automated ML classification (for nudity, violence, copyright via Content ID, and spam) with human review queues for borderline cases. Processing must balance thoroughness with time-to-publish — creators expect uploads to be live within minutes.

public class ContentModerationPipeline
{
    private readonly IVideoAnalyzer _analyzer;
    private readonly IAudioTranscriber _transcriber;
    private readonly IHumanReviewQueue _reviewQueue;

    public async Task<ModerationResult> ModerateAsync(
        VideoUpload upload)
    {
        // Parallel automated analysis
        var visualTask = _analyzer.AnalyzeVisualAsync(
            upload.VideoUrl);
        var audioTask = _transcriber.TranscribeAsync(
            upload.AudioUrl);
        var metadataTask = _analyzer
            .AnalyzeMetadataAsync(upload.Metadata);

        await Task.WhenAll(visualTask, audioTask, metadataTask);

        var visual = await visualTask;
        var audio = await audioTask;
        var metadata = await metadataTask;

        // Combine scores from all classifiers
        var combinedScore = new ModerationScore
        {
            Nudity = Math.Max(visual.NudityScore,
                audio.NudityKeywords ? 0.8 : 0),
            Violence = visual.ViolenceScore,
            CopyrightRisk = visual.ContentIdMatchScore,
            Spam = metadata.SpamScore
        };

        if (combinedScore.MaxCategory > 0.9)
            return ModerationResult.Rejected(
                combinedScore.WorstCategory);
        if (combinedScore.MaxCategory > 0.6)
            return await _reviewQueue
                .SubmitForHumanReviewAsync(upload, combinedScore);

        return ModerationResult.Approved();
    }
}

Moderation SLA Targets