How to Design TikTok Short-Form Video Platform — A Senior+ Guide
A complete system design deep-dive into building a TikTok-scale short-form video platform from scratch
1. Introduction
TikTok has fundamentally transformed how the world consumes and creates media content. With over 1.5 billion monthly active users globally, TikTok stands as one of the most dominant social media platforms ever built. The platform core experience revolves around short-form vertical videos ranging from 15 to 60 seconds, served through an extraordinarily intelligent recommendation engine known as the For You Page. Unlike traditional social networks where your feed is driven by people you follow, the For You Page algorithm surfaces content based on predicted user interest, creating an addictive and deeply personalized experience that keeps users engaged for an average of 95 minutes per day.
Building a TikTok-scale platform presents extraordinary engineering challenges that span every layer of the technology stack. The system must ingest millions of video uploads daily, transcode them in near real-time to multiple formats and resolutions, understand their content using computer vision and natural language processing, rank them using a sophisticated machine learning pipeline, and serve billions of video views per day with sub-second latency. All of this must happen while maintaining content safety across dozens of languages, supporting creator monetization, and operating across dozens of countries with varying regulatory requirements.
In this comprehensive system design guide, we will break down every major component of a TikTok-like platform in exhaustive detail. We will cover the video upload pipeline with chunked resumable uploads, the transcoding infrastructure optimized for sub-5-second processing, the recommendation algorithm with its progressive promotion system, the content moderation pipeline with both automated and human review, the sound and effects engine, the caching strategy across multiple tiers, the database architecture using polyglot persistence, and the multi-region deployment model with data sovereignty compliance.
This guide is written for senior engineers and engineering managers preparing for system design interviews or planning real-world video platform architectures. The scope of this design covers the most critical and complex subsystems that make TikTok work at global scale. We will estimate capacity requirements in concrete numbers, design data models with proper relationships and indexing, sketch high-level architectures using Mermaid diagrams, implement key services in C#, and address the trade-offs that arise at every decision point.
What makes TikTok particularly interesting from a system design perspective is the tight coupling between content understanding, recommendation quality, and user retention. A mediocre algorithm leads to irrelevant content, which causes users to leave. A great algorithm requires real-time signals, diverse content pools, and sophisticated ranking models. The infrastructure supporting this must be both fast and cost-effective, which introduces fascinating engineering trade-offs across storage, compute, and networking. By the end of this guide, you will have a thorough understanding of how to design a short-form video platform capable of serving billions of users at global scale.
2. Requirements Gathering
Before designing any system, we must clearly define the functional and non-functional requirements. For a TikTok-like platform, the requirements span content creation, content consumption, social interaction, and platform operations.
Functional Requirements
Video Upload and Creation: Users must be able to record or upload short-form videos (15-60 seconds) directly from their mobile devices. The upload process should support resumable chunked uploads to handle unreliable network conditions. Videos should be uploadable with captions, hashtags, sounds, and location tags. The system must support parallel chunk uploads for maximum throughput on fast connections and sequential uploads on cellular to conserve bandwidth.
Video Feed (For You Page): Users must receive a personalized, infinite-scroll feed of short-form video content. The feed should be algorithmically curated based on user interests, engagement history, and trending content. New users should receive a diverse set of popular videos to bootstrap the cold-start problem. The feed must load in under 200 milliseconds and the first video must begin playing within 500 milliseconds of the user swiping.
Following Feed: Users should have a separate feed showing content exclusively from accounts they follow, sorted by recency. This feed provides a traditional social media experience alongside the algorithmically curated For You Page.
Social Interactions: Users must be able to like, comment, share, and save videos. Users can follow other creators, send direct messages, and create Duet or Stitch responses to existing videos. All interaction counts must be displayed as approximate numbers with eventual consistency to ensure system performance at scale.
Sound and Music: Users must be able to browse a library of licensed music and original sounds, use them in their videos, and discover trending sounds. When other users reuse a sound, attribution must link back to the original creator or rights holder.
Effects and Filters: The platform must offer a library of augmented reality effects, face filters, green screen effects, speed controls, and text overlays that can be applied during recording or post-production. Effects must run in real-time at 30 FPS on mid-range devices.
Content Moderation: All uploaded content must pass through automated moderation (computer vision, audio analysis, text analysis) and human review queues before being distributed widely. The moderation pipeline must process millions of videos daily with minimal false positive and false negative rates.
Non-Functional Requirements
| Requirement | Target | Rationale |
|---|---|---|
| Availability | 99.99% | Consumer social platform with global users across all time zones |
| Feed Load Latency | < 200ms p99 | Smooth scrolling experience without perceptible delay |
| Video Start Latency | < 500ms p95 | Immediate playback on swipe with pre-buffered segments |
| Upload Success Rate | > 99.5% | Resumable chunked uploads handle network failures gracefully |
| Feed Throughput | 10M+ read requests/sec | Billion-user scale with multiple sessions per user daily |
| Storage Durability | 99.999999999% (11 nines) | Creator content must never be lost under any circumstances |
| Consistency Model | Eventual for feeds, Strong for payments | Feed data can be slightly stale; financial transactions require strict consistency |
| Transcoding Latency | < 5 seconds for 30s video | Creators expect near-instant publishing after upload |
Scope Boundaries
This design focuses on the core video feed, upload pipeline, recommendation algorithm, and content moderation. We will touch on live streaming and monetization but will not deep-dive into payment processing internals, ad auction systems, or detailed creator analytics dashboards, as those are large standalone systems that warrant separate design documents.
3. Capacity Estimation
Capacity estimation grounds our design in reality by translating abstract requirements into concrete infrastructure needs. Let us estimate the key numbers assuming a platform with 1.5 billion monthly active users and a standard social media engagement profile.
Read and Write QPS
Daily Active Users: 1.5 billion MAU with roughly 80% DAU gives us 1.2 billion daily active users. This is our baseline for all per-user calculations.
Feed Requests: If each user opens the app 8 times per day and scrolls through 100 videos per session, that produces 120 billion video views per day. Converting to QPS: 120 billion divided by 86,400 seconds per day gives approximately 1.4 million video view requests per second. This is the dominant read path in the system and drives most of our infrastructure sizing.
Video Uploads: If 5% of daily active users upload one video per day, that is 60 million uploads per day, or roughly 700 uploads per second. Each upload involves multiple write operations: the chunked file upload to object storage, metadata writes to the database, and a transcoding job enqueue. The effective write QPS per upload is approximately 3-5x the raw upload count.
Social Actions: Likes, comments, shares, and follows generate approximately 10 actions per user per day, totaling 12 billion actions per day or about 140,000 writes per second. Comments are the most expensive action because they involve both a write to the comments table and an update to the video comment count.
Storage Estimation
| Component | Calculation | Daily Result |
|---|---|---|
| Raw video uploads | 60M videos x 30MB average | 1.8 PB/day |
| Transcoded variants (4x) | 1.8 PB x 4 resolution variants | 7.2 PB/day |
| Thumbnail images | 60M x 5 thumbnails x 200KB | 60 TB/day |
| Audio extractions | 60M x 2MB audio-only | 120 TB/day |
| Metadata records | 60M x 5KB per record | 300 GB/day |
| Interaction events | 12B events x 200 bytes | 2.4 TB/day |
| Annual storage growth | Approximately 8 PB/day raw x 365 | ~2.9 EB/year |
These numbers are staggering and explain why TikTok is one of the largest consumers of storage infrastructure on the planet. A TikTok-scale platform requires exabyte-scale object storage and a tiered storage strategy that moves older, less-accessed content to colder, cheaper storage tiers automatically.
Bandwidth Estimation
Ingress (Uploads): 60 million uploads at 30 MB average equals 1.8 PB per day ingress, or roughly 170 Gbps average. Peak upload times (evening hours in major markets) can see 5-10x this throughput, requiring 1-2 Tbps of ingress capacity.
Egress (Playback): 120 billion views at 3 MB per video (transcoded, adaptive bitrate) equals 360 PB per day egress. This translates to roughly 33 Tbps average bandwidth. Peak usage during evening hours in major markets can see 2-3x the average, requiring 60-100 Tbps of egress capacity. This is one of the largest egress footprints of any internet service, surpassing even Netflix in raw data transfer volume.
Compute Estimation
Transcoding: 60 million videos per day with an average processing time of 30 seconds per video on a dedicated transcoding core requires roughly 20,000 CPU cores running continuously for transcoding alone. GPU-accelerated transcoding using NVIDIA NVENC can reduce this to roughly 5,000 GPU workers, which is more cost-effective but requires careful capacity management.
ML Inference: Content understanding (object detection, text extraction, audio analysis) and feed ranking models require substantial GPU clusters. Each feed request involves a candidate generation model and a ranking model inference, consuming roughly 10-50ms of GPU time per request. At 1.4 million requests per second, this requires approximately 14,000-70,000 GPU-seconds of inference capacity per second, or roughly 14,000-70,000 GPUs dedicated to inference.
4. Data Model
The data model for a short-form video platform is rich and deeply interconnected. We have users, videos, sounds, hashtags, comments, likes, follows, and various interaction events. Let us design the core entities, their attributes, and the relationships between them.
Core Entities
User: Each user has a unique identifier, username, display name, profile picture URL, bio text, follower count, following count, total likes received, verification status, preferred languages, geographic region, and account creation timestamp. Users also have privacy settings controlling who can view their content, who can comment, and whether their liked videos are publicly visible. The user entity also includes an embedded interests object that stores category affinities and content preferences computed by the recommendation system.
Video: Each video has a unique identifier, the creator user ID, a caption string, associated hashtags, a sound reference, duration in milliseconds, visibility status (public, friends-only, private), view count, like count, comment count, share count, completion count (how many users watched to the end), moderation status, content metadata (extracted by the content understanding pipeline), and creation timestamp. The video entity also references its transcoded media files across multiple quality tiers and its thumbnail images.
Sound: Sounds can be original audio uploaded by a user or licensed music from a music library. Each sound has a unique identifier, title, artist name, duration, a reference to the audio file, the original creator (if user-generated), usage count, trending score, and license information. The sound entity supports an attribution chain that traces derivation history for original sounds.
Hashtag: Each hashtag has a unique name (lowercase, normalized), creation date, total usage count, an optional description, and a trending score. Hashtags link to trending topics and challenges, enabling content discovery beyond the recommendation algorithm.
Comment: Comments are threaded, with each comment referencing a video, the commenter user ID, an optional parent comment ID (for replies), text content, like count, and creation timestamp. The comment entity supports nested replies up to three levels deep.
Entity Relationship Diagram
Video Interaction Events
Beyond simple likes and comments, TikTok tracks granular interaction signals for the recommendation algorithm. These events are typically stored in an event stream (Kafka) and processed into feature stores for ML model training and real-time inference.
| Event Type | Key Fields | Weight for Algorithm |
|---|---|---|
| video_view | user_id, video_id, watch_duration_ms, is_replay | Medium |
| video_complete | user_id, video_id, replay_count | High |
| like | user_id, video_id, timestamp | High |
| comment | user_id, video_id, comment_text, sentiment | High |
| share | user_id, video_id, share_platform | Very High |
| follow_from_video | user_id, creator_id, video_id | Very High |
| sound_use | user_id, sound_id, video_id | Medium |
| profile_visit | user_id, creator_id, visit_duration | Medium |
| long_press | user_id, video_id | Low |
| not_interested | user_id, video_id, reason | Negative (strong) |
| video_save | user_id, video_id | Very High |
| search_click | user_id, video_id, search_query | High |
5. API Design
The API layer for a TikTok-like platform must be designed for high throughput, low latency, and mobile-first consumption patterns. We design RESTful APIs with cursor-based pagination, optional GraphQL for complex data fetching needs, and gRPC for internal service-to-service communication.
Core API Endpoints
HTTP
# Feed Endpoints
GET /api/v1/feed/for-you?page_size=20&cursor={cursor}
GET /api/v1/feed/following?page_size=20&cursor={cursor}
# Video CRUD Operations
POST /api/v1/videos/upload-init
POST /api/v1/videos/upload-chunk
POST /api/v1/videos/upload-complete
GET /api/v1/videos/{video_id}
DELETE /api/v1/videos/{video_id}
PUT /api/v1/videos/{video_id}
# Video Interactions
POST /api/v1/videos/{video_id}/like
DELETE /api/v1/videos/{video_id}/like
GET /api/v1/videos/{video_id}/comments?cursor={cursor}
POST /api/v1/videos/{video_id}/comments
POST /api/v1/videos/{video_id}/share
POST /api/v1/videos/{video_id}/report
# User Profile and Social
GET /api/v1/users/{user_id}/profile
GET /api/v1/users/{user_id}/videos?page_size=20&cursor={cursor}
POST /api/v1/users/{user_id}/follow
DELETE /api/v1/users/{user_id}/follow
GET /api/v1/users/me/followers?cursor={cursor}
GET /api/v1/users/me/following?cursor={cursor}
# Sound Operations
GET /api/v1/sounds/trending?page_size=20
GET /api/v1/sounds/{sound_id}
GET /api/v1/sounds/{sound_id}/videos?page_size=20
# Search and Discovery
GET /api/v1/search?q={query}&type=video|user|sound|hashtag
GET /api/v1/hashtags/trending
GET /api/v1/hashtags/{name}/videos?page_size=20
# Notifications
GET /api/v1/notifications?page_size=20&cursor={cursor}
PUT /api/v1/notifications/read-all
PUT /api/v1/notifications/settings
Feed API Response Schema
JSON
{
"status": "success",
"data": {
"items": [
{
"video_id": "v_8x7k2m9p",
"author": {
"user_id": "u_3n5j8k2l",
"username": "creator_name",
"display_name": "Creator Display",
"avatar_url": "https://cdn.example.com/avatars/u_3n5j8k2l.webp",
"is_verified": true
},
"caption": "Check out this amazing view! #travel #nature",
"sound": {
"sound_id": "s_1a2b3c4d",
"title": "Original Sound",
"artist": "creator_name",
"is_original": true
},
"video_urls": {
"play_url": "https://cdn.example.com/videos/v_8x7k2m9p/master.m3u8",
"cover_url": "https://cdn.example.com/covers/v_8x7k2m9p.webp",
"thumbnail_urls": [
"https://cdn.example.com/thumbs/v_8x7k2m9p_1.webp",
"https://cdn.example.com/thumbs/v_8x7k2m9p_2.webp"
]
},
"stats": {
"view_count": 1250000,
"like_count": 89000,
"comment_count": 3200,
"share_count": 15000
},
"duration_ms": 15000,
"hashtags": ["travel", "nature"],
"created_at": "2026-06-28T14:30:00Z",
"has_liked": false,
"has_bookmarked": false
}
],
"has_more": true,
"cursor": "eyJ2aWRlb19pZCI6InZfOHg3azJtOXAifQ=="
}
}
Upload API Design
Video uploads use a three-phase protocol: init, chunked upload, and complete. This design supports resumable uploads and handles unreliable mobile networks gracefully.
HTTP
# Phase 1: Initialize Upload
POST /api/v1/videos/upload-init
Body: {
"file_name": "my_video.mp4",
"file_size_bytes": 31457280,
"duration_ms": 15000,
"mime_type": "video/mp4",
"caption": "My amazing video!",
"sound_id": null,
"hashtags": ["fun", "viral"]
}
Response: {
"upload_id": "up_abc123",
"chunk_size_bytes": 5242880,
"total_chunks": 6,
"upload_urls": [
"https://upload.example.com/chunk?token=xyz1",
"https://upload.example.com/chunk?token=xyz2"
]
}
# Phase 2: Upload Chunks (parallel with retry)
PUT /api/v1/videos/upload-chunk
Headers: { "X-Upload-ID": "up_abc123", "X-Chunk-Index": "0" }
Body: [binary chunk data]
# Phase 3: Complete Upload
POST /api/v1/videos/upload-complete
Body: { "upload_id": "up_abc123" }
Response: {
"video_id": "v_new123",
"status": "processing",
"estimated_processing_time_ms": 30000
}
6. High-Level Architecture
The high-level architecture of a TikTok-like platform consists of several major subsystems: the client layer, the API gateway, microservices, the video processing pipeline, the recommendation engine, the storage layer, and the CDN. Each subsystem is independently scalable and deployable.
Service Responsibilities
| Service | Responsibility | Key Technology |
|---|---|---|
| API Gateway | Authentication, rate limiting, request routing, response caching | Envoy or Kong |
| Feed Service | Assembles personalized feeds by calling candidate generation and ranking | Go or Rust |
| Upload Service | Manages chunked upload sessions, triggers processing pipeline | Go |
| Transcoding Pipeline | Converts uploaded video to multiple formats, resolutions, and codecs | FFmpeg + GPU workers |
| Content Analysis | Extracts metadata, detects policy violations, generates content embeddings | Python with ML models |
| Ranking Model | Scores candidate videos for each user based on predicted engagement | Python with TensorFlow or PyTorch |
| Feature Store | Serves pre-computed and real-time features for ML models | Redis + Spark |
| Moderation Service | Automated and human moderation pipeline for content safety | Python + workflow engine |
| Notification Service | Push notifications, in-app notifications, email digests | Go + FCM and APNs |
| Search Service | Full-text search, hashtag search, user search, video search | Elasticsearch |
Communication Patterns
Services communicate through a combination of synchronous REST or gRPC calls for user-facing requests and asynchronous Kafka events for background processing. The upload service triggers transcoding jobs through a message queue rather than synchronous calls, ensuring the upload API responds quickly even when the transcoding pipeline is under heavy load. The feed service uses gRPC for internal communication with the ranking model to minimize serialization overhead and latency.
7. Video Upload Pipeline
The video upload pipeline is one of the most critical paths in the system. It must be reliable, fast, and capable of handling massive throughput while ensuring content is safely stored and efficiently processed. A failure in the upload pipeline directly impacts creator experience and content supply.
Upload Flow Architecture
Chunked Upload Protocol
The upload client divides the video file into fixed-size chunks (typically 5MB each) and uploads them in parallel using pre-signed URLs. This approach provides several critical advantages for mobile-first upload scenarios:
- Resumability: If a chunk fails due to network interruption, only that chunk needs to be retried. The upload session tracks which chunks have been successfully received using a bitmap or set data structure, allowing the client to resume from exactly where it left off without re-uploading successful chunks.
- Parallelism: Multiple chunks can be uploaded simultaneously (typically 3-6 parallel streams on Wi-Fi), maximizing bandwidth utilization. On cellular networks, the client switches to sequential uploads to avoid network congestion and excessive battery drain.
- Direct-to-Storage: Pre-signed URLs allow the client to upload directly to object storage (S3 or GCS) without proxying through the application server. This eliminates the application server as a bottleneck for upload bandwidth and allows the upload service to focus on session management and coordination.
- Progress Tracking: The client can report upload progress accurately by counting acknowledged chunks. This enables a smooth progress bar experience for the user, with ETA calculations based on current upload speed.
- Verification: Each chunk upload returns an ETag that the client sends to the upload service on completion. The service uses ETags to verify chunk integrity and detect any corruption during transfer.
Upload Session Management
C#
public class UploadSession
{
public string UploadId { get; set; }
public long UserId { get; set; }
public string FileName { get; set; }
public long FileSizeBytes { get; set; }
public int TotalChunks { get; set; }
public int ChunkSizeBytes { get; set; } = 5 * 1024 * 1024; // 5MB
public HashSet<int> ReceivedChunks { get; set; } = new();
public Dictionary<int, string> ChunkETags { get; set; } = new();
public UploadStatus Status { get; set; }
public DateTime CreatedAt { get; set; }
public DateTime ExpiresAt { get; set; }
public VideoMetadata PendingMetadata { get; set; }
public double Progress => TotalChunks > 0
? (double)ReceivedChunks.Count / TotalChunks * 100
: 0;
public bool IsComplete => ReceivedChunks.Count == TotalChunks;
public bool IsExpired => DateTime.UtcNow > ExpiresAt;
public List<string> GeneratePreSignedUrls(string bucket)
{
var urls = new List<string>();
for (int i = 0; i < TotalChunks; i++)
{
if (!ReceivedChunks.Contains(i))
{
var key = $\"uploads/{UploadId}/chunk_{i:D4}\";
urls.Add(GeneratePresignedPutUrl(bucket, key, TimeSpan.FromHours(1)));
}
}
return urls;
}
public async Task MarkChunkReceived(int chunkIndex, string etag)
{
ReceivedChunks.Add(chunkIndex);
ChunkETags[chunkIndex] = etag;
if (IsComplete)
{
Status = UploadStatus.ReadyForProcessing;
await TriggerMergeAndProcessing();
}
}
private async Task TriggerMergeAndProcessing()
{
var mergedKey = $\"videos/{UploadId}/original.mp4\";
await _objectStorage.MergeChunks(
$\"uploads/{UploadId}/\",
TotalChunks,
mergedKey
);
await _transcodingQueue.Enqueue(new TranscodingJob
{
UploadId = UploadId,
SourceKey = mergedKey,
Metadata = PendingMetadata,
Priority = CalculatePriority(),
CreatedAt = DateTime.UtcNow
});
}
private JobPriority CalculatePriority()
{
// Verified creators and high-follower accounts get priority
if (PendingMetadata.CreatorIsVerified) return JobPriority.High;
if (PendingMetadata.CreatorFollowers > 1_000_000) return JobPriority.High;
return JobPriority.Normal;
}
}
Client-Side Upload Strategy
The mobile client implements an adaptive upload strategy based on network conditions. On Wi-Fi, it uploads all chunks in parallel (typically 6 concurrent streams) for maximum speed. On 4G or 5G cellular, it uploads sequentially (2 concurrent streams) to avoid consuming excessive bandwidth and to respect data caps. On 3G or slower connections, it defers the upload and queues it for when a faster connection becomes available. The client also performs client-side video compression before upload to reduce the upload size by 20-40% without perceptible quality loss, using hardware-accelerated encoding available on modern mobile devices.
8. Short-Form Transcoding
Transcoding is the process of converting uploaded video into multiple formats, resolutions, and bitrates suitable for playback across diverse devices and network conditions. For a short-form video platform, transcoding must be extremely fast (ideally completing in under 5 seconds) because users expect their content to be live immediately after upload. This is a fundamentally different requirement from traditional video platforms where transcoding can take minutes or hours.
Output Variants
| Variant | Resolution | Bitrate | Codec | Purpose |
|---|---|---|---|---|
| Ultra High | 1080x1920 | 6 Mbps | H.265 | High-end devices on Wi-Fi connections |
| High | 1080x1920 | 3.5 Mbps | H.264 | Standard high-quality mobile playback |
| Medium | 720x1280 | 2 Mbps | H.264 | Standard mobile playback on 4G |
| Low | 480x854 | 800 Kbps | H.264 | Weak cellular connections or data saver mode |
| Audio Only | N/A | 128 Kbps | AAC | Audio-only playback mode or background play |
Each variant is also packaged as an HLS (HTTP Live Streaming) manifest with adaptive bitrate switching. This allows the client player to dynamically switch between quality tiers based on real-time network conditions, ensuring smooth, buffer-free playback regardless of the user connection type.
Transcoding Pipeline Architecture
Fast Transcoding Strategy
Sub-5-second transcoding for 15-60 second videos requires aggressive optimization at every stage of the pipeline. The key strategies that make this possible include:
- GPU-Accelerated Encoding: NVIDIA NVENC and AMD VCE hardware encoders can transcode a 30-second video to multiple resolutions in under 3 seconds, compared to 30+ seconds on CPU alone. Each GPU can handle 8-16 concurrent encoding sessions, making this both fast and cost-effective.
- Parallel Encoding: Each resolution variant is encoded independently on separate GPU streams. A 30-second video encoded to 4 variants runs 4 parallel encoding tasks, completing in roughly the time of a single encode rather than 4x the time.
- Segmented Processing: The video is split into 2-second segments that are transcoded in parallel across a distributed worker fleet, then stitched together. This allows horizontal scaling of transcoding throughput by adding more workers.
- Hardware Decoding: The input video is hardware-decoded to avoid CPU bottlenecks on the decode side, allowing the full pipeline to run entirely on GPU resources without competing for CPU time.
- Pre-warmed Worker Pools: A pool of transcoding workers is always warm and ready to accept jobs, eliminating cold-start delays. Workers are pre-loaded with codec libraries and optimized memory pools.
- Input Optimization: The transcoding pipeline accepts only specific input formats (MP4, MOV) and rejects unsupported formats early. The input is validated and probed in milliseconds before being dispatched to workers.
HLS Packaging for Adaptive Bitrate
M3U8
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:BANDWIDTH=6000000,RESOLUTION=1080x1920,CODECS="hvc1.1.6.L150.90"
variant_1080p_high.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=3500000,RESOLUTION=1080x1920,CODECS="avc1.640028"
variant_1080p_medium.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2000000,RESOLUTION=720x1280,CODECS="avc1.64001f"
variant_720p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=800000,RESOLUTION=480x854,CODECS="avc1.64001e"
variant_480p.m3u8
The player client starts with the highest quality variant that the network can support and dynamically switches down if bandwidth decreases, or up if bandwidth increases. This provides the best possible viewing experience at any given moment. Each HLS segment is 2 seconds long, allowing the player to react quickly to changing network conditions. For short-form video, the entire video is typically buffered in advance because the file is small enough (under 5 MB even for the highest quality variant), so ABR switching primarily affects the initial quality selection.
9. Content Understanding
Content understanding is the system ability to automatically analyze and extract meaning from uploaded videos. This capability is critical for both the recommendation algorithm (which needs rich content metadata to match videos with interested users) and content moderation (which must detect policy violations before content goes viral). The content understanding pipeline processes every uploaded video and generates a rich set of metadata, embeddings, and safety scores.
Content Analysis Pipeline
Computer Vision Models
The visual analysis pipeline runs multiple neural networks on extracted video frames. We typically sample 1 frame per second from the video and process each frame through the following models in a GPU inference cluster:
- Object Detection (YOLO v8 or DETR): Identifies objects in each frame including people, animals, vehicles, products, and locations. The detected objects are aggregated across all frames to create a scene-level object inventory with confidence scores and bounding boxes.
- Scene Classification: A ResNet or EfficientNet model classifies each frame into one of hundreds of predefined scene categories such as beach, kitchen, gym, concert, office, or outdoor market. The dominant scene across all frames becomes the primary scene classification for the video.
- Face Detection and Recognition: RetinaFace or MTCNN detects faces in each frame, and a face embedding model (ArcFace) generates 512-dimensional face vectors for each detected face. These vectors enable face-based search, creator identification, and duplicate detection across the platform, with strict privacy controls and opt-out mechanisms.
- Optical Character Recognition (OCR): PaddleOCR or Tesseract extracts text visible in the video frames. This text is then processed through NLP models to understand embedded captions, signs, product labels, and text overlays within the video content.
- NSFW Detection: A specialized classification model flags potentially inappropriate content with a safety score from 0 (completely safe) to 1 (explicit). Content above a configurable threshold (typically 0.85) is routed to human moderation for review.
Audio Analysis
The audio track is processed separately from the video frames. Speech-to-text models based on OpenAI Whisper architecture transcribe any spoken content, enabling text-based search and understanding of video dialogue. This transcription supports over 100 languages with high accuracy. Music identification models based on audio fingerprinting technology (similar to Shazam algorithm) identify any licensed music used in the video, which is critical for copyright compliance and royalty tracking. Audio scene classification models detect ambient sounds, music genres, and emotional tone from the audio track alone.
Multi-Modal Embeddings
The most powerful aspect of content understanding is multi-modal embedding. Models like CLIP (Contrastive Language-Image Pre-training) learn a shared vector space for images and text. We extend this to video by encoding visual frames, audio features, and text (caption plus hashtags) into a unified embedding vector of 1024 dimensions. This vector enables several critical capabilities: semantic video search where users can search using natural language descriptions, content-based recommendations that find visually or semantically similar videos to ones the user has enjoyed, and duplicate and near-duplicate detection using cosine similarity between content vectors to identify reposts, copies, and derivative works across the platform.
10. For You Page Algorithm
The For You Page (FYP) algorithm is the heart of TikTok user experience. It is the primary reason users spend an average of 95 minutes per day on the platform, more than any other social media app. The FYP works by predicting which videos a specific user will find engaging from a pool of billions of videos, and it does this in real-time as the user scrolls through their feed.
Recommendation Architecture
Two-Stage Ranking Pipeline
The ranking pipeline uses a two-stage approach to balance computational cost with prediction quality. This is the standard architecture for large-scale recommendation systems.
Stage 1 - Candidate Generation (Retrieval): From the full corpus of billions of videos, we retrieve roughly 1,000 to 5,000 candidate videos for each user. This stage uses fast approximate nearest neighbor (ANN) search on content and user embedding vectors, combined with collaborative filtering signals. The candidate pool is drawn from multiple sources to ensure diversity: trending videos, videos similar to previously liked content, fresh uploads from emerging creators, and content from the user social graph. Pre-filtering removes videos the user has already seen, videos in different languages (unless the user is multilingual), content that fails safety checks, and videos from blocked creators. The entire candidate generation step must complete within 20 milliseconds.
Stage 2 - Fine Ranking: The top 1,000 to 5,000 candidates are scored by a deep neural network that predicts the probability of various engagement actions: watch completion, like, comment, share, and follow. This model takes into account hundreds of features including user history, video metadata, creator signals, temporal patterns, and cross-feature interactions. The model outputs a composite engagement score that determines the final ordering. The fine ranking step must complete within 80 milliseconds for the entire candidate set.
Progressive Promotion System
TikTok uses a progressive promotion system that carefully controls how new videos are exposed to audiences. This system prevents bad content from reaching many users while ensuring good content gets discovered regardless of the creator follower count.
| Tier | Impressions | Duration | Promotion Criteria |
|---|---|---|---|
| Tier 1 (Cold Start) | 200-500 views | 1-2 hours | Completion rate above 40%, like rate above 5% |
| Tier 2 | 1,000-5,000 views | 4-8 hours | Completion rate above 50%, engagement above median |
| Tier 3 | 10,000-50,000 views | 12-24 hours | Completion rate above 55%, share rate above average |
| Tier 4 | 100,000-500,000 views | 24-48 hours | Strong engagement across all metrics |
| Tier 5 (Viral) | 1,000,000+ views | Multiple days | Exceptional engagement with broad demographic appeal |
Cold Start Problem
New users present a cold start problem because the system has no interaction history to base recommendations on. TikTok solves this through several complementary strategies. First, the system shows new users a carefully curated set of broadly popular videos across diverse categories. Second, it leverages any available signals: the user geographic location (regional trends), sign-up demographics (age, if provided), and initial follows (if the user follows accounts during onboarding). Third, it implements an exploration-exploitation strategy where the first 30-60 seconds of content are heavily weighted toward exploration (diverse categories), then rapidly converge toward exploitation (content similar to what the user engaged with). Internally, the system tracks a confidence score for the user interest profile and increases personalization as confidence grows. Most users transition from cold start to personalized recommendations within 3-5 minutes of usage.
11. Feed Ranking
Feed ranking is the mathematical core of the recommendation system. It transforms raw candidate videos into a precisely ordered feed that maximizes user satisfaction and platform engagement metrics. The ranking model must be both accurate (predicting what users will enjoy) and fast (scoring thousands of candidates in under 100 milliseconds).
Feature Engineering
The ranking model consumes hundreds of features organized into several categories. Feature engineering is often more important than model architecture for recommendation quality.
| Feature Category | Example Features | Count |
|---|---|---|
| User Features | age, country, language, account_age_days, avg_session_length, video_completion_rate, preferred_content_hours | ~50 |
| User History | last_100_video_interactions, category_distribution, sound_preferences, creator_follow_list, search_history | ~200 |
| Video Features | duration, creation_time, caption_length, hashtag_count, sound_type, is_original, moderation_score | ~40 |
| Creator Features | follower_count, avg_views, upload_frequency, engagement_rate, account_age, verification_status | ~30 |
| Context Features | time_of_day, day_of_week, device_type, network_type, current_session_length, battery_level | ~20 |
| Cross Features | user_category_affinity x video_category, user_language x video_language, user_region x creator_region | ~100 |
| Content Embeddings | video_clip_vector (512d), audio_embedding (256d), text_embedding (256d), caption_embedding (128d) | ~1000 |
Prediction Targets
The ranking model predicts multiple engagement probabilities simultaneously using a multi-task learning architecture. Each prediction target represents a different type of user engagement, and they are all predicted in a single forward pass through the model.
- P(complete): Probability the user watches the video to the end. This is the strongest positive signal because it indicates genuine interest in the content.
- P(like): Probability the user likes the video. Likes are a moderate-strength signal that correlate with content quality.
- P(comment): Probability the user leaves a comment. Comments indicate strong engagement, especially positive comments.
- P(share): Probability the user shares the video to another platform or via direct message. Shares are the highest-quality positive signal because users only share content they think others will enjoy.
- P(follow): Probability the user follows the video creator. This indicates the user wants to see more content from this creator.
- P(save): Probability the user saves the video to their favorites. Saves indicate high long-term value content.
- P(not_interested): Probability the user swipes away quickly or explicitly marks the video as not interested. This is a strong negative signal that must be weighted heavily.
Scoring Formula
The final score is a weighted combination of predicted engagement probabilities, with weights tuned through extensive A/B testing. These weights are not static; they are continuously adjusted based on platform objectives and user satisfaction metrics.
Pseudocode
score = w1 * P(complete)
+ w2 * P(like)
+ w3 * P(comment)
+ w4 * P(share)
+ w5 * P(follow)
+ w6 * P(save)
- w7 * P(not_interested)
+ diversity_bonus
+ freshness_bonus
# Typical weights (tuned via A/B testing):
# w1 = 0.35 (watch completion is king)
# w2 = 0.15
# w3 = 0.10
# w4 = 0.20 (shares indicate high viral potential)
# w5 = 0.10
# w6 = 0.05
# w7 = 0.25 (negative signal weighted heavily to avoid bad experiences)
# Diversity bonus: +0.05 for videos from categories
# the user has not seen recently (prevents echo chambers)
# Freshness bonus: +0.03 for videos less than 6 hours old
# (helps new content get distribution)
Model Architecture
The ranking model typically uses a deep-and-wide architecture (similar to Google Wide and Deep). The wide component handles high-cardinality categorical features (user ID, video ID, sound ID) through embedding lookups that learn memorization patterns. The deep component processes dense numerical features through a series of fully connected layers with batch normalization, dropout, and ReLU activations that learn generalization patterns. The two components are concatenated before the final prediction layers, allowing the model to learn both memorization and generalization patterns simultaneously.
At serving time, the model must score 1,000 to 5,000 candidate videos per user within 100 milliseconds to maintain feed loading latency targets. This requires model optimization techniques including feature pruning (removing low-importance features), quantization (reducing model precision from float32 to int8 for inference), and model distillation (training a lightweight student model from a larger teacher model to maintain accuracy with reduced compute requirements).
12. Sound and Music System
Sounds are a defining feature of TikTok content ecosystem. The platform supports two types of audio content: licensed music from record labels and distributors, and original sounds created by users. The sound system must handle discovery, attribution, trending calculations, and usage tracking at massive scale while maintaining proper licensing compliance.
Sound Data Model
C#
public class Sound
{
public string SoundId { get; set; }
public string Title { get; set; }
public string Artist { get; set; }
public SoundType Type { get; set; } // Licensed, Original, Voiceover
public string AudioUrl { get; set; }
public int DurationMs { get; set; }
public string OriginalCreatorId { get; set; } // null for licensed music
public long UsageCount { get; set; }
public long WeeklyUsageDelta { get; set; }
public double TrendingScore { get; set; }
public List<string> Albums { get; set; }
public DateTime CreatedAt { get; set; }
public SoundLicenseInfo LicenseInfo { get; set; }
}
public class SoundUsage
{
public string SoundId { get; set; }
public string VideoId { get; set; }
public string UserId { get; set; }
public int StartOffsetMs { get; set; } // Which part of the sound is used
public int DurationMs { get; set; }
public DateTime UsedAt { get; set; }
}
public class SoundTrendingMetrics
{
public string SoundId { get; set; }
public long DailyUsages { get; set; }
public long WeeklyUsages { get; set; }
public double GrowthRate { get; set; }
public double EngagementRate { get; set; }
public List<string> TopRegions { get; set; }
public DateTime LastUpdated { get; set; }
}
Trending Sound Algorithm
The trending sound algorithm identifies sounds that are rapidly gaining popularity. It considers not just raw usage count but the rate of growth, the engagement metrics of videos using the sound, and the diversity of creators using it. A sound used by thousands of diverse creators in a short period scores higher than a sound used many times by a few creators.
C#
public class SoundTrendingCalculator
{
public double CalculateTrendingScore(SoundTrendingMetrics metrics)
{
// Velocity: rate of usage growth compared to weekly average
double velocity = metrics.WeeklyUsages > 0
? (double)metrics.DailyUsages / (metrics.WeeklyUsages / 7.0)
: 0;
// Growth rate bonus (capped at 1.0)
double growthBonus = Math.Min(metrics.GrowthRate / 100.0, 1.0);
// Engagement quality: average engagement of videos using this sound
double engagementScore = metrics.EngagementRate;
// Diversity factor: penalize sounds used by few creators
double diversityFactor = CalculateDiversityFactor(metrics.SoundId);
// Recency: newer sounds get a boost for discovery
double recencyBoost = CalculateRecencyBoost(metrics.SoundId);
// Weighted combination
double score = (velocity * 0.35)
+ (growthBonus * 0.20)
+ (engagementScore * 0.20)
+ (diversityFactor * 0.15)
+ (recencyBoost * 0.10);
return Math.Round(score, 6);
}
private double CalculateDiversityFactor(string soundId)
{
long uniqueCreators = _soundRepository
.GetRecentUniqueCreators(soundId, TimeSpan.FromDays(7));
// Logarithmic scaling: more creators means higher diversity
return Math.Min(Math.Log10(uniqueCreators + 1) / 4.0, 1.0);
}
private double CalculateRecencyBoost(string soundId)
{
var createdAt = _soundRepository.GetCreatedAt(soundId);
var ageHours = (DateTime.UtcNow - createdAt).TotalHours;
// Sounds less than 48 hours old get a freshness boost
if (ageHours < 48)
return 1.0 - (ageHours / 48.0) * 0.5;
return 0;
}
}
Sound Attribution Chain
When a user creates a sound from an existing video (for example, using someone else original audio), the platform maintains an attribution chain. This chain traces back to the original creator, ensuring they receive credit and any applicable royalties. The chain is stored as a linked list of sound references, and the display UI shows the full lineage from the original creator through each derivation. The attribution chain also prevents copyright disputes by maintaining a clear record of content provenance.
| Feature | Licensed Music | Original Sound |
|---|---|---|
| Source | Record labels and distributors | User-generated audio |
| Duration Limit | 60 seconds (clip from full song) | Up to 60 seconds |
| Royalty Handling | Revenue sharing with rights holders | No royalties, creator credit only |
| Attribution Display | Artist name and album art | Original creator name and video link |
| Usage Tracking | Required for royalty calculations | For trending and discovery metrics |
| Geographic Restrictions | May be region-locked due to licensing | Global availability |
13. Effects and Filters
Effects and filters are essential creative tools that differentiate TikTok from simpler video platforms. The effects system includes face filters, body tracking effects, green screen compositing, text overlays, speed controls, timers, and a wide range of augmented reality experiences. These effects must run in real-time at 30 FPS on devices ranging from flagship phones to mid-range budget devices.
Effects Architecture
Real-Time Face Tracking
Face tracking is the foundation of most popular TikTok effects. The system uses a lightweight face mesh model (typically 468 3D facial landmarks) that runs in real-time on the device neural processing unit (NPU) or GPU. The face mesh provides landmark positions (468 points defining the face geometry, updated at 30-60 FPS), head pose estimation (3D rotation in pitch, yaw, and roll plus translation of the head), expression detection (blend shapes for common expressions like smile, wink, open mouth, and raised eyebrows that trigger effect animations), and occlusion handling (robust tracking when parts of the face are temporarily hidden by hands or objects).
Effect Packaging and Delivery
Each effect is packaged as a self-contained bundle to enable efficient delivery and instant loading on the device.
| Component | Description | Typical Size |
|---|---|---|
| Manifest | Effect metadata, dependencies, compatibility info | 2 KB |
| Shader Programs | GPU shader code (GLSL/Metal/HLSL) | 50-200 KB |
| 3D Assets | Meshes, textures, animations | 1-10 MB |
| ML Models | Face mesh, segmentation, tracking models | 2-5 MB |
| Audio Assets | Sound effects associated with the effect | 100 KB - 2 MB |
| Preview Images | Thumbnail and preview animations | 200-500 KB |
Effect bundles are served from a dedicated CDN with aggressive caching and pre-loading strategies. The client pre-fetches the next 3 to 5 effects in the effects tray while the user is recording, ensuring instant effect switching with no visible loading delay. Effects are version-controlled, and new versions are deployed by uploading new bundles with updated version numbers rather than modifying existing bundles in place.
Green Screen Effect
The green screen effect is one of TikTok most distinctive features. It uses real-time semantic segmentation to separate the foreground (person) from the background and replace it with any uploaded image or video. The segmentation model runs on the device GPU and outputs a per-pixel mask at 30 FPS, which is composited with the replacement background in the rendering pipeline. Modern implementations use a lightweight U-Net architecture optimized for mobile inference, achieving over 30 FPS on mid-range devices. The segmentation model is typically 2-5 MB in size and is fine-tuned specifically for human body segmentation rather than general semantic segmentation, which allows it to be much smaller and faster while maintaining high quality boundary detection around hair, clothing, and accessories.
14. Duet, Stitch, Remix
Duet, Stitch, and Remix are collaborative content creation features that enable users to build upon each other videos. These features are critical for viral content chains, challenges, and community building on the platform. They transform TikTok from a content consumption platform into a collaborative content creation ecosystem.
Feature Comparison
| Feature | Duet | Stitch | Remix |
|---|---|---|---|
| Layout | Side-by-side split screen | Clipped segment plus new content | Audio re-use with new video |
| Video Source | Full original video plays alongside | 1-5 second clip from original | Audio extracted from original |
| Attribution | Original creator tagged and linked | Original creator tagged and linked | Sound attributed to original |
| Approval Required | Configurable by creator settings | Configurable by creator settings | Always allowed for all users |
| Revenue Sharing | Optional creator fund split | Optional creator fund split | N/A (sound attribution only) |
| Video Length | Up to 60 seconds | Clip of 1-5 seconds plus new content up to 60 seconds | Up to 60 seconds |
Duet Implementation
The Duet feature requires splitting the display into two synchronized video players. The technical implementation involves several coordinated components. Layout management renders two video streams side-by-side with a configurable split ratio (50/50 or 70/30). Both videos must start playback at the same timestamp and stay synchronized throughout the viewing experience. Audio mixing combines both audio tracks with volume controls allowing the duet creator to balance the original and their own audio. During recording, the original video plays on one side while the camera captures the new content on the other side, requiring careful audio synchronization to prevent echo and feedback. The system maintains a directed graph of video derivations, enabling discovery of all duets, stitches, and remixes originating from any video.
DAG-Based Content Derivation Graph
The derivation graph is stored in a graph database and serves multiple critical purposes. For content discovery, it enables finding all responses to a popular video, creating a chain of creative evolution. For moderation, it enables propagating content removal decisions to derivative works automatically. For analytics, it measures the viral spread of content through the platform and identifies the most influential original videos. The graph also powers the attribution display, showing users the full creative lineage when they encounter a duet or stitch.
15. Live Streaming Integration
Live streaming is an increasingly important feature for TikTok, enabling real-time interaction between creators and their audiences. While live streaming is technically a distinct system from short-form video, it shares several infrastructure components and adds unique requirements for low-latency real-time communication, real-time chat, and virtual gift transactions.
Live Streaming Architecture
Key Differences from VOD
Live streaming introduces several challenges not present in the short-form video on-demand system. Latency requirements are much stricter: live streams must be delivered with sub-3-second latency to feel truly live, which requires WebRTC or low-latency HLS (LL-HLS) instead of standard HLS, which typically has 10-30 seconds of latency. Ingestion requires the streamer to push a continuous RTMP or WebRTC stream to the ingestion server, which must handle real-time transcoding and packaging without introducing additional delay. Real-time chat requires WebSocket connections with sub-100ms message delivery, and a popular stream may have millions of concurrent chatters requiring a distributed pub-sub system with sticky connections.
Virtual gifts are a primary monetization mechanism for live streaming. They require real-time inventory management, creator revenue calculations, and platform commission tracking with strong consistency guarantees. The system must handle millions of gift transactions per minute during popular streams without losing any transactions or creating double-spend issues. Live content must also be moderated in real-time using automated systems (audio transcription, computer vision) with the ability to quickly cut the stream if serious policy violations occur.
Co-Host and Multi-Guest
TikTok live streaming supports multiple simultaneous hosts (up to 4 guests in a single stream). This requires a selective forwarding unit (SFU) that receives streams from all hosts and efficiently distributes them to viewers. The SFU optimizes bandwidth by only forwarding the specific streams each viewer needs and handles layout switching on the client side. The SFU architecture scales to millions of concurrent viewers by distributing the forwarding load across multiple edge nodes, with each viewer connecting to their nearest geographic edge node.
16. Creator Monetization
Creator monetization is essential for retaining top content creators and ensuring a steady supply of high-quality content to the platform. A TikTok-like platform must offer multiple monetization channels to attract creators across different content niches, audience sizes, and content styles. Without effective monetization, creators will migrate to competing platforms that offer better earning potential.
Monetization Channels
| Channel | Mechanism | Revenue Source | Minimum Threshold |
|---|---|---|---|
| Creator Fund | Performance-based payments from platform pool | Platform advertising revenue allocation | 10K followers, 100K views per 30 days |
| Virtual Gifts (LIVE) | Viewers purchase and send virtual gifts during live streams | Viewer purchases with 50/50 revenue split | 1K followers for LIVE access |
| Brand Partnerships | Creator Marketplace matches brands with creators | Direct brand payments negotiated per campaign | Varies by brand requirements |
| Shopping Links | Product tags on videos linking to e-commerce | Affiliate commissions on sales | 1K followers minimum |
| Series and Premium | Paid exclusive content behind paywall | Viewer subscription payments | 10K followers for eligibility |
| Ad Revenue Sharing | Pre-roll and mid-roll ads on creator videos | Advertising revenue with 55% creator share | 10K followers, 100K views |
Creator Fund Payout Engine
The Creator Fund payout engine calculates daily earnings for each eligible creator based on their video performance. The calculation must be deterministic, auditable, and capable of processing millions of creator accounts daily. The engine distributes a fixed daily pool proportionally based on engagement scores.
C#
public class CreatorFundPayoutEngine
{
private readonly decimal DailyFundPool = 5_000_000m; // $5M daily pool
public async Task<List<CreatorPayout>> CalculateDailyPayouts(DateTime date)
{
var eligibleVideos = await _videoRepo.GetVideosPublishedOnDate(date);
var eligibleCreators = await _creatorRepo.GetEligibleCreators();
var videoScores = eligibleVideos
.Where(v => eligibleCreators.Any(c => c.UserId == v.CreatorId))
.Select(v => new
{
Video = v,
Score = CalculateEngagementScore(v)
})
.ToList();
decimal totalScore = videoScores.Sum(v => v.Score);
var payouts = videoScores
.GroupBy(v => v.Video.CreatorId)
.Select(g => new CreatorPayout
{
CreatorId = g.Key,
Date = date,
TotalEngagementScore = g.Sum(v => v.Score),
SharePercentage = g.Sum(v => v.Score) / totalScore,
GrossPayout = (g.Sum(v => v.Score) / totalScore) * DailyFundPool,
PlatformCommission = (g.Sum(v => v.Score) / totalScore)
* DailyFundPool * 0.40m,
NetPayout = (g.Sum(v => v.Score) / totalScore)
* DailyFundPool * 0.60m
})
.ToList();
return payouts;
}
private decimal CalculateEngagementScore(Video video)
{
decimal views = video.ViewCount;
decimal likes = video.LikeCount * 2;
decimal comments = video.CommentCount * 3;
decimal shares = video.ShareCount * 4;
decimal completions = video.CompletionCount * 5;
decimal qualityViews = views * (decimal)video.CompletionRate;
return qualityViews + likes + comments + shares + completions;
}
}
Virtual Gifts Economy
The virtual gifts economy is a sophisticated in-app purchase system. Viewers buy virtual coins with real money (with prices varying by region and platform fees), then use coins to purchase virtual gifts of varying values to send to creators during live streams. The platform takes a commission (typically 50%) and pays out the remainder to the creator on a regular payout schedule. This system requires real-time transaction processing with strong consistency, fraud detection to prevent money laundering and fake gift schemes, age verification for purchasers, and regulatory compliance for digital goods sales across multiple jurisdictions with different tax laws and consumer protection regulations. The gift inventory system must handle millions of concurrent purchases during popular streams without losing any transactions.
17. Content Moderation at Scale
Content moderation is arguably the most challenging operational aspect of running a global video platform. With 60 million videos uploaded daily, automated systems must handle the vast majority of moderation decisions while human moderators review edge cases and handle appeals. The moderation pipeline must be fast enough to prevent violating content from going viral while being accurate enough to avoid excessive false positives that frustrate creators.
Moderation Pipeline Architecture
Policy Violation Categories
| Category | Detection Method | Action | Confidence Threshold |
|---|---|---|---|
| CSAM (Child Safety) | PhotoDNA hash plus ML classifier | Immediate removal plus report to NCMEC | Any hash match |
| Terrorism | Terrorist content hash database plus ML | Immediate removal plus report to authorities | Any hash match |
| NSFW and Nudity | Computer vision classifier (NudeNet) | Remove or age-gate content | Confidence above 0.85 |
| Violence and Gore | Computer vision plus audio analysis | Remove or apply warning screen | Confidence above 0.80 |
| Hate Speech | Audio transcription plus NLP classifier | Remove content | Confidence above 0.90 |
| Misinformation | Fact-check database plus ML model | Add label and reduce distribution | Confidence above 0.75 |
| Spam | Behavioral analysis plus content analysis | Remove content and flag account | Confidence above 0.85 |
| Harassment | Text analysis plus context modeling | Remove or limit visibility | Confidence above 0.80 |
Hash-Based Detection
Hash-based detection is the fastest and most reliable method for known violating content. Organizations like NCMEC maintain databases of known CSAM content hashes. The platform computes perceptual hashes (pHash) of every uploaded video and compares them against these databases. A match triggers immediate removal without requiring ML inference, which is both faster and more reliable. Perceptual hashing is robust against minor modifications like resizing, compression, or cropping, making it effective against simple evasion attempts. The hash comparison runs in under 10 milliseconds per video, making it the first check in the moderation pipeline.
Human Moderation at Scale
Despite advances in automated moderation, human review remains essential for nuanced content decisions that require cultural context, sarcasm detection, and understanding of evolving social norms. At scale, this requires tens of thousands of content moderators working in shifts across multiple languages and time zones. Key operational challenges include moderator wellbeing (exposure to disturbing content causes psychological harm, requiring mental health support, mandatory breaks, and content blurring), consistency (different moderators may reach different conclusions on similar content, requiring regular calibration sessions), speed (moderation queues must be processed quickly to prevent violating content from spreading, with priority queuing for high-severity content), and lingual coverage (content must be moderated in all supported languages, requiring native-speaker moderators for each language).
18. Trending and Discover Page
The Trending and Discover page serves as a content exploration surface, complementing the personalized For You Page by showing users what is popular across the platform. Unlike the FYP, which is highly individualized, the trending page surfaces content that has broad appeal and rapid growth. It helps users discover new creators, participate in challenges, and understand what is happening in the cultural moment.
Trending Algorithm
The trending algorithm identifies content, sounds, hashtags, and creators that are experiencing rapid growth in engagement. The algorithm considers velocity of engagement growth, geographic spread of engagement, diversity of creators and audiences, and recency of the content.
C#
public class TrendingCalculator
{
public TrendingScore CalculateVideoTrendingScore(TrendingMetrics metrics)
{
double viewsVelocity = CalculateVelocity(
metrics.ViewsLast1h, metrics.ViewsLast6h, metrics.ViewsLast24h);
double engagementVelocity = CalculateVelocity(
metrics.EngagementsLast1h, metrics.EngagementsLast6h,
metrics.EngagementsLast24h);
double geoSpread = (double)metrics.UniqueRegions / TotalGlobalRegions;
double diversityScore = CalculateEngagementDiversity(
metrics.EngagingUserIds);
double ageHours = (DateTime.UtcNow - metrics.VideoCreatedAt).TotalHours;
double recencyDecay = Math.Exp(-0.1 * ageHours);
double score =
(viewsVelocity * 0.25) +
(engagementVelocity * 0.30) +
(geoSpread * 0.15) +
(diversityScore * 0.15) +
(recencyDecay * 0.15);
return new TrendingScore
{
VideoId = metrics.VideoId,
Score = score,
Category = ClassifyTrendingCategory(metrics),
TrendDirection = CalculateTrendDirection(metrics)
};
}
private double CalculateVelocity(double recent, double medium, double older)
{
if (older == 0) return recent > 0 ? 1.0 : 0;
double ratio1h = recent / (older / 24.0);
double ratio6h = medium / (older / 4.0);
return (ratio1h * 0.6) + (ratio6h * 0.4);
}
}
Discover Page Layout
The Discover page is organized into several sections to help users explore content beyond their usual interests. Trending Hashtags shows a ranked list of hashtags experiencing rapid growth, with short descriptions and representative videos. Each hashtag displays the number of videos created with it in the last 24 hours and the overall growth trajectory. Trending Sounds presents audio tracks that are rapidly gaining usage, with preview playback and links to all videos using the sound. Featured Creators highlights accounts that are growing rapidly or creating consistently high-quality content in specific categories. Category Browsing organizes content by categories such as Comedy, Dance, Sports, Food, and Education with sub-categories and curated playlists. Challenges showcases active challenges and hashtag trends with participation counts and tutorial videos. Localized Trends surface trends specific to the user region or language, ensuring cultural relevance.
Trending Data Pipeline
Trending calculations run on a streaming pipeline that processes interaction events in near real-time. A Kafka Streams application aggregates events into time windows (1-hour, 6-hour, 24-hour) and calculates trending scores for each content item. The results are written to a Redis sorted set for each trending category, enabling O(log N) retrieval of the top-N trending items. The pipeline refreshes every 5 minutes, ensuring the trending page always reflects current platform activity. For trending hashtags and sounds, the pipeline also tracks the rate of change to distinguish between steadily popular items and rapidly rising trends.
19. Notification System
The notification system drives user re-engagement by informing users about relevant activity: new followers, likes on their videos, comments, mentions, trending content from followed creators, and system announcements. A well-designed notification system balances engagement with user satisfaction by avoiding notification fatigue while ensuring users do not miss important interactions.
Notification Types and Priority
| Type | Trigger | Priority | Delivery Channel |
|---|---|---|---|
| New Follower | User A follows User B | Medium | Push notification plus In-App |
| Video Like | User likes creator video | Low | In-App only (batched hourly) |
| Comment | User comments on video | Medium | Push notification plus In-App |
| Comment Reply | Reply to user comment | High | Push notification plus In-App |
| Mention | User mentioned in comment or caption | High | Push notification plus In-App |
| Duet or Stitch | Someone duets or stitches user video | High | Push notification plus In-App |
| Trending | Creator video enters trending | Medium | Push notification plus In-App |
| Live Started | Followed creator goes live | High | Push notification only |
| Milestone | Follower count milestone reached | Low | In-App notification only |
Notification Delivery Architecture
Smart Notification Bundling
To prevent notification fatigue, the system bundles similar notifications intelligently. If a user receives 50 likes on a video within an hour, they receive a single notification saying "50 people liked your video" rather than 50 individual notifications that would be annoying and disruptive. The bundling logic groups notifications by type and time window, generating natural-language summaries of batched notifications. The bundling window varies by notification type: likes are batched every 30 minutes, comments are batched every 15 minutes, and new followers are batched every hour.
The rate limiter enforces per-user notification frequency caps to prevent over-notifying. A user might receive at most 5 push notifications per hour and 20 per day, with priority-based exceptions for high-priority events like direct messages or live stream starts from close connections. These limits are configurable per user through notification settings, and the system respects device-level notification preferences as well. The rate limiter uses a sliding window algorithm that tracks recent notification counts and rejects new notifications once the cap is reached, with an exception queue for high-priority notifications that bypass the cap.
20. Database Design
The database layer for a TikTok-like platform must handle diverse access patterns: user profiles, video metadata, social graphs, interaction records, and analytics data. No single database technology can optimally serve all these patterns, so we use a polyglot persistence approach with specialized databases for each workload.
Database Technology Selection
| Data Type | Technology | Reasoning |
|---|---|---|
| User Profiles | CockroachDB (Distributed SQL) | Strong consistency, ACID transactions, global distribution |
| Video Metadata | CockroachDB | Transactional integrity, complex joins, secondary indexes |
| Social Graph | Neo4j or Amazon Neptune | Graph traversal for followers of followers, mutual friends |
| Feed Cache | Redis Cluster | Sub-millisecond reads, TTL-based expiry, sorted sets |
| User Sessions | Redis Cluster | Fast reads and writes, session state management |
| Interaction Events | Apache Kafka to ClickHouse | High-throughput writes, analytical queries on event data |
| Content Embeddings | Milvus or Pinecone | Approximate nearest neighbor search for similar content |
| Object Storage | S3 or GCS or MinIO | Durable, cost-effective binary storage for video files |
| Search Index | Elasticsearch | Full-text search, hashtag search, user search |
Sharding Strategy
At TikTok scale, a single database instance cannot handle the read or write load. Data must be sharded across multiple instances using consistent hashing to distribute load evenly. User data is sharded by user_id, which ensures all data for a single user resides on the same shard, enabling efficient single-user queries. Video data is sharded by video_id, which distributes reads evenly because the feed service reads random videos from across all shards. Interaction data is sharded by the target entity (video_id for video interactions, user_id for user-directed notifications), which co-locates all interactions for a given entity. The social graph stores follower and following lists on the user shard. Graph traversals that cross shards use fan-out queries, but common traversals like friends-of-friends are pre-computed and cached.
Feed Storage
The feed for each user is pre-computed and stored in Redis sorted sets. Each user feed is a sorted set of video IDs ordered by the ranking score. The feed is regenerated periodically (every 30 to 60 minutes) or on-demand when the user opens the app. Pre-computed feeds reduce read latency from hundreds of milliseconds (database query plus ranking computation) to single-digit milliseconds (Redis ZRANGEBYSCORE).
C#
public class FeedStorageService
{
private readonly IRedisCluster _redis;
private const int FeedCacheSize = 500;
private const int FeedTtlHours = 2;
public async Task CacheFeed(long userId, List<RankedVideo> rankedVideos)
{
var key = $\"feed:{userId}\";
var entries = rankedVideos
.Take(FeedCacheSize)
.Select(v => new SortedSetEntry(
v.VideoId.ToString(),
v.Score))
.ToArray();
await _redis.ZAddAsync(key, entries);
await _redis.KeyExpireAsync(key, TimeSpan.FromHours(FeedTtlHours));
}
public async Task<List<long>> GetFeedPage(
long userId, int pageSize, double minScore = double.NegativeInfinity)
{
var key = $\"feed:{userId}\";
var results = await _redis.ZRevRangeByScoreAsync(
key, max: \"+inf\", min: minScore.ToString(),
skip: 0, take: pageSize);
return results
.Select(r => long.Parse(r.Element))
.ToList();
}
public async Task InvalidateFeed(long userId)
{
var key = $\"feed:{userId}\";
await _redis.KeyDeleteAsync(key);
}
}
21. Caching Strategy
Caching is critical for a TikTok-scale platform. Without aggressive caching, the sheer volume of read requests would overwhelm database capacity and result in unacceptable latency. We employ caching at multiple layers of the architecture, from the client application to the CDN edge, with each layer serving a specific purpose and optimization target.
Multi-Layer Caching Architecture
Cache Strategy by Data Type
| Data | Cache Layer | TTL | Invalidation Strategy |
|---|---|---|---|
| Video metadata | Redis plus Local | 5 min (Redis), 1 min (Local) | Write-through on update |
| Video thumbnails | CDN | 7 days | Purge on video deletion |
| Video segments (HLS) | CDN | 30 days | Never (immutable once created) |
| User profile | Redis plus Local | 10 min (Redis), 2 min (Local) | Write-through on update |
| Follower count | Redis | 1 min | Eventual via counting queue |
| Feed (ranked videos) | Redis | 2 hours | Lazy refresh on next access |
| Trending topics | Redis | 5 min | Streaming pipeline refresh |
| Effect bundles | CDN | 7 days | Version-based (new version = new URL) |
| Search results | Redis | 10 min | TTL expiry with warm refresh |
Cache Stampede Prevention
When a popular video cache expires, thousands of simultaneous requests can hit the database simultaneously, causing a cache stampede that can cascade into a full system outage. We prevent this with several complementary techniques. Request coalescing ensures that when multiple requests for the same key arrive simultaneously, only one request hits the database while the others wait and receive the same result. Stale-while-revalidate serves the stale value while asynchronously refreshing the cache in the background, ensuring zero-latency reads even during cache refresh. Jittered TTLs add a random jitter (for example, 5 minutes plus or minus 30 seconds) to prevent synchronized expiration of many related keys. Early expiration refreshes extremely hot keys (celebrity profiles, viral videos) at 80% of TTL rather than waiting for full expiration.
22. Multi-Region Design
TikTok operates in over 150 countries and regions, requiring a multi-region deployment strategy that provides low-latency access to users worldwide while complying with data sovereignty regulations. Each major region (Americas, Europe, Southeast Asia, East Asia) operates as an independent deployment with data replication for disaster recovery and content locality.
Multi-Region Architecture
Data Replication Strategy
Different data types require different replication strategies across regions based on their consistency requirements and access patterns. User profiles and social graph data are replicated asynchronously to all regions with a target lag of under 5 seconds. A user in any region can view any other user profile. Writes are routed to the user home region and replicated outward to other regions. Video metadata is replicated asynchronously so that a video uploaded in one region becomes visible globally within 10 to 30 seconds. The video content itself is served from the CDN, which has edge nodes in every region regardless of where the original file is stored.
Feed data is not replicated between regions. Each region computes feeds independently using a global feature store that aggregates interaction data from all regions. User interaction events are written locally and streamed to a global event pipeline for analytics and ML training. Eventual consistency is acceptable because interaction counts (likes, views) are displayed as approximate numbers with a few seconds of latency. This approach avoids the complexity and latency of cross-region synchronous replication for the highest-throughput data path.
Compliance and Data Sovereignty
| Regulation | Region | Key Requirements |
|---|---|---|
| GDPR | European Union | Data minimization, right to erasure, consent management, data processing agreements |
| CCPA | California, USA | Right to know, right to delete, right to opt-out of data sales |
| PIPL | China | Data localization, cross-border transfer restrictions, explicit consent requirements |
| DPDPA | India | Data fiduciary obligations, cross-border transfer restrictions |
| LDPD | Russia | Data localization requiring citizen data stored on local servers |
23. Cost Estimation
Running a TikTok-scale platform involves significant infrastructure costs. Let us estimate the monthly cloud infrastructure cost for the major cost centers based on industry benchmarks and publicly available pricing for major cloud providers.
Monthly Cost Breakdown
| Component | Resource Specification | Monthly Cost (Estimated) |
|---|---|---|
| Object Storage | 5 PB stored, 8 PB written per month | $400,000 |
| CDN Egress | 360 PB per month egress | $28,800,000 |
| Compute (Application) | 5,000 instances (8 vCPU, 32GB RAM) | $1,500,000 |
| GPU Compute (ML and Transcoding) | 2,000 GPU instances (NVIDIA A100) | $8,000,000 |
| Redis Cluster | 200 nodes (512GB RAM each, 100TB total) | $1,200,000 |
| CockroachDB | 100 nodes (8 vCPU, 32GB RAM, 1TB SSD) | $800,000 |
| Kafka | 200 brokers (8 vCPU, 32GB RAM, 4TB SSD) | $600,000 |
| ClickHouse (Analytics) | 50 nodes (16 vCPU, 64GB RAM, 10TB SSD) | $500,000 |
| Elasticsearch | 100 nodes (8 vCPU, 32GB RAM, 2TB SSD) | $700,000 |
| Milvus (Vector DB) | 50 nodes (8 vCPU, 128GB RAM, GPU) | $600,000 |
| Cross-Region Bandwidth | 10 PB per month inter-region transfer | $1,000,000 |
| Miscellaneous | DNS, WAF, logging, monitoring | $500,000 |
| Total Estimated Monthly | ~$44.6 Million |
Cost Optimization Strategies
At this scale, even small percentage reductions in cost translate to millions of dollars in annual savings. CDN optimization through self-hosted CDN infrastructure (like TikTok Pico architecture) can reduce CDN costs by 40-60% compared to third-party CDNs. Storage tiering moves videos older than 6 months with fewer than 100 views from hot SSD-backed storage to warm HDD-backed storage to cold archive storage, reducing storage costs by 80% for older content. Reserved instances for predictable workloads (databases, Kafka clusters) provide 30-60% discounts compared to on-demand pricing. Spot instances for transcod and ML training workloads offer 60-80% savings with automatic failover to on-demand if spot capacity is reclaimed. Codec efficiency improvements migrating from H.264 to H.265 or AV1 reduce video storage and bandwidth requirements by 30-50%, directly impacting both storage and CDN costs.
24. Interview Q and A
Here are 12 common system design interview questions related to building a TikTok-like platform, along with structured answers for senior-level candidates.
Question 1: How do you handle the cold start problem for new users?
For new users with no interaction history, we use a multi-pronged approach. First, we show a diverse set of globally popular videos that span multiple categories including comedy, sports, music, food, and education. Second, we leverage any available signals: the user geographic location for regional trends, sign-up demographics if provided, and initial follows if the user follows accounts during onboarding. Third, we implement an exploration-exploitation strategy where the first 30 to 60 seconds of content are heavily weighted toward exploration (diverse categories), then rapidly converge toward exploitation (content similar to what the user engaged with). Internally, we track a confidence score for the user interest profile and increase personalization as confidence grows. Most users transition from cold start to fully personalized recommendations within 3 to 5 minutes of active usage.
Question 2: How do you prevent the echo chamber effect?
Echo chambers reduce content diversity and can lead to user dissatisfaction and platform homogeneity. We address this through several mechanisms in the ranking pipeline. First, we add a diversity bonus score for videos from categories the user has not recently engaged with. Second, we enforce a maximum consecutive video limit from any single category (no more than 3 consecutive comedy videos, for example). Third, we periodically inject exploration candidates from outside the user typical interest profile, measuring whether engagement improves to justify the diversity injection. Fourth, the progressive promotion system naturally exposes users to diverse content because trending content spans many categories and creator types.
Question 3: How would you design the video feed API for sub-200ms latency?
Achieving sub-200ms feed latency requires eliminating synchronous database queries from the hot path entirely. The feed is pre-computed and stored in Redis sorted sets keyed by user ID. When a user requests the next page of their feed, the API server performs a single Redis ZREVRANGEBYSCORE query, which completes in 1 to 5 milliseconds. The feed is regenerated every 30 to 60 minutes or on-demand when the user opens the app after a long absence. The API response includes video metadata that is also cached in Redis, so no database round-trips are needed for the critical path. CDN edge caching of the API response provides an additional layer of latency reduction for geographically distant users.
Question 4: How do you handle a viral video that suddenly gets millions of views?
Viral videos create sudden traffic spikes that can overwhelm the system if not handled properly. The video content is served from CDN edge nodes, which can handle massive read throughput without impacting the origin server. The video metadata and interaction counts are cached in Redis, absorbing read traffic. Interaction counters (views, likes) use eventually consistent counting with periodic batching rather than synchronous database updates. The transcoding pipeline has auto-scaling GPU workers that handle sudden increases in derivative content (duets, stitches). Circuit breakers and rate limits prevent a single viral video from degrading the experience for other users.
Question 5: How do you detect and handle duplicate or reposted content?
Duplicate detection uses a multi-layered approach. We compute perceptual hashes (pHash) of the video visual content at upload time and compare against a database of existing hashes. We use multi-modal embeddings (CLIP vectors) to find semantically similar content. Audio fingerprinting detects reused audio tracks. Metadata analysis flags videos with the same caption, sound, and similar duration as existing popular content. Detected duplicates are either blocked at upload time or have their distribution severely limited. The original creator retains full distribution rights while derivative works receive reduced algorithmic distribution.
Question 6: How do you ensure content moderation scales to millions of uploads per day?
Scaling moderation requires a tiered automated system. Tier 1 is hash-based matching for known violating content (CSAM, terrorism), processing in under 10 milliseconds. Tier 2 is ML-based classification (NSFW, violence, hate speech) using optimized GPU inference, processing each video in under 2 seconds. Tier 3 is policy engine matching for nuanced violations, running a rules engine on extracted features. Content that passes all automated checks is served immediately but remains in a review queue for post-publication monitoring. Content triggering Tier 1 or high-confidence Tier 2 is removed before distribution. Low-confidence cases enter a human review queue with priority based on current and predicted view count.
Question 7: How do you design the recommendation system to work across different cultures?
Cultural adaptation is critical for a global platform. The recommendation system maintains separate interest models for each cultural context, considering language, region, cultural norms, and local trends. Features include local trending topics, region-specific content popularity patterns, and cultural holidays or events. The candidate generation pool includes a region-specific component that surfaces locally relevant content alongside globally popular content. The system learns cross-cultural interests: a user in Japan watching American cooking content should receive more American cooking content regardless of their location. Cultural context is treated as a feature rather than a hard filter.
Question 8: How do you handle graceful degradation when the recommendation model is slow?
Fallback strategies are essential for maintaining availability. If the primary ranking model exceeds its latency budget of 100ms, we fall back to a lightweight pre-ranked feed computed in advance. If the candidate generation service is slow, we fall back to a cached trending feed. If Redis is degraded, we serve the last-known feed from a local in-memory cache. In the worst case, we serve a static popular feed from CDN-cached responses. Each fallback level trades personalization quality for latency reduction, ensuring users always see content even if personalization is temporarily degraded.
Question 9: How would you migrate from H.264 to AV1 codec?
Codec migration is a multi-month project executed gradually. First, we implement AV1 encoding in parallel with existing H.264 encoding, serving AV1 only to clients that support it (detected via User-Agent or client capability negotiation). Over 3 to 6 months, as device support for AV1 increases across modern Android and iOS devices, we gradually shift traffic. The client player supports adaptive switching between H.264 and AV1 segments. We monitor quality metrics (startup time, buffering ratio, visual quality scores) to ensure AV1 performs better, not just smaller. Storage savings of 30-50% are realized gradually as we re-encode older popular content during idle transcoding capacity.
Question 10: How do you handle live video moderation differently from VOD?
Live video moderation is fundamentally harder because content cannot be un-seen once broadcast. The approach combines real-time audio transcription with a 3 to 5 second delay, frame sampling every 2 seconds through NSFW and violence classifiers, and automated stream termination for high-confidence violations. Human moderators watch high-risk streams in real-time with the ability to cut the stream within seconds. New live streams from accounts with prior violations have reduced reach and stricter automated monitoring. Post-stream, the full recording goes through the standard VOD moderation pipeline, and the stream may be retroactively removed.
Question 11: How do you ensure data consistency for like and follow counts?
Like and follow counts use eventually consistent counting for performance. When a user likes a video, the like is immediately recorded in the database (strong consistency for the user action), but the video like count is updated asynchronously through a counting queue. The count displayed to users is read from Redis, updated by a background process aggregating actual counts every few seconds. This means counts may be slightly stale (by up to 5 to 10 seconds), but the system avoids the performance bottleneck of synchronous count updates. For the user own content, we can read the accurate count directly from the database.
Question 12: How would you design the system to support a new country launch?
A new country launch requires deployment of a new region with full infrastructure including application servers, databases, transcoding pipeline, CDN edge nodes, and ML model serving. Key considerations include language support for content moderation and search, local content licensing for music and sound rights, regulatory compliance for data protection and content restrictions, and payment processing for creator monetization in local currency. The new region starts with no local content and serves a mix of global trending content and content in the local language from other regions. As local creators join, the recommendation system gradually shifts toward local content.
25. Full C# Implementation
Below is a substantial C# implementation covering the core services for a TikTok-like platform. This code demonstrates the key service interfaces, data models, dependency injection, and business logic for the feed, upload, ranking, notification, and moderation systems. The implementation follows clean architecture principles with proper separation of concerns.
C#
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using System.Text.Json;
using System.Collections.Concurrent;
namespace TikTokPlatform.Core.Models
{
public enum VideoVisibility { Public, FriendsOnly, Private }
public enum UploadStatus { Initialized, Uploading, Processing, Ready, Failed }
public enum SoundType { Licensed, Original, Voiceover }
public enum NotificationPriority { Low, Medium, High, Critical }
public enum ModerationStatus { Pending, Approved, Rejected, UnderReview }
public enum InteractionEventType
{
View, Complete, Like, Comment, Share,
Follow, Save, NotInterested, ProfileVisit
}
public class User
{
public long UserId { get; set; }
public string Username { get; set; }
public string DisplayName { get; set; }
public string Bio { get; set; }
public string ProfileUrl { get; set; }
public long FollowerCount { get; set; }
public long FollowingCount { get; set; }
public long TotalLikes { get; set; }
public bool IsVerified { get; set; }
public string Region { get; set; }
public DateTime CreatedAt { get; set; }
public UserInterests Interests { get; set; } = new();
}
public class UserInterests
{
public Dictionary<string, double> CategoryAffinity { get; set; } = new();
public List<string> PreferredLanguages { get; set; } = new();
public List<long> TopCreators { get; set; } = new();
public DateTime LastUpdated { get; set; }
}
public class Video
{
public long VideoId { get; set; }
public long CreatorId { get; set; }
public string Caption { get; set; }
public long? SoundId { get; set; }
public int DurationMs { get; set; }
public VideoVisibility Visibility { get; set; }
public ModerationStatus ModerationStatus { get; set; }
public long ViewCount { get; set; }
public long LikeCount { get; set; }
public long CommentCount { get; set; }
public long ShareCount { get; set; }
public long CompletionCount { get; set; }
public double CompletionRate =>
ViewCount > 0 ? (double)CompletionCount / ViewCount : 0;
public List<string> Hashtags { get; set; } = new();
public ContentMetadata ContentMetadata { get; set; }
public DateTime CreatedAt { get; set; }
public VideoUrls Urls { get; set; } = new();
}
public class ContentMetadata
{
public List<string> DetectedObjects { get; set; } = new();
public List<string> DetectedScenes { get; set; } = new();
public string Language { get; set; }
public double SafetyScore { get; set; }
public float[] ContentEmbedding { get; set; }
}
public class VideoUrls
{
public string MasterPlaylistUrl { get; set; }
public string CoverUrl { get; set; }
}
public class RankedVideo
{
public Video Video { get; set; }
public double Score { get; set; }
public string Reason { get; set; }
}
public class UploadSession
{
public string UploadId { get; set; }
public long UserId { get; set; }
public string FileName { get; set; }
public long FileSizeBytes { get; set; }
public int TotalChunks { get; set; }
public int ChunkSizeBytes { get; set; } = 5 * 1024 * 1024;
public HashSet<int> ReceivedChunks { get; set; } = new();
public UploadStatus Status { get; set; }
public DateTime CreatedAt { get; set; }
public DateTime ExpiresAt { get; set; }
public double Progress => TotalChunks > 0
? (double)ReceivedChunks.Count / TotalChunks * 100 : 0;
public bool IsComplete => ReceivedChunks.Count == TotalChunks;
public bool IsExpired => DateTime.UtcNow > ExpiresAt;
}
public class ModelPrediction
{
public double PComplete { get; set; }
public double PLike { get; set; }
public double PComment { get; set; }
public double PShare { get; set; }
public double PFollow { get; set; }
public double PSave { get; set; }
public double PNotInterested { get; set; }
}
public class Notification
{
public long NotificationId { get; set; }
public long UserId { get; set; }
public string Type { get; set; }
public string Title { get; set; }
public string Body { get; set; }
public NotificationPriority Priority { get; set; }
public bool IsRead { get; set; }
public DateTime CreatedAt { get; set; }
public Dictionary<string, string> Metadata { get; set; } = new();
}
public class ModerationResult
{
public long VideoId { get; set; }
public ModerationStatus Status { get; set; }
public double RiskScore { get; set; }
public List<string> Violations { get; set; } = new();
public string ReviewedBy { get; set; }
public DateTime ReviewedAt { get; set; }
}
}
namespace TikTokPlatform.Core.Interfaces
{
using TikTokPlatform.Core.Models;
public interface IFeedService
{
Task<FeedResponse> GetForYouFeed(long userId, int pageSize, string cursor);
Task<FeedResponse> GetFollowingFeed(long userId, int pageSize, string cursor);
Task InvalidateUserFeed(long userId);
}
public interface ICandidateGenerator
{
Task<List<Video>> GetTrendingCandidates(int count);
Task<List<Video>> GetInterestCandidates(
long userId, UserInterests interests, int count);
Task<List<Video>> GetFreshCandidates(int count);
Task<List<Video>> GetSocialCandidates(long userId, int count);
Task<List<Video>> GetExplorationCandidates(
long userId, UserInterests interests, int count);
}
public interface IRankingModel
{
Task<List<RankedVideo>> Rank(
long userId, UserInterests interests,
List<Video> candidates);
}
public interface IUploadService
{
Task<UploadSession> InitUpload(long userId, string fileName,
long fileSize, string mimeType);
Task MarkChunkReceived(string uploadId, int chunkIndex, string etag);
Task<UploadSession> CompleteUpload(string uploadId);
Task<UploadSession> GetUploadStatus(string uploadId);
}
public interface INotificationService
{
Task SendNotification(long userId, string type,
string title, string body, NotificationPriority priority);
Task<List<Notification>> GetNotifications(
long userId, int pageSize, string cursor);
Task MarkAllRead(long userId);
}
public interface IModerationService
{
Task<ModerationResult> ModerateVideo(Video video);
Task<List<ModerationResult>> GetPendingReviews(int limit);
Task SubmitAppeal(long videoId, string reason);
}
}
namespace TikTokPlatform.Core.Services
{
using TikTokPlatform.Core.Models;
using TikTokPlatform.Core.Interfaces;
public class FeedService : IFeedService
{
private readonly ICandidateGenerator _candidates;
private readonly IRankingModel _ranking;
private readonly IRedisFeedStore _feedStore;
private readonly ILogger _logger;
private const double DIVERSITY_BONUS = 0.05;
private const double FRESHNESS_BONUS = 0.03;
private const int MAX_CONSECUTIVE_SAME_CATEGORY = 3;
public FeedService(
ICandidateGenerator candidates,
IRankingModel ranking,
IRedisFeedStore feedStore,
ILogger logger)
{
_candidates = candidates;
_ranking = ranking;
_feedStore = feedStore;
_logger = logger;
}
public async Task<FeedResponse> GetForYouFeed(
long userId, int pageSize, string cursor)
{
var cached = await _feedStore.GetFeedPage(userId, pageSize, cursor);
if (cached != null && cached.Count >= pageSize)
{
return new FeedResponse
{
Items = cached,
HasMore = true,
NextCursor = cached.Last().Video.VideoId.ToString()
};
}
return await GenerateFeedOnDemand(userId, pageSize);
}
private async Task<FeedResponse> GenerateFeedOnDemand(
long userId, int pageSize)
{
var timer = System.Diagnostics.Stopwatch.StartNew();
var userInterests = await _feedStore.GetUserInterests(userId);
var trendingTask = _candidates.GetTrendingCandidates(500);
var interestTask = _candidates.GetInterestCandidates(
userId, userInterests, 300);
var freshTask = _candidates.GetFreshCandidates(200);
var socialTask = _candidates.GetSocialCandidates(userId, 100);
var exploreTask = _candidates.GetExplorationCandidates(
userId, userInterests, 100);
await Task.WhenAll(trendingTask, interestTask, freshTask,
socialTask, exploreTask);
var allCandidates = new List<Video>();
allCandidates.AddRange(trendingTask.Result);
allCandidates.AddRange(interestTask.Result);
allCandidates.AddRange(freshTask.Result);
allCandidates.AddRange(socialTask.Result);
allCandidates.AddRange(exploreTask.Result);
allCandidates = allCandidates
.GroupBy(v => v.VideoId)
.Select(g => g.First())
.ToList();
var seenIds = await _feedStore.GetSeenVideoIds(userId);
allCandidates = allCandidates
.Where(v => !seenIds.Contains(v.VideoId))
.Where(v => v.ModerationStatus == ModerationStatus.Approved)
.Where(v => v.Visibility == VideoVisibility.Public)
.ToList();
var ranked = await _ranking.Rank(userId, userInterests, allCandidates);
var diversified = ApplyDiversityReranking(ranked, pageSize * 3);
await _feedStore.CacheFeed(userId, diversified);
var page = diversified.Take(pageSize).ToList();
timer.Stop();
_logger.LogInformation(
$\"Feed generated for {userId} in {timer.ElapsedMilliseconds}ms\");
return new FeedResponse
{
Items = page,
HasMore = diversified.Count > pageSize,
NextCursor = page.LastOrDefault()?.Video.VideoId.ToString()
};
}
private List<RankedVideo> ApplyDiversityReranking(
List<RankedVideo> ranked, int targetCount)
{
var result = new List<RankedVideo>();
var categoryCounts = new Dictionary<string, int>();
foreach (var rv in ranked)
{
var cat = rv.Video.ContentMetadata?.DetectedScenes?.FirstOrDefault()
?? \"unknown\";
int consecutive = result.Count >= MAX_CONSECUTIVE_SAME_CATEGORY
? result.Skip(result.Count - MAX_CONSECUTIVE_SAME_CATEGORY)
.Count(v => (v.Video.ContentMetadata
?.DetectedScenes?.FirstOrDefault() ?? \"unknown\") == cat)
: 0;
if (consecutive >= MAX_CONSECUTIVE_SAME_CATEGORY) continue;
int catCount = categoryCounts.GetValueOrDefault(cat, 0);
if (catCount < 3)
rv.Score += DIVERSITY_BONUS;
result.Add(rv);
categoryCounts[cat] = catCount + 1;
if (result.Count >= targetCount) break;
}
return result.OrderByDescending(v => v.Score).ToList();
}
public async Task<FeedResponse> GetFollowingFeed(
long userId, int pageSize, string cursor)
{
var followingIds = await _feedStore.GetFollowingIds(userId);
var videos = await _feedStore.GetVideosByCreators(
followingIds, pageSize + 10, cursor);
var ranked = videos
.OrderByDescending(v => v.CreatedAt)
.Take(pageSize)
.Select(v => new RankedVideo
{
Video = v, Score = 1.0, Reason = \"following\"
})
.ToList();
return new FeedResponse
{
Items = ranked,
HasMore = videos.Count > pageSize,
NextCursor = ranked.LastOrDefault()?.Video.VideoId.ToString()
};
}
public async Task InvalidateUserFeed(long userId)
{
await _feedStore.InvalidateFeed(userId);
}
}
public class RankingModel : IRankingModel
{
private const double W_COMPLETE = 0.35;
private const double W_LIKE = 0.15;
private const double W_COMMENT = 0.10;
private const double W_SHARE = 0.20;
private const double W_FOLLOW = 0.10;
private const double W_SAVE = 0.05;
private const double W_NEGATIVE = 0.25;
private readonly IMLInferenceService _mlService;
public RankingModel(IMLInferenceService mlService)
{
_mlService = mlService;
}
public async Task<List<RankedVideo>> Rank(
long userId, UserInterests interests, List<Video> candidates)
{
if (candidates.Count == 0)
return new List<RankedVideo>();
var predictions = await _mlService.PredictBatch(
userId, interests, candidates);
var ranked = new List<RankedVideo>();
for (int i = 0; i < candidates.Count; i++)
{
var pred = predictions[i];
var video = candidates[i];
double score =
W_COMPLETE * pred.PComplete +
W_LIKE * pred.PLike +
W_COMMENT * pred.PComment +
W_SHARE * pred.PShare +
W_FOLLOW * pred.PFollow +
W_SAVE * pred.PSave -
W_NEGATIVE * pred.PNotInterested;
var ageHours = (DateTime.UtcNow - video.CreatedAt).TotalHours;
if (ageHours < 6)
score += 0.03 * (1.0 - ageHours / 6.0);
ranked.Add(new RankedVideo
{
Video = video,
Score = Math.Round(score, 6),
Reason = DetermineReason(pred)
});
}
return ranked.OrderByDescending(v => v.Score).ToList();
}
private string DetermineReason(ModelPrediction pred)
{
if (pred.PShare > 0.3) return \"shareable_content\";
if (pred.PComplete > 0.7) return \"high_completion\";
if (pred.PLike > 0.4) return \"likely_to_like\";
if (pred.PFollow > 0.3) return \"new_creator_to_follow\";
return \"algorithmic_match\";
}
}
public class UploadService : IUploadService
{
private readonly IObjectStorage _storage;
private readonly ITranscodingQueue _transcoding;
private readonly ConcurrentDictionary<string, UploadSession> _sessions;
public UploadService(
IObjectStorage storage, ITranscodingQueue transcoding)
{
_storage = storage;
_transcoding = transcoding;
_sessions = new ConcurrentDictionary<string, UploadSession>();
}
public async Task<UploadSession> InitUpload(
long userId, string fileName, long fileSize, string mimeType)
{
int chunkSize = 5 * 1024 * 1024;
int totalChunks = (int)Math.Ceiling((double)fileSize / chunkSize);
var session = new UploadSession
{
UploadId = Guid.NewGuid().ToString(\"N\"),
UserId = userId,
FileName = fileName,
FileSizeBytes = fileSize,
TotalChunks = totalChunks,
ChunkSizeBytes = chunkSize,
Status = UploadStatus.Initialized,
CreatedAt = DateTime.UtcNow,
ExpiresAt = DateTime.UtcNow.AddHours(2)
};
_sessions.TryAdd(session.UploadId, session);
return await Task.FromResult(session);
}
public async Task MarkChunkReceived(
string uploadId, int chunkIndex, string etag)
{
if (!_sessions.TryGetValue(uploadId, out var session))
throw new InvalidOperationException(\"Upload session not found\");
if (session.IsExpired)
throw new InvalidOperationException(\"Upload session expired\");
session.ReceivedChunks.Add(chunkIndex);
session.Status = UploadStatus.Uploading;
if (session.IsComplete)
{
session.Status = UploadStatus.Processing;
await _storage.MergeChunks(
$\"uploads/{uploadId}/\", session.TotalChunks,
$\"videos/{uploadId}/original.mp4\");
await _transcoding.Enqueue(uploadId);
}
}
public async Task<UploadSession> CompleteUpload(string uploadId)
{
if (!_sessions.TryGetValue(uploadId, out var session))
throw new InvalidOperationException(\"Upload session not found\");
if (!session.IsComplete)
throw new InvalidOperationException(\"Upload not complete\");
return await Task.FromResult(session);
}
public async Task<UploadSession> GetUploadStatus(string uploadId)
{
_sessions.TryGetValue(uploadId, out var session);
return await Task.FromResult(session);
}
}
public class NotificationService : INotificationService
{
private readonly IPushNotificationClient _push;
private readonly INotificationStore _store;
private static readonly HashSet<string> RateLimitedTypes = new()
{ \"like\", \"view\", \"follow\" };
public NotificationService(
IPushNotificationClient push, INotificationStore store)
{
_push = push;
_store = store;
}
public async Task SendNotification(
long userId, string type, string title,
string body, NotificationPriority priority)
{
var notification = new Notification
{
NotificationId = GenerateId(),
UserId = userId,
Type = type,
Title = title,
Body = body,
Priority = priority,
IsRead = false,
CreatedAt = DateTime.UtcNow
};
await _store.Save(notification);
if (priority >= NotificationPriority.High ||
!RateLimitedTypes.Contains(type))
{
await _push.SendPush(userId, title, body, type);
}
}
public async Task<List<Notification>> GetNotifications(
long userId, int pageSize, string cursor)
{
return await _store.GetByUser(userId, pageSize, cursor);
}
public async Task MarkAllRead(long userId)
{
await _store.MarkAllAsRead(userId);
}
private long GenerateId() =>
DateTimeOffset.UtcNow.ToUnixTimeMilliseconds() * 1000
+ Random.Shared.Next(1000);
}
public class ModerationService : IModerationService
{
private readonly IContentAnalyzer _analyzer;
private readonly IModerationStore _store;
private const double AUTO_REMOVE_THRESHOLD = 0.90;
private const double AUTO_APPROVE_THRESHOLD = 0.10;
public ModerationService(
IContentAnalyzer analyzer, IModerationStore store)
{
_analyzer = analyzer;
_store = store;
}
public async Task<ModerationResult> ModerateVideo(Video video)
{
var riskScore = await _analyzer.CalculateRiskScore(video);
var violations = await _analyzer.DetectViolations(video);
var result = new ModerationResult
{
VideoId = video.VideoId,
RiskScore = riskScore,
Violations = violations,
ReviewedAt = DateTime.UtcNow
};
if (violations.Contains(\"CSAM\") || violations.Contains(\"Terrorism\"))
{
result.Status = ModerationStatus.Rejected;
result.ReviewedBy = \"auto_hash_match\";
}
else if (riskScore >= AUTO_REMOVE_THRESHOLD)
{
result.Status = ModerationStatus.Rejected;
result.ReviewedBy = \"auto_ml_high_confidence\";
}
else if (riskScore <= AUTO_APPROVE_THRESHOLD)
{
result.Status = ModerationStatus.Approved;
result.ReviewedBy = \"auto_ml_low_risk\";
}
else
{
result.Status = ModerationStatus.UnderReview;
result.ReviewedBy = \"queued_for_human\";
}
await _store.Save(result);
return result;
}
public async Task<List<ModerationResult>> GetPendingReviews(int limit)
{
return await _store.GetPending(limit);
}
public async Task SubmitAppeal(long videoId, string reason)
{
await _store.CreateAppeal(videoId, reason);
}
}
public class SoundTrendingCalculator
{
public double CalculateTrendingScore(
long dailyUsages, long weeklyUsages,
double growthRate, double engagementRate,
long uniqueCreators, double ageHours)
{
double velocity = weeklyUsages > 0
? (double)dailyUsages / (weeklyUsages / 7.0) : 0;
double growth = Math.Min(growthRate / 100.0, 1.0);
double diversity = Math.Min(
Math.Log10(uniqueCreators + 1) / 4.0, 1.0);
double recency = ageHours < 48
? 1.0 - (ageHours / 48.0) * 0.5 : 0;
return Math.Round(
(velocity * 0.35) +
(growth * 0.20) +
(engagementRate * 0.20) +
(diversity * 0.15) +
(recency * 0.10), 6);
}
}
}
namespace TikTokPlatform.Core.FeedResponseNS
{
using TikTokPlatform.Core.Models;
public class FeedResponse
{
public List<RankedVideo> Items { get; set; } = new();
public string NextCursor { get; set; }
public bool HasMore { get; set; }
}
}
26. Conclusion
Designing a TikTok-scale short-form video platform is one of the most challenging and rewarding system design problems in the industry. The system touches every major area of distributed systems engineering: high-throughput data ingestion, real-time transcoding, machine learning at scale, content delivery optimization, real-time recommendation systems, content moderation, multi-region deployment, and cost optimization at exabyte scale.
The key architectural decisions that define the platform include the chunked resumable upload pipeline that ensures reliable content ingestion from unreliable mobile networks, the GPU-accelerated transcoding system that achieves sub-5-second processing for near-instant publishing, the multi-stage recommendation pipeline that balances personalization quality with computational cost through candidate generation and two-stage ranking, the progressive promotion system that ensures content quality determines distribution rather than creator fame, and the multi-layer caching architecture that delivers sub-200ms feed latency to billions of users.
The content understanding pipeline is perhaps the most technically impressive component. By combining computer vision, audio analysis, natural language processing, and multi-modal embeddings, the system builds a rich understanding of every piece of content on the platform. This understanding powers both the recommendation algorithm (matching content with interested users) and the moderation pipeline (detecting policy violations before content spreads). The integration of these two use cases into a single content understanding platform is an elegant architectural choice that maximizes the value of the expensive ML inference infrastructure.
The recommendation algorithm itself represents a masterclass in balancing multiple objectives. It must maximize user satisfaction (showing content users enjoy), creator fairness (giving new creators a chance to be discovered), content diversity (preventing echo chambers), platform safety (avoiding harmful content), and business goals (maximizing session length and ad revenue). The multi-objective optimization problem with competing constraints is what makes recommendation system design both intellectually challenging and practically important.
From a cost perspective, the platform demonstrates the extraordinary expense of operating at global scale. With estimated monthly infrastructure costs exceeding $44 million, even small optimizations in codec efficiency, caching hit rates, or CDN routing can save millions of dollars annually. The trade-offs between performance and cost are present at every layer, from the choice of transcoding hardware to the storage tiering strategy for older content.
Looking forward, several trends will shape the next generation of short-form video platforms. On-device AI models will enable more sophisticated effects and real-time content understanding without server round-trips. Generative AI will create new content creation paradigms where users can create videos from text prompts. WebAssembly and edge computing will push more processing to the CDN edge, reducing latency further. And new video codecs like AV1 and VVC will continue to improve compression efficiency, reducing both storage costs and bandwidth requirements.
The skills required to design and operate a system like this span the full breadth of software engineering: distributed systems, machine learning, mobile development, media processing, data engineering, security, and operations. Mastering these skills and understanding how they interact in a complex, real-world system is what separates senior engineers from the rest. This guide has provided the foundational knowledge; the next step is to practice applying these concepts in system design interviews and real-world projects.