How to Design Video Conferencing like Zoom
Building real-time video, screen sharing, and large-scale meetings at 300M+ daily participant scale
Table of Contents
- Introduction — Zoom at Scale
- Requirements Gathering
- Capacity Estimation
- Data Model
- API Design
- High-Level Architecture
- WebRTC & Media Server Architecture
- Signaling Server (SDP & ICE)
- SFU vs MCU vs Mesh Topology
- Video Encoding & Decoding
- Adaptive Bitrate & Bandwidth Estimation
- Screen Sharing
- Audio Processing
- Meeting Recording & Cloud Storage
- Chat & Reactions During Meeting
- Waiting Room & Security
- Breakout Rooms
- Virtual Background & Effects
- Large Meeting & Webinar Mode
- Database Sharding
- Caching Strategy
- Multi-Region Design
- Cost Estimation
- Interview Q&A
- Full C# Implementation
- Zoom Rooms & Conference Room Systems
- Zoom Phone & Unified Communications
- Zoom Whiteboard & Collaboration
- Zoom SDK & Platform Integration
- Conclusion
1. Introduction — Zoom at Scale
Zoom Video Communications has fundamentally transformed how the world communicates. From a simple video conferencing tool founded by Eric Yuan in 2011, Zoom has grown into a platform handling over 300 million daily meeting participants, processing billions of minutes of video every month. The COVID-19 pandemic accelerated adoption from roughly 10 million to 300 million daily participants in a matter of months — a 30x scaling challenge that tested every layer of their architecture.
At its core, Zoom is built on WebRTC (Web Real-Time Communication), an open standard that enables peer-to-peer audio, video, and data streaming directly in web browsers without plugins. But WebRTC alone is insufficient for building a system at Zoom's scale. The real engineering marvel lies in the orchestration of media servers, signaling protocols, adaptive bitrate algorithms, recording pipelines, security layers, and global infrastructure that together deliver sub-200ms latency for millions of concurrent sessions.
In this system design deep-dive, we will architect a video conferencing platform from the ground up — covering WebRTC fundamentals, media server topologies, signaling protocols, adaptive streaming, recording, security, and the infrastructure required to serve hundreds of millions of users. Whether you are preparing for a senior+ system design interview or building the next generation of real-time communication infrastructure, this guide covers every critical detail.
2. Requirements Gathering
Functional Requirements
- One-on-One Video Calls: Two participants can establish a real-time audio/video call with HD quality (720p/1080p).
- Group Video Meetings: Support meetings with 2–50 participants (standard) and up to 1,000 in webinar mode.
- Screen Sharing: Any participant can share their entire screen or a specific application window.
- In-Meeting Chat: Text messaging during a meeting (public and private).
- Meeting Recording: Cloud and local recording with playback capability.
- Virtual Backgrounds: AI-powered background replacement and blur effects.
- Breakout Rooms: Split a large meeting into smaller sub-groups.
- Waiting Room: Lobby system for host-controlled admission.
- Reactions & Hand Raise: Non-verbal feedback mechanisms.
- Meeting Scheduling: Calendar integration and recurring meetings.
Non-Functional Requirements
| Requirement | Target | Rationale |
|---|---|---|
| Availability | 99.99% (52 min/year downtime) | Mission-critical for business communications |
| Latency (E2E) | <200ms audio, <300ms video | Conversational flow requires sub-400ms |
| Video Quality | 720p standard, 1080p premium | HD is baseline expectation |
| Audio Quality | 48kHz, Opus codec | Wideband audio for clarity |
| Scalability | 50M concurrent sessions | Peak global usage |
| Durability | No meeting data loss | Recordings and chat must be reliable |
| Security | E2E encryption, SOC2, GDPR | Enterprise compliance requirements |
| Consistency | Strong for scheduling, eventual for presence | Meeting state varies by criticality |
3. Capacity Estimation
Bandwidth Calculations
Per-participant bandwidth (receiving):
- Video (720p, 30fps): ~2.5 Mbps
- Audio (Opus, 48kHz): ~128 Kbps
- Screen share (1080p): ~1.5 Mbps
- Total per participant: ~4 Mbps down / ~3 Mbps up
Global capacity at scale:
- 300M daily participants / 8 peak hours = ~37.5M concurrent participants
- Peak concurrent (2x average): ~75M participants
- Aggregate bandwidth: 75M × 4 Mbps = 300 Tbps egress
- Storage (recordings): 10M hours/day × 1 GB/hour = 10 PB/day
QPS Estimation
| Operation | Daily QPS | Peak QPS (5x) |
|---|---|---|
| Meeting Creation | ~50M / 86400 ≈ 580 | ~2,900 |
| Join/Leave Events | ~600M / 86400 ≈ 6,900 | ~34,500 |
| Signaling Messages | ~5B / 86400 ≈ 58,000 | ~290,000 |
| Chat Messages | ~2B / 86400 ≈ 23,000 | ~115,000 |
| Recording Uploads | ~5M / 86400 ≈ 58 | ~290 |
| Auth Requests | ~100M / 86400 ≈ 1,160 | ~5,800 |
4. Data Model
Core Entities
-- User accounts and profiles
CREATE TABLE Users (
user_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
email VARCHAR(255) UNIQUE NOT NULL,
display_name VARCHAR(128) NOT NULL,
password_hash VARCHAR(512) NOT NULL,
avatar_url TEXT,
plan_tier VARCHAR(32) DEFAULT 'free',
mfa_enabled BOOLEAN DEFAULT false,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
-- Meetings with scheduling and configuration
CREATE TABLE Meetings (
meeting_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
host_user_id UUID NOT NULL REFERENCES Users(user_id),
title VARCHAR(255),
meeting_number BIGINT UNIQUE NOT NULL,
meeting_password VARCHAR(128),
start_time TIMESTAMPTZ,
end_time TIMESTAMPTZ,
duration_minutes INT DEFAULT 60,
max_participants INT DEFAULT 100,
is_recurring BOOLEAN DEFAULT false,
recurrence_rule VARCHAR(255),
waiting_room BOOLEAN DEFAULT true,
allow_record BOOLEAN DEFAULT true,
is_webinar BOOLEAN DEFAULT false,
e2e_encrypted BOOLEAN DEFAULT false,
status VARCHAR(20) DEFAULT 'scheduled',
created_at TIMESTAMPTZ DEFAULT NOW()
);
-- Participant tracking and roles
CREATE TABLE MeetingParticipants (
participant_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
meeting_id UUID NOT NULL REFERENCES Meetings(meeting_id),
user_id UUID REFERENCES Users(user_id),
guest_name VARCHAR(128),
role VARCHAR(20) DEFAULT 'participant',
join_time TIMESTAMPTZ,
leave_time TIMESTAMPTZ,
device_info JSONB,
connection_quality JSONB,
is_muted BOOLEAN DEFAULT true,
is_video_on BOOLEAN DEFAULT false,
is_screen_sharing BOOLEAN DEFAULT false,
created_at TIMESTAMPTZ DEFAULT NOW()
);
-- Cloud recordings storage
CREATE TABLE Recordings (
recording_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
meeting_id UUID NOT NULL REFERENCES Meetings(meeting_id),
host_user_id UUID NOT NULL REFERENCES Users(user_id),
file_key TEXT NOT NULL,
file_size_bytes BIGINT,
duration_seconds INT,
format VARCHAR(20) DEFAULT 'mp4',
resolution VARCHAR(20),
recording_type VARCHAR(20),
status VARCHAR(20) DEFAULT 'processing',
thumbnail_url TEXT,
created_at TIMESTAMPTZ DEFAULT NOW()
);
-- Virtual meeting rooms (persistent rooms)
CREATE TABLE MeetingRooms (
room_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
room_name VARCHAR(255) NOT NULL,
owner_user_id UUID NOT NULL REFERENCES Users(user_id),
room_type VARCHAR(20),
max_capacity INT DEFAULT 50,
sfu_cluster VARCHAR(64),
is_active BOOLEAN DEFAULT false,
current_participants INT DEFAULT 0,
settings JSONB DEFAULT '{}',
created_at TIMESTAMPTZ DEFAULT NOW()
);
-- Contact and directory management
CREATE TABLE Contacts (
contact_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
owner_user_id UUID NOT NULL REFERENCES Users(user_id),
contact_user_id UUID REFERENCES Users(user_id),
display_name VARCHAR(128),
email VARCHAR(255),
phone VARCHAR(32),
company VARCHAR(255),
is_favorite BOOLEAN DEFAULT false,
groups TEXT[],
created_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(owner_user_id, contact_user_id)
);
-- Chat messages within meetings
CREATE TABLE MeetingChatMessages (
message_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
meeting_id UUID NOT NULL REFERENCES Meetings(meeting_id),
sender_user_id UUID REFERENCES Users(user_id),
recipient_user_id UUID,
message_type VARCHAR(20) DEFAULT 'text',
content TEXT NOT NULL,
is_deleted BOOLEAN DEFAULT false,
created_at TIMESTAMPTZ DEFAULT NOW()
);
- UUID primary keys: Enable distributed ID generation without coordination.
- 10-digit meeting numbers: Human-readable dial-in numbers easy to type.
- JSONB for device_info and settings: Flexible schema for heterogeneous client devices.
- Separate Participants table: Enables historical analytics and connection quality tracking.
5. API Design
REST API Endpoints
| Method | Endpoint | Description | Auth |
|---|---|---|---|
| POST | /api/v1/auth/login | User authentication | None |
| POST | /api/v1/auth/token/refresh | Refresh JWT token | Refresh token |
| POST | /api/v1/meetings | Create/schedule a meeting | Bearer JWT |
| GET | /api/v1/meetings/{id} | Get meeting details | Bearer JWT |
| PATCH | /api/v1/meetings/{id} | Update meeting settings | Bearer JWT (host) |
| DELETE | /api/v1/meetings/{id} | Cancel/delete meeting | Bearer JWT (host) |
| POST | /api/v1/meetings/{id}/join | Join a meeting | Bearer JWT or guest |
| POST | /api/v1/meetings/{id}/end | End meeting (host) | Bearer JWT (host) |
| GET | /api/v1/meetings/{id}/participants | List participants | Bearer JWT |
| POST | /api/v1/meetings/{id}/recordings/start | Start cloud recording | Bearer JWT (host) |
| POST | /api/v1/meetings/{id}/recordings/stop | Stop cloud recording | Bearer JWT (host) |
| GET | /api/v1/meetings/{id}/recordings | List recordings | Bearer JWT |
| GET | /api/v1/users/me | Get current user profile | Bearer JWT |
| GET | /api/v1/contacts | List user contacts | Bearer JWT |
| POST | /api/v1/contacts | Add contact | Bearer JWT |
| POST | /api/v1/meetings/{id}/breakout/create | Create breakout rooms | Bearer JWT (host) |
| POST | /api/v1/meetings/{id}/breakout/assign | Assign participants | Bearer JWT (host) |
| POST | /api/v1/waiting-room/{id}/admit | Admit from waiting room | Bearer JWT (host) |
WebSocket Signaling Protocol
{
"type": "offer",
"meetingId": "abc-123",
"senderId": "user-456",
"payload": {
"sdp": "v=0\r\no=- 4611731400 2 IN IP4 127.0.0.1...",
"type": "offer"
},
"targetId": "user-789"
}
{
"type": "ice-candidate",
"meetingId": "abc-123",
"senderId": "user-456",
"payload": {
"candidate": "candidate:1 1 UDP 2122252543...",
"sdpMLineIndex": 0,
"sdpMid": "0"
}
}
{
"type": "mute-state",
"meetingId": "abc-123",
"senderId": "user-456",
"payload": {
"audioMuted": true,
"videoMuted": false
}
}
6. High-Level Architecture
- Media plane separated from signaling plane: Video/audio flows through SFU clusters, signaling through WebSocket servers — independently scalable.
- Geo-distributed media processing: SFU clusters in each major region ensure minimal latency.
- Stateless application servers: All session state in Redis, allowing horizontal scaling.
- Event-driven recording: Recording workers subscribe to streams on-demand, decoupled from live media.
7. WebRTC & Media Server Architecture
WebRTC is the foundational protocol for browser-based real-time communication. It provides APIs for capturing media streams, establishing peer connections, and transmitting audio/video/data. However, WebRTC's native peer-to-peer model only works well for 1-on-1 calls. For group meetings, we need a media server architecture.
WebRTC Connection Lifecycle
WebRTC Stack Components
| Component | Role | Implementation |
|---|---|---|
| ICE | NAT traversal and candidate gathering | STUN/TURN server cluster |
| DTLS | Key exchange for SRTP encryption | Built into WebRTC stack |
| SRTP | Encrypted media transport | AES-128-CM encryption |
| SCTP | Reliable data channel | Used for chat and file transfer |
| RTP/RTCP | Media packetization and feedback | VP8/H.264 payload types |
Media Server Selection
For production at scale, the industry has converged on Selective Forwarding Units (SFUs) as the optimal topology. Popular open-source SFU implementations include:
- mediasoup: Node.js-based SFU with C++ worker processes
- Janus Gateway: C-based general-purpose WebRTC gateway
- Pion/TURN: Go-based WebRTC stack for custom implementations
- LiveKit: Open-source SFU with built-in recording and E2EE
8. Signaling Server (SDP Exchange & ICE Candidates)
The signaling server is the coordination hub that enables WebRTC peers to discover each other and negotiate connection parameters. WebRTC deliberately leaves signaling unspecified, giving us flexibility. Zoom uses WebSocket-based signaling for low-latency bidirectional communication.
Signaling Flow Diagram
SDP (Session Description Protocol) Exchange
// SDP Offer from participant (simplified)
const offerSdp = {
sdp: `v=0
o=- 9223372036854775807 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0 1
a=msid-semantic: WMS stream_label
m=audio 9 UDP/TLS/RTP/SAVPF 111 103
c=IN IP4 0.0.0.0
a=ice-ufrag:abc123
a=ice-pwd:def456longpassword
a=fingerprint:sha-256 AA:BB:CC:...
a=mid:0
a=rtpmap:111 opus/48000/2
a=fmtp:111 minptime=10;useinbandfec=1
a=sendrecv
m=video 9 UDP/TLS/RTP/SAVPF 96 97
c=IN IP4 0.0.0.0
a=ice-ufrag:abc123
a=ice-pwd:def456longpassword
a=fingerprint:sha-256 AA:BB:CC:...
a=mid:1
a=rtpmap:96 VP8/90000
a=rtpmap:97 H264/90000
a=sendrecv
a=ssrc:1234567890 cname:unique_cname`,
type: 'offer'
};
ICE Candidate Processing
ICE discovers the best network path between two peers. Candidates are gathered in three categories:
| Candidate Type | Description | Priority | Success Rate |
|---|---|---|---|
| Host | Local network interface IP | Highest | Same LAN only |
| Server Reflexive (srflx) | Public IP via STUN server | High | ~85% |
| Relay (TURN) | Relayed through TURN server | Lowest | ~99% |
9. SFU vs MCU vs Mesh Topology
The choice of media server topology is one of the most critical architectural decisions. Each approach has distinct trade-offs in bandwidth, CPU usage, scalability, and video quality.
| Property | Mesh | MCU | SFU |
|---|---|---|---|
| Server CPU | None | Very High | Low |
| Client Upload | N-1 streams | 1 stream | 1 stream |
| Client Download | N-1 streams | 1 mixed | N-1 forwarded |
| Video Quality | Best | Worst (re-encode) | Best |
| Max Participants | 4-6 | 100+ | 1000+ |
| Latency | Lowest | Highest | Low |
| Scalability | Poor | Moderate | Excellent |
Cascading SFU Architecture for Large Rooms
In a cascaded SFU architecture, each instance handles a subset of participants. Media streams are forwarded between SFU instances via high-bandwidth inter-datacenter links, allowing thousands of participants in a single meeting while keeping each participant connected to a geographically nearby SFU.
10. Video Encoding & Decoding (VP8, H.264, AV1)
Video codec selection impacts quality, bandwidth, CPU usage, and hardware acceleration compatibility.
| Codec | Bitrate (720p30) | Quality (VMAF) | HW Acceleration | License |
|---|---|---|---|---|
| VP8 | 1.5 Mbps | 85 | Partial | Free |
| VP9 | 1.0 Mbps | 90 | Intel QSV, NVIDIA | Free |
| H.264 | 1.5 Mbps | 87 | Universal | Licensed |
| H.265/HEVC | 0.8 Mbps | 92 | Modern GPUs | Licensed |
| AV1 | 0.7 Mbps | 93 | RTX 40+, Intel Arc | Free (royalty-free) |
Codec Negotiation Strategy
Scalable Video Coding (SVC): VP9 and AV1 support SVC, encoding multiple quality layers in a single stream. The SFU forwards only appropriate layers to each subscriber based on bandwidth and screen size.
- Base layer: 360p at 500 Kbps — minimum viable quality
- Enhancement 1: +480p at +300 Kbps — HD for desktop
- Enhancement 2: +720p at +500 Kbps — full HD
- Enhancement 3: +1080p at +1.0 Mbps — premium
11. Adaptive Bitrate & Bandwidth Estimation
Adaptive bitrate (ABR) is essential for maintaining call quality across varying network conditions.
WebRTC includes a built-in Bandwidth Estimation module based on the GCC (Google Congestion Control) algorithm using transport-wide sequence numbers and delay-based estimation.
| Parameter | Value | Purpose |
|---|---|---|
| Minimum bitrate | 100 Kbps | Audio-only floor |
| Maximum bitrate | 3 Mbps | Prevent overwhelming receivers |
| Initial bitrate | 300 Kbps | Conservative start |
| Loss threshold | 2% | Trigger downshift |
| RTT threshold | 300ms | Quality warning |
12. Screen Sharing
Screen sharing presents unique challenges — high-resolution content (1080p–4K), sharp text and UI elements, and different motion characteristics require specialized encoding.
Optimization Strategies:
- Content-aware encoding: Detect video vs static text and adjust codec parameters
- Region of Interest: Higher bitrate near cursor, lower for static areas
- Lossless mode: For pixel-perfect text (design tools, code editors)
- Frame rate adaptation: 5–10 fps for static, 15–30 fps for video playback
13. Audio Processing (Noise Suppression, Echo Cancellation)
Audio quality is often more critical than video — users can tolerate degraded video but will immediately notice poor audio.
| Stage | Algorithm | Purpose | Latency |
|---|---|---|---|
| AEC | Adaptive filter (NLMS) | Remove speaker output from mic | ~5ms |
| Noise Suppression | RNNoise (deep learning) | Remove background noise | ~3ms |
| AGC | RMS normalization | Normalize volume levels | ~2ms |
| VAD | Energy + spectral analysis | Detect speech for bandwidth saving | ~1ms |
| Stereo | Spatial audio processing | Speaker identification cues | ~5ms |
Zoom's AI Audio: AI noise suppression trained on thousands of noise profiles (keyboard, baby crying, construction, coffee shop), music mode that disables suppression, and multi-path audio combining multiple devices.
14. Meeting Recording & Cloud Storage
Cloud recording captures meeting audio, video, screen share, and chat into downloadable files.
| Resolution | Bitrate | Storage/Hour | Daily (10M hours) |
|---|---|---|---|
| 1080p | 3 Mbps | 1.35 GB | 13.5 PB |
| 720p | 1.5 Mbps | 675 MB | 6.75 PB |
| 480p | 600 Kbps | 270 MB | 2.7 PB |
| Audio only | 128 Kbps | 57.6 MB | 576 TB |
15. Chat & Reactions During Meeting
In-meeting chat operates on a separate data plane from media, using WebSocket connections for real-time delivery.
Reactions: Ephemeral overlays (thumbs up, heart, clap) flow through the signaling WebSocket using lightweight messages optimized for high-frequency, low-latency delivery.
{
"type": "reaction",
"meetingId": "abc-123",
"senderId": "user-456",
"payload": {
"emoji": "thumbsup",
"timestamp": 1689345678,
"expiresAt": 1689345683
}
}
16. Waiting Room & Security
Security Features:
- E2E Encryption: Insertable Streams API for participant-to-participant encryption
- Meeting passwords: Minimum 6 characters with complexity requirements
- Lock meeting: Prevent new participants after start
- Remove participant: Eject and block re-entry
- Watermarking: Invisible screen share watermarking for leak tracing
- Compliance: SOC 2 Type II, HIPAA, GDPR, FedRAMP
17. Breakout Rooms
Breakout rooms split a large meeting into smaller groups — one of the most complex features from a systems perspective.
Each breakout room is a separate meeting with its own SFU assignment. The host can broadcast to all rooms, join any room, and close all rooms. Target transition latency: under 2 seconds.
18. Virtual Background & Effects
| Component | Model | Inference Time | Platform |
|---|---|---|---|
| Person Segmentation | MediaPipe Selfie Segmentation | ~8ms/frame | WebGL / WASM |
| Portrait Distinction | Custom CNN | ~3ms/frame | Core ML / NNAPI |
| Background Replacement | Alpha blending + blur | ~2ms/frame | GPU Shader |
| Lighting Normalization | Color transfer network | ~5ms/frame | Metal / Vulkan |
Supported Effects: Virtual background images/videos, background blur (adjustable), AR video filters, appearance touch-up (skin smoothing), and studio effects (eyebrow, lip color — premium).
19. Large Meeting & Webinar Mode
| Feature | Standard (≤50) | Large (≤1000) | Webinar (≤50K) |
|---|---|---|---|
| Video Distribution | SFU forwarding | Cascading SFU | CDN + WebRTC hybrid |
| Participant Video | Self-selected | Host-controlled gallery | Panelists only |
| Audio Input | All participants | Muted by default | Panelists only |
| Chat | Public + private | Public + host-only | Moderated Q&A |
20. Database Sharding
| Table | Shard Key | Strategy | Shards |
|---|---|---|---|
| Users | user_id (UUID) | Consistent hashing | 64 |
| Meetings | meeting_id (UUID) | Consistent hashing | 128 |
| MeetingParticipants | meeting_id | Co-locate with Meetings | 128 |
| Recordings | host_user_id | Co-locate with Users | 64 |
| MeetingRooms | room_id (UUID) | Consistent hashing | 32 |
| Contacts | owner_user_id | Co-locate with Users | 64 |
| ChatMessages | meeting_id | Time-bucketed | 256 |
Co-location Strategy: Related tables on the same shard enable single-shard joins. Hot partitions from popular meetings are mitigated with read replicas, Redis caching, and request coalescing.
21. Caching Strategy
| Layer | Technology | TTL | Cached Data |
|---|---|---|---|
| L1 - Browser | Service Worker + IndexedDB | Session | Codecs, UI assets, preferences |
| L2 - CDN | CloudFront / Akamai | 1hr–7 days | Assets, thumbnails, files |
| L3 - Application | Redis Cluster | 5min–24hr | Sessions, meeting state, SFU assignments |
| L4 - Database | PostgreSQL buffer | N/A | Hot rows |
Redis Data Structures for Meeting State
// Redis key patterns for real-time meeting state
string meetingKey = $"meeting:{meetingId}";
await redis.HashSetAsync(meetingKey, new HashEntry[] {
new("status", "active"),
new("host_id", hostUserId.ToString()),
new("participant_count", "42"),
new("sfu_cluster", "us-east-1")
});
await redis.KeyExpireAsync(meetingKey, TimeSpan.FromHours(4));
// Participant set
string participantsKey = $"meeting:{meetingId}:participants";
await redis.SetAddAsync(participantsKey,
participantIds.Select(id => (RedisValue)id.ToString()).ToArray());
// SFU assignment map
string sfuKey = $"meeting:{meetingId}:sfu:assignments";
await redis.HashSetAsync(sfuKey, participantIds.Select(id =>
new HashEntry(id.ToString(), "sfu-us-east-1a")).ToArray());
// Signaling message queue
string signalQueueKey = $"signal:{participantId}";
await redis.ListLeftPushAsync(signalQueueKey, serializedMessage);
await redis.KeyExpireAsync(signalQueueKey, TimeSpan.FromMinutes(5));
22. Multi-Region Design
Strategy: GeoDNS for user routing, active-active SFU clusters for media, active-passive API control plane, and async Cassandra replication for eventual consistency (<100ms across regions).
23. Cost Estimation
| Component | Monthly Cost | Notes |
|---|---|---|
| SFU Servers | $8–12M | ~5,000 bare-metal with 10Gbps NICs |
| TURN Servers | $3–5M | ~1,000 high-bandwidth servers |
| STUN Servers | $200K | ~500 lightweight at CDN PoPs |
| API Servers | $1–2M | Kubernetes clusters |
| Signaling Servers | $500K–1M | Stateless WebSocket handlers |
| PostgreSQL | $500K–1M | Multi-AZ RDS, read replicas |
| Redis Cluster | $300K–500K | In-memory session state |
| Cassandra | $500K–1M | Multi-region, high writes |
| S3 Storage | $5–8M | Petabyte-scale with lifecycle |
| CDN | $2–3M | Global edge caching |
| Transcoding GPU | $2–3M | AWS G5 instances |
| Bandwidth Egress | $10–15M | 300 Tbps peak |
| AI/ML | $1–2M | Virtual BG, noise suppression |
| Monitoring | $500K–1M | Datadog, PagerDuty |
| Total | $35–55M/mo | ~$420–660M annually |
24. Interview Q&A
25. Full C# Implementation
Production-grade C# implementation of the core video conferencing system components including signaling server, SFU session manager, meeting orchestrator, recording service, and adaptive bitrate controller.
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;
using System.Net.WebSockets;
using System.Text;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Logging;
namespace VideoConferencing.Core
{
// =============================================
// Domain Models
// =============================================
public enum ParticipantRole { Host, CoHost, Participant, Panelist, Viewer }
public enum MeetingStatus { Scheduled, Active, Ended, Cancelled }
public enum VideoCodec { VP8, VP9, H264, H265, AV1 }
public enum RecordingStatus { Processing, Ready, Failed, Deleted }
public enum SignalMessageType
{
Join, Leave, Offer, Answer, IceCandidate,
MuteState, ScreenShare, Reaction, Chat,
Broadcast, EndMeeting, BreakoutAssign
}
public record IceCandidate(string Candidate, int SdpMLineIndex, string SdpMid);
public record SdpPayload(string Sdp, string Type);
public record MuteState(bool AudioMuted, bool VideoMuted);
public record ReactionPayload(string Emoji, long ExpiresAt);
public record DeviceInfo(
string Platform, string Browser, string OsVersion,
bool HasGpu, bool SupportsAv1, bool SupportsVp9,
int CpuCores, long MemoryMb, bool HasCamera, bool HasMicrophone);
public record ConnectionQuality(
double BitrateKbps, double PacketLossPercent,
double RoundTripMs, double JitterMs);
public class Participant
{
public Guid Id { get; init; } = Guid.NewGuid();
public Guid? UserId { get; init; }
public string DisplayName { get; init; } = string.Empty;
public ParticipantRole Role { get; init; } = ParticipantRole.Participant;
public WebSocket SignalingSocket { get; set; } = null!;
public DeviceInfo? Device { get; set; }
public ConnectionQuality? Quality { get; set; }
public bool IsAudioMuted { get; set; } = true;
public bool IsVideoOn { get; set; } = false;
public bool IsScreenSharing { get; set; } = false;
public string SfuTransportId { get; set; } = string.Empty;
public DateTime JoinedAt { get; init; } = DateTime.UtcNow;
public DateTime? LeftAt { get; set; }
}
public class Meeting
{
public Guid Id { get; init; } = Guid.NewGuid();
public long MeetingNumber { get; init; }
public Guid HostUserId { get; init; }
public string Title { get; init; } = string.Empty;
public string? Password { get; init; }
public MeetingStatus Status { get; set; } = MeetingStatus.Scheduled;
public int MaxParticipants { get; init; } = 100;
public bool WaitingRoomEnabled { get; init; } = true;
public bool RecordingEnabled { get; init; } = true;
public bool IsWebinar { get; init; } = false;
public bool E2eEncrypted { get; init; } = false;
public string AssignedSfuCluster { get; set; } = string.Empty;
public DateTime CreatedAt { get; init; } = DateTime.UtcNow;
public DateTime? StartedAt { get; set; }
public DateTime? EndedAt { get; set; }
public List<BreakoutRoom> BreakoutRooms { get; init; } = new();
}
public class BreakoutRoom
{
public Guid RoomId { get; init; } = Guid.NewGuid();
public string Name { get; init; } = string.Empty;
public string AssignedSfu { get; set; } = string.Empty;
public List<Guid> AssignedParticipants { get; init; } = new();
public bool IsActive { get; set; } = false;
}
public class Recording
{
public Guid Id { get; init; } = Guid.NewGuid();
public Guid MeetingId { get; init; }
public Guid HostUserId { get; init; }
public string FileKey { get; set; } = string.Empty;
public long FileSizeBytes { get; set; }
public int DurationSeconds { get; set; }
public RecordingStatus Status { get; set; } = RecordingStatus.Processing;
public DateTime CreatedAt { get; init; } = DateTime.UtcNow;
}
// =============================================
// Codec Negotiation and Adaptive Bitrate
// =============================================
public class CodecProfile
{
public VideoCodec Codec { get; init; }
public int MaxBitrateKbps { get; init; }
public int MinBitrateKbps { get; init; }
public int[] SupportedResolutions { get; init; } = Array.Empty<int>();
public bool SupportsSVC { get; init; }
public bool HasHardwareAcceleration { get; init; }
}
public class BandwidthEstimator
{
private double _currentBitrateKbps = 300;
private double _targetBitrateKbps = 300;
private readonly double _minBitrateKbps = 100;
private readonly double _maxBitrateKbps = 3000;
private readonly double _lossThresholdPercent = 2.0;
private readonly double _rttThresholdMs = 300.0;
private readonly object _lock = new();
public double CurrentBitrateKbps
{
get { lock (_lock) return _currentBitrateKbps; }
}
public void UpdateEstimate(double measuredBandwidthKbps,
double packetLossPercent, double rttMs, double jitterMs)
{
lock (_lock)
{
if (packetLossPercent > _lossThresholdPercent)
{
_targetBitrateKbps = Math.Max(_minBitrateKbps,
_currentBitrateKbps * 0.7);
}
else if (rttMs > _rttThresholdMs)
{
_targetBitrateKbps = Math.Max(_minBitrateKbps,
_currentBitrateKbps * 0.85);
}
else
{
_targetBitrateKbps = Math.Min(_maxBitrateKbps,
measuredBandwidthKbps * 0.85);
}
_currentBitrateKbps += (_targetBitrateKbps - _currentBitrateKbps) * 0.15;
_currentBitrateKbps = Math.Clamp(_currentBitrateKbps,
_minBitrateKbps, _maxBitrateKbps);
}
}
public (int resolution, int fps, int bitrateKbps) GetEncodingParams()
{
var bitrate = CurrentBitrateKbps;
return bitrate switch
{
>= 2500 => (1080, 30, (int)bitrate),
>= 1200 => (720, 30, (int)bitrate),
>= 500 => (480, 20, (int)bitrate),
>= 200 => (360, 15, (int)bitrate),
_ => (240, 7, (int)bitrate)
};
}
}
public class CodecNegotiator
{
private static readonly List<CodecProfile> CodecProfiles = new()
{
new CodecProfile
{
Codec = VideoCodec.AV1, MaxBitrateKbps = 3000,
MinBitrateKbps = 100,
SupportedResolutions = new[] { 240, 360, 480, 720, 1080 },
SupportsSVC = true, HasHardwareAcceleration = true
},
new CodecProfile
{
Codec = VideoCodec.VP9, MaxBitrateKbps = 3000,
MinBitrateKbps = 100,
SupportedResolutions = new[] { 240, 360, 480, 720, 1080 },
SupportsSVC = true, HasHardwareAcceleration = true
},
new CodecProfile
{
Codec = VideoCodec.H264, MaxBitrateKbps = 3000,
MinBitrateKbps = 100,
SupportedResolutions = new[] { 240, 360, 480, 720, 1080 },
SupportsSVC = false, HasHardwareAcceleration = true
},
new CodecProfile
{
Codec = VideoCodec.VP8, MaxBitrateKbps = 2000,
MinBitrateKbps = 100,
SupportedResolutions = new[] { 240, 360, 480, 720 },
SupportsSVC = false, HasHardwareAcceleration = false
}
};
public CodecProfile Negotiate(DeviceInfo localDevice, DeviceInfo remoteDevice)
{
var localSupported = GetLocallySupportedCodecs(localDevice);
var remoteSupported = GetLocallySupportedCodecs(remoteDevice);
return localSupported.FirstOrDefault(c =>
remoteSupported.Any(r => r.Codec == c.Codec))
?? CodecProfiles.Last();
}
private List<CodecProfile> GetLocallySupportedCodecs(DeviceInfo device)
{
var codecs = new List<CodecProfile>();
if (device.SupportsAv1)
codecs.Add(CodecProfiles.First(c => c.Codec == VideoCodec.AV1));
if (device.SupportsVp9)
codecs.Add(CodecProfiles.First(c => c.Codec == VideoCodec.VP9));
codecs.Add(CodecProfiles.First(c => c.Codec == VideoCodec.H264));
codecs.Add(CodecProfiles.First(c => c.Codec == VideoCodec.VP8));
return codecs;
}
}
// =============================================
// Signaling Server
// =============================================
public class SignalMessage
{
public SignalMessageType Type { get; init; }
public Guid MeetingId { get; init; }
public Guid SenderId { get; init; }
public Guid? TargetId { get; init; }
public string Payload { get; init; } = string.Empty;
public long Timestamp { get; init; } =
DateTimeOffset.UtcNow.ToUnixTimeMilliseconds();
}
public interface ISignalingServer
{
Task HandleConnectionAsync(WebSocket socket,
Guid meetingId, Guid participantId);
Task BroadcastToMeetingAsync(Guid meetingId,
SignalMessage message, Guid? excludeId = null);
Task SendToParticipantAsync(Guid participantId, SignalMessage message);
}
public class SignalingServer : ISignalingServer
{
private readonly ILogger<SignalingServer> _logger;
private readonly ConcurrentDictionary<Guid,
ConcurrentDictionary<Guid, WebSocket>> _meetingSockets = new();
private readonly ConcurrentDictionary<Guid, Guid> _participantToMeeting = new();
public SignalingServer(ILogger<SignalingServer> logger)
{
_logger = logger;
}
public async Task HandleConnectionAsync(WebSocket socket,
Guid meetingId, Guid participantId)
{
_meetingSockets.GetOrAdd(meetingId, _ => new())
.TryAdd(participantId, socket);
_participantToMeeting.TryAdd(participantId, meetingId);
_logger.LogInformation(
"Participant {P} connected to meeting {M}",
participantId, meetingId);
var buffer = new byte[65536];
try
{
while (socket.State == WebSocketState.Open)
{
var result = await socket.ReceiveAsync(
new ArraySegment<byte>(buffer),
CancellationToken.None);
if (result.MessageType == WebSocketMessageType.Close)
{
await socket.CloseAsync(
WebSocketCloseStatus.NormalClosure,
"Client closed", CancellationToken.None);
break;
}
if (result.MessageType == WebSocketMessageType.Text)
{
var json = Encoding.UTF8.GetString(
buffer, 0, result.Count);
var message = JsonSerializer
.Deserialize<SignalMessage>(json);
if (message != null)
await ProcessSignalMessageAsync(message);
}
}
}
catch (WebSocketException ex)
{
_logger.LogWarning(ex,
"WebSocket error for participant {P}", participantId);
}
finally
{
RemoveParticipant(meetingId, participantId);
_participantToMeeting.TryRemove(participantId, out _);
_logger.LogInformation(
"Participant {P} disconnected from {M}",
participantId, meetingId);
}
}
private async Task ProcessSignalMessageAsync(SignalMessage message)
{
switch (message.Type)
{
case SignalMessageType.Offer:
case SignalMessageType.Answer:
case SignalMessageType.IceCandidate:
if (message.TargetId.HasValue)
await SendToParticipantAsync(
message.TargetId.Value, message);
break;
case SignalMessageType.MuteState:
case SignalMessageType.ScreenShare:
case SignalMessageType.Reaction:
await BroadcastToMeetingAsync(
message.MeetingId, message, message.SenderId);
break;
case SignalMessageType.Chat:
if (message.TargetId.HasValue)
await SendToParticipantAsync(
message.TargetId.Value, message);
else
await BroadcastToMeetingAsync(
message.MeetingId, message, message.SenderId);
break;
case SignalMessageType.Broadcast:
await BroadcastToMeetingAsync(
message.MeetingId, message);
break;
case SignalMessageType.EndMeeting:
await BroadcastToMeetingAsync(
message.MeetingId, message);
if (_meetingSockets.TryRemove(
message.MeetingId, out var sockets))
{
foreach (var kvp in sockets)
{
try { await kvp.Value.CloseAsync(
WebSocketCloseStatus.NormalClosure,
"Meeting ended", CancellationToken.None); }
catch { }
}
}
break;
}
}
public async Task BroadcastToMeetingAsync(Guid meetingId,
SignalMessage message, Guid? excludeId = null)
{
if (!_meetingSockets.TryGetValue(meetingId, out var sockets))
return;
var json = JsonSerializer.Serialize(message);
var bytes = Encoding.UTF8.GetBytes(json);
var tasks = sockets.Where(kvp =>
kvp.Key != excludeId &&
kvp.Value.State == WebSocketState.Open
).Select(async kvp =>
{
try
{
await kvp.Value.SendAsync(
new ArraySegment<byte>(bytes),
WebSocketMessageType.Text, true,
CancellationToken.None);
}
catch (Exception ex)
{
_logger.LogWarning(ex,
"Failed to send to {Id}", kvp.Key);
}
});
await Task.WhenAll(tasks);
}
public async Task SendToParticipantAsync(
Guid participantId, SignalMessage message)
{
if (_participantToMeeting.TryGetValue(
participantId, out var meetingId) &&
_meetingSockets.TryGetValue(meetingId, out var sockets) &&
sockets.TryGetValue(participantId, out var socket) &&
socket.State == WebSocketState.Open)
{
var json = JsonSerializer.Serialize(message);
var bytes = Encoding.UTF8.GetBytes(json);
await socket.SendAsync(
new ArraySegment<byte>(bytes),
WebSocketMessageType.Text, true,
CancellationToken.None);
}
}
public int GetMeetingParticipantCount(Guid meetingId)
{
return _meetingSockets.TryGetValue(meetingId, out var s)
? s.Count(kvp => kvp.Value.State == WebSocketState.Open)
: 0;
}
private void RemoveParticipant(Guid meetingId, Guid participantId)
{
if (_meetingSockets.TryGetValue(meetingId, out var sockets))
{
sockets.TryRemove(participantId, out _);
if (sockets.IsEmpty)
_meetingSockets.TryRemove(meetingId, out _);
}
}
}
// =============================================
// SFU Session Manager
// =============================================
public class SfuSession
{
public string TransportId { get; init; } =
Guid.NewGuid().ToString("N");
public Guid ParticipantId { get; init; }
public string IceUfrag { get; init; } = RandomString(4);
public string IcePwd { get; init; } = RandomString(22);
public string DtlsFingerprint { get; init; } =
"sha-256 " + string.Join(":",
Enumerable.Range(0, 32)
.Select(_ => Random.Shared.Next(256)
.ToString("X2")));
public bool IsConnected { get; set; } = false;
public DateTime CreatedAt { get; init; } = DateTime.UtcNow;
private static string RandomString(int len)
{
const string c = "ABCDEFGHIJKLMNOPQRSTUVWXYZ" +
"abcdefghijklmnopqrstuvwxyz0123456789";
return new string(Enumerable.Repeat(c, len)
.Select(s => s[Random.Shared.Next(s.Length)]).ToArray());
}
}
public class SfuClusterInfo(
string name, int maxCap, double bwGbps)
{
public string Name { get; } = name;
public int MaxCapacity { get; } = maxCap;
public double BandwidthGbps { get; } = bwGbps;
public int CurrentLoad { get; set; }
}
public interface ISfuManager
{
Task<SfuSession> AllocateTransportAsync(
Guid meetingId, Guid participantId);
Task DeallocateTransportAsync(string transportId);
Task<string> GenerateSdpOfferAsync(
string transportId, VideoCodec codec);
Task ProcessSdpAnswerAsync(string transportId, string sdp);
Task<string> GetBestSfuClusterAsync(string region);
}
public class SfuManager : ISfuManager
{
private readonly ILogger<SfuManager> _logger;
private readonly ConcurrentDictionary<string, SfuSession>
_sessions = new();
private readonly ConcurrentDictionary<Guid, List<string>>
_meetingTransports = new();
private readonly Dictionary<string, SfuClusterInfo>
_clusters = new()
{
["us-east-1"] = new("us-east-1", 5000, 12000),
["us-west-2"] = new("us-west-2", 4000, 10000),
["eu-west-1"] = new("eu-west-1", 4500, 11000),
["ap-south-1"] = new("ap-south-1", 3000, 8000),
["ap-northeast-1"] = new("ap-northeast-1", 3500, 9500),
};
public SfuManager(ILogger<SfuManager> logger)
{
_logger = logger;
}
public Task<SfuSession> AllocateTransportAsync(
Guid meetingId, Guid participantId)
{
var session = new SfuSession
{
ParticipantId = participantId
};
_sessions.TryAdd(session.TransportId, session);
_meetingTransports.GetOrAdd(meetingId, _ => new())
.Add(session.TransportId);
_logger.LogInformation(
"Allocated SFU transport {T} for {P} in {M}",
session.TransportId, participantId, meetingId);
return Task.FromResult(session);
}
public Task DeallocateTransportAsync(string transportId)
{
if (_sessions.TryRemove(transportId, out var session))
{
_logger.LogInformation(
"Deallocated SFU transport {T}", transportId);
if (_meetingTransports.TryGetValue(
session.ParticipantId, out var t))
t.Remove(transportId);
}
return Task.CompletedTask;
}
public Task<string> GenerateSdpOfferAsync(
string transportId, VideoCodec codec)
{
if (!_sessions.TryGetValue(transportId, out var s))
throw new InvalidOperationException(
$"Transport {transportId} not found");
var pt = codec switch
{
VideoCodec.VP8 => 96,
VideoCodec.VP9 => 98,
VideoCodec.H264 => 102,
VideoCodec.AV1 => 110,
_ => 96
};
var cn = codec switch
{
VideoCodec.VP8 => "VP8/90000",
VideoCodec.VP9 => "VP9/90000",
VideoCodec.H264 => "H264/90000",
VideoCodec.AV1 => "AV1/90000",
_ => "VP8/90000"
};
var sdp = $"v=0\n" +
$"o=- {DateTimeOffset.UtcNow.ToUnixTimeSeconds()}" +
$" 2 IN IP4 127.0.0.1\n" +
$"s=SFU\n" +
$"t=0 0\n" +
$"a=group:BUNDLE 0 1\n" +
$"m=audio 9 UDP/TLS/RTP/SAVPF 111\n" +
$"c=IN IP4 0.0.0.0\n" +
$"a=ice-ufrag:{s.IceUfrag}\n" +
$"a=ice-pwd:{s.IcePwd}\n" +
$"a=fingerprint:sha-256 {s.DtlsFingerprint}\n" +
$"a=mid:0\n" +
$"a=rtpmap:111 opus/48000/2\n" +
$"a=sendrecv\n" +
$"m=video 9 UDP/TLS/RTP/SAVPF {pt}\n" +
$"c=IN IP4 0.0.0.0\n" +
$"a=ice-ufrag:{s.IceUfrag}\n" +
$"a=ice-pwd:{s.IcePwd}\n" +
$"a=fingerprint:sha-256 {s.DtlsFingerprint}\n" +
$"a=mid:1\n" +
$"a=rtpmap:{pt} {cn}\n" +
$"a=sendrecv";
return Task.FromResult(sdp);
}
public Task ProcessSdpAnswerAsync(
string transportId, string sdp)
{
if (_sessions.TryGetValue(transportId, out var s))
{
s.IsConnected = true;
_logger.LogInformation(
"SDP answer for {T}", transportId);
}
return Task.CompletedTask;
}
public Task<string> GetBestSfuClusterAsync(string region)
{
var mapping = new Dictionary<string, string>
{
["us"] = "us-east-1", ["ca"] = "us-east-1",
["eu"] = "eu-west-1", ["uk"] = "eu-west-1",
["de"] = "eu-west-1", ["in"] = "ap-south-1",
["sg"] = "ap-northeast-1", ["jp"] = "ap-northeast-1",
};
var cluster = mapping.TryGetValue(
region.ToLower(), out var m) ? m : "us-east-1";
if (_clusters.TryGetValue(cluster, out var info) &&
info.CurrentLoad >= info.MaxCapacity * 0.9)
{
cluster = _clusters
.OrderBy(c => (double)c.Value.CurrentLoad /
c.Value.MaxCapacity).First().Key;
}
return Task.FromResult(cluster);
}
}
// =============================================
// Meeting Orchestrator
// =============================================
public interface IMeetingOrchestrator
{
Task<Meeting> CreateMeetingAsync(Guid hostUserId, string title,
string? password = null, int maxParticipants = 100,
bool waitingRoom = true);
Task<Participant> JoinMeetingAsync(long meetingNumber,
Guid? userId, string displayName, string password,
DeviceInfo device, string region);
Task<bool> AdmitFromWaitingRoomAsync(
Guid meetingId, Guid participantId);
Task EndMeetingAsync(Guid meetingId, Guid hostUserId);
Task<BreakoutRoom> CreateBreakoutRoomAsync(
Guid meetingId, string roomName);
Task AssignToBreakoutAsync(Guid meetingId,
Guid participantId, Guid roomId);
Task CloseAllBreakoutsAsync(Guid meetingId);
}
public class MeetingOrchestrator : IMeetingOrchestrator
{
private readonly ILogger<MeetingOrchestrator> _logger;
private readonly ISignalingServer _signaling;
private readonly ISfuManager _sfuManager;
private readonly CodecNegotiator _codecNeg = new();
private readonly Random _random = new();
private readonly ConcurrentDictionary<Guid, Meeting>
_meetings = new();
private readonly ConcurrentDictionary<long, Guid>
_meetingNumberMap = new();
private readonly ConcurrentDictionary<Guid,
ConcurrentDictionary<Guid, Participant>> _participants = new();
private readonly ConcurrentDictionary<Guid,
ConcurrentDictionary<Guid, Participant>> _waitingRoom = new();
public MeetingOrchestrator(
ILogger<MeetingOrchestrator> logger,
ISignalingServer signaling, ISfuManager sfuManager)
{
_logger = logger;
_signaling = signaling;
_sfuManager = sfuManager;
}
public async Task<Meeting> CreateMeetingAsync(
Guid hostUserId, string title, string? password = null,
int maxParticipants = 100, bool waitingRoom = true)
{
var num = GenerateMeetingNumber();
var cluster = await _sfuManager
.GetBestSfuClusterAsync("us");
var meeting = new Meeting
{
HostUserId = hostUserId,
Title = title,
MeetingNumber = num,
Password = password,
MaxParticipants = maxParticipants,
WaitingRoomEnabled = waitingRoom,
AssignedSfuCluster = cluster,
Status = MeetingStatus.Scheduled
};
_meetings.TryAdd(meeting.Id, meeting);
_meetingNumberMap.TryAdd(num, meeting.Id);
_logger.LogInformation(
"Created meeting {Id} (#{N}) on {C}",
meeting.Id, num, cluster);
return meeting;
}
public async Task<Participant> JoinMeetingAsync(
long meetingNumber, Guid? userId, string displayName,
string password, DeviceInfo device, string clientRegion)
{
if (!_meetingNumberMap.TryGetValue(
meetingNumber, out var mid))
throw new InvalidOperationException(
"Meeting not found");
if (!_meetings.TryGetValue(mid, out var meeting))
throw new InvalidOperationException(
"Meeting not found");
if (meeting.Status == MeetingStatus.Ended)
throw new InvalidOperationException(
"Meeting has ended");
if (meeting.Password != null &&
meeting.Password != password)
throw new UnauthorizedAccessException(
"Invalid password");
var list = _participants.GetOrAdd(mid, _ => new());
if (list.Count >= meeting.MaxParticipants)
throw new InvalidOperationException("Meeting is full");
var role = (userId == meeting.HostUserId)
? ParticipantRole.Host
: ParticipantRole.Participant;
var participant = new Participant
{
UserId = userId,
DisplayName = displayName,
Role = role,
Device = device
};
if (meeting.WaitingRoomEnabled &&
role != ParticipantRole.Host)
{
var w = _waitingRoom.GetOrAdd(mid, _ => new());
w.TryAdd(participant.Id, participant);
_logger.LogInformation(
"{Name} in waiting room for {M}",
displayName, mid);
return participant;
}
return await AdmitParticipantAsync(
meeting, participant);
}
public async Task<bool> AdmitFromWaitingRoomAsync(
Guid meetingId, Guid participantId)
{
if (!_waitingRoom.TryGetValue(meetingId, out var w))
return false;
if (!w.TryRemove(participantId, out var p))
return false;
if (!_meetings.TryGetValue(meetingId, out var m))
return false;
await AdmitParticipantAsync(m, p);
return true;
}
private async Task<Participant> AdmitParticipantAsync(
Meeting meeting, Participant participant)
{
var list = _participants.GetOrAdd(
meeting.Id, _ => new());
list.TryAdd(participant.Id, participant);
if (meeting.Status == MeetingStatus.Scheduled)
{
meeting.Status = MeetingStatus.Active;
meeting.StartedAt = DateTime.UtcNow;
}
var sfu = await _sfuManager.AllocateTransportAsync(
meeting.Id, participant.Id);
participant.SfuTransportId = sfu.TransportId;
var dev = participant.Device ?? new DeviceInfo(
"unknown", "unknown", "unknown",
false, false, false, 4, 4096, true, true);
var codec = _codecNeg.Negotiate(dev, dev);
var offer = await _sfuManager.GenerateSdpOfferAsync(
sfu.TransportId, codec.Codec);
await _signaling.SendToParticipantAsync(
participant.Id, new SignalMessage
{
Type = SignalMessageType.Offer,
MeetingId = meeting.Id,
SenderId = meeting.HostUserId,
TargetId = participant.Id,
Payload = offer
});
await _signaling.BroadcastToMeetingAsync(
meeting.Id, new SignalMessage
{
Type = SignalMessageType.Join,
MeetingId = meeting.Id,
SenderId = participant.Id,
Payload = JsonSerializer.Serialize(new
{
participantId = participant.Id,
displayName = participant.DisplayName,
role = participant.Role.ToString()
})
}, participant.Id);
_logger.LogInformation(
"{Name} joined meeting {M}",
participant.DisplayName, meeting.Id);
return participant;
}
public async Task EndMeetingAsync(
Guid meetingId, Guid hostUserId)
{
if (!_meetings.TryGetValue(meetingId, out var m))
throw new InvalidOperationException(
"Meeting not found");
if (m.HostUserId != hostUserId)
throw new UnauthorizedAccessException(
"Only host can end");
m.Status = MeetingStatus.Ended;
m.EndedAt = DateTime.UtcNow;
await _signaling.BroadcastToMeetingAsync(
meetingId, new SignalMessage
{
Type = SignalMessageType.EndMeeting,
MeetingId = meetingId,
SenderId = hostUserId
});
if (_participants.TryRemove(meetingId, out var parts))
{
foreach (var kvp in parts)
await _sfuManager.DeallocateTransportAsync(
kvp.Value.SfuTransportId);
}
_waitingRoom.TryRemove(meetingId, out _);
var dur = m.EndedAt.Value.Subtract(
m.StartedAt ?? m.CreatedAt).TotalMinutes;
_logger.LogInformation(
"Meeting {Id} ended after {D} min",
meetingId, dur);
}
public Task<BreakoutRoom> CreateBreakoutRoomAsync(
Guid meetingId, string roomName)
{
if (!_meetings.TryGetValue(meetingId, out var m))
throw new InvalidOperationException(
"Meeting not found");
var room = new BreakoutRoom { Name = roomName };
m.BreakoutRooms.Add(room);
_logger.LogInformation(
"Created breakout '{N}' for {M}",
roomName, meetingId);
return Task.FromResult(room);
}
public async Task AssignToBreakoutAsync(
Guid meetingId, Guid participantId, Guid roomId)
{
if (!_meetings.TryGetValue(meetingId, out var m))
throw new InvalidOperationException(
"Meeting not found");
var room = m.BreakoutRooms.FirstOrDefault(
r => r.RoomId == roomId)
?? throw new InvalidOperationException(
"Room not found");
if (_participants.TryGetValue(meetingId, out var parts) &&
parts.TryGetValue(participantId, out var p))
{
room.AssignedParticipants.Add(participantId);
await _signaling.SendToParticipantAsync(
participantId, new SignalMessage
{
Type = SignalMessageType.BreakoutAssign,
MeetingId = meetingId,
SenderId = m.HostUserId,
TargetId = participantId,
Payload = JsonSerializer.Serialize(new
{
roomId = room.RoomId,
roomName = room.Name
})
});
_logger.LogInformation(
"Assigned {N} to breakout '{B}'",
p.DisplayName, room.Name);
}
}
public async Task CloseAllBreakoutsAsync(Guid meetingId)
{
if (!_meetings.TryGetValue(meetingId, out var m))
return;
await _signaling.BroadcastToMeetingAsync(
meetingId, new SignalMessage
{
Type = SignalMessageType.Broadcast,
MeetingId = meetingId,
SenderId = m.HostUserId,
Payload = JsonSerializer.Serialize(new
{
action = "return_to_main"
})
});
m.BreakoutRooms.Clear();
_logger.LogInformation(
"Breakouts closed for {M}", meetingId);
}
private long GenerateMeetingNumber()
{
long num;
do
{
num = 1000000000L +
(long)(_random.NextDouble() * 8999999999L);
} while (_meetingNumberMap.ContainsKey(num));
return num;
}
}
// =============================================
// Recording Service
// =============================================
public interface IRecordingService
{
Task<Recording> StartRecordingAsync(
Guid meetingId, Guid hostUserId);
Task StopRecordingAsync(Guid recordingId);
Task<List<Recording>> GetRecordingsAsync(
Guid meetingId);
}
public class RecordingService : IRecordingService
{
private readonly ILogger<RecordingService> _logger;
private readonly ConcurrentDictionary<Guid, Recording>
_recordings = new();
private readonly ConcurrentDictionary<Guid, DateTime>
_active = new();
public RecordingService(
ILogger<RecordingService> logger)
{
_logger = logger;
}
public Task<Recording> StartRecordingAsync(
Guid meetingId, Guid hostUserId)
{
var rec = new Recording
{
MeetingId = meetingId,
HostUserId = hostUserId,
FileKey = $"recordings/{meetingId}/" +
$"{Guid.NewGuid()}.mp4",
Status = RecordingStatus.Processing
};
_recordings.TryAdd(rec.Id, rec);
_active.TryAdd(meetingId, DateTime.UtcNow);
_logger.LogInformation(
"Recording {Id} started for {M}",
rec.Id, meetingId);
return Task.FromResult(rec);
}
public Task StopRecordingAsync(Guid recordingId)
{
if (_recordings.TryGetValue(recordingId, out var rec))
{
rec.Status = RecordingStatus.Ready;
rec.DurationSeconds =
(int)(DateTime.UtcNow - rec.CreatedAt)
.TotalSeconds;
rec.FileSizeBytes =
rec.DurationSeconds * 200_000;
_active.TryRemove(rec.MeetingId, out _);
_logger.LogInformation(
"Recording {Id} stopped: {D}s, {S} bytes",
rec.Id, rec.DurationSeconds, rec.FileSizeBytes);
}
return Task.CompletedTask;
}
public Task<List<Recording>> GetRecordingsAsync(
Guid meetingId)
{
var list = _recordings.Values
.Where(r => r.MeetingId == meetingId)
.OrderBy(r => r.CreatedAt).ToList();
return Task.FromResult(list);
}
}
// =============================================
// Main Entry Point
// =============================================
public class Program
{
public static async Task Main(string[] args)
{
using var lf = LoggerFactory.Create(b =>
b.AddConsole()
.SetMinimumLevel(LogLevel.Information));
var sig = new SignalingServer(
lf.CreateLogger<SignalingServer>());
var sfu = new SfuManager(
lf.CreateLogger<SfuManager>());
var orch = new MeetingOrchestrator(
lf.CreateLogger<MeetingOrchestrator>(),
sig, sfu);
var rec = new RecordingService(
lf.CreateLogger<RecordingService>());
// Create a meeting
var hostId = Guid.NewGuid();
var meeting = await orch.CreateMeetingAsync(
hostId, "System Design Interview Prep",
password: "zoom2026", maxParticipants: 50);
Console.WriteLine(
$"Meeting created: #{meeting.MeetingNumber}");
// Create breakout rooms
var room1 = await orch.CreateBreakoutRoomAsync(
meeting.Id, "Backend Deep Dive");
var room2 = await orch.CreateBreakoutRoomAsync(
meeting.Id, "Frontend Discussion");
var room3 = await orch.CreateBreakoutRoomAsync(
meeting.Id, "Behavioral Prep");
// Simulate participants joining
var device = new DeviceInfo(
"Windows", "Chrome", "120",
true, true, true, 8, 16384, true, true);
var alice = await orch.JoinMeetingAsync(
meeting.MeetingNumber, Guid.NewGuid(),
"Alice Chen", "zoom2026", device, "us");
Console.WriteLine(
$"Alice joined: {alice.Id}");
var bob = await orch.JoinMeetingAsync(
meeting.MeetingNumber, Guid.NewGuid(),
"Bob Smith", "zoom2026", device, "eu");
Console.WriteLine(
$"Bob joined: {bob.Id}");
var carol = await orch.JoinMeetingAsync(
meeting.MeetingNumber, Guid.NewGuid(),
"Carol Davis", "zoom2026", device, "ap");
Console.WriteLine(
$"Carol joined: {carol.Id}");
// Assign to breakout rooms
await orch.AssignToBreakoutAsync(
meeting.Id, alice.Id, room1.RoomId);
await orch.AssignToBreakoutAsync(
meeting.Id, bob.Id, room2.RoomId);
await orch.AssignToBreakoutAsync(
meeting.Id, carol.Id, room3.RoomId);
// Start recording
var recording = await rec.StartRecordingAsync(
meeting.Id, hostId);
Console.WriteLine(
$"Recording started: {recording.Id}");
// Adaptive bitrate demo
var estimator = new BandwidthEstimator();
var random = new Random();
for (int i = 0; i < 10; i++)
{
var bw = 500 + random.NextDouble() * 3000;
var loss = random.NextDouble() * 5;
var rtt = 50 + random.NextDouble() * 400;
var jitter = random.NextDouble() * 30;
estimator.UpdateEstimate(bw, loss, rtt, jitter);
var (res, fps, br) = estimator
.GetEncodingParams();
Console.WriteLine(
$" ABR [{i}]: BW={bw:F0}kbps " +
$"Loss={loss:F1}% RTT={rtt:F0}ms " +
$"-> {res}p @{fps}fps {br}kbps");
}
// Stop recording
await rec.StopRecordingAsync(recording.Id);
// Close breakout rooms
await orch.CloseAllBreakoutsAsync(meeting.Id);
Console.WriteLine(
"All breakout rooms closed");
// End meeting
await orch.EndMeetingAsync(meeting.Id, hostId);
Console.WriteLine(
"Meeting ended successfully");
}
}
}
27. Zoom Rooms & Conference Room Systems
Zoom Rooms extend the video conferencing platform from personal devices into dedicated conference room hardware, creating a seamless hybrid work experience. A Zoom Room is a software-based room system running on certified hardware that provides one-touch-to-join meeting capability, wireless screen sharing, room scheduling, and integrated digital signage. Designing this subsystem requires bridging the digital conferencing infrastructure with physical IoT devices, room booking systems, and enterprise AV equipment across thousands of room configurations worldwide.
Room System Architecture
The Zoom Rooms controller runs a dedicated application on an iPad, Android tablet, or dedicated touch panel that communicates with the Zoom cloud via a persistent WebSocket connection. This controller manages the room's AV peripherals — cameras, microphones, speakers, displays, and content cameras — through USB, HDMI, and network interfaces. The controller authenticates using a unique room token derived from the room's provisioning key, establishing a secure channel to the Room Service in the cloud.
Hardware Integration & AV Pipeline
Conference room hardware presents unique challenges that personal devices do not face. Room systems must handle multi-camera switching, ceiling microphone arrays with beamforming, acoustic echo cancellation adapted for large room acoustics, and automatic camera framing that tracks active speakers. The AV pipeline processes raw HDMI/USB feeds, encodes them into WebRTC-compatible streams, and routes them through the assigned SFU cluster just like any other participant — but with significantly higher production quality requirements.
The content camera is a specialized device (such as the Huddly L1) that captures whiteboards and physical content using AI-powered content enhancement. It detects the whiteboard surface, applies perspective correction, enhances contrast for readability, and outputs a clean digital feed that remote participants can view as a separate video stream in the meeting. This requires dedicated GPU processing either on the local controller or offloaded to an edge compute node for rooms without sufficient local processing power.
Room Booking & Calendar Sync
The booking subsystem synchronizes with enterprise calendar systems (Microsoft 365, Google Workspace, Exchange) using OAuth 2.0 delegated access and webhook-based change notifications. When a meeting is scheduled in Outlook, the booking service receives a calendar event webhook, matches the room's email address to a provisioned Zoom Room, and reserves the room's time slot. The scheduling panel outside the room displays real-time availability, upcoming meetings, and provides one-touch ad-hoc booking. Conflict resolution uses optimistic concurrency with last-write-wins for the same room, while cross-room conflicts are detected and resolved at the application layer.
| Component | Specification | Purpose | Latency Requirement |
|---|---|---|---|
| PTZ Camera | 4K 60fps, 12x optical zoom, USB 3.0 | Speaker tracking + auto-framing | <100ms exposure to encode |
| Microphone Array | Multi-element beamforming, AEC, 48kHz | Ceiling pickup for 20ft radius | <5ms DSP processing |
| Audio DSP | Hardware AEC + AGC + NS, Dante/AES67 | Room acoustic echo cancellation | <10ms round-trip |
| Content Camera | AI whiteboard enhancement, USB/UVC | Physical content capture for remote | <150ms processing |
| Scheduling Panel | 10" touch, PoE, room availability display | Outside-room booking and check-in | <2s sync from cloud |
| IoT Sensors | PIR + UWB occupancy, CO2, temperature | Room utilization analytics + HVAC | <30s telemetry interval |
| Digital Signage | 4K HDMI output, CMS-managed content | Corporate communications when idle | On-demand, no real-time requirement |
IoT Sensors & Smart Room Integration
Modern conference rooms integrate IoT sensors for occupancy detection, environmental monitoring, and automated room control. Passive infrared (PIR) sensors and ultra-wideband (UWB) radars detect room occupancy, enabling automatic meeting start when participants enter and automatic cleanup when the room empties. CO2 and temperature sensors feed into HVAC controllers to maintain comfortable meeting conditions, while occupancy data feeds analytics dashboards that help facilities teams optimize room sizing and allocation across the organization.
The IoT subsystem publishes sensor telemetry to a cloud MQTT broker, which routes messages to the IoT Device Registry service. This service maintains device health, firmware versions, and configuration state for every sensor in every room. Firmware over-the-air (FOTA) updates are delivered through signed binary packages, with staged rollouts and automatic rollback on failure detection. Device provisioning uses a zero-touch enrollment model: when a new sensor is connected to the room network, it broadcasts a mDNS advertisement, the Zoom Room controller discovers and authenticates it, and cloud provisioning automatically registers the device and applies its configuration profile.
28. Zoom Phone & Unified Communications
Zoom Phone extends the platform beyond video meetings into full unified communications as a service (UCaaS), providing cloud-based telephony that replaces traditional PBX systems. Zoom Phone handles VoIP calling, PSTN integration, interactive voice response (IVR) systems, intelligent call routing, voicemail transcription, and SMS/messaging — all unified under a single user identity that spans meetings, phone calls, and chat. This section explores the telephony subsystem architecture, covering call signaling, media handling, PSTN gateway integration, and the real-time processing pipelines that enable carrier-grade voice quality at global scale.
Telephony Architecture
The core telephony engine manages call state machines that track every call through its lifecycle: INVITE, ringing, answered, held, transferred, conferenced, and terminated. Each call leg is modeled as a state machine with explicit transitions, enabling the system to handle complex scenarios like consultative transfers, call parking, and multi-party conferencing. The Session Border Controller (SBC) serves as the security and protocol translation boundary between Zoom's internal SIP infrastructure and external PSTN providers, handling TLS termination, codec transcoding, and DTMF relay.
Call Routing Engine
The call routing engine processes incoming calls through a configurable rule chain that determines how each call is handled. Routing rules are evaluated in priority order: time-of-day routing (business hours vs after-hours), department-based routing (sales, support, engineering), skills-based routing (language, expertise level), and load-based routing (distribute across available agents). Each rule can invoke actions like play announcement, route to queue, transfer to extension, send to voicemail, or reject with busy signal.
public class CallRoutingEngine
{
private readonly List<IRoutingRule> _rules = new();
private readonly ILogger<CallRoutingEngine> _logger;
public CallRoutingEngine(ILogger<CallRoutingEngine> logger)
{
_logger = logger;
}
public void AddRule(int priority, IRoutingRule rule)
{
rule.Priority = priority;
_rules.Add(rule);
_rules.Sort((a, b) => a.Priority.CompareTo(b.Priority));
}
public async Task<RoutingResult> RouteCallAsync(IncomingCall call)
{
_logger.LogInformation(
"Routing call {CallId} from {Caller}",
call.CallId, call.CallerNumber);
foreach (var rule in _rules)
{
if (await rule.MatchesAsync(call))
{
var result = await rule.ExecuteAsync(call);
_logger.LogInformation(
"Call {CallId} matched rule '{Rule}' -> {Action}",
call.CallId, rule.Name, result.Action);
return result;
}
}
return new RoutingResult(
RoutingAction.SendToVoicemail,
target: call.OwnerExtension);
}
}
public interface IRoutingRule
{
int Priority { get; set; }
string Name { get; }
Task<bool> MatchesAsync(IncomingCall call);
Task<RoutingResult> ExecuteAsync(IncomingCall call);
}
public class BusinessHoursRule : IRoutingRule
{
public int Priority { get; set; }
public string Name => "BusinessHours";
public Task<bool> MatchesAsync(IncomingCall call)
{
var now = DateTimeOffset.UtcNow;
var localTime = TimeZoneInfo.ConvertTime(now, call.OwnerTimeZone);
bool isWeekday = localTime.DayOfWeek is
>= DayOfWeek.Monday and <= DayOfWeek.Friday;
bool isBusinessHours = localTime.Hour >= 9 && localTime.Hour < 17;
return Task.FromResult(isWeekday && isBusinessHours);
}
public Task<RoutingResult> ExecuteAsync(IncomingCall call)
{
return Task.FromResult(new RoutingResult(
RoutingAction.RouteToQueue,
queueId: call.DepartmentQueueId));
}
}
public class SkillsBasedRoutingRule : IRoutingRule
{
public int Priority { get; set; }
public string Name => "SkillsBased";
public async Task<bool> MatchesAsync(IncomingCall call)
{
if (string.IsNullOrEmpty(call.DepartmentQueueId))
return false;
var queue = await GetQueueAsync(call.DepartmentQueueId);
return queue.Agents.Any(a => a.IsAvailable);
}
public async Task<RoutingResult> ExecuteAsync(IncomingCall call)
{
var queue = await GetQueueAsync(call.DepartmentQueueId);
var callerLang = await DetectLanguageAsync(call.CallerNumber);
var bestAgent = queue.Agents
.Where(a => a.IsAvailable)
.OrderByDescending(a => a.HasSkill(callerLang))
.ThenBy(a => a.CurrentCallCount)
.First();
return new RoutingResult(
RoutingAction.TransferToExtension,
target: bestAgent.Extension);
}
private Task<CallQueue> GetQueueAsync(string id) =>
Task.FromResult(new CallQueue());
private Task<string> DetectLanguageAsync(string number) =>
Task.FromResult("en-US");
}
public enum RoutingAction
{
RouteToQueue, TransferToExtension,
SendToVoicemail, PlayAnnouncement, Reject
}
public record RoutingResult(
RoutingAction Action,
string? target = null,
string? queueId = null);
public record IncomingCall(
string CallId, string CallerNumber,
string OwnerExtension, string? DepartmentQueueId,
TimeZoneInfo OwnerTimeZone);
Voicemail Transcription Pipeline
When a call reaches voicemail, the audio stream is simultaneously recorded and streamed to the ASR (Automatic Speech Recognition) pipeline for real-time transcription. The recording service captures the RTP audio stream, strips headers, and writes raw PCM to a buffer. The ASR engine — based on Whisper or Deepgram — processes the audio in chunks, producing a streaming transcription that is displayed in near-real-time on the user's Zoom Phone dashboard. After the voicemail completes, the full audio file is uploaded to S3, the complete transcript is stored in Elasticsearch for full-text search, and a push notification is sent to the user's devices with the transcript preview and audio playback link.
| Feature | Implementation | Protocol | SLA |
|---|---|---|---|
| VoIP Calling | Zoom Phone App (WebRTC + SIP) | SIP over TLS + SRTP media | <150ms one-way audio |
| PSTN Integration | SIP trunking via Twilio/BT/NTT | SIP over TCP + RTP/SRTP | 99.99% call completion |
| IVR / Auto-Attendant | Configurable menu trees with TTS | In-dialog SIP INFO + RTP | <500ms menu response |
| Call Queues | Skills-based routing with callbacks | SIP INVITE queue with ACD | <30s average speed of answer |
| Voicemail | Cloud recording + ASR transcription | RTP stream tap + S3 upload | <30s transcription delivery |
| Call Recording | Barge-in recording via media fork | SIP re-INVITE for fork | <200ms recording start |
| SMS / MMS | A2P messaging via carrier gateways | HTTP REST + SMPP | <5s message delivery |
| E911 | Dynamic location tracking per device | HELO/LOKI + PIDF-LO | Regulatory compliance |
29. Zoom Whiteboard & Collaboration
Zoom Whiteboard provides a persistent, real-time collaborative canvas that participants can use during and outside of meetings for brainstorming, diagramming, and visual collaboration. Unlike static screen sharing, the whiteboard maintains a shared state that all participants can edit simultaneously, with changes synchronized in real-time through conflict-free replicated data types (CRDTs). The whiteboard system integrates sticky notes, freehand drawing, text boxes, shapes, connectors, image insertion, and AI-powered diagramming suggestions — all backed by a scalable real-time collaboration engine.
Real-Time Collaboration Architecture
The collaboration engine uses a CRDT-based approach for whiteboard operations. Each element on the canvas — whether a freehand stroke, a sticky note, a shape, or a connector — is identified by a globally unique ID combined with the creating client's identifier to ensure uniqueness without coordination. Operations like insert, move, rotate, resize, delete, and update properties are expressed as operations on the CRDT document. The system uses an operation-based CRDT where each operation is broadcast to all peers via WebSocket, applied locally using the operation's transform function, and persisted asynchronously to the backend for durability.
The key insight for whiteboard synchronization is that drawing operations have temporal and spatial locality — users typically work on different areas of the canvas or take turns editing the same element. The system exploits this by using operational transformation (OT) for concurrent edits to the same element (e.g., two users simultaneously editing the text in a sticky note) and CRDT merge semantics for independent operations on different elements. This hybrid approach provides the responsiveness of OT for the most common editing patterns while guaranteeing eventual consistency through CRDT convergence.
Drawing & Annotation Engine
The client-side rendering engine uses HTML5 Canvas or WebGL to render the whiteboard content. Freehand drawing captures pointer events at 60fps, applies smoothing algorithms (Catmull-Rom splines for stroke interpolation), and renders the stroke locally with zero perceived latency. Stroke data — a sequence of points with pressure sensitivity values — is batched and sent to the server every 50ms as a single atomic operation. The server broadcasts the complete stroke to other participants, who render it as it arrives, creating the illusion of real-time collaborative drawing even when network latency is 100-200ms.
public class WhiteboardCanvas
{
private readonly ConcurrentDictionary<Guid, CanvasElement>
_elements = new();
private readonly List<WhiteboardOperation> _operationLog = new();
private readonly object _logLock = new();
private Guid _canvasId;
public WhiteboardCanvas(Guid canvasId)
{
_canvasId = canvasId;
}
public WhiteboardOperation ApplyOperation(WhiteboardOperation op)
{
lock (_logLock)
{
op.SequenceNumber = _operationLog.Count;
op.Timestamp = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds();
switch (op.Type)
{
case OperationType.InsertStroke:
var stroke = JsonSerializer
.Deserialize<StrokeElement>(op.Payload)!;
_elements.TryAdd(op.ElementId, stroke);
break;
case OperationType.InsertSticky:
var sticky = JsonSerializer
.Deserialize<StickyNoteElement>(op.Payload)!;
_elements.TryAdd(op.ElementId, sticky);
break;
case OperationType.InsertShape:
var shape = JsonSerializer
.Deserialize<ShapeElement>(op.Payload)!;
_elements.TryAdd(op.ElementId, shape);
break;
case OperationType.Move:
if (_elements.TryGetValue(op.ElementId, out var el))
{
var pos = JsonSerializer
.Deserialize<Position>(op.Payload)!;
el.X = pos.X;
el.Y = pos.Y;
}
break;
case OperationType.UpdateText:
if (_elements.TryGetValue(op.ElementId, out var txt) &&
txt is TextElement te)
{
te.Content = op.Payload;
}
break;
case OperationType.Delete:
_elements.TryRemove(op.ElementId, out _);
break;
case OperationType.UpdateColor:
if (_elements.TryGetValue(op.ElementId, out var ce))
{
ce.BackgroundColor = op.Payload;
}
break;
}
_operationLog.Add(op);
}
return op;
}
public WhiteboardOperation Transform(
WhiteboardOperation local,
WhiteboardOperation remote)
{
if (local.Type == OperationType.Move &&
remote.Type == OperationType.Move &&
local.ElementId == remote.ElementId)
{
var localPos = JsonSerializer
.Deserialize<Position>(local.Payload)!;
var remotePos = JsonSerializer
.Deserialize<Position>(remote.Payload)!;
localPos.X += remotePos.X - localPos.X;
localPos.Y += remotePos.Y - localPos.Y;
local.Payload = JsonSerializer.Serialize(localPos);
}
return local;
}
public CanvasSnapshot CreateSnapshot()
{
lock (_logLock)
{
return new CanvasSnapshot
{
CanvasId = _canvasId,
Elements = _elements.Values.ToList(),
OperationCount = _operationLog.Count,
CreatedAt = DateTimeOffset.UtcNow
};
}
}
}
public class CanvasElement
{
public Guid Id { get; init; }
public double X { get; set; }
public double Y { get; set; }
public double Width { get; set; }
public double Height { get; set; }
public string BackgroundColor { get; set; } = "#FFEB3B";
public int ZIndex { get; set; }
}
public class StrokeElement : CanvasElement
{
public List<Point> Points { get; init; } = new();
public string StrokeColor { get; set; } = "#000000";
public double StrokeWidth { get; set; } = 2.0;
public bool IsSmoothing { get; set; } = true;
}
public class StickyNoteElement : CanvasElement
{
public string Content { get; set; } = string.Empty;
public string FontSize { get; set; } = "16px";
public double Rotation { get; set; }
}
public class ShapeElement : CanvasElement
{
public string ShapeType { get; init; } = "rectangle";
public string StrokeColor { get; set; } = "#333333";
public double StrokeWidth { get; set; } = 2.0;
public List<Guid> ConnectedTo { get; init; } = new();
}
public class TextElement : CanvasElement
{
public string Content { get; set; } = string.Empty;
}
public record Position(double X, double Y);
public record Point(double X, double Y, double Pressure);
public class WhiteboardOperation
{
public Guid ElementId { get; init; }
public OperationType Type { get; init; }
public string Payload { get; init; } = string.Empty;
public Guid AuthorId { get; init; }
public long Timestamp { get; set; }
public int SequenceNumber { get; set; }
}
public enum OperationType
{
InsertStroke, InsertSticky, InsertShape,
Move, Resize, UpdateText, UpdateColor,
Delete, BringToFront, SendToBack
}
public class CanvasSnapshot
{
public Guid CanvasId { get; init; }
public List<CanvasElement> Elements { get; init; } = new();
public int OperationCount { get; set; }
public DateTimeOffset CreatedAt { get; set; }
}
| Feature | Implementation | Sync Strategy | Max Elements |
|---|---|---|---|
| Freehand Drawing | Pointer events + Catmull-Rom smoothing | Batched ops every 50ms | 10,000 strokes per canvas |
| Sticky Notes | Rich text editor with Markdown | CRDT per-note text | 500 notes per canvas |
| Shapes & Connectors | SVG-based with snap-to-grid | CRDT position + properties | 2,000 shapes per canvas |
| Image Insertion | Upload to S3 + CDN delivery | Reference-based (URL sync) | 100 images per canvas |
| Diagrams | AI-assisted layout from text prompts | Batch element creation | Auto-generated diagrams |
| Templates | Pre-built layouts (flowchart, kanban, mind map) | One-shot element batch insert | 20+ templates |
| Version History | Operational log replay from Kafka | Snapshot + delta chain | 30-day operation retention |
30. Zoom SDK & Platform Integration
The Zoom Developer Platform provides SDKs, APIs, and marketplace integrations that enable third-party developers to embed Zoom's video, phone, chat, and whiteboard capabilities into their own applications. The platform consists of the Meeting SDK (formerly Meeting SDK), the Contact Center SDK, embedded meeting capabilities for web, and a marketplace of over 2,500 applications. This section explores the SDK architecture, authentication flows, embedded meeting patterns, and the developer ecosystem that extends Zoom's core platform into every vertical and workflow.
SDK Architecture & Authentication
The Meeting SDK uses a two-layer authentication model. Server-to-server communication uses OAuth 2.0 with client credentials grants, obtaining short-lived access tokens (typically 1 hour TTL) that authenticate REST API calls for meeting management, recording retrieval, and user operations. Client-side SDK instances authenticate using a JWT (JSON Web Token) signed with the application's SDK secret, containing the user's identity, meeting number, role permissions, and expiration time. This JWT is generated server-side and passed to the client SDK during initialization, preventing secret exposure in client code.
The Web SDK operates differently from native SDKs — it loads the Zoom meeting client as a JavaScript bundle that establishes WebRTC connections directly from the browser. The SDK connects to Zoom's signaling servers via WebSocket, negotiates media through the standard SDP/ICE flow, and renders video in a specified DOM element. Developers can customize the UI by hiding default controls and implementing their own overlay using the SDK's event callbacks for participant changes, video state, and audio level updates.
public class ZoomMeetingSdkService
{
private readonly HttpClient _http;
private readonly IConfiguration _config;
private readonly ILogger<ZoomMeetingSdkService> _logger;
public ZoomMeetingSdkService(
HttpClient http, IConfiguration config,
ILogger<ZoomMeetingSdkService> logger)
{
_http = http;
_config = config;
_logger = logger;
}
public async Task<string> GenerateSdkSignatureAsync(
string meetingNumber, string role, string userName)
{
var token = await GetAccessTokenAsync();
var meetingDetails = await GetMeetingAsync(
meetingNumber, token);
var payload = new
{
sdkKey = _config["Zoom:SdkKey"],
meetingNumber,
role, // 0 = attendee, 1 = host
userName,
password = meetingDetails.Password,
custId = Guid.NewGuid().ToString("N"),
sessionExpire = 1800,
noRecording = false
};
var json = JsonSerializer.Serialize(payload);
var signature = ComputeHmacSha256(
json, _config["Zoom:SdkSecret"]);
_logger.LogInformation(
"Generated SDK signature for meeting {MN}, role {R}",
meetingNumber, role);
return Convert.ToBase64String(
System.Text.Encoding.UTF8.GetBytes(
$"{_config["Zoom:SdkKey"]}.{json}.{signature}"));
}
private async Task<string> GetAccessTokenAsync()
{
var credentials = Convert.ToBase64String(
System.Text.Encoding.UTF8.GetBytes(
$"{_config["Zoom:ClientId"]}:{_config["Zoom:ClientSecret"]}"));
var response = await _http.PostAsync(
"https://zoom.us/oauth/token", new FormUrlEncodedContent(
new Dictionary<string, string>
{
["grant_type"] = "account_credentials",
["account_id"] = _config["Zoom:AccountId"]
}));
var body = await response.Content
.ReadFromJsonAsync<TokenResponse>();
return body!.AccessToken;
}
private async Task<ZoomMeetingDetails> GetMeetingAsync(
string meetingNumber, string token)
{
var response = await _http.GetAsync(
$"https://api.zoom.us/v2/meetings/{meetingNumber}");
response.EnsureSuccessStatusCode();
return (await response.Content
.ReadFromJsonAsync<ZoomMeetingDetails>())!;
}
private static string ComputeHmacSha256(
string data, string secret)
{
using var hmac = new System.Security.Cryptography
.HMACSHA256(System.Text.Encoding.UTF8.GetBytes(secret));
var hash = hmac.ComputeHash(
System.Text.Encoding.UTF8.GetBytes(data));
return Convert.ToBase64String(hash);
}
}
public record TokenResponse(
[JsonPropertyName("access_token")] string AccessToken,
[JsonPropertyName("expires_in")] int ExpiresIn);
public record ZoomMeetingDetails(
[JsonPropertyName("id")] long Id,
[JsonPropertyName("topic")] string Topic,
[JsonPropertyName("password")] string Password);
Embedded Meetings & Marketplace
Embedded meetings allow third-party applications to host Zoom meetings within their own UI using the Web SDK's "client view" mode. The SDK renders the meeting interface inside a specified HTML container, hiding Zoom branding and allowing the host application to control the surrounding UI. This enables platforms like Salesforce, HubSpot, and Epic EHR to integrate video consultations directly into their workflows without context-switching to the Zoom application. The embedded SDK supports custom controls, event-driven UI updates, and programmatic participant management through the JavaScript API.
The Zoom App Marketplace hosts over 2,500 integrations spanning project management (Jira, Asana), CRM (Salesforce, HubSpot), healthcare (Epic, Cerner), education (Canvas, Blackboard), and developer tools (GitHub, Jira). Each marketplace app integrates through a combination of OAuth scopes, webhook event subscriptions, and in-meeting SDK components. The marketplace handles app discovery, authentication consent, billing for paid apps, and compliance review to ensure apps meet Zoom's security and privacy requirements.
| SDK / Component | Platform | Key Capabilities | Auth Method |
|---|---|---|---|
| Meeting SDK (Web) | JavaScript / TypeScript | Embed video meetings in web apps | JWT signature + OAuth |
| Meeting SDK (Native) | Windows, macOS, iOS, Android | Full-featured native meeting client | SDK Key + JWT |
| Contact Center SDK | JavaScript + REST | Omnichannel contact center integration | OAuth 2.0 |
| Phone SDK | Web + Native | Embed telephony into third-party apps | OAuth 2.0 + JWT |
| Whiteboard SDK | JavaScript | Embed collaborative whiteboard canvas | OAuth 2.0 |
| REST API v2 | HTTP / JSON | Meeting, user, recording, phone management | OAuth 2.0 |
| Webhooks | HTTP POST callbacks | Real-time event notifications | HMAC-SHA256 signature |
| Chatbot SDK | WebSocket + REST | Build Zoom Chat bots and apps | OAuth 2.0 |
31. Conclusion
Designing a video conferencing system at Zoom's scale is one of the most challenging problems in distributed systems engineering. It requires deep expertise across real-time networking (WebRTC, ICE, SRTP), media processing (codecs, encoding, adaptive bitrate), distributed systems (SFU cascading, multi-region deployment, database sharding), security (E2EE, compliance), and infrastructure cost optimization.
The key architectural insights that emerge from this analysis are:
- SFU over MCU: Selective Forwarding Units are the optimal media server topology for modern video conferencing, trading bandwidth for massive CPU savings and video quality preservation.
- Separation of planes: Media, signaling, and data planes should be independently scalable, allowing each to grow according to its specific demand patterns.
- Adaptive everything: From video bitrate to screen share frame rates to audio processing intensity, the system must continuously adapt to changing network and device conditions.
- Geo-distribution is non-negotiable: Sub-200ms latency requirements demand media processing servers in every major region, with intelligent routing via latency-based DNS.
- Cost is dominated by bandwidth and storage: At 300 Tbps peak bandwidth and 10 PB/day recording storage, infrastructure costs reach $500M+ annually, making optimization strategies critical.
For system design interviews, the most important skill is not memorizing every detail but understanding the trade-offs between different design choices and being able to justify your decisions with clear reasoning. This article provides the foundation to do exactly that — whether you are designing for 10 users or 300 million.
- Always start with requirements and capacity estimation before jumping into architecture
- Explain WHY you chose SFU over MCU/Mesh with specific trade-offs
- Discuss how each component scales independently
- Address failure scenarios: what happens when a participant's network drops? When an SFU crashes? When a region goes down?
- Know the cost implications — interviewers love candidates who consider operational costs