1. System Overview & Requirements

Video conferencing has become one of the most bandwidth-intensive, latency-sensitive, and infrastructure-heavy systems in modern software engineering. Unlike request-response web applications, a video conferencing system demands sub-200ms end-to-end latency, sustained high-throughput media delivery, and graceful degradation under adverse network conditions. Designing such a system requires a deep understanding of real-time networking, media codecs, distributed systems, and security.

In this guide, we will design a video conferencing platform capable of supporting meetings with up to 1,000 participants, featuring real-time audio/video, screen sharing, chat, breakout rooms, recording, virtual backgrounds, and end-to-end encryption. We will walk through every major subsystem — from WebRTC's peer-to-peer origins to geo-distributed SFU cascades — and discuss trade-offs that matter in production.

Functional Requirements

Non-Functional Requirements

2. High-Level Architecture

The video conferencing system consists of five major subsystems: the Client Layer (web/mobile SDK), the Signaling Layer (WebSocket-based session management), the Media Layer (SFU cluster for audio/video routing), the Supporting Services (recording, chat, presence), and the Infrastructure Layer (STUN/TURN, storage, monitoring).

mermaid
graph TB
    subgraph "Client Layer"
        A[Web App / Electron]
        B[Mobile SDK - iOS]
        C[Mobile SDK - Android]
    end

    subgraph "API Gateway & Auth"
        D[API Gateway]
        E[Auth Service - OAuth2/SSO]
        F[Rate Limiter]
    end

    subgraph "Signaling Layer"
        G[Signaling Server - WebSocket]
        H[Room Manager]
        I[Presence Service]
    end

    subgraph "Media Layer"
        J[SFU Cluster]
        K[SFU Node 1]
        L[SFU Node 2]
        M[SFU Node N]
        N[TURN Server Pool]
        O[STUN Server Pool]
    end

    subgraph "Supporting Services"
        P[Recording / Egress Service]
        Q[Chat Service]
        R[Screen Share Controller]
        S[Breakout Room Manager]
        T[Virtual Background ML Service]
    end

    subgraph "Data & Storage"
        U[Redis - Room State]
        V[PostgreSQL - User/Meeting Data]
        W[S3 - Recordings & Assets]
        X[Prometheus + Grafana]
    end

    A & B & C --> D
    D --> E & F
    D --> G
    G --> H & I
    H --> J
    J --> K & L & M
    K & L & M --> N & O
    G --> P & Q & R & S
    T --> J
    H --> U
    D --> V
    P --> W
    K & L & M --> X
    

Data Flow for a Typical Meeting

1. Join: User authenticates via the API Gateway, receives a session token, and connects to the Signaling Server via WebSocket.
2. Room Assignment: The Room Manager assigns the user to an SFU node based on geographic proximity and current load.
3. SDP Exchange: The client and SFU exchange SDP offers/answers via the Signaling Server to establish media tracks.
4. ICE Connectivity: The client gathers ICE candidates (host, srflx, relay) and the SFU responds with its own candidates. Connectivity checks are performed.
5. Media Flow: Once ICE succeeds, DTLS handshake completes, and encrypted RTP (SRTP) media flows directly between the client and the assigned SFU node.
6. Fanout: The SFU receives each participant's media and forwards (subscribes) relevant tracks to other participants based on their viewport and bandwidth constraints.

3. WebRTC Deep Dive

WebRTC (Web Real-Time Communication) is the foundation of browser-based video conferencing. It is a W3C standard that provides browsers and mobile applications with real-time communication capabilities via simple JavaScript APIs. Under the hood, WebRTC encompasses a sophisticated stack of protocols and algorithms that handle everything from codec negotiation to network traversal.

The WebRTC Protocol Stack

mermaid
graph TB
    A["Application Layer - getUserMedia(), RTCPeerConnection"] --> B["SDP - Session Description Protocol"]
    B --> C["DTLS - Datagram Transport Layer Security"]
    C --> D["SRTP - Secure Real-time Transport Protocol"]
    D --> E["ICE - Interactive Connectivity Establishment"]
    E --> F["STUN / TURN - NAT Traversal"]
    E --> G["UDP - User Datagram Protocol"]
    G --> H["IP Network"]
    

RTCPeerConnection Lifecycle

The RTCPeerConnection object is the core API for managing a WebRTC connection. It handles the entire lifecycle from ICE gathering through media exchange to connection teardown.

1. Create PeerConnection: Initialize with ICE servers (STUN/TURN) configuration.
2. Add Local Tracks: Get media via getUserMedia() and add tracks to the connection via addTrack().
3. ICE Gathering: The client gathers ICE candidates (host, server-reflexive, relay) and signals them to the remote peer via the signaling server.
4. SDP Offer/Answer: One side creates an SDP offer describing its media capabilities; the other responds with an SDP answer.
5. Connectivity Checks: ICE performs STUN binding requests to test candidate pairs and find the best path.
6. DTLS Handshake: Once a candidate pair is selected, DTLS establishes encryption keys for SRTP.
7. Media Exchange: Encrypted RTP flows over the selected candidate pair.

C# Signaling Server Implementation

Below is a C# implementation of a WebSocket-based signaling server using ASP.NET Core. This server facilitates SDP exchange and ICE candidate relay between clients.

csharp
public class SignalingHub : Hub
{
    private static readonly ConcurrentDictionary<string, RoomState> _rooms = new();
    private readonly IRoomManager _roomManager;
    private readonly ILogger<SignalingHub> _logger;

    public SignalingHub(IRoomManager roomManager, ILogger<SignalingHub> logger)
    {
        _roomManager = roomManager;
        _logger = logger;
    }

    public override async Task OnConnectedAsync()
    {
        var roomId = Context.GetHttpContext()?.Request.Query["roomId"].FirstOrDefault();
        var userId = Context.UserIdentifier;

        if (string.IsNullOrEmpty(roomId) || string.IsNullOrEmpty(userId))
        {
            Context.Abort();
            return;
        }

        await Groups.AddToGroupAsync(Context.ConnectionId, roomId);
        var room = _rooms.GetOrAdd(roomId, _ => new RoomState(roomId));
        room.Participants.TryAdd(userId, new ParticipantInfo
        {
            ConnectionId = Context.ConnectionId,
            JoinedAt = DateTimeOffset.UtcNow,
            Role = ParticipantRole.Attendee
        });

        await Clients.Group(roomId).SendAsync("ParticipantJoined", new
        {
            UserId = userId,
            ParticipantCount = room.Participants.Count
        });

        _logger.LogInformation("User {UserId} joined room {RoomId}", userId, roomId);
    }

    public async Task SendOffer(string targetUserId, RTCSessionDescription offer)
    {
        var roomId = Context.GetHttpContext()?.Request.Query["roomId"].FirstOrDefault();
        var room = _rooms.GetValueOrDefault(roomId);
        if (room == null) return;

        var target = room.Participants.GetValueOrDefault(targetUserId);
        if (target == null) return;

        await Clients.Client(target.ConnectionId).SendAsync("ReceiveOffer", new
        {
            FromUserId = Context.UserIdentifier,
            Offer = offer
        });
    }

    public async Task SendAnswer(string targetUserId, RTCSessionDescription answer)
    {
        var roomId = Context.GetHttpContext()?.Request.Query["roomId"].FirstOrDefault();
        var room = _rooms.GetValueOrDefault(roomId);
        if (room == null) return;

        var target = room.Participants.GetValueOrDefault(targetUserId);
        if (target == null) return;

        await Clients.Client(target.ConnectionId).SendAsync("ReceiveAnswer", new
        {
            FromUserId = Context.UserIdentifier,
            Answer = answer
        });
    }

    public async Task RelayIceCandidate(string targetUserId, RTCIceCandidate candidate)
    {
        var roomId = Context.GetHttpContext()?.Request.Query["roomId"].FirstOrDefault();
        var room = _rooms.GetValueOrDefault(roomId);
        if (room == null) return;

        var target = room.Participants.GetValueOrDefault(targetUserId);
        if (target == null) return;

        await Clients.Client(target.ConnectionId).SendAsync("ReceiveIceCandidate", new
        {
            FromUserId = Context.UserIdentifier,
            Candidate = candidate
        });
    }

    public async Task PublishMediaTrack(string trackId, MediaTrackInfo trackInfo)
    {
        var roomId = Context.GetHttpContext()?.Request.Query["roomId"].FirstOrDefault();
        await Clients.GroupExcept(roomId, Context.ConnectionId)
            .SendAsync("TrackPublished", new
            {
                UserId = Context.UserIdentifier,
                TrackId = trackId,
                TrackInfo = trackInfo
            });
    }

    public override async Task OnDisconnectedAsync(Exception? exception)
    {
        var roomId = Context.GetHttpContext()?.Request.Query["roomId"].FirstOrDefault();
        var userId = Context.UserIdentifier;

        if (!string.IsNullOrEmpty(roomId) && _rooms.TryGetValue(roomId, out var room))
        {
            room.Participants.TryRemove(userId, out _);
            await Clients.Group(roomId).SendAsync("ParticipantLeft", new
            {
                UserId = userId,
                ParticipantCount = room.Participants.Count
            });

            if (room.Participants.IsEmpty)
            {
                _rooms.TryRemove(roomId, out _);
                await _roomManager.CleanupRoom(roomId);
            }
        }
    }
}

public class RoomState
{
    public string RoomId { get; }
    public ConcurrentDictionary<string, ParticipantInfo> Participants { get; } = new();
    public DateTimeOffset CreatedAt { get; } = DateTimeOffset.UtcNow;

    public RoomState(string roomId) => RoomId = roomId;
}

public class ParticipantInfo
{
    public string ConnectionId { get; set; } = string.Empty;
    public DateTimeOffset JoinedAt { get; set; }
    public ParticipantRole Role { get; set; }
    public bool IsMuted { get; set; }
    public bool IsVideoOff { get; set; }
}

public enum ParticipantRole
{
    Host,
    CoHost,
    Attendee
}

public class RTCSessionDescription
{
    public string Type { get; set; } = string.Empty;
    public string Sdp { get; set; } = string.Empty;
}

public class RTCIceCandidate
{
    public string Candidate { get; set; } = string.Empty;
    public string SdpMid { get; set; } = string.Empty;
    public int SdpMLineIndex { get; set; }
}

public class MediaTrackInfo
{
    public string Kind { get; set; } = string.Empty;
    public string Label { get; set; } = string.Empty;
    public bool Enabled { get; set; }
}

Insertable Streams for E2EE

WebRTC's Insertable Streams API (also known as "Encoded Transform") allows applications to intercept and encrypt media frames before they are sent over the wire, and decrypt them on the receiving end. This provides true end-to-end encryption where even the SFU cannot inspect the media content.

csharp
public class E2EKeyManager
{
    private readonly IDistributedCache _cache;
    private readonly TimeSpan _keyRotationInterval = TimeSpan.FromHours(24);

    public async Task<EncryptionKey> GetOrCreateRoomKey(string roomId)
    {
        var cached = await _cache.GetAsync($"e2ee:{roomId}");
        if (cached != null)
        {
            return JsonSerializer.Deserialize<EncryptionKey>(cached)!;
        }

        var key = new EncryptionKey
        {
            KeyId = Guid.NewGuid().ToString("N"),
            Material = new byte[32],
            CreatedAt = DateTimeOffset.UtcNow,
            ExpiresAt = DateTimeOffset.UtcNow.Add(_keyRotationInterval)
        };

        using var rng = RandomNumberGenerator.Create();
        rng.GetBytes(key.Material);

        await _cache.SetAsync($"e2ee:{roomId}",
            JsonSerializer.SerializeToUtf8Bytes(key),
            new DistributedCacheEntryOptions
            {
                AbsoluteExpiration = key.ExpiresAt
            });

        return key;
    }

    public async Task RotateKey(string roomId)
    {
        await _cache.RemoveAsync($"e2ee:{roomId}");
    }
}

public class EncryptionKey
{
    public string KeyId { get; set; } = string.Empty;
    public byte[] Material { get; set; } = Array.Empty<byte>();
    public DateTimeOffset CreatedAt { get; set; }
    public DateTimeOffset ExpiresAt { get; set; }
}

4. Signaling Server Design

The signaling server is the control plane of the video conferencing system. It does not handle any media — it is responsible for session negotiation, participant management, room state, and relaying SDP offers/answers and ICE candidates between peers. Signaling must be reliable, ordered, and low-latency.

Protocol Choice: WebSocket vs. HTTP Long-Polling

We use WebSocket as the primary signaling transport due to its low-latency bidirectional nature, with automatic fallback to HTTP long-polling for restrictive network environments.

Room State Machine

mermaid
stateDiagram-v2
    [*] --> Created: Host creates meeting
    Created --> WaitingRoom: First participant joins
    WaitingRoom --> Active: Host admits participant
    Active --> Active: Participants join/leave
    Active --> Breakout: Host creates breakout rooms
    Breakout --> Active: Participants return
    Active --> Recording: Recording started
    Recording --> Active: Recording stopped
    Active --> Ended: Host ends meeting
    WaitingRoom --> Ended: Host ends meeting
    Ended --> [*]
    

Signaling Server Scalability

The signaling server must handle millions of concurrent WebSocket connections. Key scaling strategies include:

• Horizontal Scaling: Multiple signaling server instances behind a load balancer with sticky sessions (by IP or a custom header).
• Redis Pub/Sub: For cross-instance message routing when participants in the same room connect to different signaling servers.
• Connection Multiplexing: Each client maintains a single WebSocket connection; all signaling messages (SDP, ICE, chat, presence) are multiplexed over it with message type headers.
• Graceful Reconnection: Clients implement exponential backoff reconnection with state resynchronization. The server maintains a session buffer for recently disconnected participants.

csharp
public class RoomManager : IRoomManager
{
    private readonly IConnectionMultiplexer _redis;
    private readonly ILogger<RoomManager> _logger;

    public async Task<SFUNodeInfo> AssignSFUNode(string roomId, GeoLocation clientLocation)
    {
        var availableNodes = await GetAvailableSFUNodes();

        var bestNode = availableNodes
            .OrderBy(n => CalculateDistance(clientLocation, n.Location))
            .ThenBy(n => n.CurrentLoad / (double)n.MaxCapacity)
            .FirstOrDefault();

        if (bestNode == null)
        {
            throw new ServiceUnavailableException("No SFU nodes available");
        }

        var db = _redis.GetDatabase();
        await db.HashSetAsync($"room:{roomId}:sfu", new HashEntry[]
        {
            new("nodeId", bestNode.NodeId),
            new("assignedAt", DateTimeOffset.UtcNow.ToUnixTimeSeconds())
        });

        await db.StringIncrementAsync($"sfu:{bestNode.NodeId}:connections");

        _logger.LogInformation(
            "Assigned SFU node {NodeId} to room {RoomId} (load: {Load}/{Max})",
            bestNode.NodeId, roomId, bestNode.CurrentLoad, bestNode.MaxCapacity);

        return bestNode;
    }

    private double CalculateDistance(GeoLocation a, GeoLocation b)
    {
        const double R = 6371;
        var dLat = ToRadians(b.Latitude - a.Latitude);
        var dLon = ToRadians(b.Longitude - a.Longitude);
        var lat1 = ToRadians(a.Latitude);
        var lat2 = ToRadians(b.Latitude);

        var h = Math.Sin(dLat / 2) * Math.Sin(dLat / 2) +
                Math.Cos(lat1) * Math.Cos(lat2) *
                Math.Sin(dLon / 2) * Math.Sin(dLon / 2);

        return R * 2 * Math.Asin(Math.Sqrt(h));
    }

    private double ToRadians(double deg) => deg * Math.PI / 180;
}

5. ICE, STUN, and TURN Servers

NAT (Network Address Translation) traversal is one of the most challenging aspects of real-time communication. Most devices are behind NATs or firewalls that prevent direct peer-to-peer connectivity. The ICE (Interactive Connectivity Establishment) framework, combined with STUN and TURN servers, solves this problem.

How ICE Works

1. Candidate Gathering: Each peer gathers ICE candidates of three types:
   • Host candidates: Local IP addresses and ports on the device.
   • Server-reflexive (srflx) candidates: Public IP:port as seen by a STUN server.
   • Relay candidates: Public IP:port allocated on a TURN server for relayed traffic.
2. Candidate Exchange: Both peers send their candidate lists to each other via the signaling server.
3. Connectivity Checks: Each peer sends STUN binding requests to every candidate pair. Successful responses indicate a working path.
4. Candidate Pair Selection: ICE selects the best-performing candidate pair based on priority calculations (preferring direct paths over relayed ones).

mermaid
sequenceDiagram
    participant A as Peer A
    participant STUN as STUN Server
    participant TURN as TURN Server
    participant B as Peer B

    A->>STUN: STUN Binding Request (discover srflx)
    STUN-->>A: Response with public IP:port

    A->>TURN: Allocate relay endpoint
    TURN-->>A: Relay candidate (IP:port on TURN)

    Note over A,B: Exchange candidates via signaling server

    A->>B: STUN Binding Request (host candidate)
    B-->>A: Success (direct path works!)

    A->>B: STUN Binding Request (srflx candidate)
    B-->>A: Success (NAT traversal works!)

    Note over A,B: Best pair selected, DTLS handshake begins
    

STUN Server Implementation

csharp
public class StunServer
{
    private readonly UdpClient _udp;
    private readonly ILogger<StunServer> _logger;
    private const int StunPort = 3478;

    public StunServer(int port = StunPort)
    {
        _udp = new UdpClient(port);
        _logger = LoggerFactory.Create(b => b.AddConsole()).CreateLogger<StunServer>();
    }

    public async Task StartAsync(CancellationToken ct)
    {
        _logger.LogInformation("STUN server listening on port {Port}", StunPort);

        while (!ct.IsCancellationRequested)
        {
            var result = await _udp.ReceiveAsync(ct);
            _ = Task.Run(() => HandleStunMessage(result.Buffer, result.RemoteEndPoint));
        }
    }

    private async Task HandleStunMessage(byte[] data, IPEndPoint remoteEndPoint)
    {
        if (data.Length < 20) return;

        var msgType = (StunMessageType)((data[0] << 8) | data[1]);
        if (msgType != StunMessageType.BindingRequest) return;

        var response = new byte[28];
        response[0] = 0x01; response[1] = 0x01; // Binding Response
        response[2] = 0x00; response[3] = 0x08; // Length: 8

        // Magic cookie
        response[4] = 0x21; response[5] = 0x12;
        response[6] = 0xA4; response[7] = 0x42;

        // Transaction ID from request
        Array.Copy(data, 8, response, 8, 12);

        // XOR-MAPPED-ADDRESS
        var addrBytes = remoteEndPoint.Address.GetAddressBytes();
        var xoredAddr = new byte[4];
        var magicCookie = BitConverter.GetBytes(0x2112A442);
        for (int i = 0; i < 4; i++)
            xoredAddr[i] = (byte)(addrBytes[i] ^ magicCookie[i]);

        var xoredPort = (ushort)(remoteEndPoint.Port ^ 0x2112);

        var fullResponse = new byte[response.Length + 12];
        Array.Copy(response, fullResponse, response.Length);
        fullResponse[20] = 0x00; fullResponse[21] = 0x20; // XOR-MAPPED-ADDRESS
        fullResponse[22] = 0x00; fullResponse[23] = 0x08;
        fullResponse[24] = 0x00; fullResponse[25] = 0x01; // IPv4
        fullResponse[26] = (byte)(xoredPort >> 8);
        fullResponse[27] = (byte)(xoredPort & 0xFF);
        Array.Copy(xoredAddr, 0, fullResponse, 28, 4);

        await _udp.SendAsync(fullResponse, fullResponse.Length, remoteEndPoint);
    }
}

TURN Server Deployment

TURN servers are expensive because they relay all media traffic through them. We deploy coturn (the industry-standard open-source TURN server) in each geographic region with the following configuration:

6. Media Topologies: P2P, MCU, SFU

Choosing the right media topology is one of the most impactful architectural decisions. The three primary topologies each have distinct trade-offs in terms of server cost, client bandwidth, video quality, and scalability.

Peer-to-Peer (P2P)

Each participant sends their media directly to every other participant. No server is involved in media routing — only the signaling server coordinates the connection.

• Pros: Lowest latency (direct path), no server media cost, simplest architecture.
• Cons: Doesn't scale beyond 4-6 participants due to upload bandwidth requirements. With N participants, each client must send (N-1) copies of their stream and receive (N-1) streams.
• Best for: 1:1 calls, small group calls (2-4 participants).

Multipoint Control Unit (MCU)

A central server receives all participants' streams, decodes them, composites (mixes) them into a single layout, re-encodes, and sends the composite stream to each participant.

• Pros: Clients only send/receive one stream regardless of participant count. Simple client implementation.
• Cons: Extremely CPU-intensive server (decoding + compositing + encoding for every participant). High latency (decode + encode adds 100-200ms). Everyone sees the same layout.
• Best for: Legacy telephony systems, legacy hardware endpoints.

Selective Forwarding Unit (SFU)

A central server receives each participant's streams and selectively forwards (routes) them to other participants. No decoding or encoding occurs on the server.

• Pros: Low server CPU (just packet forwarding), supports simulcast (multiple quality layers), clients choose which streams to subscribe to, scales to thousands with cascading.
• Cons: Higher client-side bandwidth than MCU (each client downloads individual streams), requires more sophisticated client for layout rendering.
• Best for: Modern video conferencing at any scale. This is the standard approach.

mermaid
graph LR
    subgraph "P2P"
        A1[Peer A] <--> B1[Peer B]
        A1 <--> C1[Peer C]
        B1 <--> C1
    end

    subgraph "MCU"
        A2[Peer A] --> MCU[MCU Server]
        B2[Peer B] --> MCU
        C2[Peer C] --> MCU
        MCU --> A2
        MCU --> B2
        MCU --> C2
    end

    subgraph "SFU"
        A3[Peer A] --> SFU[SFU Server]
        B3[Peer B] --> SFU
        C3[Peer C] --> SFU
        SFU -->|A stream| B3
        SFU -->|A stream| C3
        SFU -->|B stream| A3
        SFU -->|B stream| C3
        SFU -->|C stream| A3
        SFU -->|C stream| B3
    end
    

Bandwidth Comparison

7. SFU Cascading & Geo-Distribution

For meetings with hundreds or thousands of participants, a single SFU node becomes a bottleneck. A single SFU with 500 participants sending 720p video must forward 500 x 499 = 250,000 video streams — far beyond what a single machine can handle. SFU cascading solves this by distributing participants across multiple interconnected SFU nodes.

Cascaded SFU Architecture

mermaid
graph TB
    subgraph "US-East Region"
        SFU1[SFU Primary - us-east-1]
        P1[Participant 1]
        P2[Participant 2]
        P3[Participant 3]
        P4[Participant 4]
        P5[Participant 5]
    end

    subgraph "EU-West Region"
        SFU2[SFU Cascade - eu-west-1]
        P6[Participant 6]
        P7[Participant 7]
        P8[Participant 8]
    end

    subgraph "AP-Southeast Region"
        SFU3[SFU Cascade - ap-southeast-1]
        P9[Participant 9]
        P10[Participant 10]
    end

    P1 & P2 & P3 & P4 & P5 --> SFU1
    P6 & P7 & P8 --> SFU2
    P9 & P10 --> SFU3

    SFU1 <--> SFU2
    SFU1 <--> SFU3
    SFU2 <--> SFU3
    

How SFU Cascading Works

1. Participant Assignment: Each participant is connected to the nearest SFU node based on their geographic location.
2. Intra-SFU Forwarding: Within a single SFU node, media is forwarded between locally-connected participants using the standard SFU logic.
3. Inter-SFU Bridge: Each SFU subscribes to the media streams of participants on other SFU nodes. These "upstream" subscriptions are forwarded across inter-SFU links.
4. Layer Selection: SFU cascading uses simulcast layers to optimize inter-SFU bandwidth. The primary SFU sends only the appropriate simulcast layer to each cascade SFU.

csharp
public class SFUCascadingManager
{
    private readonly ISFUNodeRegistry _nodeRegistry;
    private readonly IInterSFUTransport _transport;

    public async Task<CascadePlan> CreateCascadePlan(string roomId)
    {
        var regions = await _nodeRegistry.GetRegionsWithParticipants(roomId);
        var plan = new CascadePlan(roomId);

        var primaryRegion = regions
            .OrderByDescending(r => r.ParticipantCount)
            .First();

        plan.PrimarySFU = primaryRegion.SFUNode;

        foreach (var region in regions.Where(r => r.Region != primaryRegion.Region))
        {
            var subscription = new InterSFUSubscription
            {
                FromSFU = plan.PrimarySFU,
                ToSFU = region.SFUNode,
                Tracks = await GetTracksForRegion(roomId, primaryRegion.Region),
                SimulcastLayer = SelectOptimalLayer(region.ParticipantCount),
                Direction = SubscriptionDirection.Bidirectional
            };

            plan.Subscriptions.Add(subscription);
        }

        return plan;
    }

    private SimulcastLayer SelectOptimalLayer(int participantCount)
    {
        return participantCount switch
        {
            <= 10 => SimulcastLayer.High,
            <= 50 => SimulcastLayer.Medium,
            <= 200 => SimulcastLayer.Low,
            _ => SimulcastLayer.Lowest
        };
    }
}

public enum SimulcastLayer
{
    Lowest, // ~90p, ~100kbps
    Low,    // ~180p, ~250kbps
    Medium, // ~480p, ~750kbps
    High    // ~720p, ~2Mbps
}

public class InterSFUSubscription
{
    public SFUNodeInfo FromSFU { get; set; } = new();
    public SFUNodeInfo ToSFU { get; set; } = new();
    public List<TrackSubscription> Tracks { get; set; } = new();
    public SimulcastLayer SimulcastLayer { get; set; }
    public SubscriptionDirection Direction { get; set; }
}

Latency Considerations in Cascading

Inter-SFU links add latency. A participant on SFU-node-A receiving media that originates from a participant on SFU-node-B must traverse the inter-SFU link. Typical inter-datacenter latency is 1-5ms for same-region and 50-200ms for cross-continent. To minimize cascading latency:

• Prefer same-region SFU connections for participants with high audio interaction.
• Use regional SFU clusters with dedicated inter-region links (not the public internet).
• Implement intelligent participant grouping — participants who interact frequently (e.g., same breakout room) should be on the same SFU.

8. Video & Audio Codec Selection

Codec selection directly impacts video quality, bandwidth usage, CPU cost, and browser compatibility. The choice of video and audio codecs is a critical design decision with significant trade-offs.

Video Codecs Comparison

Audio Codecs Comparison

Our Codec Strategy

mermaid
graph TB
    A[Client Codec Preference] --> B{Negotiation}
    B -->|Video| C{Browser Support Check}
    C -->|AV1 supported| D[Use AV1]
    C -->|VP9 supported| E[Use VP9]
    C -->|Legacy browser| F[Use H.264]
    B -->|Audio| G[Always Opus]
    D --> H[Apply Simulcast Layers]
    E --> H
    F --> H
    G --> I[Apply DTX and FEC]
    

Codec Configuration for Optimal Quality

csharp
public class CodecConfigurator
{
    public VideoCodecConfig GetVideoConfig(MeetingSize size, NetworkCondition network)
    {
        var config = new VideoCodecConfig
        {
            CodecPriority = new[] { "VP9", "H264", "VP8" },
            ClockRate = 90000,
            MaxBitrate = network switch
            {
                NetworkCondition.Excellent => 2_500_000,
                NetworkCondition.Good => 1_500_000,
                NetworkCondition.Fair => 750_000,
                NetworkCondition.Poor => 300_000,
                _ => 1_000_000
            },
            FrameRate = size switch
            {
                MeetingSize.Small => 30,
                MeetingSize.Medium => 24,
                MeetingSize.Large => 15,
                _ => 15
            },
            Resolution = size switch
            {
                MeetingSize.Small => VideoResolution.p720,
                MeetingSize.Medium => VideoResolution.p480,
                MeetingSize.Large => VideoResolution.p360,
                _ => VideoResolution.p360
            }
        };

        return config;
    }

    public AudioCodecConfig GetAudioConfig()
    {
        return new AudioCodecConfig
        {
            Codec = "opus",
            ClockRate = 48000,
            Channels = 1,
            Bitrate = 64_000,
            UseFEC = true,
            UseDTX = true,
            MinBitrate = 12_000,
            MaxBitrate = 128_000
        };
    }
}

public enum MeetingSize
{
    Small,  // 2-5 participants
    Medium, // 6-25 participants
    Large   // 26-1000 participants
}

public enum NetworkCondition
{
    Excellent, // > 5 Mbps, < 50ms RTT
    Good,      // 2-5 Mbps, 50-100ms RTT
    Fair,      // 500 kbps-2 Mbps, 100-200ms RTT
    Poor       // < 500 kbps, > 200ms RTT
}

9. Simulcast, SVC & Adaptive Bitrate

In a multi-participant meeting, each participant has different network conditions and screen sizes. Sending the same high-quality stream to everyone wastes bandwidth for participants on slow connections or viewing small tiles. Simulcast and SVC (Scalable Video Coding) solve this problem by allowing the SFU to deliver different quality levels to different subscribers.

Simulcast vs. SVC

Simulcast Layer Configuration

csharp
public class SimulcastConfigurator
{
    public List<SimulcastLayerConfig> GetSimulcastLayers(VideoResolution maxResolution)
    {
        return new List<SimulcastLayerConfig>
        {
            new()
            {
                Layer = SimulcastLayer.High,
                Width = 1280, Height = 720,
                FrameRate = 30,
                MaxBitrate = 2_000_000,
                ScaleFactor = 1,
                ScalabilityMode = ScalabilityMode.L1T3
            },
            new()
            {
                Layer = SimulcastLayer.Medium,
                Width = 640, Height = 360,
                FrameRate = 24,
                MaxBitrate = 750_000,
                ScaleFactor = 2,
                ScalabilityMode = ScalabilityMode.L1T2
            },
            new()
            {
                Layer = SimulcastLayer.Low,
                Width = 320, Height = 180,
                FrameRate = 15,
                MaxBitrate = 200_000,
                ScaleFactor = 4,
                ScalabilityMode = ScalabilityMode.L1T1
            }
        };
    }

    public SimulcastLayer SelectLayerForSubscriber(
        SubscriberInfo subscriber,
        PublisherInfo publisher)
    {
        var viewportArea = subscriber.ViewportWidth * subscriber.ViewportHeight;
        var availableBandwidth = subscriber.EstimatedBandwidth;

        if (viewportArea >= 1280 * 720 && availableBandwidth >= 2_000_000)
            return SimulcastLayer.High;
        if (viewportArea >= 640 * 360 && availableBandwidth >= 750_000)
            return SimulcastLayer.Medium;

        return SimulcastLayer.Low;
    }
}

public class SimulcastLayerConfig
{
    public SimulcastLayer Layer { get; set; }
    public int Width { get; set; }
    public int Height { get; set; }
    public int FrameRate { get; set; }
    public int MaxBitrate { get; set; }
    public int ScaleFactor { get; set; }
    public string ScalabilityMode { get; set; } = string.Empty;
}

Adaptive Bitrate (ABR) Algorithm

The SFU continuously monitors each subscriber's network conditions and adjusts the simulcast layer accordingly. The ABR algorithm considers:

• Available bandwidth estimation: Based on packet loss, RTT, and throughput measurements.
• Receiver viewport: The subscriber's visible grid size (a participant viewing a 2x2 grid doesn't need 1080p for each tile).
• Receiver CPU: Decoding 20 high-resolution streams simultaneously may overwhelm the receiver's CPU.
• Network congestion indicators: Delay-based congestion detection (GCC algorithm) and loss-based indicators.

csharp
public class AdaptiveBitrateController
{
    private readonly ConcurrentDictionary<string, SubscriberState> _subscribers = new();

    public SimulcastLayer EvaluateAndAdjust(SubscriberMetrics metrics)
    {
        var state = _subscribers.GetOrAdd(metrics.SubscriberId, _ => new SubscriberState());

        state.BandwidthEstimate = UpdateBandwidthEstimate(
            metrics.RoundTripTime,
            metrics.PacketLossRate,
            metrics.Jitter,
            state.BandwidthEstimate);

        var optimalLayer = DetermineOptimalLayer(state, metrics);

        if (optimalLayer != state.CurrentLayer)
        {
            state.LayerChanges++;
            var sustainedTime = DateTime.UtcNow - state.ConditionChangedAt;
            var minHoldTime = state.LayerChanges > 3
                ? TimeSpan.FromSeconds(10)
                : TimeSpan.FromSeconds(3);

            if (sustainedTime >= minHoldTime)
            {
                state.CurrentLayer = optimalLayer;
                state.ConditionChangedAt = DateTime.UtcNow;
            }
        }

        return state.CurrentLayer;
    }

    private double UpdateBandwidthEstimate(
        TimeSpan rtt, double lossRate, double jitter, double currentEstimate)
    {
        var delayBasedEstimate = currentEstimate * Math.Max(0.5, 1.0 - jitter.TotalMilliseconds / 200.0);
        var lossBasedEstimate = currentEstimate * (1 - lossRate * 2.5);
        return Math.Min(delayBasedEstimate, lossBasedEstimate);
    }

    private SimulcastLayer DetermineOptimalLayer(
        SubscriberState state, SubscriberMetrics metrics)
    {
        var bw = state.BandwidthEstimate;
        var viewportArea = metrics.ViewportWidth * metrics.ViewportHeight;

        if (bw >= 2_000_000 && viewportArea >= 921_600)
            return SimulcastLayer.High;
        if (bw >= 750_000 && viewportArea >= 230_400)
            return SimulcastLayer.Medium;

        return SimulcastLayer.Low;
    }
}

10. Audio Processing Pipeline

Audio quality is arguably more important than video quality in a video conference — participants will tolerate degraded video but not garbled audio. The audio processing pipeline must handle echo cancellation, noise suppression, automatic gain control, and voice activity detection.

Audio Processing Pipeline

mermaid
graph LR
    A[Microphone Input] --> B[Noise Suppression]
    B --> C[Automatic Gain Control]
    C --> D[Echo Cancellation]
    D --> E[Voice Activity Detection]
    E --> F[Comfort Noise Generation]
    F --> G[Opus Encoder]
    G --> H[Network Transport]
    H --> I[Opus Decoder]
    I --> J[Playout Buffer]
    J --> K[Speaker Output]
    

Echo Cancellation (AEC)

When a participant uses speakers (not headphones), the microphone picks up the speaker output, creating an echo that other participants hear. Acoustic Echo Cancellation (AEC) uses the far-end audio signal as a reference to identify and remove the echo from the microphone signal.

csharp
public class AudioProcessor
{
    private readonly EchoCanceller _aec;
    private readonly NoiseSuppressor _ns;
    private readonly AutomaticGainController _agc;
    private readonly VoiceActivityDetector _vad;
    private readonly ComfortNoiseGenerator _cng;

    public AudioProcessor()
    {
        _aec = new EchoCanceller(new EchoCancellerConfig
        {
            SuppressionLevel = EchoSuppressionLevel.High,
            SampleRate = 48000,
            FramesPerBuffer = 960,
            TailLength = 250
        });

        _ns = new NoiseSuppressor(new NoiseSuppressorConfig
        {
            SuppressionLevel = NoiseSuppressionLevel.High,
            SampleRate = 48000
        });

        _agc = new AutomaticGainController(new AGCConfig
        {
            TargetLevel = -3,
            CompressionGainDb = 12,
            LimiterEnable = true
        });

        _vad = new VoiceActivityDetector(new VADConfig
        {
            Sensitivity = 0.85,
            FrameSizeMs = 20
        });

        _cng = new ComfortNoiseGenerator(new CNGConfig
        {
            NoiseLevel = -60
        });
    }

    public AudioFrame ProcessFrame(AudioFrame inputFrame, AudioFrame? farEndReference)
    {
        var aecOutput = _aec.Process(inputFrame, farEndReference);
        var nsOutput = _ns.Process(aecOutput);
        var agcOutput = _agc.Process(nsOutput);
        var isSpeaking = _vad.Process(agcOutput);

        if (!isSpeaking)
        {
            return _cng.Generate();
        }

        return agcOutput;
    }
}

Jitter Buffer

The jitter buffer is a critical component on the receiver side that smooths out network jitter by buffering incoming audio packets before playback. A well-tuned jitter buffer balances latency against audio continuity:

• Static jitter buffer: Fixed delay (e.g., 60ms). Simple but suboptimal.
• Adaptive jitter buffer: Dynamically adjusts buffer depth based on measured jitter. Prefers lower latency when network is stable, increases buffer during congestion.
• Target buffer depth: Typically 60-120ms for VoIP. Beyond 200ms, participants notice conversational overlap issues.

11. Recording Architecture & Egress Service

Cloud recording is a critical feature for business video conferencing. Unlike local recording (which runs on the client), cloud recording involves a server-side component that subscribes to media streams and writes them to storage. This is complex because it requires a dedicated SFU subscriber (the "egress service") that joins the meeting as a virtual participant.

Recording Architecture

mermaid
graph TB
    subgraph "Meeting SFU"
        P1[Participant 1 Stream]
        P2[Participant 2 Stream]
        P3[Participant 3 Stream]
    end

    subgraph "Egress Service"
        E1[Egress Worker 1]
        E2[Egress Worker 2]
        R[Recording Coordinator]
    end

    subgraph "Storage"
        S3[S3 - Raw Segments]
        MP4[MP4/WebM Final File]
        DB[PostgreSQL - Recording Metadata]
    end

    P1 & P2 & P3 --> E1
    P1 & P2 & P3 --> E2
    E1 --> S3
    E2 --> S3
    R --> E1 & E2
    S3 --> MP4
    R --> DB
    

Egress Worker Implementation

csharp
public class EgressWorker : BackgroundService
{
    private readonly ISFUClient _sfuClient;
    private readonly IRecordingStorage _storage;
    private readonly ILogger<EgressWorker> _logger;

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            var job = await GetNextRecordingJob(stoppingToken);
            if (job == null)
            {
                await Task.Delay(1000, stoppingToken);
                continue;
            }

            try
            {
                await ProcessRecordingJob(job, stoppingToken);
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "Failed recording job {JobId}", job.JobId);
                await MarkJobFailed(job.JobId, ex.Message);
            }
        }
    }

    private async Task ProcessRecordingJob(RecordingJob job, CancellationToken ct)
    {
        _logger.LogInformation("Starting recording for room {RoomId}", job.RoomId);

        var session = await _sfuClient.JoinAsSubscriber(new SFUSubscriberConfig
        {
            RoomId = job.RoomId,
            SubscriptionType = SubscriptionType.AllSpeakers,
            SimulcastLayer = SimulcastLayer.High,
            AudioEnabled = true,
            VideoEnabled = true
        });

        var muxer = job.Format switch
        {
            RecordingFormat.WebM => new WebMMuxer(job.OutputPath),
            RecordingFormat.MP4 => new MP4Muxer(job.OutputPath),
            RecordingFormat.Zoom => new ZoomTSVMuxer(job.OutputPath),
            _ => throw new ArgumentException($"Unsupported format: {job.Format}")
        };

        var segmentDuration = TimeSpan.FromMinutes(5);
        var currentSegmentStart = DateTime.UtcNow;
        var segmentIndex = 0;

        await foreach (var mediaFrame in session.GetMediaFrames(ct))
        {
            await muxer.WriteFrame(mediaFrame);

            if (DateTime.UtcNow - currentSegmentStart >= segmentDuration)
            {
                await muxer.FinalizeSegment();

                await _storage.UploadSegment(new SegmentUpload
                {
                    RecordingId = job.RecordingId,
                    SegmentIndex = segmentIndex,
                    FilePath = muxer.CurrentFilePath,
                    Duration = segmentDuration
                });

                segmentIndex++;
                currentSegmentStart = DateTime.UtcNow;
                muxer.StartNewSegment();
            }

            await UpdateRecordingProgress(job.RecordingId, session.GetStats());
        }

        await muxer.FinalizeSegment();
        await _storage.UploadSegment(new SegmentUpload
        {
            RecordingId = job.RecordingId,
            SegmentIndex = segmentIndex,
            FilePath = muxer.CurrentFilePath,
            Duration = DateTime.UtcNow - currentSegmentStart
        });

        await _storage.ConcatenateSegments(job.RecordingId);
        await MarkJobCompleted(job.RecordingId);

        _logger.LogInformation(
            "Completed recording for room {RoomId}, {Segments} segments",
            job.RoomId, segmentIndex + 1);
    }
}

Recording Storage Strategy

12. Screen Sharing

Screen sharing is one of the most bandwidth-intensive features in video conferencing. A screen share of a 4K monitor with motion (e.g., video playback) can require 8-20 Mbps. The system must handle screen sharing differently from camera video to ensure quality and performance.

Screen Share Architecture

mermaid
graph TB
    A[User clicks Share Screen] --> B[getDisplayMedia API]
    B --> C{Share Type}
    C -->|Entire Screen| D[High Resolution Capture]
    C -->|Window| E[Window-specific Capture]
    C -->|Browser Tab| F[Tab Capture with Audio]

    D --> G[Screen Capture Encoder]
    E --> G
    F --> G

    G --> H[High Bitrate Track - 5-15 Mbps]
    H --> I[SFU]
    I --> J[Subscribers with Adaptive Quality]
    

Screen Share Configuration

csharp
public class ScreenShareConfigurator
{
    public ScreenShareConfig GetConfig(SharingContext context)
    {
        return context switch
        {
            SharingContext.StaticContent => new ScreenShareConfig
            {
                MaxResolution = new Resolution(1920, 1080),
                FrameRate = 5,
                MaxBitrate = 2_000_000,
                Codec = "VP9",
                ContentType = "detail"
            },
            SharingContext.DynamicContent => new ScreenShareConfig
            {
                MaxResolution = new Resolution(1920, 1080),
                FrameRate = 30,
                MaxBitrate = 10_000_000,
                Codec = "VP9",
                ContentType = "motion"
            },
            SharingContext.LowBandwidth => new ScreenShareConfig
            {
                MaxResolution = new Resolution(1280, 720),
                FrameRate = 15,
                MaxBitrate = 1_500_000,
                Codec = "H264",
                ContentType = "detail"
            },
            _ => throw new ArgumentException($"Unknown context: {context}")
        };
    }
}

13. In-Meeting Chat

The chat subsystem handles text messaging during meetings — public messages to all participants, private messages, and file sharing. While chat seems simple, it must handle concurrent messages, message ordering, read receipts, and persistence.

Chat Architecture

csharp
public class ChatService
{
    private readonly IConnectionMultiplexer _redis;
    private readonly IMessageStore _messageStore;
    private readonly IHubContext<ChatHub> _chatHub;

    public async Task<ChatMessage> SendMessage(ChatMessageRequest request)
    {
        var roomState = await GetRoomState(request.RoomId);
        if (!roomState.Participants.ContainsKey(request.SenderId))
        {
            throw new UnauthorizedAccessException("Not in room");
        }

        var sequenceNumber = await _redis.GetDatabase()
            .StringIncrementAsync($"chat:{request.RoomId}:seq");

        var message = new ChatMessage
        {
            Id = Guid.NewGuid().ToString("N"),
            RoomId = request.RoomId,
            SenderId = request.SenderId,
            SenderName = request.SenderName,
            Content = SanitizeContent(request.Content),
            Type = request.Type,
            TargetUserId = request.TargetUserId,
            SequenceNumber = sequenceNumber,
            CreatedAt = DateTimeOffset.UtcNow,
            Reactions = new List<ChatReaction>()
        };

        await _messageStore.SaveMessage(message);

        if (string.IsNullOrEmpty(request.TargetUserId))
        {
            await _chatHub.Clients.Group(request.RoomId)
                .SendAsync("NewMessage", message);
        }
        else
        {
            var senderConn = roomState.Participants[request.SenderId].ConnectionId;
            var targetConn = roomState.Participants[request.TargetUserId].ConnectionId;
            await _chatHub.Clients.Clients(senderConn, targetConn)
                .SendAsync("NewMessage", message);
        }

        return message;
    }

    private string SanitizeContent(string content)
    {
        content = Regex.Replace(content, "<[^>]*>", string.Empty);
        content = content.Length > 5000 ? content[..5000] : content;
        return content;
    }
}

14. Breakout Rooms

Breakout rooms allow a host to split meeting participants into smaller groups for focused discussion. This is a complex feature because it requires splitting the media topology, managing multiple simultaneous rooms, and allowing participants to move between rooms.

Breakout Room Architecture

mermaid
graph TB
    subgraph "Main Meeting"
        SFU_M[SFU - Main Room]
        H[Host]
    end

    subgraph "Breakout Room 1"
        SFU_B1[SFU - Breakout 1]
        P1[Participant A]
        P2[Participant B]
    end

    subgraph "Breakout Room 2"
        SFU_B2[SFU - Breakout 2]
        P3[Participant C]
        P4[Participant D]
    end

    H --> SFU_M
    P1 & P2 --> SFU_B1
    P3 & P4 --> SFU_B2

    SFU_M -.->|Broadcast Announcement| SFU_B1
    SFU_M -.->|Broadcast Announcement| SFU_B2
    SFU_B1 -.->|Return to Main| SFU_M
    SFU_B2 -.->|Return to Main| SFU_M
    

Breakout Room Implementation

csharp
public class BreakoutRoomManager
{
    private readonly ISFUPool _sfuPool;
    private readonly IRoomManager _roomManager;

    public async Task<BreakoutRoomPlan> CreateBreakoutRooms(
        string parentRoomId,
        BreakoutRoomRequest request)
    {
        var plan = new BreakoutRoomPlan
        {
            ParentRoomId = parentRoomId,
            CreatedAt = DateTimeOffset.UtcNow,
            Duration = request.Duration
        };

        for (int i = 0; i < request.RoomCount; i++)
        {
            var sfuNode = await _sfuPool.AllocateNode();
            var breakoutRoom = new BreakoutRoom
            {
                Id = Guid.NewGuid().ToString("N"),
                Name = $"Breakout Room {i + 1}",
                SFUNodeId = sfuNode.NodeId,
                ParentRoomId = parentRoomId,
                Participants = new List<string>()
            };

            plan.BreakoutRooms.Add(breakoutRoom);
        }

        var participants = await _roomManager.GetParticipants(parentRoomId);
        if (request.AutoAssign)
        {
            var shuffled = participants.OrderBy(_ => Random.Shared.Next()).ToList();
            for (int i = 0; i < shuffled.Count; i++)
            {
                var roomIndex = i % plan.BreakoutRooms.Count;
                plan.BreakoutRooms[roomIndex].Participants.Add(shuffled[i].UserId);
            }
        }
        else
        {
            foreach (var assignment in request.Assignments)
            {
                var room = plan.BreakoutRooms.FirstOrDefault(r => r.Id == assignment.RoomId);
                room?.Participants.AddRange(assignment.UserIds);
            }
        }

        return plan;
    }

    public async Task MoveParticipant(
        string participantId,
        string fromRoomId,
        string toRoomId)
    {
        var fromSFU = await _roomManager.GetSFUNode(fromRoomId);
        await fromSFU.RemoveParticipant(participantId);

        var toSFU = await _roomManager.GetSFUNode(toRoomId);
        var iceServers = await GetICEServers();

        await SignalNewConnection(participantId, toSFU, iceServers);
    }

    public async Task BroadcastToAllRooms(string parentRoomId, string message)
    {
        var breakoutRooms = await GetBreakoutRooms(parentRoomId);

        foreach (var room in breakoutRooms)
        {
            var sfu = await _sfuPool.GetNode(room.SFUNodeId);
            await sfu.SendDataChannelMessage(new DataChannelMessage
            {
                Type = MessageType.Broadcast,
                Content = message,
                Sender = "Host"
            });
        }
    }
}

15. Virtual Backgrounds (ML-Based)

Virtual backgrounds use machine learning models for real-time person segmentation, separating the user from their physical background and replacing it with an image, video, or blur effect. This feature must run entirely on the client (browser/desktop) with low latency and minimal CPU usage.

ML Segmentation Pipeline

mermaid
graph LR
    A[Camera Frame] --> B[Person Segmentation Model]
    B --> C[Binary Mask]
    C --> D{Background Mode}
    D -->|Blur| E[Gaussian Blur]
    D -->|Image| F[Background Image]
    D -->|Video| G[Background Video]
    E --> H[Composite Frame]
    F --> H
    G --> H
    H --> I[Encoder Input]
    

Client-Side Implementation

csharp
public class VirtualBackgroundProcessor
{
    private readonly SegmentationModel _model;
    private readonly WebGLRenderer _renderer;
    private readonly FrameBuffer _outputBuffer;

    public VirtualBackgroundProcessor()
    {
        _model = new SegmentationModel(new ModelConfig
        {
            ModelUrl = "https://cdn.example.com/models/selfie_segmenter.tflite",
            InputWidth = 256,
            InputHeight = 256,
            OutputWidth = 256,
            OutputHeight = 256,
            TFLite = true
        });

        _renderer = new WebGLRenderer();
        _outputBuffer = new FrameBuffer(1280, 720);
    }

    public async Task<VideoFrame> ProcessFrame(VideoFrame inputFrame)
    {
        var segmentationResult = await _model.Predict(inputFrame);
        var mask = GenerateMask(segmentationResult, featherRadius: 5);

        var outputTexture = _renderer.ApplyBackground(
            inputTexture: inputFrame.Texture,
            maskTexture: mask.Texture,
            backgroundTexture: _currentBackground.Texture,
            mode: _backgroundMode,
            blurStrength: 15.0f);

        return await _outputBuffer.ReadFromTexture(outputTexture);
    }

    private BinaryMask GenerateMask(SegmentationResult result, int featherRadius)
    {
        var rawMask = result.ProbabilityMap;
        var resizedMask = ResizeMask(rawMask, 1280, 720);
        var featheredMask = ApplyGaussianBlur(resizedMask, featherRadius);
        return new BinaryMask(featheredMask, threshold: 0.7f);
    }
}

public enum BackgroundMode
{
    None,
    Blur,
    Image,
    Video
}

16. Waiting Rooms & Participant Limits

Waiting rooms provide a security layer where participants wait in a virtual lobby until the host admits them. This prevents unauthorized access and allows hosts to vet participants before they join the meeting.

Waiting Room Flow

mermaid
sequenceDiagram
    participant P as Participant
    participant S as Signaling Server
    participant H as Host
    participant SFU as SFU Server

    P->>S: Join Meeting Request
    S->>S: Check Waiting Room Policy

    alt Waiting Room Enabled
        S-->>P: You are in the waiting room
        S->>H: New participant waiting
        H->>S: Admit Participant
        S->>SFU: Assign SFU node
        S-->>P: You have been admitted
    else Waiting Room Disabled
        S->>SFU: Assign SFU node
        S-->>P: Join confirmed
    end

    P->>SFU: Connect media (ICE + SDP)
    SFU-->>P: Media flowing
    

Participant Limits

csharp
public class ParticipantLimitService
{
    private readonly IPlanRepository _planRepo;

    public async Task<ParticipantLimitResult> CheckLimit(string roomId, string userId)
    {
        var meeting = await GetMeeting(roomId);
        var plan = await _planRepo.GetPlan(meeting.OrganizerId);

        var currentCount = await GetParticipantCount(roomId);
        var limit = plan.MaxParticipants;

        if (currentCount >= limit)
        {
            return new ParticipantLimitResult
            {
                Allowed = false,
                Reason = ParticipantLimitReason.MeetingFull,
                CurrentCount = currentCount,
                MaxAllowed = limit,
                WaitlistPosition = await GetWaitlistPosition(roomId)
            };
        }

        if (meeting.HasWaitingRoom && meeting.Role == MeetingRole.Webinar)
        {
            return new ParticipantLimitResult
            {
                Allowed = false,
                Reason = ParticipantLimitReason.WaitingRoom,
                RequiresHostApproval = true
            };
        }

        return new ParticipantLimitResult
        {
            Allowed = true,
            CurrentCount = currentCount + 1,
            MaxAllowed = limit
        };
    }
}

17. QoS Metrics & Monitoring

Quality of Service (QoS) monitoring is essential for maintaining a high-quality video conferencing experience. The system must continuously collect, aggregate, and analyze metrics from clients, SFUs, and network infrastructure.

Key QoS Metrics

Monitoring Stack

mermaid
graph TB
    subgraph "Data Collection"
        A[Client WebRTC Stats]
        B[SFU RTCP Reports]
        C[TURN Bandwidth Logs]
        D[Signaling Events]
    end

    subgraph "Metrics Pipeline"
        E[Prometheus]
        F[Grafana Dashboards]
        G[AlertManager]
    end

    A & B & C & D --> E
    E --> F & G
    

Client-Side Metrics Collection

csharp
public class QoSMetricsCollector
{
    private readonly RTCPeerConnection _peerConnection;
    private readonly Timer _collectionTimer;
    private readonly IMetricsReporter _reporter;

    public QoSMetricsCollector(RTCPeerConnection peerConnection, IMetricsReporter reporter)
    {
        _peerConnection = peerConnection;
        _reporter = reporter;
        _collectionTimer = new Timer(CollectMetrics, null,
            TimeSpan.Zero, TimeSpan.FromSeconds(5));
    }

    private async void CollectMetrics(object? state)
    {
        var stats = await _peerConnection.GetStats();
        var metrics = new QoSMetrics
        {
            Timestamp = DateTimeOffset.UtcNow,
            RoomId = _roomId,
            UserId = _userId
        };

        foreach (var stat in stats.Stats.Values)
        {
            switch (stat)
            {
                case InboundRTPVideoStats video:
                    metrics.VideoJitter = video.Jitter;
                    metrics.VideoPacketLoss = video.PacketsLost / (double)video.PacketsReceived;
                    metrics.VideoFrameRate = video.FramesPerSecond;
                    metrics.VideoWidth = video.FrameWidth;
                    metrics.VideoHeight = video.FrameHeight;
                    break;

                case InboundRTPAudioStats audio:
                    metrics.AudioJitter = audio.Jitter;
                    metrics.AudioPacketLoss = audio.PacketsLost / (double)audio.PacketsReceived;
                    metrics.AudioLevel = audio.AudioLevel;
                    metrics.AudioConcealment = audio.ConcealedSamples / (double)audio.TotalSamples;
                    break;

                case CandidatePairStats pair:
                    metrics.RTT = pair.CurrentRoundTripTime;
                    metrics.AvailableOutgoingBitrate = pair.AvailableOutgoingBitrate;
                    metrics.AvailableIncomingBitrate = pair.AvailableIncomingBitrate;
                    break;
            }
        }

        metrics.AudioMOS = CalculateMOS(
            metrics.RTT, metrics.AudioPacketLoss, metrics.AudioJitter);

        metrics.QualityScore = CalculateQualityScore(metrics);

        await _reporter.Report(metrics);
    }

    private double CalculateMOS(double rtt, double packetLoss, double jitter)
    {
        var effectiveLatency = rtt + jitter * 2 + 10;
        var rFactor = 93.2 - effectiveLatency / 40.0 - packetLoss * 2.5;

        if (rFactor < 0) rFactor = 0;
        if (rFactor > 100) rFactor = 100;

        if (rFactor < 6.5) return 1.0;
        return 1.0 + 0.035 * rFactor
             + rFactor * (rFactor - 60.0) * (100.0 - rFactor) * 7e-6;
    }
}

Alerting Rules

18. Security & Encryption

Security in video conferencing is paramount, especially for enterprise and healthcare use cases. The system must protect media in transit, at rest, and during processing. A layered security approach is essential.

Security Architecture

mermaid
graph TB
    subgraph "Transport Security"
        A[TLS 1.3 - Signaling]
        B[DTLS-SRTP - Media]
        C[E2EE - Insertable Streams]
    end

    subgraph "Authentication"
        D[OAuth 2.0 / SAML SSO]
        E[JWT Tokens]
        F[MFA - TOTP/WebAuthn]
    end

    subgraph "Authorization"
        G[Role-Based Access Control]
        H[Meeting Passwords]
        I[Waiting Rooms]
        J[Domain Restriction]
    end

    subgraph "Data Security"
        K[Encryption at Rest - AES-256]
        L[Recording Access Controls]
        M[Data Retention Policies]
        N[Audit Logging]
    end

    A & B & C --> K
    D & E & F --> G
    G --> H & I & J
    

DTLS-SRTP for Media Encryption

DTLS-SRTP is the standard mechanism for encrypting RTP media in WebRTC. It works as follows:

1. DTLS Handshake: After ICE selects a candidate pair, a DTLS handshake occurs over the same UDP port. This establishes shared secret keys.
2. Key Derivation: The DTLS handshake generates encryption keys using SRTP key derivation functions. Different keys are used for encryption, authentication, and salting.
3. SRTP Encryption: Each RTP packet is encrypted using AES-128-CM (Counter Mode) and authenticated using HMAC-SHA1.
4. SRTCP: RTCP control packets are also encrypted and authenticated using SRTCP.

SSO Integration

csharp
public class SSOService
{
    private readonly IConfiguration _config;

    public async Task<AuthResult> AuthenticateWithSSO(string provider, string redirectUri)
    {
        return provider.ToLower() switch
        {
            "google" => await AuthenticateGoogle(redirectUri),
            "microsoft" => await AuthenticateMicrosoft(redirectUri),
            "okta" => await AuthenticateSAML(redirectUri, "okta"),
            "azure-ad" => await AuthenticateAzureAD(redirectUri),
            _ => throw new ArgumentException($"Unsupported SSO provider: {provider}")
        };
    }

    private async Task<AuthResult> AuthenticateGoogle(string redirectUri)
    {
        var settings = new GoogleOpenIdConnectSettings
        {
            ClientId = _config["SSO:Google:ClientId"],
            ClientSecret = _config["SSO:Google:ClientSecret"],
            RedirectUri = redirectUri,
            Scopes = new[] { "openid", "profile", "email" }
        };

        var flow = new AuthorizationCodeFlow(settings);
        var token = await flow.ExchangeAuthorizationCodeAsync(
            await GetAuthorizationCode(settings));

        var payload = await VerifyGoogleToken(token.IdToken);

        return new AuthResult
        {
            UserId = payload.Subject,
            Email = payload.Email,
            Name = payload.Name,
            Domain = payload.Email.Split('@').Last(),
            TokenExpiry = DateTimeOffset.FromUnixTimeSeconds(payload.ExpirationTime)
        };
    }

    private async Task<AuthResult> AuthenticateAzureAD(string redirectUri)
    {
        var confidentialClient = ConfidentialClientApplicationBuilder
            .Create(_config["SSO:AzureAD:ClientId"])
            .WithClientSecret(_config["SSO:AzureAD:ClientSecret"])
            .WithRedirectUri(redirectUri)
            .WithAuthority($"https://login.microsoftonline.com/{_config["SSO:AzureAD:TenantId"]}")
            .Build();

        var authCode = await GetAuthorizationCode(confidentialClient);
        var result = await confidentialClient
            .AcquireTokenByAuthorizationCode(authCode)
            .WithScopes("openid", "profile", "email")
            .ExecuteAsync();

        return new AuthResult
        {
            UserId = result.UniqueId,
            Email = result.Account.Username,
            Name = result.Account.Name,
            Domain = result.Account.Username.Split('@').Last()
        };
    }
}

Access Control Matrix

19. Compliance (HIPAA, GDPR)

Video conferencing systems often handle sensitive personal and health information. Compliance with regulations like HIPAA (Health Insurance Portability and Accountability Act) and GDPR (General Data Protection Regulation) is not optional — it is a legal requirement for many use cases.

HIPAA Requirements for Video Conferencing

GDPR Data Processing

csharp
public class GDPRComplianceService
{
    private readonly IDataSubjectRepository _dataRepo;
    private readonly IAuditLogger _auditLogger;
    private readonly IRecordingStorage _recordingStorage;

    // Right to Access (Article 15)
    public async Task<DataExportResult> ExportUserData(string userId)
    {
        var userData = new DataExportResult
        {
            UserId = userId,
            ExportedAt = DateTimeOffset.UtcNow,
            PersonalData = await _dataRepo.GetPersonalData(userId),
            MeetingHistory = await _dataRepo.GetMeetingHistory(userId),
            ChatMessages = await _dataRepo.GetChatMessages(userId),
            Recordings = await _recordingStorage.GetUserRecordings(userId)
        };

        await _auditLogger.Log(new AuditEvent
        {
            Action = "GDPR_DATA_EXPORT",
            UserId = userId,
            Timestamp = DateTimeOffset.UtcNow
        });

        return userData;
    }

    // Right to Erasure / Right to be Forgotten (Article 17)
    public async Task EraseUserData(string userId, ErasureRequest request)
    {
        await _dataRepo.AnonymizePersonalData(userId);

        var recordings = await _recordingStorage.GetUserRecordings(userId);
        foreach (var recording in recordings)
        {
            await _recordingStorage.DeleteRecording(recording.Id, immediate: true);
        }

        if (request.DeleteChatMessages)
        {
            await _dataRepo.DeleteUserChatMessages(userId);
        }
        else
        {
            await _dataRepo.AnonymizeChatMessages(userId);
        }

        await _dataRepo.DeleteAccount(userId);

        await _auditLogger.Log(new AuditEvent
        {
            Action = "GDPR_DATA_ERASURE",
            UserId = userId,
            Details = request.ToString(),
            Timestamp = DateTimeOffset.UtcNow
        });

        await NotifyThirdPartyProcessors(userId, DataProcessingAction.Erasure);
    }

    // Data Protection Impact Assessment (DPIA)
    public DPIAReport GenerateDPIA()
    {
        return new DPIAReport
        {
            ProcessingActivity = "Video Conferencing",
            DataCategories = new[]
            {
                "Biometric data (face images in video)",
                "Audio recordings",
                "Location data (approximate, via IP)",
                "Device information",
                "Meeting metadata (duration, participants)"
            },
            LegalBasis = "Legitimate interest (business communication) + Consent (recording)",
            Risks = new[]
            {
                "Unauthorized access to meeting recordings",
                "Eavesdropping on live meetings",
                "Data breach exposing PII of participants"
            },
            Mitigations = new[]
            {
                "E2EE for live meetings",
                "AES-256 encryption for recordings at rest",
                "Role-based access control",
                "Audit logging for all data access",
                "Automatic data retention enforcement"
            }
        };
    }
}

20. API Design

The video conferencing platform exposes a RESTful API for meeting management, participant control, and recording operations. The API follows standard REST conventions with JWT authentication and consistent error handling.

Core API Endpoints

API Implementation

csharp
[ApiController]
[Route("api/v1/meetings")]
[Authorize]
public class MeetingsController : ControllerBase
{
    private readonly IMeetingService _meetingService;
    private readonly IAuthService _authService;

    [HttpPost]
    public async Task<ActionResult<MeetingResponse>> CreateMeeting(
        [FromBody] CreateMeetingRequest request)
    {
        var userId = _authService.GetUserId(User);

        var meeting = await _meetingService.CreateMeeting(new CreateMeetingCommand
        {
            OrganizerId = userId,
            Title = request.Title,
            ScheduledAt = request.ScheduledAt,
            Duration = request.Duration ?? TimeSpan.FromHours(1),
            Settings = new MeetingSettings
            {
                WaitingRoom = request.WaitingRoom ?? false,
                EnableRecording = request.EnableRecording ?? true,
                EnableChat = request.EnableChat ?? true,
                MaxParticipants = request.MaxParticipants ?? 100,
                Password = GenerateSecurePassword(),
                LobbyMessage = request.LobbyMessage
            }
        });

        return Ok(new MeetingResponse
        {
            MeetingId = meeting.Id,
            JoinUrl = $"{_config["App:BaseURL"]}/join/{meeting.Id}",
            Password = meeting.Settings.Password,
            CreatedAt = meeting.CreatedAt
        });
    }

    [HttpPost("{meetingId}/join")]
    public async Task<ActionResult<JoinResponse>> JoinMeeting(
        string meetingId,
        [FromBody] JoinMeetingRequest request)
    {
        var userId = _authService.GetUserId(User);

        var result = await _meetingService.JoinMeeting(new JoinMeetingCommand
        {
            MeetingId = meetingId,
            UserId = userId,
            Password = request.Password,
            DisplayName = request.DisplayName
        });

        if (result.RequiresWaitingRoom)
        {
            return Ok(new JoinResponse
            {
                Status = JoinStatus.WaitingRoom,
                Message = "Please wait for the host to admit you"
            });
        }

        return Ok(new JoinResponse
        {
            Status = JoinStatus.Admitted,
            Token = result.SessionToken,
            SFUEndpoint = result.SFUEndpoint,
            ICEConfig = result.ICEServers,
            RoomId = result.RoomId
        });
    }

    [HttpPost("{meetingId}/recording/start")]
    [Authorize(Roles = "Host,CoHost")]
    public async Task<ActionResult<RecordingResponse>> StartRecording(
        string meetingId,
        [FromBody] StartRecordingRequest request)
    {
        var recording = await _meetingService.StartRecording(new StartRecordingCommand
        {
            MeetingId = meetingId,
            Format = request.Format ?? RecordingFormat.MP4,
            Layout = request.Layout ?? RecordingLayout.Speaker,
            Quality = request.Quality ?? RecordingQuality.High
        });

        return Ok(new RecordingResponse
        {
            RecordingId = recording.Id,
            Status = RecordingStatus.Recording,
            StartedAt = recording.StartedAt
        });
    }
}

public record CreateMeetingRequest
{
    public string Title { get; init; } = string.Empty;
    public DateTimeOffset? ScheduledAt { get; init; }
    public TimeSpan? Duration { get; init; }
    public bool? WaitingRoom { get; init; }
    public bool? EnableRecording { get; init; }
    public bool? EnableChat { get; init; }
    public int? MaxParticipants { get; init; }
    public string? LobbyMessage { get; init; }
}

public record JoinResponse
{
    public JoinStatus Status { get; init; }
    public string? Token { get; init; }
    public string? SFUEndpoint { get; init; }
    public List<ICEServerConfig>? ICEConfig { get; init; }
    public string? RoomId { get; init; }
    public string? Message { get; init; }
}

WebSocket Signaling Protocol

csharp
public interface ISignalingClient
{
    Task Connect(string roomId, string token);
    Task SendOffer(string targetUserId, RTCSessionDescription offer);
    Task SendAnswer(string targetUserId, RTCSessionDescription answer);
    Task SendIceCandidate(string targetUserId, RTCIceCandidate candidate);
    Task PublishTrack(MediaTrackInfo track);
    Task UnpublishTrack(string trackId);
}

public interface ISignalingEvents
{
    Task OnParticipantJoined(ParticipantInfo participant);
    Task OnParticipantLeft(string userId);
    Task OnReceiveOffer(string fromUserId, RTCSessionDescription offer);
    Task OnReceiveAnswer(string fromUserId, RTCSessionDescription answer);
    Task OnReceiveIceCandidate(string fromUserId, RTCIceCandidate candidate);
    Task OnTrackPublished(string userId, MediaTrackInfo track);
    Task OnTrackUnpublished(string userId, string trackId);
    Task OnMeetingEnded(string reason);
    Task OnMuted(string userId, string trackKind);
}

21. Testing Strategy

Testing a video conferencing system is exceptionally challenging because it involves real-time media, network variability, and concurrent users. A comprehensive testing strategy requires multiple layers.

Testing Pyramid

mermaid
graph TB
    subgraph "Testing Layers"
        E2E["E2E Tests - Playwright/Cypress"]
        Integration["Integration Tests - SFU + Signaling"]
        Load["Load Tests - k6/Locust"]
        Chaos["Chaos Tests - SFU Failures"]
        Unit["Unit Tests - Codec, ABR, Auth"]
    end

    E2E --> Integration
    Integration --> Load
    Load --> Chaos
    Chaos --> Unit
    

Network Condition Simulation

csharp
public class NetworkConditionSimulator
{
    public static NetworkProfile[] TestProfiles = new[]
    {
        new NetworkProfile("Excellent", 50, 0.001, 10_000_000),
        new NetworkProfile("Good WiFi", 80, 0.01, 5_000_000),
        new NetworkProfile("Congested WiFi", 150, 0.03, 2_000_000),
        new NetworkProfile("4G Mobile", 120, 0.02, 3_000_000),
        new NetworkProfile("3G Mobile", 300, 0.05, 500_000),
        new NetworkProfile("Satellite", 600, 0.01, 1_000_000),
        new NetworkProfile("Packet Loss Spike", 100, 0.25, 5_000_000),
    };

    public static async Task RunLoadTest()
    {
        var config = new LoadTestConfig
        {
            TargetUrl = "wss://signaling.example.com",
            MaxConcurrentUsers = 1000,
            RampUpDuration = TimeSpan.FromMinutes(5),
            TestDuration = TimeSpan.FromMinutes(30),
            Scenarios = new[]
            {
                new LoadScenario
                {
                    Name = "Join and Hold",
                    Percentage = 80,
                    Action = async (client) =>
                    {
                        await client.CreateMeeting();
                        await client.JoinMeeting();
                        await Task.Delay(TimeSpan.FromMinutes(30));
                    }
                },
                new LoadScenario
                {
                    Name = "Join, Share, Leave",
                    Percentage = 15,
                    Action = async (client) =>
                    {
                        await client.JoinMeeting();
                        await client.ShareScreen();
                        await Task.Delay(TimeSpan.FromMinutes(5));
                        await client.LeaveMeeting();
                    }
                },
                new LoadScenario
                {
                    Name = "Chat Activity",
                    Percentage = 5,
                    Action = async (client) =>
                    {
                        await client.JoinMeeting();
                        for (int i = 0; i < 50; i++)
                        {
                            await client.SendChat($"Message {i}");
                            await Task.Delay(Random.Shared.Next(1000, 5000));
                        }
                    }
                }
            }
        };

        var results = await LoadTestRunner.Execute(config);

        Assert.Multiple(() =>
        {
            Assert.True(results.AverageJoinTime < TimeSpan.FromSeconds(2),
                $"Average join time {results.AverageJoinTime} exceeds 2s");
            Assert.True(results.P99JoinTime < TimeSpan.FromSeconds(5),
                $"P99 join time {results.P99JoinTime} exceeds 5s");
            Assert.True(results.PacketLossRate < 0.01,
                $"Packet loss rate {results.PacketLossRate} exceeds 1%");
        });
    }
}

Testing Matrix

22. Cost Estimation

Understanding the cost structure of a video conferencing platform is essential for pricing decisions and infrastructure planning. The primary cost drivers are SFU compute, TURN bandwidth, storage, and signaling infrastructure.

Cost Breakdown per Participant-Minute

Infrastructure Cost for Scale

csharp
public class CostEstimator
{
    public CostEstimate CalculateMonthlyCost(ScaleParameters parameters)
    {
        var estimate = new CostEstimate();

        // SFU Costs
        var sfuNodesNeeded = (int)Math.Ceiling(
            parameters.ConcurrentMeetings * parameters.AvgParticipantsPerMeeting / 500.0);

        estimate.SFUCompute = new ComputeCost
        {
            InstanceType = "c5.xlarge",
            InstanceCount = (int)(sfuNodesNeeded * 1.3),
            HourlyRate = 0.17,
            MonthlyCost = (int)(sfuNodesNeeded * 1.3) * 0.17 * 730,
            ReservedDiscount = 0.35
        };

        estimate.SFUCompute.ReservedMonthlyCost =
            estimate.SFUCompute.MonthlyCost * (1 - estimate.SFUCompute.ReservedDiscount);

        // Bandwidth Costs
        var monthlyParticipantMinutes =
            parameters.DailyMeetings *
            parameters.AvgMeetingDurationMinutes *
            parameters.AvgParticipantsPerMeeting * 30;

        var totalBandwidthGB = monthlyParticipantMinutes * 1.5 * 60 / 8 / 1000;

        estimate.Bandwidth = new BandwidthCost
        {
            TotalGB = totalBandwidthGB,
            PricePerGB = 0.085,
            MonthlyCost = totalBandwidthGB * 0.085
        };

        // Storage Costs
        var recordingsPerMonth = parameters.DailyMeetings *
            parameters.RecordingRate * 30;
        var avgRecordingSizeGB = 0.5;

        estimate.Storage = new StorageCost
        {
            S3StandardGB = recordingsPerMonth * avgRecordingSizeGB,
            S3GlacierGB = recordingsPerMonth * avgRecordingSizeGB * 2,
            S3StandardPrice = 0.023,
            S3GlacierPrice = 0.00099,
            MonthlyCost = recordingsPerMonth * avgRecordingSizeGB * 0.023
        };

        // Signaling Servers
        estimate.Signaling = new ComputeCost
        {
            InstanceType = "c5.large",
            InstanceCount = Math.Max(3, (int)(parameters.ConcurrentMeetings / 1000)),
            MonthlyCost = Math.Max(3, (int)(parameters.ConcurrentMeetings / 1000)) * 0.085 * 730
        };

        estimate.TotalMonthlyCost =
            estimate.SFUCompute.ReservedMonthlyCost +
            estimate.Bandwidth.MonthlyCost +
            estimate.Storage.MonthlyCost +
            estimate.Signaling.MonthlyCost;

        estimate.CostPerParticipantMinute =
            estimate.TotalMonthlyCost / monthlyParticipantMinutes;

        return estimate;
    }
}

// Example: 10,000 daily meetings, avg 20 participants, 30 min each
// Monthly participant-minutes: 10,000 x 30 x 20 x 30 = 180M
// SFU nodes needed: 10,000 x 20 / 500 = 400 nodes
// SFU compute (reserved): ~400 x 0.17 x 730 x 0.65 = $32,253/month
// Bandwidth: 180M x 1.5 x 60 / 8 / 1000 = 2,025 GB x $0.085 = $172/month
// Storage: 5,000 recordings x 0.5 GB x $0.023 = $57.50/month
// Total: ~$32,500/month = $0.00018 per participant-minute

23. Interview Q&A

These questions cover the most frequently asked system design interview topics related to video conferencing systems. Each answer highlights key trade-offs and demonstrates senior-level understanding.

Architecture Questions

Networking Questions

Scale and Reliability Questions

Security Questions

Design Summary

When discussing video conferencing in interviews, always emphasize the following key points:

• SFU over MCU: SFUs are the modern standard. Explain why (CPU savings, simulcast support, scalability).
• WebRTC is the foundation: Understand the full stack from getUserMedia to DTLS-SRTP.
• ICE/STUN/TURN is critical: NAT traversal is the hardest networking problem in real-time communication.
• Simulcast is the scaling trick: It trades sender bandwidth for receiver quality optimization.
• Audio matters more than video: Prioritize audio quality, echo cancellation, and low latency.
• Security is non-negotiable: DTLS-SRTP, E2EE, SSO, and compliance features are expected in enterprise.
• Cost is dominated by bandwidth: TURN relay and SFU data transfer are the primary cost drivers.