Designing a Multiplayer Game Server
Building Real-Time Competitive Infrastructure at Fortnite/Valorant Scale
Table of Contents
- Introduction — Why Multiplayer Game Servers Are Hard
- Functional & Non-Functional Requirements
- Capacity Estimation & Back-of-the-Envelope
- Data Model & Storage Design
- High-Level Architecture
- Game Loop, Tick Rate & Determinism
- Network Protocol — UDP vs TCP
- State Synchronization & Replication
- Lag Compensation, Prediction & Rollback
- Matchmaking System
- Game Session & Lobby Management
- Anti-Cheat System
- Leaderboard & Ranking — Elo / Glicko-2
- Real-Time Chat System
- Spectator Mode & Live Observers
- Replay System & Playback
- Reliability, Failure Modes & Disaster Recovery
- Cost Estimation
- Interview Questions & Answers
- Conclusion & Further Reading
1. Introduction — Why Multiplayer Game Servers Are Hard
Multiplayer game servers sit at the intersection of distributed systems, real-time networking, and physics simulation. Unlike a typical web application where a 200ms response time is perfectly acceptable, a competitive multiplayer shooter like Fortnite or Valorant demands sub-50ms server processing, deterministic physics, anti-cheat enforcement, and state synchronization for up to 100 concurrent players per match — all running across dozens of simultaneous matches in a single data center region.
The fundamental challenge is this: the game world must appear consistent and responsive to all players simultaneously, even though each player's client is running on a different machine, with different network latency, and possibly with adversarial modifications. The server is the single source of truth, and it must reconcile conflicting inputs, compensate for network delays, detect cheats, and maintain a fair experience — all within the tight budget of a single tick, often just 16.6 milliseconds at 60Hz or 33.3ms at 30Hz.
This guide walks through the complete system design of a production multiplayer game server infrastructure, covering everything from network protocol selection and lag compensation algorithms to matchmaking, anti-cheat, replay systems, and cost modeling. We draw on techniques used by studios like Epic Games (Fortnite), Riot Games (Valorant), Valve (CS2), and id Software (Quake) to build a comprehensive reference architecture suitable for senior, staff, and principal engineers preparing for system design interviews or evaluating real production architectures.
Why This Matters at Scale
Consider the operational demands: Fortnite峰值 during Chapter events has sustained over 12 million concurrent players. Valorant regularly supports millions of ranked matches per day. Each match requires a dedicated server instance running the authoritative game simulation, network replication to all participants, anti-cheat analysis, and post-match result processing. The infrastructure must scale horizontally, survive regional outages, and maintain consistent performance regardless of player population spikes.
Unlike stateless microservices that can be freely horizontally scaled, game server instances carry heavy per-instance state (the entire game world snapshot) and must maintain persistent network connections with bounded latency. This creates unique scaling, reliability, and operational challenges that do not arise in typical cloud application architectures.
2. Functional & Non-Functional Requirements
Functional Requirements
- Match Creation: Players can queue for matches, be matched by skill, and enter a lobby that transitions to an active game session.
- Real-Time Gameplay: The server runs an authoritative simulation of the game world. Player inputs (movement, shooting, building, abilities) are processed every tick and broadcast to all participants.
- State Synchronization: All clients receive a consistent, up-to-date view of the game state. Discrepancies due to latency are resolved through prediction, interpolation, and reconciliation.
- Matchmaking: Players are grouped into matches based on skill rating, queue time, party size, and server region to ensure balanced, low-latency games.
- Ranking & Leaderboards: Post-match rating adjustments using a skill-rating algorithm (Elo, Glicko-2, or TrueSkill). Global and regional leaderboards.
- Anti-Cheat: Detection and prevention of cheating through server-side validation, statistical analysis, and client-side integrity verification.
- Replay & Spectator: Record and playback game matches for review, content creation, and live spectating of tournaments.
- Real-Time Chat: In-match text and voice communication between teammates and between all players in a lobby.
- Progression & Inventory: Persistent player accounts, progression tracking, cosmetic items, battle pass progress, and statistics.
Non-Functional Requirements
| Requirement | Target | Rationale |
|---|---|---|
| Latency (tick processing) | < 5ms per server tick | Must leave headroom within the tick interval for network send/receive |
| Tick Rate | 30–64 Hz (server), 60 Hz (client render) | Competitive games require high tick rates for smooth, responsive gameplay |
| Concurrent Players | 10M+ online, 100K+ concurrent matches | Scale of major titles like Fortnite, Valorant, Apex Legends |
| Match Capacity | 2–100 players per match | Squad-based (4), duo (2), or battle royale (100) game modes |
| Packet Loss Tolerance | Resilient up to 5% loss | Typical consumer internet conditions |
| Jitter Tolerance | Up to 30ms variation | Mobile and congested network scenarios |
| Availability | 99.9% per region | Game servers are stateful; regional failover is the primary strategy |
| Security | Server-authoritative, cheat-resistant | Competitive integrity requires server as single source of truth |
3. Capacity Estimation & Back-of-the-Envelope
Player & Match Projections
Assume a global title targeting the scale of a mid-tier competitive shooter:
- Daily Active Users (DAU): 5 million
- Peak Concurrent Users (PCU): 1.5 million (typically 20–30% of DAU)
- Average Session Length: 45 minutes
- Average Players per Match: 50 (battle royale mode with squads)
- Peak Concurrent Matches: 1.5M / 50 = 30,000 simultaneous matches
Server Instance Sizing
Each match requires one authoritative server instance. At 30,000 peak concurrent matches, we need 30,000 game server instances running simultaneously. A single bare-metal or high-CPU cloud machine can typically host 2–4 game server instances depending on the game complexity:
| Parameter | Value | Notes |
|---|---|---|
| Game server CPU per instance | 2–4 vCPU (high clock) | Physics, game logic, anti-cheat per tick |
| Game server RAM per instance | 2–4 GB | World state, player snapshots, buffers |
| Instances per machine | 2–4 | Depends on game complexity and machine spec |
| Total machines needed (peak) | 7,500–15,000 | 30,000 matches / 2–4 per machine |
| Network bandwidth per instance | ~500 Kbps up, ~2 Mbps down | ~20 packets/sec × 100 players × ~1KB per packet |
Network Bandwidth Estimation
Each game server instance sends state updates to all connected clients. For a 50-player match at 30Hz:
- Outbound per tick: 50 players × 200 bytes/player state = 10 KB per tick
- Ticks per second: 30
- Outbound bandwidth per instance: 10 KB × 30 = 300 KB/s = 2.4 Mbps
- Total at 30K matches: 30,000 × 2.4 Mbps = 72 Gbps aggregate outbound
Storage Requirements
- Match results: ~50 players × 30K matches/day × 2 KB per result = 3 GB/day of metadata
- Replay data: ~500 KB per player per minute × 20 min avg × 5% of matches recorded = ~1.5 TB/day if recording all ranked matches
- Player profiles: 5M players × 10 KB = 50 GB (fits easily in a relational database or document store)
- Leaderboards: Sorted sets in Redis, ~50 MB for 5M ranked players
4. Data Model & Storage Design
The data model for a multiplayer game server splits cleanly into two domains: hot real-time state that lives in memory on the game server instance during a match, and persistent cold state stored in databases for profiles, progression, match history, and rankings.
Persistent Data Model (PostgreSQL / DynamoDB)
SQL
-- Player accounts and profiles
CREATE TABLE players (
player_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
username VARCHAR(32) UNIQUE NOT NULL,
email VARCHAR(255) UNIQUE NOT NULL,
password_hash VARCHAR(255) NOT NULL,
display_name VARCHAR(64) NOT NULL,
avatar_url TEXT,
created_at TIMESTAMPTZ DEFAULT NOW(),
last_login TIMESTAMPTZ,
is_banned BOOLEAN DEFAULT FALSE,
ban_reason TEXT,
ban_expires_at TIMESTAMPTZ
);
-- Skill ratings per game mode
CREATE TABLE player_ratings (
player_id UUID REFERENCES players(player_id),
game_mode VARCHAR(32) NOT NULL,
rating FLOAT NOT NULL DEFAULT 1500.0,
rating_dev FLOAT NOT NULL DEFAULT 350.0, -- Glicko-2 RD
volatility FLOAT NOT NULL DEFAULT 0.06, -- Glicko-2 sigma
games_played INT DEFAULT 0,
wins INT DEFAULT 0,
losses INT DEFAULT 0,
win_streak INT DEFAULT 0,
last_match_at TIMESTAMPTZ,
updated_at TIMESTAMPTZ DEFAULT NOW(),
PRIMARY KEY (player_id, game_mode)
);
-- Match results (append-only)
CREATE TABLE match_results (
match_id UUID PRIMARY KEY,
game_mode VARCHAR(32) NOT NULL,
region VARCHAR(16) NOT NULL,
map_name VARCHAR(64) NOT NULL,
server_id VARCHAR(64),
started_at TIMESTAMPTZ NOT NULL,
ended_at TIMESTAMPTZ NOT NULL,
duration_sec INT NOT NULL,
winning_team INT,
player_count INT NOT NULL,
replay_url TEXT,
metadata JSONB DEFAULT '{}'::jsonb
);
-- Individual player results per match
CREATE TABLE match_player_results (
match_id UUID REFERENCES match_results(match_id),
player_id UUID REFERENCES players(player_id),
team INT,
placement INT, -- 1st place in BR, etc.
kills INT DEFAULT 0,
deaths INT DEFAULT 0,
assists INT DEFAULT 0,
damage_dealt FLOAT DEFAULT 0,
damage_taken FLOAT DEFAULT 0,
survival_time FLOAT DEFAULT 0,
rating_before FLOAT,
rating_after FLOAT,
rating_change FLOAT,
PRIMARY KEY (match_id, player_id)
);
-- Battle pass and progression
CREATE TABLE player_progression (
player_id UUID REFERENCES players(player_id),
season_id INT NOT NULL,
xp BIGINT DEFAULT 0,
battle_pass_tier INT DEFAULT 0,
unlocks JSONB DEFAULT '[]'::jsonb,
updated_at TIMESTAMPTZ DEFAULT NOW(),
PRIMARY KEY (player_id, season_id)
);
CREATE INDEX idx_match_results_region_time ON match_results(region, started_at DESC);
CREATE INDEX idx_match_results_mode ON match_results(game_mode, started_at DESC);
CREATE INDEX idx_player_ratings_rating ON player_ratings(game_mode, rating DESC);
Hot State (In-Memory on Game Server)
The authoritative game state held in memory during a live match includes:
C#
public class MatchState
{
public Guid MatchId { get; set; }
public ulong ServerTick { get; set; } // Current tick number
public float TickRate { get; set; } // e.g., 30 Hz
public MatchPhase Phase { get; set; } // Waiting, InProgress, PostGame
public GameMode Mode { get; set; }
public MapDefinition Map { get; set; }
public DateTime MatchStartTime { get; set; }
public Dictionary<Guid, PlayerState> Players { get; set; } = new();
public List<ProjectileState> Projectiles { get; set; } = new();
public WorldState World { get; set; } // Buildings, items, vehicles, zones
public InputBuffer InputBuffer { get; set; } // Ring buffer of pending inputs
public EventLog Events { get; set; } // For replay recording
}
public class PlayerState
{
public Guid PlayerId { get; set; }
public int SlotIndex { get; set; }
public int TeamId { get; set; }
public Vector3 Position { get; set; }
public Quaternion Rotation { get; set; }
public Vector3 Velocity { get; set; }
public float Health { get; set; }
public float Shield { get; set; }
public bool IsAlive { get; set; }
public int Inventory { get; set; }
public InventoryItems InventoryItems { get; set; }
public PlayerInput LatestInput { get; set; }
public int LastProcessedTick { get; set; }
public MovementState MovementState { get; set; }
public BuildState BuildingState { get; set; } // Fortnite-specific
public Queue<PlayerInput> PendingInputs { get; set; } = new();
}
public enum MatchPhase
{
WaitingForPlayers,
Countdown,
InProgress,
EndGame,
PostMatch
}
Redis Layer
Redis serves multiple roles in the architecture: matchmaking queues, session lookups, real-time leaderboards, distributed locks, and pub/sub for cross-service communication.
C#
public class RedisGameCache
{
// Matchmaking queue: sorted set scored by rating
// ZADD matchmaking:ranked <rating> <playerId>
public async Task EnqueuePlayerAsync(string mode, Guid playerId, double rating)
{
await _redis.SortedSetAddAsync($"matchmaking:{mode}", playerId.ToString(), rating);
}
// Active match lookup: player -> server instance
public async Task<string?> GetPlayerMatchServerAsync(Guid playerId)
{
return await _redis.HashGetAsync("player:server:map", playerId.ToString());
}
// Session token -> player data for auth
public async Task<PlayerSession?> GetSessionAsync(string token)
{
var data = await _redis.HashGetAllAsync($"session:{token}");
return data.Length > 0 ? Deserialize<PlayerSession>(data) : null;
}
// Live leaderboard via sorted set
public async Task UpdateLeaderboardAsync(string mode, Guid playerId, double rating)
{
await _redis.SortedSetAddAsync($"leaderboard:{mode}", playerId.ToString(), rating);
}
// Retrieve top N players
public async Task<List<(string playerId, double rating)>> GetTopPlayersAsync(string mode, int count)
{
var results = await _redis.SortedSetRangeByScoreWithScoresAsync(
$"leaderboard:{mode}", 0, count - 1, order: Order.Descending);
return results.Select(r => (r.Element, r.Score)).ToList();
}
}
5. High-Level Architecture
The architecture is organized into several layers: the client-facing edge, the matchmaking and session orchestration layer, the game server fleet, the persistence and analytics layer, and the supporting services (chat, anti-cheat, replay).
Component Responsibilities
Game Gateway
The gateway handles initial client connections, authentication (JWT validation), and routes the client to the appropriate matchmaking queue or game server. It maintains a mapping of player-to-server-instance in Redis so reconnects and late joins can be handled without full re-authentication. The gateway also enforces rate limiting and DDoS protection at the edge.
Matchmaker Service
The matchmaker collects players into queues, groups them into balanced matches based on skill rating, party size, and latency, and then requests available server capacity from the Server Resolver. It uses an ELO-based bucketing system where players within a configurable rating range (e.g., ±200 ELO) are eligible for the same match. Queue timeout logic progressively widens the rating window to prevent indefinite waits.
Server Resolver
The Server Resolver maintains real-time capacity information for every game server instance in every region. When the matchmaker produces a candidate match, the resolver identifies the optimal server instance based on: (1) available capacity, (2) geographic proximity to the majority of players, (3) current load, and (4) network quality metrics. It communicates with the Game Server Daemon, which manages the lifecycle of individual server instances on each physical or virtual machine.
Game Server Daemon
The Game Server Daemon (GSD) runs on every machine hosting game server instances. It manages the lifecycle of server processes — starting new instances, monitoring health, collecting metrics, and terminating completed matches. The GSD reports capacity and load metrics to the Server Resolver via a persistent connection and supports rolling deployments of game updates without affecting active matches.
Game Server Instance
Each instance is the authoritative simulation for a single match. It runs the game loop, processes player inputs, executes physics, manages world state, runs anti-cheat checks, records replay data, and replicates state to all connected clients. The instance is completely isolated — no instance communicates with another instance during gameplay, which simplifies consistency guarantees significantly.
6. Game Loop, Tick Rate & Determinism
The game loop is the heartbeat of the server. Every tick, the server must: receive and buffer player inputs, process each input through the game logic, run physics simulation, check win conditions, log events for replay, and broadcast the updated state to all clients. The budget for all of this is the tick interval — 16.6ms at 60Hz or 33.3ms at 30Hz.
Server Game Loop
C#
public class GameServerLoop
{
private readonly MatchState _state;
private readonly NetworkManager _network;
private readonly PhysicsEngine _physics;
private readonly AntiCheatEngine _antiCheat;
private readonly ReplayRecorder _replay;
private readonly float _tickRate;
private readonly float _tickInterval; // seconds per tick
private bool _running;
private ulong _currentTick;
public GameServerLoop(MatchState state, NetworkManager network,
PhysicsEngine physics, AntiCheatEngine antiCheat,
ReplayRecorder replay, float tickRate = 30f)
{
_state = state;
_network = network;
_physics = physics;
_antiCheat = antiCheat;
_replay = replay;
_tickRate = tickRate;
_tickInterval = 1.0f / tickRate;
}
public async Task RunAsync(CancellationToken ct)
{
_running = true;
_currentTick = 0;
double nextTickTime = Stopwatch.GetTimestamp();
while (_running && !ct.IsCancellationRequested)
{
double currentTime = Stopwatch.GetTimestamp();
double elapsed = (currentTime - nextTickTime) / Stopwatch.Frequency;
if (elapsed < 0)
{
// Not time for next tick yet — yield CPU
await Task.Delay(1);
continue;
}
// Accumulator-based fixed timestep
double accumulator = elapsed;
nextTickTime += _tickInterval * Stopwatch.Frequency;
while (accumulator >= _tickInterval)
{
Tick(_currentTick);
_currentTick++;
accumulator -= _tickInterval;
}
// Anti-cheat runs asynchronously after each tick
_antiCheat.AnalyzeTick(_state);
}
}
private void Tick(ulong tick)
{
// 1. Receive and buffer all player inputs
var inputs = _network.CollectInputs();
// 2. Apply inputs to each player
foreach (var (playerId, input) in inputs)
{
if (_state.Players.TryGetValue(playerId, out var player) && player.IsAlive)
{
player.LatestInput = input;
player.LastProcessedTick = (int)tick;
_replay.RecordInput(tick, playerId, input);
}
}
// 3. Process game logic per player
foreach (var player in _state.Players.Values)
{
if (!player.IsAlive) continue;
ProcessPlayerMovement(player, _state);
ProcessPlayerActions(player, _state);
ProcessInventory(player, _state);
}
// 4. Process projectiles and collisions
_physics.StepSimulation(_state, _tickInterval);
// 5. Process game-specific systems
ProcessZoneShrinkage(_state, tick); // Battle royale zone
ProcessBuildingDecay(_state, tick); // Fortnite builds
ProcessAbilityCooldowns(_state, tick); // Valorant abilities
// 6. Check win conditions
var result = CheckWinConditions(_state);
if (result != null)
{
EndMatch(_state, result);
_running = false;
return;
}
// 7. Record snapshot for replay
_replay.RecordSnapshot(tick, _state);
// 8. Broadcast state to all clients
_network.BroadcastState(_state, tick);
// 9. Collect telemetry
CollectTickMetrics(tick);
}
private void ProcessPlayerMovement(PlayerState player, MatchState state)
{
var input = player.LatestInput;
// Server-authoritative movement: apply input on server
var desiredVelocity = CalculateDesiredVelocity(input, player.MovementState);
var newRotation = Quaternion.Lerp(
player.Rotation,
input.LookRotation,
_tickInterval * player.MovementState.TurnSpeed);
var collisionResult = _physics.MoveCharacter(
player.Position, desiredVelocity, _tickInterval);
player.Position = collisionResult.Position;
player.Rotation = newRotation;
player.Velocity = collisionResult.Velocity;
player.MovementState = collisionResult.NewMovementState;
}
}
Tick Rate Trade-offs
| Tick Rate | Interval | Input Lag Added | CPU Cost per Match | Use Case |
|---|---|---|---|---|
| 20 Hz | 50 ms | 25 ms avg | Baseline (1.0x) | Battle royale (large world, 100 players) |
| 30 Hz | 33 ms | 16.5 ms avg | 1.5x | Competitive BR, squad-based shooters |
| 64 Hz | 15.6 ms | 7.8 ms avg | 3.2x | Tactical shooters (CS2) |
| 128 Hz | 7.8 ms | 3.9 ms avg | 6.4x | Top-tier competitive (Valorant Premier) |
Determinism Considerations
While a dedicated server model does not strictly require deterministic simulation (since the server is the sole authority), determinism is valuable for replay playback and anti-cheat verification. Determinism means that given the same sequence of inputs, the server produces the same sequence of states. This requires careful handling of floating-point math, random number generation, and data structure iteration order. Use fixed-point math for physics calculations, seed-based PRNGs for random events, and sorted collections where iteration order matters.
7. Network Protocol — UDP vs TCP
The choice of network protocol is one of the most consequential decisions in multiplayer game server design. The two primary options — UDP and TCP — have fundamentally different trade-offs that directly impact gameplay feel and responsiveness.
Why UDP is the Standard for Real-Time Games
TCP guarantees ordered, reliable delivery through its built-in retransmission and flow control mechanisms. While this sounds desirable, it is catastrophic for real-time games due to head-of-line blocking. When a TCP packet is lost, all subsequent packets are buffered at the kernel level until the lost packet is retransmitted and received. This means a single lost packet can stall all subsequent updates for 100–300ms (one round-trip), causing rubber-banding, stuttering, and inconsistent game state.
UDP has no built-in ordering or reliability. Packets are sent independently, and if one is lost, it does not affect the delivery of subsequent packets. This makes UDP ideal for game state updates, where the latest state is always more valuable than any previous state. If a state packet is lost, the next state packet (arriving 30ms later) supersedes it anyway.
Hybrid Approach: UDP + Custom Reliability Layer
Modern game networks use a hybrid approach: UDP as the transport with a custom reliability layer on top. Time-sensitive data (player positions, inputs, physics state) is sent unreliably — a lost packet simply means waiting for the next one. Critical but non-time-sensitive data (match start/end events, player join/leave notifications, chat messages) is sent reliably with sequence numbers and acknowledgments at the application layer.
C#
public enum PacketChannel : byte
{
Unreliable = 0, // Position updates, physics snapshots
UnreliableSequenced = 1, // Latest state only (old packets discarded)
ReliableOrdered = 2, // Game events, match state changes
ReliableUnordered = 3, // Chat messages, inventory updates
}
public class GamePacketHeader
{
public ushort SequenceNumber { get; set; } // Per-channel sequence
public ushort AckSequence { get; set; } // Last received sequence from peer
public uint AckBitfield { get; set; } // Bitmask of received packets before ack
public PacketChannel Channel { get; set; }
public byte PayloadType { get; set; } // StateUpdate, Input, Event, etc.
public const int HeaderSize = 12; // bytes
public byte[] Serialize()
{
var buffer = new byte[HeaderSize];
var span = buffer.AsSpan();
BinaryPrimitives.WriteUInt16LittleEndian(span, SequenceNumber);
BinaryPrimitives.WriteUInt16LittleEndian(span[2..], AckSequence);
BinaryPrimitives.WriteUInt32LittleEndian(span[4..], AckBitfield);
span[8] = (byte)Channel;
span[9] = PayloadType;
// 2 bytes reserved
return buffer;
}
}
public class UnreliableChannel
{
private ushort _sequence;
public GamePacket CreatePacket(byte[] payload, byte payloadType)
{
_sequence++;
return new GamePacket
{
Header = new GamePacketHeader
{
SequenceNumber = _sequence,
Channel = PacketChannel.Unreliable,
PayloadType = payloadType
},
Payload = payload
};
}
}
public class ReliableChannel
{
private ushort _sendSequence;
private ushort _receiveSequence;
private readonly Dictionary<ushort, byte[]> _pendingAcks = new();
private readonly HashSet<ushort> _receivedSequences = new();
private readonly Queue<(ushort seq, byte[] data)> _receiveQueue = new();
public GamePacket CreateReliablePacket(byte[] payload, byte payloadType)
{
_sendSequence++;
var packet = new GamePacket
{
Header = new GamePacketHeader
{
SequenceNumber = _sendSequence,
Channel = PacketChannel.ReliableOrdered,
PayloadType = payloadType
},
Payload = payload
};
_pendingAcks[_sendSequence] = payload;
return packet;
}
public void OnAckReceived(ushort ackSequence, uint bitfield)
{
_pendingAcks.Remove(ackSequence);
for (int i = 0; i < 32; i++)
{
if ((bitfield & (1u << i)) != 0)
{
_pendingAcks.Remove((ushort)(ackSequence - 1 - i));
}
}
}
public void ResendLostPackets(Func<GamePacket, bool> sendFunc)
{
foreach (var (seq, data) in _pendingAcks)
{
var packet = new GamePacket
{
Header = new GamePacketHeader
{
SequenceNumber = seq,
Channel = PacketChannel.ReliableOrdered,
PayloadType = 0
},
Payload = data
};
sendFunc(packet);
}
}
}
Packet Structure
Game packets are compact — every byte counts when sending at high frequency. A typical state update packet structure:
| Field | Size | Description |
|---|---|---|
| Packet Header | 12 bytes | Sequence, ack, channel, type |
| Tick Number | 8 bytes | Server tick this state represents |
| Player Count | 2 bytes | Number of player states in packet |
| Per-Player State | ~150–300 bytes | Position, rotation, health, animation, etc. |
| World Events | Variable | Zone updates, building changes, item spawns |
| Checksum | 4 bytes | CRC32 of payload for corruption detection |
QUIC as an Emerging Alternative
QUIC (used by HTTP/3) is gaining traction in game networking because it operates over UDP but provides built-in streams, encryption, and ordered delivery per stream. This allows developers to use reliable ordered delivery for some data streams and unreliable delivery for others — all within the same QUIC connection. However, QUIC's connection establishment overhead and encryption add latency that may not suit the most latency-sensitive game data. Some studios (e.g., Google Stadia, Amazon Luna) have adopted QUIC-based protocols for cloud gaming where the bandwidth benefits outweigh the latency costs.
8. State Synchronization & Replication
State synchronization is the process of keeping all clients seeing a consistent game world. The server is authoritative — it computes the "true" game state each tick — and clients must be brought as close to that state as possible despite network latency, packet loss, and varying connection quality.
Delta Compression
Sending the full world state every tick is wasteful. Most of the world does not change between ticks. Delta compression sends only the differences between the last acknowledged state and the current state, dramatically reducing bandwidth.
C#
public class DeltaCompressor
{
private readonly Dictionary<Guid, PlayerSnapshot> _lastAckedSnapshots = new();
public DeltaState ComputeDelta(MatchState current, Guid playerId)
{
var delta = new DeltaState { Tick = current.ServerTick };
foreach (var (pid, currentState) in current.Players)
{
if (!_lastAckedSnapshots.TryGetValue(pid, out var lastAcked))
{
// Player is new — send full state
delta.FullStates.Add(pid, currentState.ToSnapshot());
continue;
}
var playerDelta = new PlayerDelta { PlayerId = pid };
// Position delta — only send if changed by more than threshold
float posDelta = Vector3.Distance(lastAcked.Position, currentState.Position);
if (posDelta > 0.01f)
{
playerDelta.Position = currentState.Position;
playerDelta.DeltaFlags |= DeltaFlags.PositionChanged;
}
// Rotation delta — quantized to save bandwidth
if (Quaternion.Angle(lastAcked.Rotation, currentState.Rotation) > 0.5f)
{
playerDelta.Rotation = QuantizeRotation(currentState.Rotation);
playerDelta.DeltaFlags |= DeltaFlags.RotationChanged;
}
// Health always sent (critical)
if (Math.Abs(lastAcked.Health - currentState.Health) > 0.01f)
{
playerDelta.Health = currentState.Health;
playerDelta.DeltaFlags |= DeltaFlags.HealthChanged;
}
// Shield
if (Math.Abs(lastAcked.Shield - currentState.Shield) > 0.01f)
{
playerDelta.Shield = currentState.Shield;
playerDelta.DeltaFlags |= DeltaFlags.ShieldChanged;
}
// Alive status — always sent when changed
if (lastAcked.IsAlive != currentState.IsAlive)
{
playerDelta.IsAlive = currentState.IsAlive;
playerDelta.DeltaFlags |= DeltaFlags.AliveChanged;
}
if (playerDelta.DeltaFlags != DeltaFlags.None)
{
delta.PlayerDeltas.Add(playerDelta);
}
}
// Track what we sent for this player's ack
foreach (var (pid, state) in current.Players)
{
_lastAckedSnapshots[pid] = state.ToSnapshot();
}
return delta;
}
private static ushort QuantizeRotation(Quaternion q)
{
// Compress quaternion to 16 bits using smallest-three encoding
float maxVal = Math.Max(Math.Max(Math.Abs(q.X), Math.Abs(q.Y)),
Math.Max(Math.Abs(q.Z), Math.Abs(q.W)));
int i0 = 0, i1 = 1, i2 = 2;
float sign = q.W >= 0 ? 1f : -1f;
// ... smallest-three compression logic
return (ushort)((i0 << 14) | (i1 << 12) |
((ushort)((q.X / maxVal * 0.5f + 0.5f) * 4095) << 0));
}
}
[Flags]
public enum DeltaFlags : byte
{
None = 0,
PositionChanged = 1,
RotationChanged = 2,
HealthChanged = 4,
ShieldChanged = 8,
AliveChanged = 16,
InventoryChanged = 32,
AnimationChanged = 64,
FullSnapshot = 128
}
Client-Side Interpolation
Clients receive state updates at the server's tick rate (e.g., 30Hz) but render at a higher rate (e.g., 60Hz or 144Hz). Between server ticks, the client interpolates between the last two received states to produce smooth motion. The client renders approximately one server tick behind the latest received state to always have two snapshots to interpolate between.
C#
public class ClientInterpolator
{
private readonly Queue<StateSnapshot> _snapshotBuffer = new();
private const int InterpolationDelayTicks = 2; // Render 2 ticks behind
public void OnStateReceived(StateSnapshot snapshot)
{
_snapshotBuffer.Enqueue(snapshot);
}
public PlayerRenderState InterpolatePlayer(Guid playerId, float renderTime)
{
var snapshots = _snapshotBuffer.ToArray();
if (snapshots.Length < 3)
return snapshots.Length > 0
? snapshots[^1].GetPlayerState(playerId)
: default;
// Target render tick = latest server tick - interpolation delay
ulong targetTick = snapshots[^1].Tick - InterpolationDelayTicks;
// Find the two snapshots surrounding the target tick
StateSnapshot from = null, to = null;
for (int i = 0; i < snapshots.Length - 1; i++)
{
if (snapshots[i].Tick <= targetTick && snapshots[i + 1].Tick >= targetTick)
{
from = snapshots[i];
to = snapshots[i + 1];
break;
}
}
if (from == null || to == null)
return snapshots[^1].GetPlayerState(playerId);
// Linear interpolation factor
float t = (float)(targetTick - from.Tick) / (to.Tick - from.Tick);
var fromState = from.GetPlayerState(playerId);
var toState = to.GetPlayerState(playerId);
return new PlayerRenderState
{
Position = Vector3.Lerp(fromState.Position, toState.Position, t),
Rotation = Quaternion.Slerp(fromState.Rotation, toState.Rotation, t),
Health = Mathf.Lerp(fromState.Health, toState.Health, t),
AnimationFrame = Mathf.Lerp(fromState.AnimationFrame, toState.AnimationFrame, t)
};
}
}
Priority-Based Replication
Not all entities need to be replicated to all players at the same rate. A player 500 meters away does not need per-tick position updates. Priority-based replication assigns update frequency based on relevance (distance, line of sight, importance) to each client, significantly reducing bandwidth.
9. Lag Compensation, Prediction & Rollback
Lag compensation is the most technically complex subsystem in a competitive game server. The fundamental problem: a player shoots at what they see on screen, but by the time their input reaches the server, the target has moved. Without lag compensation, the server would evaluate the shot against the target's current position — not the position the shooter saw — making it impossible to hit fast-moving targets at high latency.
Server-Side Lag Compensation
The server maintains a circular buffer of historical world states (typically the last 1 second of snapshots). When a player fires a shot, the client sends the tick number they were rendering at the moment of the shot. The server rewinds the world to that historical state and evaluates the hit detection against the rewound positions.
C#
public class LagCompensator
{
private readonly Queue<WorldSnapshot> _history = new();
private const int MaxHistoryTicks = 60; // 1 second at 60Hz
public void RecordSnapshot(ulong tick, MatchState state)
{
var snapshot = new WorldSnapshot
{
Tick = tick,
Timestamp = DateTime.UtcNow,
Players = state.Players.ToDictionary(
kvp => kvp.Key,
kvp => kvp.Value.ToSnapshot())
};
_history.Enqueue(snapshot);
while (_history.Count > MaxHistoryTicks)
_history.Dequeue();
}
public HitResult EvaluateHit(
Guid shooterId,
Vector3 shotOrigin,
Vector3 shotDirection,
float weaponRange,
ulong clientTick,
ulong serverTick)
{
// Calculate how many ticks back to rewind
ulong ticksBack = serverTick - clientTick;
if (ticksBack > MaxHistoryTicks)
ticksBack = MaxHistoryTicks; // Clamp to prevent stale lookups
// Find the historical snapshot closest to client's tick
WorldSnapshot? rewoundState = null;
foreach (var snapshot in _history)
{
if (snapshot.Tick <= clientTick)
{
rewoundState = snapshot;
break;
}
}
if (rewoundState == null)
return HitResult.Miss; // No historical data available
// Evaluate hit detection against rewound player positions
foreach (var (playerId, playerSnap) in rewoundState.Players)
{
if (playerId == shooterId) continue; // Can't shoot yourself
if (!playerSnap.IsAlive) continue;
// Sphere or capsule intersection test
var hitbox = GetPlayerHitbox(playerSnap);
if (RaycastCapsule(shotOrigin, shotDirection, weaponRange,
hitbox, out float hitDistance, out Vector3 hitPoint))
{
float damage = CalculateDamage(weaponRange, hitDistance, hitPoint);
return new HitResult
{
Hit = true,
TargetId = playerId,
Damage = damage,
HitPoint = hitPoint,
RewoundTicks = ticksBack,
RewoundPosition = playerSnap.Position
};
}
}
return HitResult.Miss;
}
private bool RaycastCapsule(Vector3 origin, Vector3 direction, float maxDistance,
Capsule hitbox, out float distance, out Vector3 hitPoint)
{
// GJK algorithm or simplified sphere-sweep for performance
hitPoint = Vector3.Zero;
distance = 0;
// Simplified: check ray vs expanded sphere (capsule as two spheres + cylinder)
float expandedRadius = hitbox.Radius;
Vector3 center = hitbox.Bottom + (hitbox.Top - hitbox.Bottom) * 0.5f;
float halfHeight = Vector3.Distance(hitbox.Bottom, hitbox.Top) * 0.5f;
// Ray-swept sphere test along capsule axis
Vector3 oc = origin - center;
float a = Vector3.Dot(direction, direction);
float b = 2f * Vector3.Dot(oc, direction);
float c = Vector3.Dot(oc, oc) - expandedRadius * expandedRadius;
float discriminant = b * b - 4 * a * c;
if (discriminant < 0) return false;
distance = (-b - Mathf.Sqrt(discriminant)) / (2 * a);
if (distance < 0 || distance > maxDistance) return false;
hitPoint = origin + direction * distance;
return true;
}
}
public struct WorldSnapshot
{
public ulong Tick { get; set; }
public DateTime Timestamp { get; set; }
public Dictionary<Guid, PlayerSnapshot> Players { get; set; }
}
public struct HitResult
{
public bool Hit { get; set; }
public Guid TargetId { get; set; }
public float Damage { get; set; }
public Vector3 HitPoint { get; set; }
public ulong RewoundTicks { get; set; }
public Vector3 RewoundPosition { get; set; }
public static HitResult Miss => new() { Hit = false };
}
Client-Side Prediction
Without prediction, the client would have to wait one round-trip for the server to acknowledge each movement input before updating the player's position on screen. This adds 30–100ms of apparent input lag. Client-side prediction resolves this by having the client immediately apply its own inputs locally, showing instant response, while simultaneously sending the input to the server.
When the server's acknowledgment arrives with the authoritative position, the client compares it to its predicted position. If they match, no correction is needed. If they diverge (due to collision with an invisible wall, a cheat detection correction, or a physics discrepancy), the client smoothly reconciles back to the server's position.
C#
public class ClientPredictor
{
private readonly Queue<PendingInput> _pendingInputs = new();
private Vector3 _lastServerPosition;
private ulong _lastServerTick;
public void OnLocalInput(PlayerInput input, ulong currentTick)
{
// Immediately apply input locally for responsive feel
ApplyInput(input);
// Store for reconciliation when server acknowledges
_pendingInputs.Enqueue(new PendingInput
{
Tick = currentTick,
Input = input,
PredictedPosition = LocalPlayer.Position,
PredictedRotation = LocalPlayer.Rotation
});
}
public void OnServerAck(Guid playerId, ulong tick, Vector3 authoritativePos,
Quaternion authoritativeRot)
{
// Remove all inputs up to and including the acknowledged tick
while (_pendingInputs.Count > 0 && _pendingInputs.Peek().Tick <= tick)
{
var acked = _pendingInputs.Dequeue();
}
// Compare predicted vs authoritative
if (_pendingInputs.Count > 0)
{
var latestPredicted = _pendingInputs.Peek().PredictedPosition;
float error = Vector3.Distance(latestPredicted, authoritativePos);
if (error > 0.01f)
{
// Server correction needed
StartReconciliation(authoritativePos, authoritativeRot);
}
}
_lastServerPosition = authoritativePos;
_lastServerTick = tick;
}
private void StartReconciliation(Vector3 targetPos, Quaternion targetRot)
{
// Snap to server position and re-apply all pending (unacknowledged) inputs
LocalPlayer.Position = targetPos;
LocalPlayer.Rotation = targetRot;
foreach (var pending in _pendingInputs)
{
ApplyInput(pending.Input);
}
}
}
Entity Interpolation for Other Players
While the local player uses prediction, other players' positions are displayed via interpolation between server-received states. This avoids the rubber-banding effect — other players appear to move smoothly between the states the server has confirmed, with no speculative prediction of their movements.
10. Matchmaking System
The matchmaking system is responsible for assembling fair, low-latency matches from a pool of queuing players. A good matchmaking system balances match quality (skill balance) against queue time — players want fair matches but do not want to wait forever to find one.
Matchmaking Algorithm
C#
public class Matchmaker
{
private readonly RedisGameCache _redis;
private readonly ServerResolver _serverResolver;
private readonly float _baseRatingRange;
private readonly float _rangeExpansionPerSecond;
private readonly int _targetTeamSize;
private readonly int _targetTeamCount;
public Matchmaker(RedisGameCache redis, ServerResolver serverResolver,
float baseRatingRange = 100f, float rangeExpansionPerSecond = 5f)
{
_redis = redis;
_serverResolver = serverResolver;
_baseRatingRange = baseRatingRange;
_rangeExpansionPerSecond = rangeExpansionPerSecond;
_targetTeamSize = 4; // Squad-based
_targetTeamCount = 25; // 25 squads = 100 players
}
public async Task<MatchCandidate?> TryFormMatchAsync(string gameMode, string region)
{
int totalPlayersNeeded = _targetTeamSize * _targetTeamCount;
// Get players in queue sorted by wait time (FIFO within rating buckets)
var queuedPlayers = await _redis.GetQueuedPlayersAsync(gameMode, region);
if (queuedPlayers.Count < totalPlayersNeeded)
return null;
// Calculate adaptive rating range based on oldest player's wait time
var oldestPlayer = queuedPlayers[0];
float waitSeconds = (float)(DateTime.UtcNow - oldestPlayer.QueuedAt).TotalSeconds;
float ratingRange = _baseRatingRange +
(waitSeconds * _rangeExpansionPerSecond);
// Try to find a match within the expanded rating range
double medianRating = queuedPlayers[queuedPlayers.Count / 2].Rating;
double minRating = medianRating - ratingRange;
double maxRating = medianRating + ratingRange;
var eligible = queuedPlayers
.Where(p => p.Rating >= minRating && p.Rating <= maxRating)
.ToList();
if (eligible.Count < totalPlayersNeeded)
return null;
// Team composition: balance teams by aggregate rating
var teams = ComposeBalancedTeams(eligible, totalPlayersNeeded);
// Find optimal server region
var playerIds = eligible.Select(p => p.PlayerId).ToList();
var serverInfo = await _serverResolver.FindBestServerAsync(
gameMode, region, playerIds);
if (serverInfo == null) return null;
// Create match candidate
return new MatchCandidate
{
MatchId = Guid.NewGuid(),
GameMode = gameMode,
Players = eligible.Select(p => new MatchPlayer
{
PlayerId = p.PlayerId,
Team = teams[p.PlayerId],
Rating = p.Rating,
PartyId = p.PartyId
}).ToList(),
ServerInstanceId = serverInfo.InstanceId,
Region = serverInfo.Region,
AverageRating = eligible.Average(p => p.Rating),
RatingSpread = eligible.Max(p => p.Rating) -
eligible.Min(p => p.Rating),
CreatedAt = DateTime.UtcNow
};
}
private Dictionary<Guid, int> ComposeBalancedTeams(
List<QueuedPlayer> players, int totalNeeded)
{
var teams = new Dictionary<Guid, int>();
int teamCount = totalNeeded / _targetTeamSize;
// Sort by rating descending for snake draft
var sorted = players.OrderBy(p => -p.Rating).ToList();
for (int i = 0; i < totalNeeded; i++)
{
int teamIndex;
int round = i / teamCount;
if (round % 2 == 0)
teamIndex = i % teamCount;
else
teamIndex = teamCount - 1 - (i % teamCount);
teams[sorted[i].PlayerId] = teamIndex;
}
return teams;
}
}
Rating Algorithm Comparison
| Algorithm | System | Strengths | Weaknesses |
|---|---|---|---|
| Elo | Chess, early LoL | Simple, well-understood, fast | Assumes 1v1, no confidence metric |
| Glicko-2 | CS:GO (MMR), Pokemon Showdown | Rating deviation (RD), volatility, handles inactivity | More complex to tune parameters |
| TrueSkill | Xbox Live, Halo | Handles team games, party queue, multi-skill | Microsoft patent, Bayesian inference is expensive |
| TrueSkill 2 | Halo Infinite, newer Xbox titles | Handles quit/AFK, fireteams, multi-objective | Complex, requires lots of tuning data |
| OpenSkill | Open-source alternative | MIT licensed, handles teams, Bayesian | Less battle-tested at scale |
11. Game Session & Lobby Management
Game session management handles the full lifecycle of a match: from when players are matched together, through the pre-game lobby, the active match, and post-match results. This state machine must be robust — players can disconnect, servers can crash, and the system must handle all edge cases gracefully.
Session State Machine
C#
public class GameSessionManager
{
private readonly RedisGameCache _redis;
private readonly Dictionary<Guid, GameSession> _sessions = new();
public async Task<GameSession> CreateSessionAsync(MatchCandidate candidate)
{
var session = new GameSession
{
MatchId = candidate.MatchId,
State = SessionState.Creating,
Players = candidate.Players.ToDictionary(
p => p.PlayerId, p => new SessionPlayer
{
PlayerId = p.PlayerId,
Team = p.Team,
State = PlayerSessionState.Connecting,
ConnectionToken = GenerateToken(),
ConnectedAt = null,
DisconnectedAt = null
}),
ServerInstanceId = candidate.ServerInstanceId,
Region = candidate.Region,
CreatedAt = DateTime.UtcNow,
StateHistory = new List<SessionStateTransition>
{
new(SessionState.Creating, DateTime.UtcNow)
}
};
_sessions[candidate.MatchId] = session;
// Register player-to-server mapping in Redis
foreach (var player in candidate.Players)
{
await _redis.SetPlayerServerAsync(
player.PlayerId, candidate.ServerInstanceId, candidate.MatchId);
}
// Notify the game server to allocate the match
var serverReady = await NotifyGameServerAllocateAsync(candidate);
if (!serverReady)
{
session.State = SessionState.Failed;
await HandleSessionFailure(session, "Server allocation failed");
return session;
}
session.State = SessionState.WaitingForPlayers;
return session;
}
public async Task OnPlayerConnected(Guid matchId, Guid playerId)
{
var session = _sessions[matchId];
var player = session.Players[playerId];
player.State = PlayerSessionState.Connected;
player.ConnectedAt = DateTime.UtcNow;
// Check if all players connected
if (session.Players.Values.All(p => p.State == PlayerSessionState.Connected))
{
session.State = SessionState.AllPlayersConnected;
session.GameStartTime = DateTime.UtcNow.AddSeconds(10); // 10s countdown
}
await UpdateSessionRedis(session);
}
public async Task OnPlayerDisconnected(Guid matchId, Guid playerId)
{
var session = _sessions[matchId];
var player = session.Players[playerId];
player.State = PlayerSessionState.Disconnected;
player.DisconnectedAt = DateTime.UtcNow;
// In competitive games, disconnects are punished ( deserter penalty )
// But the match continues — AI takes over or player is removed
// If too many players disconnect, consider ending the match
int connected = session.Players.Values
.Count(p => p.State == PlayerSessionState.Connected);
int total = session.Players.Count;
if (connected < total * 0.5)
{
// Less than 50% connected — cancel match
session.State = SessionState.Cancelled;
await HandleMatchCancellation(session);
}
await UpdateSessionRedis(session);
}
public async Task EndSession(Guid matchId, MatchResult result)
{
var session = _sessions[matchId];
session.State = SessionState.Completed;
session.Result = result;
session.EndedAt = DateTime.UtcNow;
// Process rating changes
foreach (var player in session.Players.Values)
{
if (player.State == PlayerSessionState.Disconnected)
{
// Apply deserter penalty
await ApplyDeserterPenaltyAsync(player.PlayerId);
}
}
// Publish match result event
await PublishMatchResult(session, result);
// Cleanup Redis mappings
foreach (var player in session.Players.Values)
{
await _redis.RemovePlayerServerAsync(player.PlayerId);
}
// Remove from active sessions after retention period
_ = Task.Delay(TimeSpan.FromMinutes(30)).ContinueWith(_ =>
{
_sessions.Remove(matchId);
});
}
}
public class GameSession
{
public Guid MatchId { get; set; }
public SessionState State { get; set; }
public Dictionary<Guid, SessionPlayer> Players { get; set; }
public Guid ServerInstanceId { get; set; }
public string Region { get; set; }
public DateTime CreatedAt { get; set; }
public DateTime? GameStartTime { get; set; }
public DateTime? EndedAt { get; set; }
public MatchResult? Result { get; set; }
public List<SessionStateTransition> StateHistory { get; set; }
}
public enum SessionState
{
Creating,
WaitingForPlayers,
AllPlayersConnected,
InGame,
PostGame,
Completed,
Cancelled,
Failed
}
12. Anti-Cheat System
Cheating is an existential threat to competitive multiplayer games. A single cheater can ruin the experience for 99 other players. Modern anti-cheat operates at multiple layers: client-side integrity verification, server-side behavioral analysis, and post-match statistical detection.
Multi-Layer Anti-Cheat Architecture
Server-Side Validation Code
C#
public class ServerAntiCheat
{
private readonly float _maxMovementSpeed;
private readonly float _maxJumpHeight;
private readonly float _maxFireRate;
private readonly float _maxTurnSpeed;
private readonly Dictionary<Guid, AntiCheatPlayerData> _playerData = new();
public ServerAntiCheat(GameConfig config)
{
_maxMovementSpeed = config.MaxMovementSpeed; // e.g., 600 units/sec
_maxJumpHeight = config.MaxJumpHeight; // e.g., 300 units
_maxFireRate = config.MaxFireRate; // e.g., 10 rounds/sec
_maxTurnSpeed = config.MaxTurnSpeed; // e.g., 180 degrees/sec
}
public CheatDetectionResult ValidatePlayerInput(
Guid playerId, PlayerInput input, PlayerState currentState, ulong tick)
{
var data = _playerData.GetOrAdd(playerId, _ => new AntiCheatPlayerData());
var violations = new List<CheatViolation>();
// 1. Movement speed validation
float distance = Vector3.Distance(data.LastPosition, currentState.Position);
float speed = distance * (1.0f / _tickInterval);
if (speed > _maxMovementSpeed * 1.1f) // 10% tolerance for floating point
{
violations.Add(new CheatViolation
{
Type = CheatType.SpeedHack,
Severity = ViolationSeverity.High,
Details = $"Speed {speed:F1} exceeds max {_maxMovementSpeed}"
});
}
// 2. Teleportation detection — large position jumps between ticks
if (distance > _maxMovementSpeed * _tickInterval * 3f)
{
violations.Add(new CheatViolation
{
Type = CheatType.TeleportHack,
Severity = ViolationSeverity.Critical,
Details = $"Teleported {distance:F1} units in one tick"
});
}
// 3. Fire rate validation
float timeSinceLastShot = (tick - data.LastShotTick) * _tickInterval;
if (timeSinceLastShot < 1.0f / _maxFireRate * 0.9f)
{
data.RapidFireCount++;
if (data.RapidFireCount > 3)
{
violations.Add(new CheatViolation
{
Type = CheatType.RapidFire,
Severity = ViolationSeverity.High,
Details = $"Fire rate {1f / timeSinceLastShot:F1} exceeds max {_maxFireRate}"
});
}
}
else
{
data.RapidFireCount = Math.Max(0, data.RapidFireCount - 1);
}
// 4. Turn speed validation (anti-aimbot)
float turnAngle = Quaternion.Angle(data.LastRotation, currentState.Rotation);
float turnSpeed = turnAngle / _tickInterval;
if (turnSpeed > _maxTurnSpeed * 1.2f)
{
data.SuspiciousTurnCount++;
if (data.SuspiciousTurnCount > 5)
{
violations.Add(new CheatViolation
{
Type = CheatType.Aimbot,
Severity = ViolationSeverity.Medium,
Details = $"Turn speed {turnSpeed:F0} deg/s exceeds max {_maxTurnSpeed}"
});
}
}
// 5. Line of sight check (wallhack detection)
if (input.IsFiring && !HasLineOfSight(currentState.Position, input.AimDirection))
{
violations.Add(new CheatViolation
{
Type = CheatType.Wallhack,
Severity = ViolationSeverity.Medium,
Details = "Firing without line of sight to target"
});
}
// Update tracking data
data.LastPosition = currentState.Position;
data.LastRotation = currentState.Rotation;
data.LastShotTick = tick;
// Accumulate violations for pattern detection
data.ViolationHistory.Add((tick, violations));
return new CheatDetectionResult
{
PlayerId = playerId,
Violations = violations,
ShouldKick = violations.Any(v => v.Severity == ViolationSeverity.Critical),
ShouldBan = data.ViolationHistory.Count(v =>
v.Violations.Any(x => x.Severity >= ViolationSeverity.High)) > 10,
Confidence = CalculateConfidence(data)
};
}
private float CalculateConfidence(AntiCheatPlayerData data)
{
int highSeverity = data.ViolationHistory
.Count(v => v.Violations.Any(x => x.Severity == ViolationSeverity.High));
int mediumSeverity = data.ViolationHistory
.Count(v => v.Violations.Any(x => x.Severity == ViolationSeverity.Medium));
float confidence = Math.Min(1.0f,
(highSeverity * 0.3f) + (mediumSeverity * 0.1f));
return confidence;
}
}
Post-Match Statistical Detection
Some cheats are too sophisticated to detect in real-time. Post-match analysis examines aggregate statistics across many matches to identify statistical outliers. Players with headshot rates above 95%, impossible kill-to-death ratios, or abnormally high damage-per-game metrics are flagged for manual review or automated temporary bans.
13. Leaderboard & Ranking — Elo / Glicko-2
A robust ranking system provides players with a sense of progression and competition. The ranking system must be fair (accurately reflect skill), transparent (players understand what affects their rating), and resistant to manipulation (boosting, smurfing, account sharing).
Glicko-2 Implementation
C#
public class Glicko2Rating
{
private const double DefaultRating = 1500.0;
private const double DefaultDeviation = 350.0;
private const double DefaultVolatility = 0.06;
private const double Tau = 0.5; // System volatility constraint
private const double Epsilon = 0.000001; // Convergence tolerance
public double Rating { get; set; } = DefaultRating;
public double RatingDeviation { get; set; } = DefaultDeviation;
public double Volatility { get; set; } = DefaultVolatility;
public int GamesPlayed { get; set; }
/// <summary>
/// Update rating after a set of match results using Glicko-2 algorithm
/// </summary>
public void UpdateRating(List<MatchResult_Glicko> results)
{
if (results.Count == 0) return;
// Step 1: Convert to Glicko-2 scale
double mu = (Rating - DefaultRating) / DefaultDeviation;
double phi = RatingDeviation / DefaultDeviation;
double sigma = Volatility;
// Step 2: Compute the estimated pre-rating period improvement (v)
double v = ComputeV(mu, phi, sigma, results);
// Step 3: Compute the improvement delta (delta)
double delta = ComputeDelta(mu, phi, v, results);
// Step 4: Update volatility (sigma) using Illinois algorithm
double newSigma = UpdateVolatility(phi, delta, v, sigma);
// Step 5: Update rating deviation (phi)
double phiStar = Math.Sqrt(phi * phi + newSigma * newSigma);
double newPhi = 1.0 / Math.Sqrt(1.0 / (phiStar * phiStar) + 1.0 / v);
// Step 6: Update rating (mu)
double muPrime = mu + (newPhi * newPhi / 1.0) *
ComputeGlicko2G(phi, results);
// Convert back to Glicko scale
Rating = muPrime * DefaultDeviation + DefaultRating;
RatingDeviation = newPhi * DefaultDeviation;
Volatility = newSigma;
GamesPlayed += results.Count;
// Clamp values
Rating = Math.Max(100, Math.Min(4000, Rating));
RatingDeviation = Math.Max(30, Math.Min(350, RatingDeviation));
}
private double ComputeV(double mu, double phi, double sigma,
List<MatchResult_Glicko> results)
{
double sum = 0;
foreach (var result in results)
{
double opponentMu = (result.OpponentRating - DefaultRating) / DefaultDeviation;
double opponentPhi = result.OpponentDeviation / DefaultDeviation;
double gPhi = ComputeGlicko2G(opponentPhi, null);
double expectedScore = ComputeExpectedScore(mu, opponentMu, gPhi);
sum += gPhi * gPhi * expectedScore * (1 - expectedScore);
}
return 1.0 / sum;
}
private double ComputeDelta(double mu, double phi, double v,
List<MatchResult_Glicko> results)
{
double sum = 0;
foreach (var result in results)
{
double opponentMu = (result.OpponentRating - DefaultRating) / DefaultDeviation;
double opponentPhi = result.OpponentDeviation / DefaultDeviation;
double gPhi = ComputeGlicko2G(opponentPhi, null);
double expectedScore = ComputeExpectedScore(mu, opponentMu, gPhi);
sum += gPhi * (result.Score - expectedScore);
}
return v * sum;
}
private double ComputeGlicko2G(double phi, object? _)
{
return 1.0 / Math.Sqrt(1.0 + (3.0 * phi * phi) / (Math.PI * Math.PI));
}
private double ComputeExpectedScore(double mu, double opponentMu, double g)
{
return 1.0 / (1.0 + Math.Exp(-g * (mu - opponentMu)));
}
private double UpdateVolatility(double phi, double delta, double v, double sigma)
{
// Illinois algorithm for finding new volatility
double a = Math.Log(sigma * sigma);
double A = a;
double f = x => Math.Exp(x) * (delta * delta - phi * phi - v - Math.Exp(x))
/ (2 * Math.Pow(phi * phi + v + Math.Exp(x), 2)) - (x - a) / (Tau * Tau);
double B, fB;
if (delta * delta > phi * phi + v)
{
B = Math.Log(delta * delta - phi * phi - v);
fB = f(B);
}
else
{
double k = 1;
B = a - k * Tau;
fB = f(B);
while (fB < 0)
{
k++;
B = a - k * Tau;
fB = f(B);
}
}
double fA = f(A);
while (Math.Abs(B - A) > Epsilon)
{
double C = A + (A - B) * fA / (fB - fA);
double fC = f(C);
if (fC * fB <= 0)
{
A = B;
fA = fB;
}
B = C;
fB = fC;
}
return Math.Exp(A / 2);
}
public int GetRankTier()
{
// Map rating to competitive rank tiers
if (Rating < 1200) return 1; // Bronze
if (Rating < 1500) return 2; // Silver
if (Rating < 1800) return 3; // Gold
if (Rating < 2100) return 4; // Platinum
if (Rating < 2400) return 5; // Diamond
if (Rating < 2700) return 6; // Master
return 7; // Grandmaster
}
}
Rating Distribution & Rank Tiers
| Tier | Rating Range | Percentile | Typical Population |
|---|---|---|---|
| Bronze | 0 – 1200 | Bottom 30% | 30% |
| Silver | 1200 – 1500 | 30% – 55% | 25% |
| Gold | 1500 – 1800 | 55% – 75% | 20% |
| Platinum | 1800 – 2100 | 75% – 88% | 13% |
| Diamond | 2100 – 2400 | 88% – 95% | 7% |
| Master | 2400 – 2700 | 95% – 98.5% | 3.5% |
| Grandmaster | 2700+ | Top 1.5% | 1.5% |
14. Real-Time Chat System
Real-time chat is a core social feature in multiplayer games. It includes pre-game lobby chat, in-match team voice/text chat, post-game chat, and global channels. The chat system must handle high throughput, support moderation, and integrate with the game client's UI.
Chat Architecture
Chat Service Implementation
C#
public class ChatService
{
private readonly IConnectionMultiplexer _redis;
private readonly ISubscriber _pubsub;
private readonly ChatModerationService _moderation;
private readonly ConcurrentDictionary<string, ChatChannel> _channels = new();
public ChatService(IConnectionMultiplexer redis, ChatModerationService moderation)
{
_redis = redis;
_pubsub = redis.GetSubscriber();
_moderation = moderation;
// Subscribe to Redis pub/sub for multi-node message distribution
_pubsub.Subscribe("chat:global", async (channel, message) =>
{
await HandleGlobalMessage(ChatMessage.Deserialize(message));
});
}
public async Task<bool> SendMessageAsync(ChatMessage message)
{
// 1. Rate limiting
if (!await CheckRateLimit(message.SenderId))
return false;
// 2. Moderation — profanity filter, spam detection, threat detection
var modResult = await _moderation.ModerateAsync(message);
if (modResult.Blocked)
{
await NotifySenderBlocked(message.SenderId, modResult.Reason);
return false;
}
if (modResult.Filtered)
{
message.Content = modResult.FilteredContent;
}
// 3. Route to appropriate channel
switch (message.ChannelType)
{
case ChannelType.Match:
await PublishToMatchChannel(message);
break;
case ChannelType.Team:
await PublishToTeamChannel(message);
break;
case ChannelType.Global:
await _pubsub.PublishAsync("chat:global",
message.Serialize());
break;
case ChannelType.Whisper:
await DeliverWhisperAsync(message);
break;
}
// 4. Persist for history
await PersistMessageAsync(message);
return true;
}
private async Task PublishToMatchChannel(ChatMessage message)
{
string channelKey = $"chat:match:{message.MatchId}";
await _pubsub.PublishAsync(channelKey, message.Serialize());
}
private async Task PublishToTeamChannel(ChatMessage message)
{
string channelKey = $"chat:match:{message.MatchId}:team:{message.TeamId}";
await _pubsub.PublishAsync(channelKey, message.Serialize());
}
private async Task PersistMessageAsync(ChatMessage message)
{
var db = _redis.GetDatabase();
string key = $"chat:history:{message.MatchId}";
var entry = new SortedSetEntry(message.Serialize(), message.Timestamp.Ticks);
await db.SortedSetAddAsync(key, entry);
await db.KeyExpireAsync(key, TimeSpan.FromDays(30));
}
}
public class ChatModerationService
{
private readonly List<IChatFilter> _filters = new()
{
new ProfanityFilter(),
new SpamDetector(),
new LinkFilter(),
new ThreatDetector(),
};
public async Task<ModerationResult> ModerateAsync(ChatMessage message)
{
foreach (var filter in _filters)
{
var result = await filter.CheckAsync(message);
if (result.Blocked)
return result;
if (result.Filtered)
message = result.FilteredMessage;
}
return ModerationResult.Allowed;
}
}
15. Spectator Mode & Live Observers
Spectator mode allows players and tournament organizers to watch live matches without participating. This is critical for esports broadcasts, content creation, and player education. Spectators receive a delayed version of the game state (typically 30–90 seconds behind) to prevent ghosting — the act of sharing enemy positions with active players.
Spectator Architecture
C#
public class SpectatorManager
{
private readonly Dictionary<Guid, SpectatorSession> _sessions = new();
private readonly int _spectatorDelayTicks = 900; // 30 seconds at 30Hz
public async Task<SpectatorSession?> JoinAsSpectator(
Guid matchId, Guid viewerId, bool isCaster)
{
var match = await GetMatchState(matchId);
if (match == null) return null;
// Check if spectators are allowed for this match
if (!match.AllowsSpectators && !isCaster)
return null;
// Rate limit spectator connections
if (_sessions.Count(s => s.Value.MatchId == matchId) > 50 && !isCaster)
return null;
var session = new SpectatorSession
{
SessionId = Guid.NewGuid(),
MatchId = matchId,
ViewerId = viewerId,
IsCaster = isCaster,
DelayTicks = isCaster ? 300 : _spectatorDelayTicks, // Casters get shorter delay
JoinedAt = DateTime.UtcNow,
FocusPlayerId = null, // Free camera by default
};
_sessions[session.SessionId] = session;
return session;
}
public StateSnapshot GetDelayedSnapshot(Guid matchId, ulong currentTick)
{
// Spectators see the state from N ticks ago
ulong targetTick = currentTick - (ulong)_spectatorDelayTicks;
return GetHistoricalSnapshot(matchId, targetTick);
}
public void UpdateSpectatorFocus(Guid sessionId, Guid? playerId, CameraMode mode)
{
if (_sessions.TryGetValue(sessionId, out var session))
{
session.FocusPlayerId = playerId;
session.CameraMode = mode;
}
}
}
public enum CameraMode
{
FreeCamera,
FirstPerson,
ThirdPerson,
TopDown,
Replay, // For post-match replay viewing
DirectorAI // AI-controlled camera for automated broadcasts
}
public class SpectatorSession
{
public Guid SessionId { get; set; }
public Guid MatchId { get; set; }
public Guid ViewerId { get; set; }
public bool IsCaster { get; set; }
public int DelayTicks { get; set; }
public Guid? FocusPlayerId { get; set; }
public CameraMode CameraMode { get; set; }
public DateTime JoinedAt { get; set; }
}
16. Replay System & Playback
The replay system records match data for playback, analysis, and content creation. Unlike video recording, a replay records game inputs and events, allowing the game engine to reconstruct the match from any perspective. This is vastly more storage-efficient and provides interactive viewing capabilities.
Replay Recording Architecture
C#
public class ReplayRecorder
{
private readonly ReplayBuffer _buffer;
private readonly int _flushIntervalTicks = 900; // Flush every 30 seconds at 30Hz
private int _ticksSinceFlush;
private readonly string _replayId;
private readonly List<ReplayEvent> _events = new();
private bool _recording;
public ReplayRecorder(string replayId, ReplayBuffer buffer)
{
_replayId = replayId;
_buffer = buffer;
_recording = true;
}
public void RecordSnapshot(ulong tick, MatchState state)
{
if (!_recording) return;
// Record only what's needed for replay reconstruction
var snapshot = new ReplaySnapshot
{
Tick = tick,
Timestamp = DateTime.UtcNow,
Players = state.Players.ToDictionary(
kvp => kvp.Key,
kvp => new ReplayPlayerSnapshot
{
Position = kvp.Value.Position,
Rotation = kvp.Value.Rotation,
Health = kvp.Value.Health,
Shield = kvp.Value.Shield,
IsAlive = kvp.Value.IsAlive,
WeaponId = kvp.Value.InventoryItems.CurrentWeapon,
AnimationState = kvp.Value.MovementState.Animation,
BuildState = kvp.Value.BuildingState
}),
WorldEvents = new List<WorldEvent>(),
ZoneState = state.World.ZoneState
};
_buffer.WriteSnapshot(snapshot);
_ticksSinceFlush++;
if (_ticksSinceFlush >= _flushIntervalTicks)
{
_buffer.FlushToDisk();
_ticksSinceFlush = 0;
}
}
public void RecordInput(ulong tick, Guid playerId, PlayerInput input)
{
if (!_recording) return;
_buffer.WriteInput(new ReplayInput
{
Tick = tick,
PlayerId = playerId,
Input = input
});
}
public void RecordEvent(ReplayEvent evt)
{
if (!_recording) return;
_events.Add(evt);
_buffer.WriteEvent(evt);
}
public async Task<ReplayMetadata> FinalizeAsync()
{
_recording = false;
_buffer.FlushToDisk();
var metadata = new ReplayMetadata
{
ReplayId = _replayId,
TotalTicks = _buffer.CurrentTick,
FileSize = _buffer.GetTotalBytes(),
Duration = _buffer.GetDuration(),
PlayerCount = _buffer.GetPlayerCount(),
Events = _events.Count,
CreatedAt = DateTime.UtcNow
};
// Upload to object storage
var storagePath = await UploadToStorageAsync(_buffer.GetFilePath());
metadata.StorageUrl = storagePath;
// Compress replay data
await CompressReplayAsync(storagePath);
return metadata;
}
}
public class ReplayPlayer
{
private readonly List<ReplaySnapshot> _snapshots;
private readonly List<ReplayInput> _inputs;
private int _currentIndex;
public void SeekToTick(ulong tick)
{
_currentIndex = _snapshots.FindIndex(s => s.Tick >= tick);
if (_currentIndex < 0) _currentIndex = 0;
}
public ReplaySnapshot GetCurrentSnapshot()
{
return _snapshots[_currentIndex];
}
public bool Advance()
{
if (_currentIndex < _snapshots.Count - 1)
{
_currentIndex++;
return true;
}
return false;
}
public List<ReplayInput> GetInputsForTick(ulong tick)
{
return _inputs.Where(i => i.Tick == tick).ToList();
}
}
Replay File Format
Replay files use a chunked binary format for efficient sequential reading and writing:
| Chunk Type | Content | Frequency | Size per Chunk |
|---|---|---|---|
| HEADER | Metadata, version, checksums | Once at start | ~2 KB |
| SNAPSHOT | Full world state at tick | Every tick (30Hz) | ~5–15 KB |
| INPUT | Player input for tick | Per player per tick | ~20–50 bytes |
| EVENT | Game events (kills, abilities) | As they occur | ~100–500 bytes |
| AUDIO | Voice chat recordings | Continuous | ~8 KB/s (compressed) |
17. Reliability, Failure Modes & Disaster Recovery
Multiplayer game servers face a unique reliability challenge: stateful sessions that cannot be trivially migrated. Unlike a web request that can be retried on a different server, a game match in progress is a continuous real-time simulation. The system must be designed to minimize impact when failures occur.
Failure Mode Analysis
| Failure | Impact | Detection | Mitigation |
|---|---|---|---|
| Game server crash | Match ends abruptly for all players | Heartbeat timeout (3s) | Restart match on new instance if early; end match with partial results if late |
| Player disconnect | Single player loses connection | Client heartbeat (5s timeout) | Grace period (30s) for reconnect; AI takes over; removal if prolonged |
| Matchmaker failure | No new matches created | Health check, queue depth monitoring | Matchmaker is stateless — horizontal scaling; fallback to regional queue |
| Redis failure | Session lookups fail; leaderboards stale | Redis sentinel failover | Read replicas; graceful degradation (cached session data in game server memory) |
| Database failure | Match results not persisted | Connection pool monitoring | Write-ahead log on game server; retry on recovery |
| Network partition | Region isolation | Cross-region health checks | Each region operates independently; no cross-region dependencies during gameplay |
| DDoS attack | Server overload, packet flood | Traffic anomaly detection | Anycast CDN; rate limiting; IP reputation filtering; scrubbing centers |
Graceful Degradation Strategy
C#
public class ResilienceManager
{
private readonly CircuitBreaker _matchmakerCircuit;
private readonly CircuitBreaker _databaseCircuit;
private readonly CircuitBreaker _redisCircuit;
public ResilienceManager()
{
_matchmakerCircuit = new CircuitBreaker(5, TimeSpan.FromSeconds(30));
_databaseCircuit = new CircuitBreaker(3, TimeSpan.FromSeconds(60));
_redisCircuit = new CircuitBreaker(3, TimeSpan.FromSeconds(15));
}
public async Task OnGameServerCrash(Guid matchId, List<Guid> playerIds)
{
// 1. Notify all connected players
foreach (var playerId in playerIds)
{
await NotifyPlayerMatchEnded(playerId, MatchEndReason.ServerCrash);
}
// 2. Attempt to recreate the match if it was in early stages
var matchInfo = await GetMatchInfo(matchId);
if (matchInfo.ElapsedTime < TimeSpan.FromMinutes(5))
{
// Early game — try to restart
await AttemptMatchRestart(matchId, playerIds);
}
else
{
// Late game — award partial results based on current standings
await AwardPartialResults(matchId);
}
// 3. Log the incident for operations
await LogIncidentAsync(new Incident
{
Type = IncidentType.GameServerCrash,
MatchId = matchId,
Severity = Severity.Medium,
AffectedPlayers = playerIds.Count,
Timestamp = DateTime.UtcNow
});
}
private async Task AttemptMatchRestart(Guid matchId, List<Guid> playerIds)
{
// Find a new server and restart the match
var server = await FindAvailableServer(matchId);
if (server == null)
{
// No servers available — fall back to partial results
await AwardPartialResults(matchId);
return;
}
// Re-create match with same players
// Players who reconnect within 60 seconds are placed back in
await CreateNewMatchWithPlayers(playerIds, server, TimeSpan.FromSeconds(60));
}
public async Task<T> ExecuteWithFallback<T>(
Func<Task<T>> primary,
Func<Task<T>> fallback,
CircuitBreaker circuit)
{
if (circuit.IsOpen)
{
return await fallback();
}
try
{
var result = await primary();
circuit.RecordSuccess();
return result;
}
catch (Exception ex)
{
circuit.RecordFailure();
return await fallback();
}
}
}
Rolling Deployment Strategy
Deploying updates to a live game with millions of active matches requires careful orchestration. The standard approach is a rolling deployment that: (1) stops allocating new matches to servers being updated, (2) waits for active matches to complete on those servers, (3) updates the server binaries, (4) health-checks the updated servers, (5) resumes allocation. This ensures no active match is interrupted. For emergency patches (critical bug fixes), a forced restart can be triggered, but this is reserved for severe issues only.
18. Cost Estimation
Multiplayer game servers are infrastructure-intensive. The primary cost driver is the dedicated compute required to run authoritative game simulations — these cannot be shared with other workloads and must maintain high single-thread performance.
Infrastructure Cost Breakdown
| Component | Spec | Monthly Cost (per unit) | Quantity (Peak) | Monthly Total |
|---|---|---|---|---|
| Game Servers (bare-metal) | 16-core / 3.5GHz, 32GB RAM, 10Gbps NIC | $1,200 | 10,000 | $12,000,000 |
| Matchmaker (cloud VMs) | 8 vCPU, 16GB RAM | $400 | 50 | $20,000 |
| Redis Cluster | 6-node cluster, 64GB RAM each | $2,000/node | 3 clusters (18 nodes) | $36,000 |
| PostgreSQL (RDS) | db.r6g.2xlarge, Multi-AZ | $3,500 | 4 (primary + replicas) | $14,000 |
| Chat Servers | 8 vCPU, 16GB RAM | $400 | 20 | $8,000 |
| Voice Relay Servers | 16 vCPU, 32GB RAM, high NIC | $800 | 30 | $24,000 |
| Replay Storage (S3) | ~50 TB/month | $0.023/GB | 50 TB | $1,150 |
| Bandwidth (egress) | ~100 TB/month | $0.09/GB | 100 TB | $9,000 |
| CDN / DDoS Protection | Cloudflare Enterprise / AWS Shield | Custom | 1 | $10,000 |
| Monitoring / Observability | Datadog / Grafana Cloud | Custom | 1 | $5,000 |
| Global Load Balancing | Anycast routing | Custom | 1 | $3,000 |
Total Monthly Cost Summary
C#
public class CostEstimation
{
public static void CalculateMonthlyCost()
{
var costs = new Dictionary<string, decimal>
{
["Game Servers (bare-metal fleet)"] = 12_000_000m,
["Matchmaker Cluster"] = 20_000m,
["Redis Cluster"] = 36_000m,
["PostgreSQL (RDS)"] = 14_000m,
["Chat Servers"] = 8_000m,
["Voice Relay Servers"] = 24_000m,
["Replay Storage (S3)"] = 1_150m,
["Bandwidth Egress"] = 9_000m,
["CDN / DDoS Protection"] = 10_000m,
["Monitoring / Observability"] = 5_000m,
["Global Load Balancing"] = 3_000m,
};
decimal total = costs.Values.Sum();
Console.WriteLine("=== Monthly Infrastructure Cost ===");
foreach (var (component, cost) in costs)
{
Console.WriteLine($" {component,-35} ${cost,15:N0}");
}
Console.WriteLine($" {"TOTAL",-35} ${total,15:N0}");
Console.WriteLine($"\n Annual cost: ${total * 12:N0}");
// Cost per player per month
decimal dau = 5_000_000;
Console.WriteLine($"\n Cost per DAU per month: ${total / dau:N4}");
Console.WriteLine($" Cost per DAU per year: ${(total * 12) / dau:N2}");
}
}
// Output:
// === Monthly Infrastructure Cost ===
// Game Servers (bare-metal fleet) $ 12,000,000
// Matchmaker Cluster $ 20,000
// Redis Cluster $ 36,000
// PostgreSQL (RDS) $ 14,000
// Chat Servers $ 8,000
// Voice Relay Servers $ 24,000
// Replay Storage (S3) $ 1,150
// Bandwidth Egress $ 9,000
// CDN / DDoS Protection $ 10,000
// Monitoring / Observability $ 5,000
// Global Load Balancing $ 3,000
// TOTAL $ 12,130,150
//
// Annual cost: $145,561,800
//
// Cost per DAU per month: $2.4260
// Cost per DAU per year: $29.11
Cost Optimization Strategies
- Reserved Instances: Commit to 1–3 year reserved contracts for game servers to reduce compute cost by 40–60%. Game server workloads are predictable and long-running, making them ideal for reservations.
- Spot Instances for Non-Critical Workloads: Chat, replay processing, and analytics can run on spot instances with automatic failover, reducing cost by 60–80% for these components.
- Regional Scaling: Scale game server capacity based on regional player population. During off-peak hours (e.g., 3 AM local time), reduce capacity by 60–70% through auto-scaling groups.
- Hybrid Bare-Metal + Cloud: Use bare-metal servers (e.g., Hetzner, OVH, or Equinix Metal) for the core game server fleet at $300–800/machine/month vs $1200+ on cloud, while using cloud for burst capacity and supporting services.
- Compression: Aggressive packet compression (LZ4) can reduce bandwidth costs by 40–60% with negligible CPU overhead.
19. Interview Questions & Answers
Q1: How would you handle a server crash mid-match in a battle royale?
Q2: Why UDP instead of TCP for the game protocol? What about QUIC?
Q3: How does lag compensation work, and why can it cause players to be hit "around corners"?
Q4: Design the matchmaking system for a game with 10M daily active players.
Q5: How would you implement an anti-cheat system?
Q6: How do you handle different tick rates between client and server?
Q7: Scale discussion — how do you handle 1M concurrent players?
Q8: Explain the difference between authoritative server model and peer-to-peer.
20. Conclusion & Further Reading
Designing a multiplayer game server infrastructure is one of the most challenging problems in distributed systems engineering. It combines the hard real-time constraints of embedded systems, the consistency requirements of distributed databases, the networking challenges of CDN architecture, and the adversarial security considerations of financial systems. The key takeaways from this design are:
- Server Authority is Non-Negotiable: For competitive games, the server must be the single source of truth. All game logic, physics, and state transitions happen on the server. Clients are presentation layers with prediction for responsiveness.
- UDP is the Foundation: Real-time game networking must use UDP with custom reliability semantics. TCP's head-of-line blocking makes it unsuitable for game state updates. QUIC is excellent for auxiliary services but not for the core game path.
- Lag Compensation is Essential: Without server-side rewind and client-side prediction, competitive gameplay is impossible at typical internet latencies. The server maintains historical state snapshots to evaluate hit detection fairly.
- Matchmaking Must Balance Quality and Speed: Use Glicko-2 or TrueSkill for skill ratings, adaptive queue windows for wait time management, and regional pooling for latency optimization.
- Anti-Cheat Requires Depth: No single anti-cheat technique is sufficient. Combine kernel-level client protection, server-side input validation, post-match statistical analysis, and community reporting.
- Failure Handling Must Be Graceful: Game server crashes are inevitable at scale. Design for partial results, match restarts, and fair rating adjustments. Never penalize players for infrastructure failures.
- Cost is the Primary Constraint: Game server infrastructure is expensive ($10–15M/month at scale). Optimize through bare-metal deployments, regional scaling, reserved instances, and aggressive compression.
Recommended Reading
- "Game Networking Errors" — Glenn Fiedler (gafferongames.com) — The definitive series on game networking fundamentals
- "Valve Developer Wiki: Source Multiplayer Networking" — Valve's approach to client-server networking in Source engine
- "Microsoft TrueSkill Technical Report" — Herbrich, Minka, Graepel — The original TrueSkill paper for skill rating in team games
- "Glicko-2 Paper" — Mark Glickman — The mathematical foundation of Glicko-2 rating systems
- "Quake 3 Source Code" — id Software — Open-source reference implementation of an authoritative game server with client prediction
- "Unreal Engine Network Guide" — Epic Games — Comprehensive guide to replication, relevancy, and priority in Unreal's networking system
- " Overwatch: Netcode" — Blizzard GDC Talk — Insights into building netcode for a fast-paced team shooter
- "Fortress Forever: Network Architecture" — Community analysis of TF2's networking model and lag compensation