Table of Contents

1. Introduction — The URL Shortener Landscape
2. Functional & Non-Functional Requirements
3. Capacity Estimation & Back-of-Envelope
4. Data Model & Storage Schema
5. API Design
6. High-Level Architecture
7. Short Code Generation Strategies
8. Base62 Encoding Deep Dive
9. Database Design & Sharding
10. Caching Strategy
11. Analytics & Click Tracking
12. Redirection Performance
13. Security & Abuse Prevention
14. Custom Aliases & Vanity URLs
15. Link Expiration & Cleanup
16. API Rate Limiting
17. Monitoring & Observability
18. Multi-Region Design
19. Case Studies — Production Systems
20. Cost Estimation
21. Edge Cases
22. Interview Q&A
23. A/B Testing with Short URLs
24. SEO & Social Sharing
25. Full C# Implementation
26. Conclusion

1. Introduction — The URL Shortener Landscape

URL shorteners transform long, unwieldy URLs into compact, shareable links. Bit.ly processes over 100 million link clicks per day and has created over 40 billion shortened URLs since its launch. Twitter's t.co handles 300 million shortened URLs daily. TinyURL, one of the original URL shorteners, has created over 400 million short links. These services are deceptively simple on the surface — take a long URL, return a short one — but building them at scale requires solving several distributed systems challenges: generating unique short codes without coordination, handling billions of redirections per day with sub-millisecond latency, and tracking analytics without impacting redirect performance.

URL shorteners serve many purposes beyond just shortening links: they provide click analytics (geography, devices, referrers), enable link rot protection (update the destination without changing the short URL), support A/B testing of landing pages, and power marketing campaigns with trackable links. The core challenge is designing a system that generates unique, collision-free short codes, stores the mapping durably, and serves redirections at the speed of a DNS lookup.

2. Functional & Non-Functional Requirements

Functional Requirements

Non-Functional Requirements

3. Capacity Estimation & Back-of-Envelope

Daily Volume Estimates

Short Code Space

Read vs Write Ratio Analysis

The 100:1 read-to-write ratio is the defining characteristic of URL shortener workloads. This extreme skew means that the system is fundamentally read-heavy, and optimizations should focus on the read path. For every new URL created, the system will handle approximately 100 redirect requests over its lifetime. This ratio is even higher for marketing campaigns and social media sharing, where a single short link may receive millions of clicks but is only created once.

Cache Estimates

4. Data Model & Storage Schema

Entity Relationship

PostgreSQL Schema

SQL
CREATE TABLE url_mappings (
    short_code VARCHAR(10) PRIMARY KEY,
    long_url TEXT NOT NULL,
    user_id BIGINT REFERENCES users(id),
    custom_alias VARCHAR(32) UNIQUE,
    created_at TIMESTAMP DEFAULT NOW(),
    expires_at TIMESTAMP,
    click_count BIGINT DEFAULT 0,
    is_active BOOLEAN DEFAULT TRUE,
    title VARCHAR(255),
    tags TEXT[],
    created_ip INET
);

CREATE INDEX idx_url_mappings_long_url ON url_mappings(long_url);
CREATE INDEX idx_url_mappings_user ON url_mappings(user_id, created_at DESC);
CREATE INDEX idx_url_mappings_expires ON url_mappings(expires_at)
    WHERE expires_at IS NOT NULL AND is_active = TRUE;
CREATE INDEX idx_url_mappings_active ON url_mappings(is_active, created_at DESC);

CREATE TABLE click_events (
    id BIGSERIAL PRIMARY KEY,
    short_code VARCHAR(10) REFERENCES url_mappings(short_code),
    ip_address INET,
    user_agent TEXT,
    referrer TEXT,
    country VARCHAR(2),
    region VARCHAR(100),
    city VARCHAR(100),
    device_type VARCHAR(20),
    browser VARCHAR(50),
    os VARCHAR(50),
    clicked_at TIMESTAMP DEFAULT NOW()
);

CREATE INDEX idx_click_events_short ON click_events(short_code, clicked_at DESC);
CREATE INDEX idx_click_events_time ON click_events(clicked_at DESC);
            

Click Events in ClickHouse (Analytics)

SQL
CREATE TABLE click_events_local ON CLUSTER '{cluster}'
(
    short_code String,
    ip_address IPv4,
    user_agent String,
    referrer String,
    country LowCardinality(String),
    device_type LowCardinality(String),
    browser LowCardinality(String),
    os LowCardinality(String),
    clicked_at DateTime
)
ENGINE = ReplicatedMergeTree('/clickhouse/{cluster}/tables/{shard}/click_events', '{replica}')
ORDER BY (short_code, clicked_at)
PARTITION BY toYYYYMM(clicked_at);

-- Materialized view for hourly aggregation
CREATE MATERIALIZED VIEW click_events_hourly_local ON CLUSTER '{cluster}'
ENGINE = ReplicatedSummingMergeTree(...)
ORDER BY (short_code, country, device_type, hour)
AS SELECT
    short_code,
    country,
    device_type,
    toStartOfHour(clicked_at) AS hour,
    count() AS clicks
FROM click_events_local
GROUP BY short_code, country, device_type, hour;
            

5. API Design

REST API

HTTP
// Shorten URL
POST /api/v1/shorten
{
    "long_url": "https://example.com/very/long/path?param=value&other=123",
    "custom_alias": "my-link",           // optional
    "expires_at": "2025-12-31T23:59:59Z", // optional
    "tags": ["marketing", "campaign"]     // optional
}

// Response
{
    "short_url": "https://short.link/abc1234",
    "short_code": "abc1234",
    "long_url": "https://example.com/very/long/path?param=value&other=123",
    "created_at": "2025-01-15T10:30:00Z",
    "expires_at": "2025-12-31T23:59:59Z",
    "qr_code": "https://short.link/qr/abc1234.png"
}

// Get URL info
GET /api/v1/urls/{short_code}

// Update URL destination
PUT /api/v1/urls/{short_code}
{
    "long_url": "https://example.com/new/destination"
}

// Delete URL
DELETE /api/v1/urls/{short_code}

// Get click analytics
GET /api/v1/urls/{short_code}/analytics?period=30d

// Get click analytics (aggregated)
GET /api/v1/urls/{short_code}/analytics/summary

// Bulk shorten
POST /api/v1/shorten/batch
{
    "urls": [
        { "long_url": "https://example.com/page1" },
        { "long_url": "https://example.com/page2", "custom_alias": "page2" }
    ]
}

// List user's URLs
GET /api/v1/users/{user_id}/urls?page=20&cursor=abc
            

Redirect Endpoint

HTTP
// Redirect (handled by load balancer or edge server)
GET /{short_code}
// Response: 301 Moved Permanently or 302 Found
// Location: https://example.com/very/long/path?param=value&other=123

// With analytics tracking
GET /{short_code}
// Server: Track click event (async)
// Server: Return 301/302 redirect
            

Internal APIs

HTTP
// Short code generation service
GET /internal/v1/generate?count=100
// Returns: 100 unique short codes

// Analytics ingestion
POST /internal/v1/analytics/track
{
    "short_code": "abc1234",
    "ip": "192.168.1.1",
    "user_agent": "...",
    "referrer": "...",
    "timestamp": "2025-01-15T10:30:00Z"
}

// Bulk analytics query
POST /internal/v1/analytics/bulk
{
    "short_codes": ["abc1234", "def5678"],
    "metrics": ["clicks", "unique_visitors", "countries"]
}
            

6. High-Level Architecture

Component Responsibilities

7. Short Code Generation Strategies

Generating unique, collision-free short codes is the core algorithmic challenge. There are five primary strategies, each with distinct trade-offs in terms of uniqueness guarantees, performance, and distribution characteristics.

Strategy 1: MD5/SHA256 Hashing + Base62

C#
public static string GenerateShortCode(string longUrl)
{
    using var sha256 = SHA256.Create();
    byte[] hash = sha256.ComputeHash(Encoding.UTF8.GetBytes(longUrl + DateTime.UtcNow.Ticks));
    string base64 = Convert.ToBase64String(hash)
        .Replace("+", "-").Replace("/", "_").TrimEnd('=');
    return base64.Substring(0, 7);
}
            

This approach hashes the long URL (optionally with a salt) and takes the first 7 characters of the Base62-encoded hash. The problem is collisions — two different long URLs may produce the same short code. With 7 characters and 3.52 trillion combinations, the birthday paradox tells us we need about 1.9 million codes before collision probability reaches 50%, which happens fast at scale. Each collision requires a retry, adding latency.

Strategy 2: Auto-Increment Counter

C#
public class CounterService
{
    private readonly IRedisClient _redis;

    public async Task<string> NextShortCode()
    {
        long id = await _redis.IncrementAsync("url:counter");
        return ToBase62(id);
    }

    private static string ToBase62(long value)
    {
        const string chars = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
        var sb = new StringBuilder();
        while (value > 0)
        {
            sb.Insert(0, chars[(int)(value % 62)]);
            value /= 62;
        }
        while (sb.Length < 7) sb.Insert(0, "0");
        return sb.ToString();
    }
}
            

The auto-increment approach uses a centralized counter in Redis and converts the integer to Base62. This guarantees uniqueness within a single counter, but creates a single point of failure and a hot key in Redis. You can shard the counter across multiple Redis instances, assigning each a range (e.g., shard 0 handles IDs 0-999,999, shard 1 handles 1M-1.99M, etc.).

Strategy 3: Pre-Generated ID Pool

C#
public class IdPoolService
{
    private readonly ConcurrentQueue<string> _pool;
    private readonly IDatabase _redis;

    public IdPoolService(int poolSize = 10_000)
    {
        _pool = new ConcurrentQueue<string>();
        _redis = ConnectionMultiplexer.Connect("localhost").GetDatabase();
        RefillPool(poolSize);
    }

    public string NextShortCode()
    {
        if (_pool.IsEmpty) RefillPool(5_000);
        _pool.TryDequeue(out string code);
        return code;
    }

    private void RefillPool(int count)
    {
        // Generate batch of unique codes via distributed counter
        long startId = (long)_redis.StringIncrement("url:pool_start", count);
        for (int i = 0; i < count; i++)
        {
            _pool.Enqueue(ToBase62(startId + i));
        }
    }
}
            

This approach pre-generates a batch of short codes and hands them out from an in-memory queue. When the pool runs low, it fetches another batch from a distributed counter. This combines the uniqueness guarantee of auto-increment with the performance of in-memory generation. Each application instance maintains its own pool, and the distributed counter assigns non-overlapping ranges.

Strategy 4: KSUID (K-Sortable Unique ID)

C#
public class KsuidGenerator
{
    private static readonly DateTime Epoch = new DateTime(2024, 1, 1, 0, 0, 0, DateTimeKind.Utc);

    public static string GenerateShortCode()
    {
        // 4 bytes timestamp + 8 bytes random = 12 bytes
        int secondsSinceEpoch = (int)(DateTime.UtcNow - Epoch).TotalSeconds;
        byte[] bytes = new byte[12];
        BitConverter.GetBytes(secondsSinceEpoch).CopyTo(bytes, 0);
        RandomNumberGenerator.Fill(bytes.AsSpan(4, 8));
        return ToBase62(bytes).Substring(0, 7);
    }
}
            

KSUIDs embed a timestamp prefix, making them naturally sortable by creation time. The first 4 bytes encode the timestamp and the remaining 8 bytes provide randomness. This approach enables chronological iteration of short codes without coordination and provides 2^64 random combinations per second — effectively eliminating collisions.

Strategy 5: Distributed ID Generator (Snowflake-like)

C#
public class DistributedIdGenerator
{
    private const int WorkerIdBits = 5;
    private const int SequenceBits = 12;
    private const long MaxWorkerId = (1L << WorkerIdBits) - 1;
    private const long MaxSequence = (1L << SequenceBits) - 1;

    private long _sequence = 0;
    private long _lastTimestamp = -1;
    private readonly long _workerId;

    public DistributedIdGenerator(long workerId)
    {
        if (workerId < 0 || workerId > MaxWorkerId)
            throw new ArgumentException($"Worker ID must be 0-{MaxWorkerId}");
        _workerId = workerId;
    }

    public long NextId()
    {
        long timestamp = GetCurrentTimestamp();
        if (timestamp == _lastTimestamp)
        {
            _sequence = (_sequence + 1) & MaxSequence;
            if (_sequence == 0) timestamp = WaitNextMillis();
        }
        else
        {
            _sequence = 0;
        }
        _lastTimestamp = timestamp;
        return (timestamp << (WorkerIdBits + SequenceBits))
             | (_workerId << SequenceBits)
             | _sequence;
    }
}
            

Strategy Comparison

8. Base62 Encoding Deep Dive

Base62 encoding is the foundation of short code generation. It uses the 62 alphanumeric characters (a-z, A-Z, 0-9) to represent numbers in a compact, human-readable format. Unlike Base64, it avoids special characters like + and / which are problematic in URLs. Base62 is the standard encoding for URL shorteners because it produces clean, readable strings that work without URL encoding. The encoding process converts a numeric ID into a string of alphanumeric characters, and the decoding process reverses this transformation. The key property is that every unique integer maps to a unique Base62 string, and vice versa, making it ideal for short code generation.

The choice of character set matters for usability. Base62 uses digits (0-9), lowercase letters (a-z), and uppercase letters (A-Z). This provides 62 unique characters, each carrying approximately 5.95 bits of information. When designing the character set, consider that some characters are visually similar (0/O, 1/l/I) which can cause confusion when sharing URLs verbally. Some systems use a custom character set that avoids these ambiguous characters (similar to Base58), trading a small amount of capacity for better readability.

C#
public static class Base62Encoder
{
    private const string Charset = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
    private static readonly Dictionary<char, int> ReverseMap;

    static Base62Encoder()
    {
        ReverseMap = new Dictionary<char, int>();
        for (int i = 0; i < Charset.Length; i++)
            ReverseMap[Charset[i]] = i;
    }

    public static string Encode(long value)
    {
        if (value == 0) return Charset[0].ToString();
        var sb = new StringBuilder();
        while (value > 0)
        {
            sb.Insert(0, Charset[(int)(value % 62)]);
            value /= 62;
        }
        return sb.ToString();
    }

    public static long Decode(string encoded)
    {
        long result = 0;
        foreach (char c in encoded)
        {
            result = result * 62 + ReverseMap[c];
        }
        return result;
    }

    public static string EncodeWithPadding(long value, int minLength = 7)
    {
        string result = Encode(value);
        return result.PadLeft(minLength, Charset[0]);
    }
}
            

Base62 vs Base64 Comparison

Why Not Base64?

Base64 uses + and / characters which have special meanings in URLs. While you can URL-safe encode Base64 by replacing + with - and / with _, the resulting strings are less readable. Base62 avoids this problem entirely by using only alphanumeric characters. When you share a short URL verbally — "Go to bit.ly slash ABC 1234" — Base62 characters are unambiguous and easy to type.

Base62 Capacity Table

9. Database Design & Sharding

Sharding Strategy

For a URL shortener, the primary access pattern is by short code (point lookup). This makes consistent hashing or range-based sharding by short code ideal. Each shard owns a portion of the short code space.

Shard Key Selection

For URL shorteners, hash-based sharding on short_code is recommended. The access pattern is almost exclusively point lookups (get URL by short code), and the code space is uniformly distributed across the hash ring. This avoids hot shards and provides even load distribution.

Replication Configuration

YAML
# PostgreSQL streaming replication
# Primary config (postgresql.conf)
wal_level = replica
max_wal_senders = 10
wal_keep_size = 1GB
synchronous_standby_names = 'replica1'

# Replica config
primary_conninfo = 'host=primary port=5432 user=replicator password=xxx'
hot_standby = on
            

Each shard runs with one primary and one or more read replicas. Writes go to the primary, and reads (for URLs not in cache) can be served by replicas. This provides both read scalability and failover capability. When a primary fails, a replica is promoted to primary and the failover manager updates the connection strings.

Database Partitioning

SQL
-- Partition click_events by month for efficient data lifecycle
CREATE TABLE click_events (
    id BIGSERIAL,
    short_code VARCHAR(10),
    ip_address INET,
    country VARCHAR(2),
    clicked_at TIMESTAMP NOT NULL,
    PRIMARY KEY (id, clicked_at)
) PARTITION BY RANGE (clicked_at);

-- Create monthly partitions
CREATE TABLE click_events_2025_01 PARTITION OF click_events
    FOR VALUES FROM ('2025-01-01') TO ('2025-02-01');
CREATE TABLE click_events_2025_02 PARTITION OF click_events
    FOR VALUES FROM ('2025-02-01') TO ('2025-03-01');

-- Auto-create future partitions via pg_partman
SELECT partman.create_parent(
    p_parent_table := 'public.click_events',
    p_control := 'clicked_at',
    p_type := 'range',
    p_interval := '1 month'
);
            

10. Caching Strategy

URL shorteners are extremely read-heavy. With a 100:1 read-to-write ratio, caching is essential for performance. The cache hit rate directly impacts redirect latency and database load.

Cache-Aside Pattern Implementation

C#
public class UrlCacheService
{
    private readonly IDatabase _redis;
    private readonly IUrlRepository _repository;
    private readonly TimeSpan _defaultTtl = TimeSpan.FromHours(24);
    private readonly TimeSpan _negativeCacheTtl = TimeSpan.FromMinutes(5);

    public async Task<string?> ResolveUrl(string shortCode)
    {
        // Layer 1: Local in-memory cache (per-instance)
        if (_localCache.TryGetValue(shortCode, out string? localUrl))
            return localUrl;

        // Layer 2: Redis distributed cache
        string cached = await _redis.StringGetAsync($"url:{shortCode}");
        if (!string.IsNullOrEmpty(cached))
        {
            _localCache.Set(shortCode, cached, TimeSpan.FromMinutes(5));
            return cached;
        }

        // Layer 3: Database
        var mapping = await _repository.GetByShortCode(shortCode);
        if (mapping == null)
        {
            // Negative caching: avoid repeated DB lookups for non-existent codes
            await _redis.StringSetAsync($"url:{shortCode}", "",
                _negativeCacheTtl);
            return null;
        }

        // Populate cache
        await _redis.StringSetAsync($"url:{shortCode}",
            mapping.LongUrl, _defaultTtl);
        _localCache.Set(shortCode, mapping.LongUrl, TimeSpan.FromMinutes(5));

        return mapping.LongUrl;
    }
}
            

Multi-Level Cache Architecture

Cache Warming for Viral URLs

C#
public class CacheWarmer
{
    private readonly IUrlRepository _repository;
    private readonly IDatabase _redis;

    public async Task WarmCache(List<string> shortCodes)
    {
        var pipeline = _redis.CreateBatch();
        var tasks = new List<Task>();

        foreach (var code in shortCodes)
        {
            tasks.Add(pipeline.StringGetAsync($"url:{code}"));
        }
        pipeline.Execute();
        await Task.WhenAll(tasks);

        // Pre-fetch any missing codes into cache
        for (int i = 0; i < shortCodes.Count; i++)
        {
            var result = await tasks[i];
            if (result.IsNullOrEmpty)
            {
                var mapping = await _repository.GetByShortCode(shortCodes[i]);
                if (mapping != null)
                {
                    await _redis.StringSetAsync(
                        $"url:{shortCodes[i]}",
                        mapping.LongUrl,
                        TimeSpan.FromHours(24));
                }
            }
        }
    }
}
            

Cache Eviction Strategy

11. Analytics & Click Tracking

Click analytics is a core feature of URL shorteners. Every redirect must also record a click event with metadata (timestamp, IP, user agent, referrer, country). This must not add latency to the redirect path. The key insight is that analytics recording is a side effect of the redirect, not a requirement for it. If the analytics pipeline fails, the redirect should still succeed. This separation of concerns allows the redirect path to be optimized for speed while analytics are processed asynchronously in the background.

The analytics pipeline must handle extreme write loads during viral events. When a celebrity shares a short link on social media, it can generate millions of clicks per second. Kafka acts as a buffer between the redirect servers and the analytics storage, absorbing burst traffic and allowing the storage layer to process events at its own pace. ClickHouse is ideal for this workload because it excels at high-throughput inserts and supports efficient columnar aggregation queries.

Event Schema

C#
public record ClickEvent
{
    public string ShortCode { get; init; } = "";
    public string IpAddress { get; init; } = "";
    public string UserAgent { get; init; } = "";
    public string Referrer { get; init; } = "";
    public string Country { get; init; } = "";
    public string City { get; init; } = "";
    public string DeviceType { get; init; } = ""; // mobile, desktop, tablet
    public string Browser { get; init; } = "";
    public string Os { get; init; } = "";
    public DateTime Timestamp { get; init; } = DateTime.UtcNow;
    public bool IsUniqueVisitor { get; init; }
}
            

Async Analytics with Kafka

C#
public class RedirectService
{
    private readonly IKafkaProducer _producer;

    public async Task<HttpResponseMessage> HandleRedirect(string shortCode)
    {
        var url = await _cacheService.ResolveUrl(shortCode);
        if (url == null) return new HttpResponseMessage(HttpStatusCode.NotFound);

        // Fire and forget click event
        _ = Task.Run(async () =>
        {
            var clickEvent = new ClickEvent
            {
                ShortCode = shortCode,
                IpAddress = GetCurrentIp(),
                UserAgent = GetHeader("User-Agent"),
                Referrer = GetHeader("Referer"),
                Timestamp = DateTime.UtcNow
            };
            await _producer.ProduceAsync("click-events", shortCode, clickEvent);
        });

        return Redirect(url);
    }
}
            

Real-Time Analytics with Redis

C#
public class RealTimeAnalytics
{
    private readonly IDatabase _redis;

    public async Task TrackClick(string shortCode, ClickEvent evt)
    {
        var pipeline = _redis.CreateBatch();

        // Total clicks
        pipeline.HashIncrementAsync("analytics:total", shortCode);

        // Clicks by date
        string dateKey = evt.Timestamp.ToString("yyyyMMdd");
        pipeline.HashIncrementAsync($"analytics:daily:{dateKey}", shortCode);

        // Clicks by country
        pipeline.HashIncrementAsync(
            $"analytics:country:{shortCode}", evt.Country);

        // Clicks by referrer domain
        string referrerDomain = ExtractDomain(evt.Referrer);
        pipeline.HashIncrementAsync(
            $"analytics:referrer:{shortCode}", referrerDomain);

        // Unique visitors (HyperLogLog)
        pipeline.HyperLogLogAddAsync(
            $"analytics:unique:{shortCode}",
            evt.IpAddress + evt.UserAgent);

        pipeline.Execute();
    }

    public async Task<AnalyticsSummary> GetSummary(string shortCode)
    {
        return new AnalyticsSummary
        {
            TotalClicks = (long)await _redis.HashGetAsync(
                "analytics:total", shortCode),
            UniqueVisitors = await _redis.HyperLogLogLengthAsync(
                $"analytics:unique:{shortCode}")
        };
    }
}
            

Analytics Data Pipeline

12. Redirection Performance

The redirect path is the most latency-sensitive operation. Users expect instant redirects. Every millisecond matters because the redirect adds to the perceived page load time of the destination URL.

301 vs 302 Redirect

Redirect Latency Breakdown

Performance Optimization Techniques

C#
public class OptimizedRedirectHandler
{
    // Connection pooling for database
    private static readonly NpgsqlConnectionPool _pool = new(
        connectionString, minConnections: 50, maxConnections: 200);

    // Pre-compiled URL builder
    private static readonly Func<string, string> _buildRedirect =
        CompiledExpression.Compile((string url) =>
            new StringBuilder(200)
                .Append("Location: ").Append(url)
                .Append("\r\nConnection: keep-alive")
                .Append("\r\nCache-Control: no-cache, no-store")
                .ToString());

    // HTTP/2 server push for analytics
    public async Task<HttpResponse> HandleRequest(HttpRequest request)
    {
        string shortCode = request.Path.Value.TrimStart('/');

        // Fast path: in-memory cache (L1)
        if (_localCache.TryGetValue(shortCode, out var url))
        {
            return new HttpResponse
            {
                StatusCode = 302,
                Headers = { ["Location"] = url }
            };
        }

        // Redis path (L2)
        url = await _redis.GetShortUrl(shortCode);
        if (url != null)
        {
            _localCache.Set(shortCode, url, TimeSpan.FromMinutes(5));
            return new HttpResponse { StatusCode = 302, Headers = { ["Location"] = url } };
        }

        // Database path (L3)
        url = await _repository.GetUrl(shortCode);
        if (url == null)
            return new HttpResponse { StatusCode = 404 };

        await _cacheService.SetAsync(shortCode, url);
        return new HttpResponse { StatusCode = 302, Headers = { ["Location"] = url } };
    }
}
            

13. Security & Abuse Prevention

URL shorteners are frequently abused for phishing, malware distribution, and spam. A production system must implement multiple layers of defense to protect users while maintaining performance. The challenge is balancing security with the low-latency requirement — every check adds latency to the shortening flow, and some checks (like external API calls to Google Safe Browsing) can take hundreds of milliseconds. The solution is a tiered approach where fast, local checks run first and expensive checks are deferred or applied only to suspicious submissions.

Abuse Prevention Layers

URL Validation

C#
public class UrlValidator
{
    private static readonly HashSet<string> BlockedSchemes = new() {
        "javascript", "data", "file", "ftp", "vbscript"
    };

    private static readonly HashSet<string> BlockedDomains = LoadBlocklist();

    public ValidationResult Validate(string longUrl)
    {
        if (!Uri.TryCreate(longUrl, UriKind.Absolute, out var uri))
            return ValidationResult.Invalid("Invalid URL format");

        if (BlockedSchemes.Contains(uri.Scheme.ToLower()))
            return ValidationResult.Invalid($"Scheme '{uri.Scheme}' not allowed");

        if (BlockedDomains.Contains(uri.Host.ToLower()))
            return ValidationResult.Blocked("Domain is blocklisted");

        if (uri.Host.Length < 4)
            return ValidationResult.Invalid("Domain too short");

        // Check for suspicious patterns
        if (ContainsPhishingPatterns(longUrl))
            return ValidationResult.Suspicious("URL may be phishing");

        // Check against Google Safe Browsing
        if (await IsInSafeBrowsing(longUrl))
            return ValidationResult.Blocked("URL flagged as unsafe");

        return ValidationResult.Valid();
    }

    private bool ContainsPhishingPatterns(string url)
    {
        // Detect URL shorteners used to hide real domains
        var uri = new Uri(url);
        string[] suspiciousPatterns = {
            "login", "verify", "account", "secure",
            "update", "confirm", "banking", "paypal"
        };
        return suspiciousPatterns.Any(p =>
            uri.Host.Contains(p, StringComparison.OrdinalIgnoreCase));
    }
}
            

Abuse Rate Limiting

C#
public class AbuseRateLimiter
{
    private readonly IDatabase _redis;

    public async Task<RateLimitResult> CheckLimit(string identifier, string tier)
    {
        var limits = GetLimits(tier);
        string key = $"ratelimit:{identifier}:{DateTime.UtcNow:yyyyMMddHH}";

        var pipeline = _redis.CreateBatch();
        var current = pipeline.StringIncrementAsync(key);
        pipeline.KeyExpireAsync(key, TimeSpan.FromHours(2));
        pipeline.Execute();

        int count = (int)await current;

        return new RateLimitResult
        {
            Allowed = count <= limits.Hourly,
            Remaining = Math.Max(0, limits.Hourly - count),
            RetryAfter = count > limits.Hourly
                ? TimeSpan.FromMinutes(15)
                : TimeSpan.Zero
        };
    }

    private static RateLimitConfig GetLimits(string tier) => tier switch
    {
        "free" => new(10, 100),
        "pro" => new(100, 1000),
        "enterprise" => new(1000, 10000),
        _ => new(10, 100)
    };
}
            

Security Headers for Redirects

HTTP
HTTP/1.1 302 Found
Location: https://example.com/page
Cache-Control: no-cache, no-store, must-revalidate
X-Frame-Options: DENY
X-Content-Type-Options: nosniff
Referrer-Policy: no-referrer-when-downgrade
Content-Security-Policy: default-src 'none'
Strict-Transport-Security: max-age=31536000; includeSubDomains
            

14. Custom Aliases & Vanity URLs

Custom aliases let users choose their own short codes (e.g., short.link/my-brand). This feature adds complexity because you must validate uniqueness and prevent namespace collisions with auto-generated codes.

Custom Alias Validation

C#
public class AliasValidator
{
    private static readonly Regex ValidPattern = new(@"^[a-zA-Z0-9_-]{3,32}$");

    // Reserved words that cannot be used as aliases
    private static readonly HashSet<string> ReservedWords = new()
    {
        "api", "admin", "login", "signup", "help",
        "docs", "status", "health", "metrics", "about",
        "blog", "careers", "terms", "privacy", "robots.txt",
        "favicon.ico", ".well-known", "manifest.json"
    };

    public ValidationResult ValidateAlias(string alias)
    {
        if (string.IsNullOrWhiteSpace(alias))
            return ValidationResult.Invalid("Alias cannot be empty");

        if (alias.Length < 3 || alias.Length > 32)
            return ValidationResult.Invalid("Alias must be 3-32 characters");

        if (!ValidPattern.IsMatch(alias))
            return ValidationResult.Invalid(
                "Only alphanumeric, hyphens, underscores allowed");

        if (ReservedWords.Contains(alias.ToLower()))
            return ValidationResult.Invalid("This alias is reserved");

        // Check for offensive content
        if (ProfanityFilter.ContainsProfanity(alias))
            return ValidationResult.Invalid("Alias contains inappropriate content");

        return ValidationResult.Valid();
    }

    public async Task<bool> IsAvailable(string alias)
    {
        // Check both auto-generated codes and custom aliases
        bool existsAsCode = await _repository.CodeExists(alias);
        bool existsAsAlias = await _repository.AliasExists(alias);
        return !existsAsCode && !existsAsAlias;
    }
}
            

Namespace Collision Prevention

Alias Availability Checking

C#
public class AliasAvailabilityService
{
    private readonly IDatabase _redis;

    public async Task<AliasAvailability> CheckAvailability(string alias)
    {
        // Bloom filter first (fast negative check)
        bool possiblyExists = await _redis.BloomFilterExistsAsync(
            "alias:bloom", alias);
        if (!possiblyExists)
            return new AliasAvailability { Available = true, Confidence = 0.99 };

        // Probable match — check exact set
        bool definitelyExists = await _redis.SetContainsAsync(
            "alias:exists", alias);
        return new AliasAvailability
        {
            Available = !definitelyExists,
            Confidence = 1.0
        };
    }

    public async Task<bool> ReserveAlias(string alias, string shortCode, TimeSpan ttl)
    {
        // Atomic: set alias if not exists
        bool reserved = await _redis.SetAddAsync("alias:reserved", alias);
        if (reserved)
        {
            await _redis.StringSetAsync($"alias:{alias}", shortCode, ttl);
            await _redis.BloomFilterAddAsync("alias:bloom", alias);
        }
        return reserved;
    }
}
            

Premium Alias Pricing

15. Link Expiration & Cleanup

Short links may need expiration for campaigns, temporary access, or compliance. Expired links should return a configurable response (404, custom landing page, or redirect to a default URL).

Expiration Strategies

C#
public class LinkExpirationService
{
    private readonly IUrlRepository _repository;
    private readonly IDatabase _redis;

    public async Task<string?> ResolveWithExpiration(string shortCode)
    {
        // Fast path: check Redis for TTL
        string? url = await _redis.StringGetAsync($"url:{shortCode}");
        if (url == null)
        {
            var mapping = await _repository.GetByShortCode(shortCode);
            if (mapping == null) return null;

            if (mapping.ExpiresAt.HasValue && mapping.ExpiresAt.Value < DateTime.UtcNow)
            {
                await HandleExpiredLink(shortCode, mapping);
                return null;
            }

            url = mapping.LongUrl;
            await _redis.StringSetAsync($"url:{shortCode}", url,
                TimeSpan.FromHours(24));
        }

        return url;
    }

    private async Task HandleExpiredLink(string shortCode, UrlMapping mapping)
    {
        // Option 1: Redirect to custom expired page
        await _redis.StringSetAsync($"url:{shortCode}",
            "https://short.link/expired", TimeSpan.FromMinutes(5));

        // Option 2: Soft delete
        mapping.IsActive = false;
        await _repository.Update(mapping);
    }
}
            

Background Cleanup Job

C#
public class LinkCleanupJob : IHostedService
{
    private readonly IServiceScopeFactory _scopeFactory;
    private readonly Timer _timer;

    public LinkCleanupJob(IServiceScopeFactory scopeFactory)
    {
        _scopeFactory = scopeFactory;
        _timer = new Timer(Execute, null, TimeSpan.Zero,
            TimeSpan.FromHours(6));
    }

    private async Task Execute(object? state)
    {
        using var scope = _scopeFactory.CreateScope();
        var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();

        // Soft delete expired links
        int expired = await db.UrlMappings
            .Where(u => u.ExpiresAt < DateTime.UtcNow && u.IsActive)
            .ExecuteUpdateAsync(s => s.SetProperty(u => u.IsActive, false));

        // Hard delete links expired for 30+ days (reclaim storage)
        var cutoff = DateTime.UtcNow.AddDays(-30);
        var oldExpired = await db.UrlMappings
            .Where(u => !u.IsActive && u.ExpiresAt < cutoff)
            .ToListAsync();

        db.UrlMappings.RemoveRange(oldExpired);
        await db.SaveChangesAsync();

        // Clean up Redis cache
        foreach (var mapping in oldExpired)
        {
            await _redis.KeyDeleteAsync($"url:{mapping.ShortCode}");
        }
    }
}
            

Storage Reclamation Estimates

16. API Rate Limiting

Rate limiting protects the shortening API from abuse and ensures fair resource allocation. Different tiers get different limits based on their subscription.

Tier-Based Rate Limits

Token Bucket Implementation

C#
public class TokenBucketRateLimiter
{
    private readonly IDatabase _redis;

    public async Task<RateLimitResult> CheckRateLimit(
        string apiKey, string tier)
    {
        var config = TierConfig.GetConfig(tier);
        string key = $"ratelimit:{apiKey}";

        var lua = @"
            local tokens_key = KEYS[1] .. ':tokens'
            local timestamp_key = KEYS[1] .. ':ts'
            local rate = tonumber(ARGV[1])
            local capacity = tonumber(ARGV[2])
            local now = tonumber(ARGV[3])
            local requested = tonumber(ARGV[4])
            local fill_time = capacity / rate
            local ttl = math.floor(fill_time * 2)
            local last_tokens = tonumber(redis.call('get', tokens_key))
            if last_tokens == nil then
                last_tokens = capacity
            end
            local last_refreshed = tonumber(redis.call('get', timestamp_key))
            if last_refreshed == nil then
                last_refreshed = 0
            end
            local delta = math.max(0, now - last_refreshed)
            local filled_tokens = math.min(capacity,
                last_tokens + (delta * rate))
            local allowed = filled_tokens >= requested
            local new_tokens = filled_tokens
            if allowed then
                new_tokens = filled_tokens - requested
            end
            redis.call('setex', tokens_key, ttl, new_tokens)
            redis.call('setex', timestamp_key, ttl, now)
            return { allowed and 1 or 0, new_tokens }
        ";

        var result = await _redis.ScriptEvaluateAsync(lua,
            new RedisKey[] { key },
            new RedisValue[] {
                config.RatePerSecond,
                config.BucketCapacity,
                DateTimeOffset.UtcNow.ToUnixTimeSeconds(),
                1
            });

        var results = (RedisResult[])result;
        bool allowed = (long)results[0] == 1;
        long remaining = (long)results[1];

        return new RateLimitResult
        {
            Allowed = allowed,
            Remaining = remaining,
            RetryAfter = allowed
                ? TimeSpan.Zero
                : TimeSpan.FromSeconds(1.0 / config.RatePerSecond)
        };
    }
}
            

Rate Limit Response Headers

HTTP
// Successful request
HTTP/1.1 200 OK
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 95
X-RateLimit-Reset: 1705312800

// Rate limited
HTTP/1.1 429 Too Many Requests
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1705312860
Retry-After: 60
            

17. Monitoring & Observability

A URL shortener requires comprehensive monitoring because broken redirects directly impact user trust. A 1-second delay in redirect time can reduce click-through rates by 7%.

Key Metrics

Grafana Dashboard Metrics

PromQL
# Redirect latency
histogram_quantile(0.99,
    rate(http_request_duration_seconds_bucket{
        endpoint="redirect"
    }[5m])
)

# Cache hit rate
sum(rate(redis_hits_total[5m])) /
    sum(rate(redis_lookups_total[5m])) * 100

# URLs created per second
rate(url_created_total[5m])

# Click events processed
sum(rate(click_events_processed_total[5m])) by (country)

# Active short codes
count(url_mappings{active="true"})
            

Alerting Rules

YAML
# prometheus/alerts.yml
groups:
  - name: url-shortener
    rules:
      - alert: HighRedirectLatency
        expr: histogram_quantile(0.99,
          rate(http_request_duration_seconds_bucket{
            endpoint="redirect"
          }[5m])) > 0.1
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "Redirect latency exceeds 100ms"

      - alert: LowCacheHitRate
        expr: sum(rate(redis_hits_total[5m])) /
          sum(rate(redis_lookups_total[5m])) * 100 < 90
        for: 10m
        labels:
          severity: warning
        annotations:
          summary: "Cache hit rate below 90%"

      - alert: High404Rate
        expr: rate(http_requests_total{
          status="404", endpoint="redirect"
        }[5m]) > 0.01
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "High 404 rate on redirects"
            

SLO Definition

18. Multi-Region Design

For global URL shorteners, redirects must be served from the nearest region to minimize latency. A user in Tokyo should not have to wait for a round-trip to US-East to resolve a redirect. This requires a multi-region deployment with proper data synchronization. The challenge is that URL mapping writes must go to a primary region (for consistency), but reads must be served locally (for performance). This is a classic primary-replica problem where the replica lag determines the consistency model.

GeoDNS routing (using Route 53 or Cloudflare) directs users to the nearest region based on their IP geolocation. Each region maintains its own Redis cache and read replica of the database. When a URL is created in the primary region, the mapping is asynchronously replicated to all regions. For the brief replication lag window (typically 1-5 seconds), a redirect miss in the regional cache triggers a query to the primary region's cache before falling back to the database.

Data Replication Strategy

GeoDNS Routing

Terraform
resource "aws_route53_record" "short_url" {
  zone_id = "Z1234567890"
  name    = "short.link"
  type    = "A"

  set_identifier = "us-east"
  geolocation_routing_policy {
    continent = "NA"
  }

  alias {
    name                   = "us-east-1.short.link.elb.amazonaws.com"
    zone_id               = "Z35SXDOTRQ7X7K"
    evaluate_target_health = true
  }
}

resource "aws_route53_record" "short_url_eu" {
  zone_id = "Z1234567890"
  name    = "short.link"
  type    = "A"

  set_identifier = "eu-west"
  geolocation_routing_policy {
    continent = "EU"
  }

  alias {
    name                   = "eu-west-1.short.link.elb.amazonaws.com"
    zone_id               = "Z35SXDOTRQ7X7K"
    evaluate_target_health = true
  }
}
            

Multi-Region C# Implementation

C#
public class GeoAwareRedirectService
{
    private readonly Dictionary<string, ICacheService> _regionalCaches;
    private readonly ICacheService _primaryCache;

    public async Task<string?> ResolveUrl(string shortCode, string clientRegion)
    {
        // Try regional cache first
        if (_regionalCaches.TryGetValue(clientRegion, out var regionalCache))
        {
            string? url = await regionalCache.GetShortUrl(shortCode);
            if (url != null) return url;
        }

        // Fall back to primary region cache
        string? primaryUrl = await _primaryCache.GetShortUrl(shortCode);
        if (primaryUrl != null)
        {
            // Populate regional cache for next time
            if (_regionalCaches.TryGetValue(clientRegion, out var rc))
                await rc.SetShortUrl(shortCode, primaryUrl, TimeSpan.FromHours(24));
            return primaryUrl;
        }

        // Cache miss: query nearest database replica
        var db = GetRegionalDb(clientRegion);
        var mapping = await db.GetByShortCode(shortCode);
        if (mapping != null)
        {
            await _primaryCache.SetShortUrl(shortCode, mapping.LongUrl,
                TimeSpan.FromHours(24));
            return mapping.LongUrl;
        }

        return null;
    }
}
            

19. Case Studies — Production Systems

Bit.ly Architecture

Twitter t.co Architecture

Key Differences Between Systems

20. Cost Estimation

Monthly Infrastructure Cost (100M URLs/day)

Cost Optimization Strategies

21. Edge Cases

Edge Cases and Solutions

Recursive Shortening Detection

C#
public class RedirectChainResolver
{
    private const int MaxRedirectDepth = 3;

    public async Task<string?> ResolveFinalUrl(
        string shortCode, int depth = 0)
    {
        if (depth >= MaxRedirectDepth)
            throw new RedirectLoopException(
                $"Redirect chain exceeds {MaxRedirectDepth} hops");

        string? url = await _cacheService.ResolveUrl(shortCode);
        if (url == null) return null;

        // Check if the destination is another short URL
        if (IsShortUrl(url))
        {
            string nestedCode = ExtractShortCode(url);
            return await ResolveFinalUrl(nestedCode, depth + 1);
        }

        return url;
    }

    private bool IsShortUrl(string url)
    {
        return url.StartsWith("https://short.link/") ||
               url.StartsWith("http://short.link/");
    }

    private string ExtractShortCode(string url)
    {
        return url.Split('/').Last();
    }
}
            

22. Interview Q&A

Senior Engineer Questions

Staff/Principal Engineer Questions

System Design Framework

24. A/B Testing with Short URLs

URL shorteners enable powerful A/B testing by routing different users to different destinations based on the same short code. This is widely used in marketing campaigns where the same short link serves different landing pages for different audiences.

A/B Testing Architecture

A/B Rule Configuration

C#
public class AbTestRule
{
    public string ShortCode { get; set; } = "";
    public List<AbTestVariant> Variants { get; set; } = new();
    public AbTestTargeting? Targeting { get; set; }
}

public class AbTestVariant
{
    public string Name { get; set; } = "";
    public string DestinationUrl { get; set; } = "";
    public double Weight { get; set; } // 0.0 to 1.0
}

public class AbTestTargeting
{
    public List<string> Countries { get; set; } = new();
    public List<string> Devices { get; set; } = new();
    public string? ReferrerDomain { get; set; }
}

public class AbTestRouter
{
    private readonly IDatabase _redis;

    public async Task<string> ResolveVariant(
        string shortCode, string country, string device, string referrer)
    {
        var rules = await _redis.JsonGetAsync<AbTestRule>(
            $"abtest:{shortCode}");
        if (rules == null)
            return await GetDefaultDestination(shortCode);

        // Check targeting criteria
        if (rules.Targeting != null)
        {
            if (rules.Targeting.Countries.Any() &&
                !rules.Targeting.Countries.Contains(country))
                return await GetDefaultDestination(shortCode);

            if (rules.Targeting.Devices.Any() &&
                !rules.Targeting.Devices.Contains(device))
                return await GetDefaultDestination(shortCode);
        }

        // Weighted random selection
        double random = Random.Shared.NextDouble();
        double cumulative = 0;
        foreach (var variant in rules.Variants)
        {
            cumulative += variant.Weight;
            if (random <= cumulative)
            {
                await TrackAbTestImpression(shortCode, variant.Name);
                return variant.DestinationUrl;
            }
        }

        return rules.Variants.Last().DestinationUrl;
    }
}
            

A/B Test Analytics

SQL
-- A/B test conversion analysis
SELECT
    ab_test_id,
    variant_name,
    COUNT(DISTINCT short_code) AS total_impressions,
    COUNT(DISTINCT CASE WHEN converted THEN visitor_id END) AS conversions,
    ROUND(COUNT(DISTINCT CASE WHEN converted THEN visitor_id END) * 100.0
        / COUNT(DISTINCT visitor_id), 2) AS conversion_rate,
    ROUND(AVG(session_duration_seconds), 1) AS avg_session_duration
FROM ab_test_events
WHERE test_id = 'promo_q1_2025'
    AND clicked_at BETWEEN '2025-01-01' AND '2025-03-31'
GROUP BY ab_test_id, variant_name
ORDER BY conversion_rate DESC;
            

Common A/B Testing Patterns

25. SEO & Social Sharing Considerations

Short URLs impact search engine optimization and social media sharing. Proper implementation ensures that shortened links pass SEO value and display rich previews on social platforms.

301 vs 302 SEO Impact

Open Graph Meta Tags for Short URLs

HTML
<!-- When serving a preview page for short URLs -->
<meta property="og:title" content="Shared Link" />
<meta property="og:description" content="Click to visit the destination" />
<meta property="og:url" content="https://short.link/abc1234" />
<meta property="og:type" content="website" />
<meta property="og:image" content="https://short.link/preview/abc1234.png" />

<!-- Twitter Card -->
<meta name="twitter:card" content="summary_large_image" />
<meta name="twitter:title" content="Shared Link" />
<meta name="twitter:description" content="Click to visit the destination" />
<meta name="twitter:image" content="https://short.link/preview/abc1234.png" />
            

Social Media Preview Generation

C#
public class SocialPreviewGenerator
{
    private readonly HttpClient _http;
    private readonly IImageService _imageService;

    public async Task<SocialPreview> GeneratePreview(string longUrl)
    {
        // Fetch the destination page to extract metadata
        var response = await _http.GetAsync(longUrl);
        var html = await response.Content.ReadAsStringAsync();

        var ogTitle = ExtractMeta(html, "og:title") ?? ExtractTitle(html);
        var ogDescription = ExtractMeta(html, "og:description")
            ?? ExtractMeta(html, "description");
        var ogImage = ExtractMeta(html, "og:image");

        // Generate preview image if none exists
        if (string.IsNullOrEmpty(ogImage))
        {
            ogImage = await _imageService.GenerateLinkPreview(
                ogTitle, longUrl);
        }

        return new SocialPreview
        {
            Title = ogTitle,
            Description = ogDescription,
            ImageUrl = ogImage,
            Domain = new Uri(longUrl).Host
        };
    }
}
            

SEO Best Practices for URL Shorteners

26. Full C# Implementation

Here is the complete production-ready implementation tying together all the components discussed throughout this article.

C#
public class UrlShortenerService : IUrlShortenerService
{
    private readonly IUrlRepository _repository;
    private readonly ICacheService _cache;
    private readonly IdPoolService _idPool;
    private readonly IKafkaProducer _kafka;
    private readonly UrlValidator _validator;
    private readonly AbuseRateLimiter _rateLimiter;
    private readonly ILogger<UrlShortenerService> _logger;

    public async Task<ShortenResult> ShortenUrl(
        ShortenRequest request, string userId, string apiKey)
    {
        // 1. Rate limit check
        var rateLimit = await _rateLimiter.CheckLimit(apiKey, "pro");
        if (!rateLimit.Allowed)
            throw new RateLimitExceededException(rateLimit.RetryAfter);

        // 2. URL validation
        var validation = _validator.Validate(request.LongUrl);
        if (!validation.IsValid)
            throw new ValidationException(validation.Error);

        // 3. Check if URL already shortened (deduplication)
        string? existingCode = await _cache.GetCodeForUrl(request.LongUrl);
        if (existingCode != null && request.CustomAlias == null)
        {
            return new ShortenResult
            {
                ShortCode = existingCode,
                ShortUrl = $"https://short.link/{existingCode}",
                IsDuplicate = true
            };
        }

        // 4. Determine short code
        string shortCode;
        if (request.CustomAlias != null)
        {
            var aliasValidation = _validator.ValidateAlias(request.CustomAlias);
            if (!aliasValidation.IsValid)
                throw new ValidationException(aliasValidation.Error);

            bool available = await _repository.IsAliasAvailable(
                request.CustomAlias);
            if (!available)
                throw new ConflictException("Alias already taken");

            shortCode = request.CustomAlias;
        }
        else
        {
            shortCode = await _idPool.NextShortCode();
        }

        // 5. Create mapping
        var mapping = new UrlMapping
        {
            ShortCode = shortCode,
            LongUrl = request.LongUrl,
            UserId = userId,
            CreatedAt = DateTime.UtcNow,
            ExpiresAt = request.ExpiresAt,
            IsActive = true,
            Tags = request.Tags?.ToArray() ?? Array.Empty<string>()
        };

        await _repository.Create(mapping);

        // 6. Populate cache
        await _cache.SetShortUrl(shortCode, request.LongUrl,
            TimeSpan.FromHours(24));

        _logger.LogInformation(
            "Short URL created: {Code} → {Url} by user {User}",
            shortCode, request.LongUrl, userId);

        return new ShortenResult
        {
            ShortCode = shortCode,
            ShortUrl = $"https://short.link/{shortCode}",
            LongUrl = request.LongUrl,
            CreatedAt = mapping.CreatedAt,
            ExpiresAt = mapping.ExpiresAt
        };
    }

    public async Task<RedirectResult> ResolveRedirect(
        string shortCode, HttpRequest request)
    {
        // 1. Fast path: resolve URL
        string? longUrl = await _cache.ResolveUrl(shortCode);
        if (longUrl == null)
            return RedirectResult.NotFound();

        // 2. Check expiration
        if (longUrl == "EXPIRED")
            return RedirectResult.Expired("https://short.link/expired");

        // 3. Track click event (fire and forget)
        _ = TrackClickAsync(shortCode, request);

        return RedirectResult.Redirect(longUrl);
    }

    private async Task TrackClickAsync(string shortCode, HttpRequest request)
    {
        try
        {
            var clickEvent = new ClickEvent
            {
                ShortCode = shortCode,
                IpAddress = request.HttpContext.Connection.RemoteIpAddress?.ToString() ?? "",
                UserAgent = request.Headers.UserAgent.ToString(),
                Referrer = request.Headers.Referer.ToString(),
                Timestamp = DateTime.UtcNow
            };
            await _kafka.ProduceAsync("click-events", shortCode, clickEvent);
        }
        catch (Exception ex)
        {
            _logger.LogWarning(ex, "Failed to track click for {Code}", shortCode);
        }
    }
}
            

27. Conclusion

Designing a URL shortener at scale is a masterclass in building read-heavy distributed systems. The key challenges — unique code generation, sub-millisecond redirects, and real-time analytics — are applicable to many other systems. The pre-generated ID pool strategy provides the best balance of uniqueness guarantees and performance. Multi-level caching (L1 in-memory, L2 Redis, L3 database) is essential for achieving the 5ms p50 redirect target. Async analytics with Kafka and ClickHouse ensures that click tracking never adds latency to the redirect path.

Key Numbers to Remember

Production Checklist

• Choose code generation strategy: pre-generated pool for most cases, KSUID for sortable codes
• Implement 3-level cache: in-memory (L1) → Redis (L2) → PostgreSQL (L3)
• Use 302 redirects for analytics, 301 for SEO-only use cases
• Async click event tracking via Kafka to keep redirect path fast
• Multi-layer abuse prevention: rate limiting → Safe Browsing → blocklists → ML
• Global GeoDNS routing with regional read replicas
• Monitor redirect latency, cache hit rate, and 404 rate as top SLOs
• Implement link expiration with both lazy and active cleanup
• Custom aliases with Bloom filter pre-check and reserved word protection
• Recursive redirect detection (max depth = 3)

Common Interview Mistakes to Avoid

• Using MD5 hash without collision handling — always have a retry strategy or use deterministic ID generation instead
• Ignoring the read-heavy nature — the 100:1 ratio means caching is not optional, it is mandatory
• Using 301 redirects when analytics are needed — browsers cache 301s and skip your server on subsequent clicks
• Making analytics synchronous — every millisecond on the redirect path matters, write click events to Kafka asynchronously
• Forgetting about custom alias namespace collisions — always check against both auto-generated codes and user-chosen aliases
• Not discussing cache stampede mitigation — a viral URL can crash your database if cache expires simultaneously for millions of requests
• Ignoring geo-distribution — a single-region deployment adds 100-300ms of latency for international users
• Skipping the abuse prevention discussion — URL shorteners are prime targets for phishing and must have multi-layer defense

Key Takeaways

A URL shortener is more than just a mapping from short codes to long URLs. It is a distributed system that must handle extreme read-to-write ratios, provide sub-millisecond redirect latency, track analytics without impacting performance, and defend against abuse. The architecture decisions you make — from ID generation strategy to caching layers to analytics pipeline — all flow from these core requirements. Mastering this design gives you a solid foundation for building any read-heavy distributed system, from CDN edge caching to API gateway routing to content delivery networks.

Whether you are building a simple link shortener for internal use or designing a global platform that handles billions of redirects per day, the fundamental principles remain the same: choose a deterministic ID generation strategy, layer your caches aggressively, separate your write path from your read path, and never let analytics processing slow down the redirect. The companies that have mastered these principles — Bit.ly, Twitter's t.co, and TinyURL — have built some of the most reliable and performant systems on the internet, serving billions of users with sub-millisecond latency and 99.99% availability.

28. Real-Time URL Analytics Dashboard

URL shorteners provide immense value through click analytics: geographic distribution, referrer tracking, device breakdown, and temporal patterns. The analytics pipeline must process click events in real-time while maintaining exact counts for billing and approximate counts for dashboards. This requires a Lambda architecture combining real-time stream processing with batch aggregation.

public class UrlAnalyticsService
{
    private readonly IKafkaProducer<string, ClickEvent> _producer;
    private readonly IRedisCluster _redis;
    private readonly ITimeSeriesDb _timeSeriesDb;

    public async Task TrackClickAsync(ClickEvent click)
    {
        // Real-time counters in Redis (sub-millisecond updates)
        var pipeline = _redis.CreatePipeline();
        var date = click.Timestamp.ToString("yyyyMMdd");

        pipeline.IncrementCounterAsync($"clicks:{click.ShortCode}:total");
        pipeline.IncrementCounterAsync($"clicks:{click.ShortCode}:date:{date}");
        pipeline.IncrementCounterAsync($"clicks:{click.ShortCode}:country:{click.CountryCode}");
        pipeline.IncrementCounterAsync($"clicks:{click.ShortCode}:ref:{click.ReferrerDomain}");
        pipeline.IncrementCounterAsync($"clicks:{click.ShortCode}:device:{click.DeviceType}");
        pipeline.SortedSetAddAsync(
            $"clicks:{click.ShortCode}:hourly",
            click.Timestamp.ToString("yyyyMMddHH"),
            1);

        await pipeline.ExecuteAsync();

        // Async: publish to Kafka for detailed analytics
        await _producer.ProduceAsync("click-events",
            new Message<string, ClickEvent>
            {
                Key = click.ShortCode,
                Value = click
            });
    }

    public async Task<UrlAnalyticsDashboard> GetDashboardAsync(
        string shortCode, DateTime from, DateTime to)
    {
        var totalCount = await _redis.GetAsync<long>(
            $"clicks:{shortCode}:total");

        var countryBreakdown = await _redis.GetHashAllAsync(
            $"clicks:{shortCode}:country");
        var referrerBreakdown = await _redis.GetHashAllAsync(
            $"clicks:{shortCode}:ref");
        var deviceBreakdown = await _redis.GetHashAllAsync(
            $"clicks:{shortCode}:device");

        var hourlyData = await _timeSeriesDb.QueryAsync(
            $"clicks_{shortCode}",
            from, to, granularity: TimeSpan.FromHours(1));

        return new UrlAnalyticsDashboard
        {
            ShortCode = shortCode,
            TotalClicks = totalCount,
            CountryBreakdown = countryBreakdown,
            ReferrerBreakdown = referrerBreakdown,
            DeviceBreakdown = deviceBreakdown,
            HourlyTimeline = hourlyData,
            TopCountries = countryBreakdown
                .OrderByDescending(kv => kv.Value)
                .Take(10)
                .Select(kv => new CountryStat { Code = kv.Key, Clicks = kv.Value })
                .ToList()
        };
    }
}

Analytics Dashboard Metrics