1. Introduction & Why DNS is Hard

Every single internet request begins with DNS. Before your browser can fetch a web page, before your mobile app can call an API, before an email can be delivered, a domain name must be resolved into an IP address. The Domain Name System is the invisible backbone of the internet — handling an estimated 1.1 trillion queries per day across the globe, yet most engineers rarely think about it until something breaks.

Designing a distributed DNS resolution system is one of the most challenging problems in system design. It demands simultaneous mastery of distributed systems, networking, security, caching theory, and global infrastructure. A DNS outage doesn't just affect one service — it can take down every service that depends on a domain, potentially affecting millions of users across the world in seconds.

Consider the scale: Cloudflare's 1.1.1.1 resolver alone processes over 1.5 trillion queries per day. Google's Public DNS handles more than 400 billion queries monthly. These are not numbers that a single server, a single data center, or even a single region can handle. The system must be globally distributed, extremely low-latency (sub-10ms for cache hits), highly available (99.999% uptime), and resilient against some of the most voluminous DDoS attacks ever recorded.

The Unique Challenges

• Latency sensitivity: DNS is on the critical path of every internet connection. An extra 50ms of DNS resolution time means 50ms added to every single connection a user makes.
• Availability requirements: A DNS failure cascades into failures for every service behind those names. There is no fallback — without DNS, the internet simply stops working.
• Attack surface: DNS is one of the oldest protocols on the internet, designed in an era of trust. It remains one of the primary targets for DDoS attacks, cache poisoning, and data exfiltration.
• Distributed consensus: Zone data must be consistent across thousands of nameservers worldwide, yet updates must propagate quickly enough to be useful.
• Protocol legacy: DNS was designed in 1983 (RFC 882/883) and has been extended repeatedly. Any new system must remain backward-compatible with UDP-based DNS over port 53 while supporting modern encrypted transports.

What We Are Building

In this article, we will design a complete distributed DNS resolution system from the ground up. We will cover every layer: from recursive resolvers that accept client queries, through authoritative nameservers that hold zone data, to the caching hierarchies, load balancing strategies, security mechanisms, and operational tooling that make the system production-ready. By the end, you will understand how to design a system capable of handling billions of queries per day with sub-10ms latency, five-nines availability, and resistance to sophisticated attacks.

2. Requirements (Functional & Non-Functional)

Functional Requirements

1. Domain Resolution: Resolve domain names to IP addresses (A/AAAA records), mail servers (MX records), name servers (NS records), canonical names (CNAME), and all other standard DNS record types.
2. Support All Record Types: A, AAAA, CNAME, MX, NS, TXT, SRV, CAA, SOA, PTR, NAPTR, DNSKEY, DS, RRSIG, NSEC, NSEC3.
3. Recursive Resolution: Perform full recursive resolution from root servers down to authoritative nameservers when answers are not cached.
4. Authoritative Resolution: Serve authoritative answers for zones we host, with support for zone transfers (AXFR/IXFR) and dynamic updates.
5. DNSSEC Validation: Validate DNSSEC signatures and provide DNSSEC-signed responses for hosted zones.
6. Encrypted Transports: Support DNS over HTTPS (DoH, RFC 8484) and DNS over TLS (DoT, RFC 7858) in addition to traditional UDP/TCP port 53.
7. Reverse DNS: Support PTR record lookups for IP-to-domain mapping.
8. Zone Management: Provide APIs and UIs for domain registration, zone editing, record management, and DNSSEC key management.
9. Split-Horizon DNS: Serve different answers based on the source IP address or network location for private/internal DNS.
10. Health Checking & Failover: Monitor backend health and automatically update DNS records when failures are detected.

Non-Functional Requirements

3. DNS Fundamentals

Recursive vs. Iterative Resolution

Understanding the difference between recursive and iterative DNS resolution is fundamental to our design.

Recursive Resolution

In recursive resolution, the client asks its DNS resolver to do all the work. The resolver performs the full chain of queries — from root servers to TLD servers to authoritative nameservers — and returns the final answer. The client simply waits for the complete response. This is how most end-user devices interact with DNS.

Iterative Resolution

In iterative resolution, each nameserver returns the best answer it has, typically a referral to the next nameserver in the chain. The querying party is responsible for following the referrals. Authoritative nameservers typically respond iteratively — they do not perform recursion themselves. This is how nameservers communicate with each other during the resolution process.

The Resolution Process

When a user types www.example.com into their browser, the following chain of events occurs:

1. The browser checks its own DNS cache. If found, skip to step 7.
2. The operating system checks its DNS cache (stub resolver). If found, skip to step 7.
3. The OS sends the query to the configured recursive resolver (ISP resolver or public resolver like 1.1.1.1).
4. The recursive resolver checks its cache. If found, return the answer. Otherwise, begin recursive resolution.
5. The resolver queries a root nameserver (. root hints file), which returns a referral to the .com TLD nameservers.
6. The resolver queries a .com TLD nameserver, which returns a referral to example.com's authoritative nameservers.
7. The resolver queries example.com's authoritative nameserver, which returns the final A record.
8. The resolver caches the response (subject to TTL), sends the answer back to the client.
9. The client's browser opens a TCP connection to the resolved IP address.

DNS Record Types

DNS Query Types

• Standard Query (QR=0, OPCODE=0): The normal recursive or iterative query. This is what we handle for 99.9% of traffic.
• Zone Transfer (AXFR/IXFR): Used to replicate zone data between primary and secondary nameservers. Requires TCP and authentication.
• Dynamic Update (UPDATE): RFC 2136 allows programmatic updates to zone data without full zone transfers.
• Notify (NOTIFY): RFC 1996 allows primary servers to notify secondaries of zone changes, triggering immediate zone transfers instead of waiting for SOA refresh intervals.

4. High-Level Architecture

Layer-by-Layer Overview

Client Layer

Clients connect via traditional DNS (UDP/TCP port 53), DNS over HTTPS (DoH on port 443), or DNS over TLS (DoT on port 853). Each transport is handled by the appropriate gateway before being normalized into an internal query format.

Transport & Load Balancing

Anycast IP addresses route clients to the nearest data center. An L4 load balancer distributes queries across the recursive resolver pool within each data center. We use consistent hashing based on the query name to maximize cache locality while maintaining even distribution.

Recursive Resolver Cluster

The recursive resolver is the workhorse of our system. It performs multi-level caching (L1 in-memory, L2 distributed), selects upstream forwarders, implements DNSSEC validation, and returns answers to clients. Each resolver instance maintains its own local cache while sharing a distributed cache layer for cross-instance cache hits.

Authoritative Layer

For zones we host, our authoritative nameservers serve responses directly. For external resolution, the recursive resolver queries root servers, TLD servers, and external authoritative nameservers using iterative queries. We maintain a copy of root hints and periodically refresh them.

Zone Management

Zone data is stored in a distributed SQL database (CockroachDB) for strong consistency. The primary nameserver reads from this database and serves zone transfers to secondary nameservers. Zone changes propagate via NOTIFY + IXFR for fast, incremental updates.

Observability

Every query is logged to a high-throughput analytics pipeline (ClickHouse) for debugging, security analysis, and compliance. Health checkers continuously monitor backend servers, and metrics flow into Prometheus/Grafana for real-time dashboards and alerting.

5. Recursive Resolver Design

The recursive resolver is the most complex component in our DNS system. It must handle millions of queries per second, perform multi-level caching, validate DNSSEC signatures, implement sophisticated upstream selection logic, and do all of this in under 5 milliseconds for cache hits.

Core Resolution Pipeline

Resolver Implementation

C#
public class RecursiveResolver
{
    private readonly ICacheLayer _l1Cache;
    private readonly IDistributedCache _l2Cache;
    private readonly IUpstreamSelector _upstreamSelector;
    private readonly IDnssecValidator _dnssecValidator;
    private readonly IQueryLogger _queryLogger;
    private readonly ResolverConfig _config;

    public RecursiveResolver(
        ICacheLayer l1Cache,
        IDistributedCache l2Cache,
        IUpstreamSelector upstreamSelector,
        IDnssecValidator dnssecValidator,
        IQueryLogger queryLogger,
        ResolverConfig config)
    {
        _l1Cache = l1Cache;
        _l2Cache = l2Cache;
        _upstreamSelector = upstreamSelector;
        _dnssecValidator = dnssecValidator;
        _queryLogger = queryLogger;
        _config = config;
    }

    public async Task<DnsResponse> ResolveAsync(DnsQuery query, CancellationToken ct)
    {
        var stopwatch = Stopwatch.StartNew();
        var cacheKey = CacheKey.From(query);

        // Step 1: Check L1 in-process cache
        var cached = await _l1Cache.GetAsync(cacheKey, ct);
        if (cached != null && !cached.IsExpired)
        {
            stopwatch.Stop();
            await _queryLogger.LogAsync(query, cached, CacheLevel.L1, stopwatch.Elapsed, ct);
            return cached;
        }

        // Step 2: Check L2 distributed cache (Redis)
        cached = await _l2Cache.GetAsync(cacheKey, ct);
        if (cached != null && !cached.IsExpired)
        {
            // Promote to L1
            await _l1Cache.SetAsync(cacheKey, cached, ct);
            stopwatch.Stop();
            await _queryLogger.LogAsync(query, cached, CacheLevel.L2, stopwatch.Elapsed, ct);
            return cached;
        }

        // Step 3: Perform recursive resolution
        var answer = await ResolveRecursiveAsync(query, _config.MaxDepth, ct);

        // Step 4: Validate DNSSEC if configured
        if (_config.EnableDnssecValidation && answer.AnswerRecords.Any())
        {
            var isValid = await _dnssecValidator.ValidateAsync(answer, ct);
            if (!isValid)
            {
                answer = DnsResponse.ServerFailure(query, "DNSSEC validation failed");
            }
        }

        // Step 5: Cache the response
        if (answer.RCode == RCode.NoError || answer.RCode == RCode.NXDomain)
        {
            var ttl = answer.AnswerRecords.Any()
                ? answer.AnswerRecords.Min(r => r.TTL)
                : _config.NegativeCacheTtl;
            var cacheEntry = new CacheEntry(answer, TimeSpan.FromSeconds(ttl));

            await _l1Cache.SetAsync(cacheKey, cacheEntry, ct);
            await _l2Cache.SetAsync(cacheKey, cacheEntry, ct);
        }

        stopwatch.Stop();
        await _queryLogger.LogAsync(query, answer, CacheLevel.Miss, stopwatch.Elapsed, ct);
        return answer;
    }

    private async Task<DnsResponse> ResolveRecursiveAsync(
        DnsQuery query, int maxDepth, CancellationToken ct)
    {
        var currentQuery = query;
        var currentServer = RootServerHints.GetRootServers().First();

        for (int depth = 0; depth < maxDepth; depth++)
        {
            var upstream = _upstreamSelector.Select(currentServer, query);
            var response = await upstream.QueryAsync(currentQuery, _config.QueryTimeout, ct);

            if (response.RCode == RCode.NoError && response.AnswerRecords.Any())
                return response;

            if (response.RCode == RCode.NXDomain)
                return response;

            if (response.Referral == null)
                return response;

            currentServer = response.Referral.Nameservers.First();
        }

        return DnsResponse.ServerFailure(query, "Max recursion depth exceeded");
    }
}

Upstream Selection Algorithm

When the recursive resolver needs to query upstream nameservers, it must choose which server to query. This decision significantly impacts latency and resilience.

C#
public class IntelligentUpstreamSelector : IUpstreamSelector
{
    private readonly ConcurrentDictionary<string, ServerHealth> _healthMap = new();
    private readonly ILogger<IntelligentUpstreamSelector> _logger;
    private readonly UpstreamConfig _config;

    public async Task<IUpstream> SelectAsync(
        IReadOnlyList<DnsServer> candidates, DnsQuery query, CancellationToken ct)
    {
        var healthy = candidates
            .Where(s => IsHealthy(s))
            .OrderBy(s => GetLatency(s))
            .ThenBy(s => GetLoad(s))
            .ToList();

        if (!healthy.Any())
        {
            _logger.LogWarning("All upstreams unhealthy for {Query}, using best-effort", query.Name);
            return candidates.OrderBy(s => GetLatency(s)).First();
        }

        // If multiple healthy servers are within 2ms, pick randomly (load spread)
        var bestLatency = healthy.First().MeasuredLatency;
        var closeServers = healthy
            .Where(s => s.MeasuredLatency <= bestLatency + TimeSpan.FromMilliseconds(2))
            .ToList();

        return closeServers[Random.Shared.Next(closeServers.Count)];
    }

    private bool IsHealthy(DnsServer server)
    {
        var health = _healthMap.GetOrAdd(server.Address, _ => new ServerHealth());
        return health.FailureRate < _config.MaxFailureRate
            && health.ConsecutiveFailures < _config.MaxConsecutiveFailures;
    }

    private TimeSpan GetLatency(DnsServer server)
    {
        var health = _healthMap.GetOrAdd(server.Address, _ => new ServerHealth());
        return health.P99Latency;
    }

    private double GetLoad(DnsServer server)
    {
        return server.CurrentQueryCount / (double)server.MaxCapacity;
    }
}

6. Authoritative Nameserver Design

The authoritative nameserver is responsible for serving definitive answers for zones it hosts. Unlike the recursive resolver, it never queries other nameservers — it either knows the answer or returns a referral/NXDOMAIN. Our authoritative nameserver must handle high query volumes, serve DNSSEC-signed responses, support zone transfers, and maintain consistency across global replicas.

Architecture

C#
public class AuthoritativeNameserver : IDnsResponder
{
    private readonly IZoneStore _zoneStore;
    private readonly IDnssecSigner _signer;
    private readonly IMetricsCollector _metrics;

    public async Task<DnsResponse> HandleQueryAsync(
        DnsQuery query, CancellationToken ct)
    {
        _metrics.IncrementCounter("dns.authoritative.queries",
            new[] { ("type", query.Type.ToString()), ("zone", GetZone(query.Name)) });

        var zone = await _zoneStore.GetZoneAsync(query.Name, ct);
        if (zone == null)
        {
            return ReferralResponse(query);
        }

        var records = await zone.LookupAsync(query.Name, query.Type, ct);

        if (records.Count == 0)
        {
            return NXDOMAIN(query, zone);
        }

        var response = new DnsResponse
        {
            Query = query,
            RCode = RCode.NoError,
            Authoritative = true,
            AnswerRecords = records.Select(r => ToResourceRecord(r, query.Type)).ToList(),
            NameServers = zone.GetNSRecords(),
            SOA = zone.GetSOARecord()
        };

        if (zone.IsSigned && _signer != null)
        {
            response = await _signer.SignResponseAsync(response, zone, ct);
        }

        return response;
    }

    private List<DnsRecord> ApplyZoneSplit(
        Zone zone, DnsQuery query, IPAddress clientIp)
    {
        if (!zone.HasSplitHorizon)
            return zone.GetRecords(query.Name, query.Type);

        var views = zone.SplitHorizonViews
            .Where(v => v.Matcher.Matches(clientIp))
            .OrderByDescending(v => v.Priority)
            .ToList();

        if (views.Any())
            return views.First().GetRecords(query.Name, query.Type);

        return zone.GetRecords(query.Name, query.Type);
    }
}

Zone Data Storage

We store zone data in CockroachDB, a distributed SQL database that provides strong consistency, automatic replication, and horizontal scaling. Zone data is stored in a normalized schema optimized for both read-heavy query serving and efficient zone transfers.

SQL
CREATE TABLE zones (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    name STRING NOT NULL UNIQUE,
    signed BOOLEAN NOT NULL DEFAULT FALSE,
    soa_serial BIGINT NOT NULL DEFAULT 1,
    soa_refresh INTERVAL NOT NULL DEFAULT '3600s',
    soa_retry INTERVAL NOT NULL DEFAULT '900s',
    soa_expire INTERVAL NOT NULL DEFAULT '604800s',
    soa_minimum_ttl INTERVAL NOT NULL DEFAULT '300s',
    created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
    updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE TABLE dns_records (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    zone_id UUID NOT NULL REFERENCES zones(id) ON DELETE CASCADE,
    name STRING NOT NULL,
    type STRING NOT NULL,
    ttl INT NOT NULL DEFAULT 3600,
    data JSONB NOT NULL,
    priority INT,
    weight INT,
    UNIQUE(zone_id, name, type, data)
);

CREATE INDEX idx_records_lookup ON dns_records (zone_id, name, type);
CREATE INDEX idx_records_zone_transfer ON dns_records (zone_id, name) INCLUDE (type, ttl, data, priority, weight);

7. DNS Cache Hierarchy (L1 / L2 / L3)

Caching is the single most important performance optimization in DNS. Without caching, every DNS query would require multiple round trips to root servers, TLD servers, and authoritative nameservers — adding hundreds of milliseconds to every internet connection. Our three-level cache hierarchy is designed to maximize cache hit rates while keeping response times minimal.

Cache Level Comparison

L1 Cache Implementation

C#
public class L1Cache : ICacheLayer
{
    private readonly ConcurrentDictionary<string, CacheEntry> _cache = new();
    private readonly long _maxSize;
    private long _currentSize;
    private readonly Timer _cleanupTimer;

    public L1Cache(CacheConfig config)
    {
        _maxSize = config.L1MaxEntries;
        _cleanupTimer = new Timer(CleanupExpired, null,
            config.CleanupInterval, config.CleanupInterval);
    }

    public ValueTask<CacheEntry?> GetAsync(CacheKey key, CancellationToken ct)
    {
        if (_cache.TryGetValue(key.ToString(), out var entry))
        {
            if (!entry.IsExpired)
            {
                entry.Hits++;
                return ValueTask.FromResult<CacheEntry?>(entry);
            }
            // Entry expired — remove it
            _cache.TryRemove(key.ToString(), out _);
            Interlocked.Add(ref _currentSize, -entry.SizeBytes);
        }

        return ValueTask.FromResult<CacheEntry?>(null);
    }

    public ValueTask SetAsync(CacheKey key, CacheEntry entry, CancellationToken ct)
    {
        var keyStr = key.ToString();

        // Evict if at capacity using LRU
        while (_currentSize + entry.SizeBytes > _maxSize && _cache.Count > 0)
        {
            EvictLeastRecentlyUsed();
        }

        _cache.AddOrUpdate(keyStr, entry, (_, _) => entry);
        Interlocked.Add(ref _currentSize, entry.SizeBytes);

        return ValueTask.CompletedTask;
    }

    private void EvictLeastRecentlyUsed()
    {
        var leastUsed = _cache
            .OrderBy(kvp => kvp.Value.Hits)
            .ThenBy(kvp => kvp.Value.CreatedAt)
            .FirstOrDefault();

        if (leastUsed.Key != null)
        {
            _cache.TryRemove(leastUsed.Key, out var removed);
            if (removed != null)
                Interlocked.Add(ref _currentSize, -removed.SizeBytes);
        }
    }

    private void CleanupExpired(object? state)
    {
        var expired = _cache
            .Where(kvp => kvp.Value.IsExpired)
            .Select(kvp => kvp.Key)
            .ToList();

        foreach (var key in expired)
        {
            if (_cache.TryRemove(key, out var removed))
                Interlocked.Add(ref _currentSize, -removed.SizeBytes);
        }
    }
}

L2 Distributed Cache (Redis)

The L2 cache is backed by a Redis Cluster deployment within each data center. It provides shared caching across all resolver instances, dramatically increasing the effective cache size and hit rate.

C#
public class L2Cache : IDistributedCache
{
    private readonly IConnectionMultiplexer _redis;
    private readonly IDatabase _db;
    private readonly SerializerOptions _serializerOptions;

    public async Task<CacheEntry?> GetAsync(CacheKey key, CancellationToken ct)
    {
        var redisKey = $"dns:{key}";
        var value = await _db.StringGetAsync(redisKey);

        if (value.IsNullOrEmpty)
            return null;

        return JsonSerializer.Deserialize<CacheEntry>(value!, _serializerOptions);
    }

    public async Task SetAsync(CacheKey key, CacheEntry entry, CancellationToken ct)
    {
        var redisKey = $"dns:{key}";
        var serialized = JsonSerializer.Serialize(entry, _serializerOptions);

        // Set with TTL + jitter to prevent thundering herd
        var ttl = entry.TtlRemaining + GetJitter(entry.TtlRemaining);
        await _db.StringSetAsync(redisKey, serialized, ttl);
    }

    private TimeSpan GetJitter(TimeSpan baseTtl)
    {
        // Add 0-10% jitter to TTL to prevent synchronized cache stampedes
        var jitterMs = Random.Shared.Next(0, (int)(baseTtl.TotalMilliseconds * 0.1));
        return TimeSpan.FromMilliseconds(jitterMs);
    }
}

8. TTL Management & Cache Invalidation

Time-To-Live (TTL) is the fundamental mechanism by which DNS balances freshness with performance. Every DNS record carries a TTL value that tells resolvers how long to cache the response. Getting TTL management right is critical — too short and you overload authoritative servers; too long and stale records persist after changes.

TTL Strategy

Cache Invalidation Strategies

Unlike HTTP caching, DNS does not support a "purge" operation in the traditional sense. Once a record is cached, it lives until its TTL expires. This creates several challenges:

1. Emergency Rollback: If you deploy a bad DNS change, you must wait for TTL expiry or use a "cache flush" strategy where you first lower the TTL, wait for old caches to expire, make the change, then raise the TTL again.
2. Negative Caching: NXDOMAIN responses are also cached (per RFC 2308). The TTL for negative responses is typically the minimum of the SOA record's minimum TTL field and 3 hours.
3. Cache Coherency: We implement a "cache stampede" prevention mechanism using probabilistic early recomputation (PER). When a cache entry is about to expire, there's a small probability it will be recomputed early, spreading the refresh load over time.

C#
public class CacheStampedePrevention
{
    private readonly ConcurrentDictionary<string, Mutex> _locks = new();

    public async Task<CacheEntry?> GetOrRecomputeAsync(
        CacheKey key,
        Func<CancellationToken, Task<CacheEntry>> recompute,
        TimeSpan ttl,
        CancellationToken ct)
    {
        var entry = await _cache.GetAsync(key, ct);
        if (entry != null && !entry.IsExpired)
            return entry;

        // Probabilistic early recomputation
        // As TTL approaches 0, probability of recomputation increases
        if (entry != null)
        {
            var elapsed = DateTime.UtcNow - entry.CreatedAt;
            var remaining = entry.ExpiresAt - DateTime.UtcNow;
            var probability = remaining.TotalSeconds / ttl.TotalSeconds;

            if (Random.Shared.NextDouble() > probability)
            {
                // Return stale entry while recomputing in background
                _ = Task.Run(async () =>
                {
                    var fresh = await recompute(ct);
                    await _cache.SetAsync(key, fresh, ct);
                }, ct);
                return entry;
            }
        }

        // Use a mutex to prevent duplicate recomputation
        var mutex = _locks.GetOrAdd(key.ToString(), _ => new Mutex());
        await mutex.WaitAsync(ct);
        try
        {
            // Double-check after acquiring lock
            entry = await _cache.GetAsync(key, ct);
            if (entry != null && !entry.IsExpired)
                return entry;

            entry = await recompute(ct);
            await _cache.SetAsync(key, entry, ct);
            return entry;
        }
        finally
        {
            mutex.Release();
        }
    }
}

9. DNS Load Balancing Strategies

DNS load balancing is one of the most powerful tools for distributing traffic across multiple servers. Since DNS is the first step in every connection, controlling DNS responses gives us control over traffic distribution. We implement multiple load balancing strategies, each optimized for different use cases.

Strategy Comparison

Load Balancer Implementation

C#
public interface ILoadBalancingStrategy
{
    IReadOnlyList<IPAddress> SelectAddresses(
        IReadOnlyList<ServerEntry> servers,
        DnsQuery query,
        ClientContext client);
}

public class GeoBasedLoadBalancer : ILoadBalancingStrategy
{
    private readonly IGeoIpLookup _geoIp;
    private readonly INetworkTopologyMap _topologyMap;

    public IReadOnlyList<IPAddress> SelectAddresses(
        IReadOnlyList<ServerEntry> servers,
        DnsQuery query,
        ClientContext client)
    {
        var clientGeo = _geoIp.Lookup(client.SourceIp);
        var clientNetwork = _topologyMap.GetNetworkInfo(client.SourceIp);

        return servers
            .OrderBy(s =>
            {
                // Primary sort: geographic distance
                var geoDistance = CalculateGeoDistance(clientGeo, s.Location);

                // Secondary sort: network proximity (AS path length)
                var networkCost = CalculateNetworkCost(clientNetwork, s.NetworkInfo);

                // Weighted combination
                return (geoDistance * 0.4) + (networkCost * 0.6);
            })
            .ThenBy(s => s.CurrentLoad)
            .Take(3) // Return top 3 candidates for client-side failover
            .Select(s => s.IpAddress)
            .ToList();
    }
}

public class WeightedRoundRobin : ILoadBalancingStrategy
{
    private readonly ConcurrentDictionary<string, int> _counters = new();
    private readonly Random _random = new();

    public IReadOnlyList<IPAddress> SelectAddresses(
        IReadOnlyList<ServerEntry> servers,
        DnsQuery query,
        ClientContext client)
    {
        // Weighted random selection
        var totalWeight = servers.Sum(s => s.Weight);
        var selected = new List<IPAddress>();
        var remaining = servers.ToList();

        while (remaining.Count > 0 && selected.Count < Math.Min(3, remaining.Count))
        {
            var roll = _random.Next(totalWeight);
            var cumulative = 0;

            for (int i = 0; i < remaining.Count; i++)
            {
                cumulative += remaining[i].Weight;
                if (roll < cumulative)
                {
                    selected.Add(remaining[i].IpAddress);
                    totalWeight -= remaining[i].Weight;
                    remaining.RemoveAt(i);
                    break;
                }
            }
        }

        return selected;
    }
}

10. Anycast Routing for DNS

Anycast is a network addressing and routing methodology where a single IP address is announced from multiple locations simultaneously. BGP routing naturally directs each client to the topologically nearest data center. Anycast is the foundational technology that enables global DNS infrastructure — all 13 root server clusters use anycast, and it is essential for our design.

How Anycast Works

When we announce our DNS service IP (e.g., 1.1.1.1) from 200+ locations via BGP, each client's queries automatically reach the closest data center without any client-side configuration. The BGP routing protocol selects the path with the best AS-path length, which generally correlates with geographic and network proximity.

Implementation Requirements

• BGP Sessions: Each data center establishes BGP sessions with upstream providers and announces our anycast prefixes. We use BGP communities to control route propagation and implement traffic engineering.
• Route Monitoring: We continuously monitor BGP route advertisements to detect hijacks, leaks, or misconfigurations that could redirect DNS traffic to unintended locations.
• Health-Based Withdrawal: If a data center fails health checks, it automatically withdraws its BGP routes, causing traffic to failover to the next-closest location within seconds.
• DDoS Resilience: Anycast distributes DDoS attack traffic across all announcing locations. An attack on one location only affects queries that would have been routed there — the rest of the network continues operating normally.

11. DNS over HTTPS (DoH) & DNS over TLS (DoT)

Traditional DNS sends queries and responses in plaintext over UDP port 53 (or TCP for large responses). This exposes DNS queries to eavesdropping, manipulation, and censorship. DoH (RFC 8484) and DoT (RFC 7858) encrypt DNS traffic, protecting user privacy and integrity.

Protocol Comparison

DoH Server Implementation

C#
[ApiController]
[Route("dns-query")]
public class DohController : ControllerBase
{
    private readonly RecursiveResolver _resolver;
    private readonly ILogger<DohController> _logger;

    [HttpPost]
    [Consumes("application/dns-message")]
    [Produces("application/dns-message")]
    public async Task<IActionResult> HandlePostQuery(
        [FromHeader(Name = "Content-Type")] string contentType,
        CancellationToken ct)
    {
        using var body = new MemoryStream();
        await Request.Body.CopyToAsync(body, ct);
        var queryBytes = body.ToArray();

        var query = DnsMessageParser.Parse(queryBytes);
        var response = await _resolver.ResolveAsync(query, ct);
        var responseBytes = DnsMessageSerializer.Serialize(response);

        Response.Headers["Content-Type"] = "application/dns-message";
        Response.Headers["Cache-Control"] = $"max-age={response.AnswerRecords.MinOrDefault(r => r.TTL)}";

        return File(responseBytes, "application/dns-message");
    }

    [HttpGet]
    [Produces("application/dns-message")]
    public async Task<IActionResult> HandleGetQuery(
        [FromQuery(Name = "dns")] string dnsParam,
        CancellationToken ct)
    {
        var queryBytes = Base64UrlEncoder.Decode(dnsParam);
        var query = DnsMessageParser.Parse(queryBytes);
        var response = await _resolver.ResolveAsync(query, ct);
        var responseBytes = DnsMessageSerializer.Serialize(response);

        return File(responseBytes, "application/dns-message");
    }
}

DoT Server Implementation

C#
public class DotServer
{
    private readonly TcpListener _listener;
    private readonly X509Certificate2 _certificate;
    private readonly RecursiveResolver _resolver;
    private readonly ILogger<DotServer> _logger;

    public async Task StartAsync(CancellationToken ct)
    {
        _listener = new TcpListener(IPAddress.Any, 853);
        _listener.Start();

        while (!ct.IsCancellationRequested)
        {
            var client = await _listener.AcceptTcpClientAsync(ct);
            _ = HandleClientAsync(client, ct);
        }
    }

    private async Task HandleClientAsync(TcpClient client, CancellationToken ct)
    {
        try
        {
            await using var networkStream = client.GetStream();
            var sslStream = new SslStream(networkStream, false);

            await sslStream.AuthenticateAsServerAsync(
                new SslServerAuthenticationOptions
                {
                    ServerCertificate = _certificate,
                    ClientCertificateRequired = false,
                    EnabledSslProtocols = SslProtocols.Tls12 | SslProtocols.Tls13,
                }, ct);

            // DNS over TLS uses length-prefixed wire format
            while (client.Connected && !ct.IsCancellationRequested)
            {
                var lengthBuffer = new byte[2];
                await sslStream.ReadExactlyAsync(lengthBuffer, ct);
                var messageLength = BinaryPrimitives.ReadUInt16BigEndian(lengthBuffer);

                var messageBuffer = new byte[messageLength];
                await sslStream.ReadExactlyAsync(messageBuffer, ct);

                var query = DnsMessageParser.Parse(messageBuffer);
                var response = await _resolver.ResolveAsync(query, ct);
                var responseBytes = DnsMessageSerializer.Serialize(response);

                var responseLength = new byte[2];
                BinaryPrimitives.WriteUInt16BigEndian(responseLength, (ushort)responseBytes.Length);
                await sslStream.WriteAsync(responseLength, ct);
                await sslStream.WriteAsync(responseBytes, ct);
            }
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "DoT connection error");
        }
        finally
        {
            client.Dispose();
        }
    }
}

12. DNSSEC (Signing, Validation & Chain of Trust)

DNS Security Extensions (DNSSEC) provide cryptographic authentication and integrity for DNS data. Without DNSSEC, an attacker can forge DNS responses (cache poisoning) and redirect users to malicious servers without detection. DNSSEC adds digital signatures to DNS records, allowing resolvers to verify that responses are authentic and unmodified.

How DNSSEC Works

Key Concepts

• Zone Signing Key (ZSK): Signs the actual DNS records (RRSets) in the zone. Rotated frequently (every 1-3 months).
• Key Signing Key (KSK): Signs the DNSKEY RRSet. Rotated less frequently (every 1-2 years). Its hash is published as a DS record in the parent zone.
• RRSIG Records: Digital signatures attached to each RRSet. Created by the ZSK. Contains the signature, key tag, algorithm, expiration, and inception timestamps.
• NSEC/NSEC3:Authenticated denial of existence. When a name doesn't exist, NSEC/NSEC3 proves it cryptographically. NSEC3 uses hashed names to prevent zone enumeration.
• DS Records: Delegation Signer records published in the parent zone. They contain a hash of the child zone's KSK, forming the chain of trust from root to leaf.

DNSSEC Signing Implementation

C#
public class DnssecSigner
{
    private readonly IKeyStore _keyStore;
    private readonly IDnssecConfig _config;

    public async Task<DnsResponse> SignResponseAsync(
        DnsResponse response, Zone zone, CancellationToken ct)
    {
        var zsk = await _keyStore.GetZskAsync(zone.Name, ct);
        var ksk = await _keyStore.GetKskAsync(zone.Name, ct);

        // Sign each answer RRSet
        foreach (var rrset in response.AnswerRecords.GroupBy(r => r.Type))
        {
            var rrsig = CreateRRSIG(
                rrset.Key,
                rrset.ToList(),
                zsk,
                zone.Name,
                response.SOA?.Serial ?? 0);

            response.AddExtraRecord(rrsig);
        }

        // Sign the NS RRSet in authority section
        if (response.NameServers.Any())
        {
            var nsRrsig = CreateRRSIG(
                DnsRecordType.NS,
                response.NameServers,
                zsk,
                zone.Name,
                response.SOA?.Serial ?? 0);

            response.AddExtraRecord(nsRrsig);
        }

        // Add DNSKEY records and their signature
        var dnskeyRrsig = CreateRRSIG(
            DnsRecordType.DNSKEY,
            new[] { zsk.PublicKeyRecord, ksk.PublicKeyRecord },
            ksk, // KSK signs the DNSKEY set
            zone.Name,
            response.SOA?.Serial ?? 0);

        response.AddExtraRecord(zsk.PublicKeyRecord);
        response.AddExtraRecord(ksk.PublicKeyRecord);
        response.AddExtraRecord(dnskeyRrsig);

        // Add NSEC3PARAM for authenticated denial of existence
        var nsec3Param = CreateNSEC3PARAM(zone.Name);
        response.AddExtraRecord(nsec3Param);

        return response;
    }

    private RRSIGRecord CreateRRSIG(
        DnsRecordType typeCovered,
        IReadOnlyList<DnsResourceRecord> records,
        DnssecKey key,
        string zoneName,
        uint serial)
    {
        var wireData = records.OrderBy(r => r.Name).Select(r => r.ToWireFormat());
        var signature = key.Sign(typeCovered, wireData);

        return new RRSIGRecord
        {
            TypeCovered = typeCovered,
            Algorithm = key.Algorithm,
            Labels = CountLabels(zoneName),
            OriginalTTL = records.First().TTL,
            SignatureExpiration = DateTime.UtcNow.AddDays(_config.SignatureValidityDays),
            SignatureInception = DateTime.UtcNow,
            KeyTag = key.KeyTag,
            SignerName = zoneName,
            Signature = signature
        };
    }
}

DNSSEC Validation

C#
public class DnssecValidator
{
    private readonly ITrustAnchorStore _trustAnchors;

    public async Task<ValidationResult> ValidateAsync(
        DnsResponse response, CancellationToken ct)
    {
        // Step 1: Find the DNSKEY records
        var dnskeyRecords = response.AdditionalRecords
            .OfType<DNSKEYRecord>()
            .ToList();

        if (!dnskeyRecords.Any())
            return ValidationResult.Insecure("No DNSKEY records found");

        // Step 2: Validate DNSKEY against DS record from parent
        var dsRecord = await GetDSRecordAsync(response.Query.Name, ct);
        if (dsRecord != null)
        {
            var ksk = dnskeyRecords.FirstOrDefault(k => k.Flags.HasFlag(DnsKeyFlags.SecureEntryPoint));
            if (ksk == null)
                return ValidationResult.Bogus("KSK missing but DS record exists");

            if (!ValidateDS(ksk, dsRecord))
                return ValidationResult.Bogus("DS validation failed");
        }

        // Step 3: Validate each RRSIG
        var rrsigRecords = response.AdditionalRecords
            .OfType<RRSIGRecord>()
            .ToList();

        foreach (var rrsig in rrsigRecords)
        {
            var rrset = GetRRSet(response, rrsig.TypeCovered, rrsig.SignerName);
            var dnskey = dnskeyRecords.FirstOrDefault(k => k.KeyTag == rrsig.KeyTag);

            if (dnskey == null)
                return ValidationResult.Bogus($"DNSKEY not found for tag {rrsig.KeyTag}");

            if (rrsig.SignatureExpiration < DateTime.UtcNow)
                return ValidationResult.Bogus("RRSIG has expired");

            if (!VerifySignature(rrsig, rrset, dnskey))
                return ValidationResult.Bogus($"RRSIG verification failed for {rrsig.TypeCovered}");
        }

        return ValidationResult.Secure;
    }
}

13. Zone Management & Zone Transfers

Zone management is the operational backbone of any authoritative DNS service. Zone transfers (AXFR and IXFR) are the mechanisms by which zone data is replicated from primary to secondary nameservers, ensuring consistency and redundancy.

AXFR vs. IXFR

Zone Transfer Implementation

C#
public class ZoneTransferService
{
    private readonly IZoneStore _zoneStore;
    private readonly IAclManager _aclManager;
    private readonly IMetricsCollector _metrics;

    public async Task HandleAxfrAsync(
        TcpClient client, string zoneName, IPAddress remoteIp, CancellationToken ct)
    {
        if (!_aclManager.IsTransferAllowed(zoneName, remoteIp))
        {
            await SendRefused(client, ct);
            return;
        }

        var zone = await _zoneStore.GetZoneAsync(zoneName, ct);
        if (zone == null)
        {
            await SendRefused(client, ct);
            return;
        }

        var stream = client.GetStream();

        // SOA record starts the transfer
        var soa = zone.GetSOARecord();
        await SendRecord(stream, soa, ct);

        // Send all records in canonical order
        var records = await zone.GetAllRecordsAsync(ct);
        foreach (var record in records.OrderBy(r => r.Name).ThenBy(r => r.Type))
        {
            await SendRecord(stream, record, ct);
        }

        // SOA record ends the transfer
        await SendRecord(stream, soa, ct);

        _metrics.IncrementCounter("dns.zone_transfer.axfr.completed",
            new[] { ("zone", zoneName) });
    }

    public async Task HandleIxfrAsync(
        TcpClient client, string zoneName, uint fromSerial,
        IPAddress remoteIp, CancellationToken ct)
    {
        if (!_aclManager.IsTransferAllowed(zoneName, remoteIp))
        {
            await SendRefused(client, ct);
            return;
        }

        var zone = await _zoneStore.GetZoneAsync(zoneName, ct);
        var changes = await _zoneStore.GetChangesSinceAsync(zoneName, fromSerial, ct);

        if (changes == null || !changes.Any())
        {
            // No changes — send current SOA
            await SendSoaOnly(client, zone, ct);
            return;
        }

        var stream = client.GetStream();
        var soa = zone.GetSOARecord();

        // Group changes into deletions and additions
        foreach (var changeGroup in changes.GroupBy(c => c.Serial))
        {
            // Deletions
            var deletions = changeGroup.Where(c => c.ChangeType == ChangeType.Delete);
            if (deletions.Any())
            {
                await SendIxfrSection(stream, IXFRSection.Deletion, deletions, ct);
            }

            // Additions
            var additions = changeGroup.Where(c => c.ChangeType == ChangeType.Add);
            if (additions.Any())
            {
                await SendIxfrSection(stream, IXFRSection.Addition, additions, ct);
            }
        }

        // Final SOA
        await SendRecord(stream, soa, ct);
    }
}

14. DNS Failover & Health Checking

DNS failover automatically updates DNS records when backend servers become unhealthy. This is critical for maintaining availability — if a server goes down, DNS should stop directing traffic to it. Our health checking system continuously monitors backend servers and updates DNS records in real-time.

Health Check Architecture

C#
public class HealthCheckManager
{
    private readonly IHealthCheckRegistry _registry;
    private readonly IDnsRecordUpdater _recordUpdater;
    private readonly IStateStore _stateStore;
    private readonly Timer _checkTimer;

    public async Task ExecuteChecksAsync(CancellationToken ct)
    {
        var targets = await _registry.GetAllTargetsAsync(ct);

        var checkTasks = targets.Select(target => CheckTargetAsync(target, ct));
        var results = await Task.WhenAll(checkTasks);

        foreach (var (target, isHealthy) in results)
        {
            var previousState = await _stateStore.GetHealthAsync(target.Id, ct);
            var currentState = isHealthy ? HealthState.Healthy : HealthState.Unhealthy;

            if (previousState != currentState)
            {
                await _stateStore.SetHealthAsync(target.Id, currentState, ct);

                // Update DNS records based on new state
                if (currentState == HealthState.Unhealthy)
                {
                    await _recordUpdater.MarkUnhealthyAsync(target.DnsRecords, ct);
                    LogWarning($"Health check failed for {target.Id}: removing from DNS");
                }
                else
                {
                    await _recordUpdater.MarkHealthyAsync(target.DnsRecords, ct);
                    LogInfo($"Health check recovered for {target.Id}: restoring to DNS");
                }
            }
        }
    }

    private async Task<(HealthTarget target, bool isHealthy)> CheckTargetAsync(
        HealthTarget target, CancellationToken ct)
    {
        var checkType = target.CheckType;
        var timeout = target.Timeout ?? TimeSpan.FromSeconds(5);

        try
        {
            using var cts = CancellationTokenSource.CreateLinkedTokenSource(ct);
            cts.CancelAfter(timeout);

            switch (checkType)
            {
                case HealthCheckType.TcpConnect:
                    return (target, await CheckTcpConnectAsync(target.Host, target.Port, cts.Token));

                case HealthCheckType.HttpEndpoint:
                    return (target, await CheckHttpEndpointAsync(target.HealthUrl, cts.Token));

                case HealthCheckType.DnsQuery:
                    return (target, await CheckDnsQueryAsync(target.ExpectedAnswer, cts.Token));

                case HealthCheckType.IcmpPing:
                    return (target, await CheckIcmpPingAsync(target.Host, cts.Token));

                default:
                    return (target, false);
            }
        }
        catch (Exception)
        {
            return (target, false);
        }
    }
}

15. Rate Limiting & DDoS Protection

DNS is one of the most heavily attacked protocols on the internet. The 2016 Dyn attack reached 1.2 Tbps of traffic. Our system must absorb massive volumetric attacks while continuing to serve legitimate queries. We implement defense at multiple layers.

Multi-Layer Defense Strategy

Response Rate Limiting (RRL)

C#
public class ResponseRateLimiter
{
    private readonly ConcurrentDictionary<string, TokenBucket> _buckets = new();
    private readonly RateLimitConfig _config;

    public bool ShouldRateLimit(DnsQuery query, IPAddress clientIp)
    {
        // Key by: source IP + response type + query name (truncated)
        // This groups amplification traffic together
        var key = GetRateLimitKey(query, clientIp);

        var bucket = _buckets.GetOrAdd(key, _ => new TokenBucket(
            _config.RateLimitPerSecond,
            _config.BurstSize));

        return !bucket.TryConsume(1);
    }

    private string GetRateLimitKey(DnsQuery query, IPAddress clientIp)
    {
        // For amplification detection, group by:
        // - Source IP (or /24 subnet for spoofed traffic)
        // - Query type (large responses are amplified)
        // - Response size category

        var subnet = GetSubnet(clientIp, 24);
        var responseCategory = GetResponseCategory(query.Type);

        return $"{subnet}:{responseCategory}";
    }

    private ResponseCategory GetResponseCategory(DnsRecordType type)
    {
        // DNSSEC-signed ANY queries produce large responses (amplification vector)
        return type switch
        {
            DnsRecordType.ANY => ResponseCategory.LargeAmplification,
            DnsRecordType.DNSKEY => ResponseCategory.LargeAmplification,
            DnsRecordType.DS => ResponseCategory.Medium,
            DnsRecordType.A => ResponseCategory.Small,
            DnsRecordType.AAAA => ResponseCategory.Small,
            _ => ResponseCategory.Small
        };
    }
}

public class TokenBucket
{
    private double _tokens;
    private readonly double _maxTokens;
    private readonly double _refillRate;
    private DateTime _lastRefill;
    private readonly object _lock = new();

    public TokenBucket(double refillRatePerSecond, double burstSize)
    {
        _refillRate = refillRatePerSecond;
        _maxTokens = burstSize;
        _tokens = burstSize;
        _lastRefill = DateTime.UtcNow;
    }

    public bool TryConsume(int tokens)
    {
        lock (_lock)
        {
            Refill();
            if (_tokens >= tokens)
            {
                _tokens -= tokens;
                return true;
            }
            return false;
        }
    }

    private void Refill()
    {
        var now = DateTime.UtcNow;
        var elapsed = (now - _lastRefill).TotalSeconds;
        _tokens = Math.Min(_maxTokens, _tokens + elapsed * _refillRate);
        _lastRefill = now;
    }
}

16. DNS Traffic Analytics & Logging

Comprehensive DNS logging and analytics are essential for security monitoring, performance optimization, capacity planning, and compliance. We log every query and response, then analyze the data in real-time for anomalies and trends.

Logging Architecture

Log Schema

SQL
CREATE TABLE dns_query_log ON CLUSTER '{cluster}' (
    query_id UUID DEFAULT generateUUIDv4(),
    timestamp DateTime64(3) DEFAULT now64(3),
    source_ip IPv6,
    query_name String,
    query_type Enum8(
        'A' = 1, 'AAAA' = 2, 'CNAME' = 3, 'MX' = 4,
        'NS' = 5, 'TXT' = 6, 'SRV' = 7, 'CAA' = 8,
        'SOA' = 9, 'PTR' = 10, 'DNSKEY' = 11, 'DS' = 12,
        'ANY' = 255
    ),
    rcode Enum8(
        'NOERROR' = 0, 'FORMERR' = 1, 'SERVFAIL' = 2,
        'NXDOMAIN' = 3, 'NOTIMP' = 4, 'REFUSED' = 5
    ),
    response_code UInt8,
    answer_count UInt16,
    authority_count UInt16,
    additional_count UInt16,
    response_size_bytes UInt32,
    latency_ms Float32,
    cache_hit Enum8('L1' = 1, 'L2' = 2, 'MISS' = 3, 'AUTHORITATIVE' = 4),
    transport Enum8('UDP' = 1, 'TCP' = 2, 'DOH' = 3, 'DOT' = 4),
    dnssec_valid Enum8('VALID' = 1, 'INVALID' = 2, 'INSECURE' = 3, 'NOT_CHECKED' = 4),
    datacenter LowCardinality(String),
    resolver_instance LowCardinality(String)
) ENGINE = MergeTree()
PARTITION BY toYYYYMMDD(timestamp)
ORDER BY (timestamp, source_ip, query_name, query_type)
TTL timestamp + INTERVAL 30 DAY;

Real-Time Analytics Queries

SQL
-- Top queried domains in the last hour
SELECT query_name, count() as query_count
FROM dns_query_log
WHERE timestamp > now() - INTERVAL 1 HOUR
GROUP BY query_name
ORDER BY query_count DESC
LIMIT 20;

-- Queries per second by datacenter
SELECT datacenter, toStartOfMinute(timestamp) as minute,
       count() / 60 as qps
FROM dns_query_log
WHERE timestamp > now() - INTERVAL 1 HOUR
GROUP BY datacenter, minute
ORDER BY minute;

-- Cache hit rate
SELECT
    cache_hit,
    count() as count,
    round(count() * 100.0 / sum(count()) OVER (), 2) as percentage
FROM dns_query_log
WHERE timestamp > now() - INTERVAL 1 HOUR
GROUP BY cache_hit;

-- Potential DDoS detection: top source IPs
SELECT source_ip, count() as queries,
       uniq(query_name) as unique_domains
FROM dns_query_log
WHERE timestamp > now() - INTERVAL 5 MINUTE
GROUP BY source_ip
HAVING queries > 10000
ORDER BY queries DESC;

-- NXDOMAIN ratio (potential random subdomain attack)
SELECT
    rcode,
    count() as count,
    round(count() * 100.0 / sum(count()) OVER (), 2) as percentage
FROM dns_query_log
WHERE timestamp > now() - INTERVAL 1 HOUR
GROUP BY rcode;

17. Domain Registration & Management

Domain registration is the process of reserving a domain name through a registrar accredited by ICANN. As a DNS provider, we offer domain registration services integrated with our DNS platform, allowing customers to register domains and have them automatically configured with our nameservers.

Registration Flow

EPP Integration

C#
public class DomainRegistrationService
{
    private readonly IEppClient _eppClient;
    private readonly IZoneProvisioner _zoneProvisioner;
    private readonly IDnssecKeyManager _keyManager;

    public async Task<RegistrationResult> RegisterDomainAsync(
        DomainRegistrationRequest request, CancellationToken ct)
    {
        // Step 1: Check availability
        var availability = await _eppClient.CheckDomainAsync(request.DomainName, ct);
        if (!availability.Available)
        {
            return RegistrationResult.Failure("Domain is not available");
        }

        // Step 2: Create EPP registration
        var eppResult = await _eppClient.CreateDomainAsync(new DomainCreate
        {
            Name = request.DomainName,
            Registrant = request.RegistrantId,
            Nameservers = await GetDefaultNameserversAsync(ct),
            Period = request.Period,
            AuthInfo = GenerateAuthInfo()
        }, ct);

        if (!eppResult.Success)
        {
            return RegistrationResult.Failure(eppResult.ErrorMessage);
        }

        // Step 3: Auto-provision DNS zone
        var zone = await _zoneProvisioner.CreateZoneAsync(request.DomainName, ct);

        // Step 4: Optionally enable DNSSEC
        if (request.EnableDnssec)
        {
            await _keyManager.GenerateAndPublishKeysAsync(request.DomainName, ct);
        }

        return RegistrationResult.Success(new RegistrationDetails
        {
            DomainName = request.DomainName,
            ExpiryDate = eppResult.ExpiryDate,
            Nameservers = zone.Nameservers,
            DnssecEnabled = request.EnableDnssec
        });
    }
}

18. Private DNS (Internal Zones & Split-Horizon)

Private DNS extends our system to handle internal domain resolution for enterprises and organizations. Split-horizon DNS (also called DNS views) allows the same domain name to resolve to different IP addresses depending on whether the query originates from an internal or external network.

Split-Horizon Architecture

C#
public class SplitHorizonResolver
{
    private readonly Dictionary<string, ZoneView> _views = new();

    public async Task<IReadOnlyList<DnsRecord>> ResolveAsync(
        string zoneName, string queryName, DnsRecordType type,
        IPAddress clientIp, CancellationToken ct)
    {
        var zone = await GetZoneAsync(zoneName, ct);
        if (zone == null || !zone.HasSplitHorizon)
        {
            return await zone?.GetRecordsAsync(queryName, type, ct)
                ?? Array.Empty<DnsRecord>();
        }

        // Determine which view to use based on client IP
        var matchingViews = zone.Views
            .Where(v => v.Networks.Any(n => n.Contains(clientIp)))
            .OrderByDescending(v => v.Priority)
            .ToList();

        var view = matchingViews.FirstOrDefault()?.Name ?? zone.DefaultView;
        var zoneData = zone.GetView(view);

        return await zoneData.GetRecordsAsync(queryName, type, ct);
    }
}

public class PrivateZoneConfig
{
    public string ZoneName { get; set; }
    public List<NetworkRange> InternalNetworks { get; set; }
    public List<NetworkRange> ExternalNetworks { get; set; }
    public Dictionary<string, List<DnsRecord>> InternalRecords { get; set; }
    public Dictionary<string, List<DnsRecord>> ExternalRecords { get; set; }
    public bool AllowTransfersFrom { get; set; }
    public List<string> TransferSources { get; set; }
}

// Example configuration for a corporate domain
// Internal: api.example.com -> 10.0.1.50 (private VPC)
// External: api.example.com -> 203.0.113.10 (public LB)

19. Monitoring & Observability

A DNS system without comprehensive monitoring is flying blind. We monitor at every layer: network (BGP routes, anycast reachability), transport (connection counts, retransmissions), application (query rates, cache hit rates, latency percentiles), and business (domain count, zone updates per hour).

Key Metrics Dashboard

Prometheus Integration

C#
public class DnsMetricsCollector
{
    private readonly Counter _queryCounter;
    private readonly Histogram _queryLatency;
    private readonly Gauge _cacheHitRate;
    private readonly Gauge _activeConnections;
    private readonly Counter _dnssecFailures;

    public DnsMetricsCollector()
    {
        _queryCounter = Metrics.CreateCounter("dns_queries_total",
            "Total DNS queries processed",
            new[] { "type", "rcode", "transport", "datacenter" });

        _queryLatency = Metrics.CreateHistogram("dns_query_latency_seconds",
            "DNS query latency in seconds",
            new[] { "cache_level" },
            new double[] { 0.001, 0.002, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0 });

        _cacheHitRate = Metrics.CreateGauge("dns_cache_hit_rate",
            "Cache hit rate percentage",
            new[] { "cache_level" });

        _activeConnections = Metrics.CreateGauge("dns_active_connections",
            "Number of active DNS connections");

        _dnssecFailures = Metrics.CreateCounter("dnssec_validation_failures_total",
            "DNSSEC validation failures",
            new[] { "reason" });
    }

    public void RecordQuery(DnsQuery query, DnsResponse response,
        CacheLevel cacheLevel, TimeSpan latency, string datacenter)
    {
        _queryCounter.WithLabels(
            query.Type.ToString(),
            response.RCode.ToString(),
            response.Transport.ToString(),
            datacenter
        ).Inc();

        _queryLatency.WithLabels(cacheLevel.ToString())
            .Observe(latency.TotalSeconds);
    }
}

Alerting Rules

YAML
groups:
  - name: dns_alerts
    rules:
      - alert: HighServFailRate
        expr: rate(dns_response_rcode_total{rcode="SERVFAIL"}[5m]) / rate(dns_queries_total[5m]) > 0.01
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: "High SERVFAIL rate detected (>1%)"

      - alert: LowCacheHitRate
        expr: dns_cache_hit_rate{cache_level="L1"} < 50
        for: 10m
        labels:
          severity: warning
        annotations:
          summary: "L1 cache hit rate dropped below 50%"

      - alert: HighQueryLatency
        expr: histogram_quantile(0.99, rate(dns_query_latency_seconds_bucket[5m])) > 0.01
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "P99 query latency exceeds 10ms"

      - alert: UpstreamServerDown
        expr: dns_upstream_failure_rate > 0.1
        for: 1m
        labels:
          severity: critical
        annotations:
          summary: "Upstream DNS server failure rate > 10%"

20. Security (Cache Poisoning, Kaminsky Attack, DNS Rebinding)

DNS security is a multi-faceted challenge. The protocol was designed in an era of network trust, and decades of extensions have introduced numerous attack vectors. Our system must defend against well-known attacks while maintaining backward compatibility.

Attack Vectors & Defenses

1. DNS Cache Poisoning (Kaminsky Attack)

Dan Kaminsky's 2008 attack exploited the predictability of DNS transaction IDs and source ports. An attacker floods a resolver with forged DNS responses, trying to guess the correct transaction ID to inject a fake record into the cache. If successful, the attacker can redirect all traffic for a domain to a malicious server.

2. DNS Amplification / Reflection Attacks

Attackers send small queries with a spoofed source IP (the victim's IP) to open resolvers. The resolvers send large responses to the victim, amplifying the attack traffic. DNSSEC-signed responses can amplify traffic by 40-70x.

C#
public class AmplificationPrevention
{
    public DnsResponse ApplyAntiAmplification(DnsQuery query, DnsResponse response, IPAddress clientIp)
    {
        // Truncate response if it exceeds threshold
        if (response.EstimatedSize > _config.MaxResponseSize)
        {
            response = response.Truncate();
            response.Flags.Truncated = true; // Set TC bit to signal client to use TCP
        }

        // Disable EDNS0 amplification for suspicious sources
        if (_rateLimiter.IsSuspicious(clientIp))
        {
            response.EDNS0 = null;
        }

        // Refuse ANY queries (RFC 8482) to prevent amplification
        if (query.Type == DnsRecordType.ANY)
        {
            return RefuseAnyQuery(query);
        }

        return response;
    }
}

3. DNS Rebinding Attacks

DNS rebinding attacks exploit the fact that DNS records have TTLs. An attacker registers a domain that resolves to a legitimate IP, waits for the victim to access it, then changes the DNS record to point to an internal IP. The victim's browser, thinking it's still talking to the legitimate server, sends requests to the internal network.

Defenses:

• EPSS (DNS Error Possible Spoofing): We validate that DNS responses contain only public IP addresses for external queries.
• Short TTL enforcement: For newly registered domains, we enforce minimum TTLs to limit the window of rebinding attacks.
• DoH/DoT: Encrypted DNS prevents network-level interception and manipulation of DNS responses.

21. Compliance & Privacy

As a DNS provider handling billions of queries daily, we bear significant responsibility for user privacy and must comply with regulations across multiple jurisdictions. DNS data is particularly sensitive because it reveals every domain a user visits.

Privacy Considerations

• Query Logging Policy: We log query metadata (source IP, query name, response, timestamp) for security and debugging, but aggregate and anonymize data after 7 days. Full query logs are retained for 30 days, then purged.
• No Personal Data Sales: We never sell or share individual query data with third parties.
• Data Encryption: All query data is encrypted at rest (AES-256) and in transit (TLS 1.3).
• Access Controls: Query logs are accessible only to the security team and require MFA + justification for access.

Regulatory Compliance

DNS Privacy Extensions

• Padding (RFC 7830): We pad DNS-over-TLS messages to fixed sizes (512 bytes) to prevent traffic analysis that could identify query types by message size.
• Oblivious DNS (ODoH): We support ODoH (RFC 9230) where a proxy separates the client's IP from the query content, providing stronger privacy guarantees than DoH alone.
• QNAME Minimization (RFC 7816): Our recursive resolver sends only the minimum amount of information to each nameserver in the resolution chain, reducing information leakage.

22. Cost Estimation

Building and operating a global DNS infrastructure requires significant investment. Below is a realistic cost estimate for a system handling 10 billion queries per day across 20 data centers.

Infrastructure Costs (Monthly)

23. API Design

The zone management API allows customers to create zones, manage DNS records, configure DNSSEC, set up health checks, and view analytics. We follow RESTful conventions with JSON payloads and OAuth 2.0 authentication.

Core API Endpoints

API Implementation

C#
[ApiController]
[Route("api/v1/zones")]
[Authorize]
public class ZonesController : ControllerBase
{
    private readonly IZoneService _zoneService;
    private readonly IAuditLogger _auditLogger;

    [HttpPost]
    [Authorize(Policy = "ZoneCreate")]
    public async Task<ActionResult<ZoneResponse>> CreateZone(
        [FromBody] CreateZoneRequest request, CancellationToken ct)
    {
        var accountId = GetAuthenticatedAccountId();

        var zone = await _zoneService.CreateZoneAsync(new CreateZoneCommand
        {
            Name = request.Name,
            AccountId = accountId,
            DefaultTtl = request.DefaultTtl ?? 3600,
            EnableDnssec = request.EnableDnssec ?? false,
            Nameservers = request.Nameservers
        }, ct);

        await _auditLogger.LogAsync(accountId, "zone.create",
            new { zone.Id, zone.Name }, ct);

        return Ok(ZoneResponse.FromDomain(zone));
    }

    [HttpPost("{zoneId}/records")]
    [Authorize(Policy = "RecordModify")]
    public async Task<ActionResult<RecordResponse>> CreateRecord(
        Guid zoneId, [FromBody] CreateRecordRequest request, CancellationToken ct)
    {
        var accountId = GetAuthenticatedAccountId();

        var record = await _zoneService.CreateRecordAsync(new CreateRecordCommand
        {
            ZoneId = zoneId,
            AccountId = accountId,
            Name = request.Name,
            Type = request.Type,
            Ttl = request.Ttl ?? 3600,
            Data = request.Data,
            Priority = request.Priority,
            Weight = request.Weight
        }, ct);

        await _auditLogger.LogAsync(accountId, "record.create",
            new { zoneId, record.Id, record.Type, record.Name }, ct);

        return Ok(RecordResponse.FromDomain(record));
    }
}

24. Testing Strategy

Testing a DNS system requires a multi-layered approach. Unit tests validate individual components, integration tests verify the full resolution pipeline, load tests measure performance under stress, and chaos tests ensure resilience against failures.

Test Categories

DNS-Specific Test Examples

C#
public class RecursiveResolverTests
{
    [Fact]
    public async Task ResolveAsync_CacheHit_ReturnsWithSubMillisecondLatency()
    {
        // Arrange
        var cache = new L1Cache(new CacheConfig { MaxEntries = 10_000 });
        var query = new DnsQuery("www.example.com", DnsRecordType.A);
        var cachedResponse = CreateCachedResponse(query);
        await cache.SetAsync(CacheKey.From(query), cachedResponse, CancellationToken.None);

        var resolver = CreateResolver(l1Cache: cache);

        // Act
        var stopwatch = Stopwatch.StartNew();
        var response = await resolver.ResolveAsync(query, CancellationToken.None);
        stopwatch.Stop();

        // Assert
        Assert.Equal(RCode.NoError, response.RCode);
        Assert.True(stopwatch.ElapsedMilliseconds < 1,
            $"Cache hit took {stopwatch.ElapsedMilliseconds}ms, expected < 1ms");
    }

    [Theory]
    [InlineData(DnsRecordType.A)]
    [InlineData(DnsRecordType.AAAA)]
    [InlineData(DnsRecordType.MX)]
    [InlineData(DnsRecordType.TXT)]
    [InlineData(DnsRecordType.SRV)]
    [InlineData(DnsRecordType.CNAME)]
    [InlineData(DnsRecordType.NS)]
    public async Task ResolveAsync_AllRecordTypes_ReturnsCorrectData(DnsRecordType type)
    {
        var resolver = CreateResolver();
        var query = new DnsQuery("example.com", type);
        var response = await resolver.ResolveAsync(query, CancellationToken.None);

        Assert.Equal(RCode.NoError, response.RCode);
        Assert.NotEmpty(response.AnswerRecords);
        Assert.All(response.AnswerRecords, r =>
            Assert.Equal(type, r.Type));
    }

    [Fact]
    public async Task ResolveAsync_NonExistentDomain_ReturnsNXDOMAIN()
    {
        var resolver = CreateResolver();
        var query = new DnsQuery("this-domain-does-not-exist-xyz123.com", DnsRecordType.A);
        var response = await resolver.ResolveAsync(query, CancellationToken.None);

        Assert.Equal(RCode.NXDomain, response.RCode);
        Assert.Empty(response.AnswerRecords);
    }

    [Fact]
    public async Task ResolveAsync_UpstreamTimeout_RetriesWithNextServer()
    {
        var mockUpstream = new Mock<IUpstream>();
        mockUpstream.Setup(u => u.QueryAsync(It.IsAny<DnsQuery>(), It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>()))
            .ThrowsAsync(new TimeoutException());

        var fallbackUpstream = new Mock<IUpstream>();
        fallbackUpstream.Setup(u => u.QueryAsync(It.IsAny<DnsQuery>(), It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>()))
            .ReturnsAsync(CreateSuccessResponse());

        var resolver = CreateResolver(upstreams: new[] { mockUpstream.Object, fallbackUpstream.Object });
        var query = new DnsQuery("example.com", DnsRecordType.A);

        var response = await resolver.ResolveAsync(query, CancellationToken.None);

        Assert.Equal(RCode.NoError, response.RCode);
        fallbackUpstream.Verify(u => u.QueryAsync(It.IsAny<DnsQuery>(), It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>()), Times.Once);
    }

    [Fact]
    public async Task DnssecValidator_ExpiredSignature_ReturnsBogus()
    {
        var validator = new DnssecValidator(new TrustAnchorStore());
        var response = CreateDNSSECResponseWithExpiredSignature();

        var result = await validator.ValidateAsync(response, CancellationToken.None);

        Assert.Equal(ValidationStatus.Bogus, result.Status);
        Assert.Contains("expired", result.ErrorMessage, StringComparison.OrdinalIgnoreCase);
    }
}

Load Testing Configuration

YAML
# dnsperf configuration for load testing
server 127.0.0.1
port 53
mode udp
timeout 5
clients 100

# Query file with realistic distribution
queryfile ./queries/realistic-queries.txt

# Test parameters
limit 10000000
stats 100000

# Expected results
# QPS target: 10,000,000
# Latency target: p99 < 5ms (cache hit), p99 < 200ms (cache miss)
# Error rate target: < 0.01%

25. Interview Q&A

Below are the most common system design interview questions about DNS resolution systems, along with structured answers demonstrating senior+ level thinking.

Q1: Why does DNS use UDP instead of TCP?

Q2: How would you handle a 1 Tbps DDoS attack on your DNS infrastructure?

Q3: What happens when a DNS record's TTL expires and the new IP is unreachable?

Q4: How do you ensure consistency of zone data across globally distributed nameservers?

Q5: How does your system handle DNSSEC key rotation without causing downtime?

Q6: Compare your DNS caching strategy with a CDN's caching strategy.

Q7: Why can't we just use a single Redis instance as the DNS cache instead of a multi-level hierarchy?

Q8: How do you handle split-brain during a network partition between data centers?

Q9: How would you design the DNS system for a new continent (e.g., expanding to Africa)?

Q10: What metrics would you track to detect a DNS cache poisoning attack in real-time?

Q11: How do you handle the tradeoff between DNSSEC security and performance?