1. Introduction

Containers have fundamentally transformed the way we build, ship, and run software. What began as a lightweight alternative to virtual machines has evolved into the backbone of modern cloud-native infrastructure. Every major cloud provider — Amazon Web Services, Microsoft Azure, Google Cloud Platform — offers managed container services, and tools like Kubernetes, Docker Swarm, and Nomad have become essential orchestration platforms for teams operating at scale. The container ecosystem is not a single technology; it is a complex stack of interdependent components spanning image formats, container runtimes, networking primitives, storage drivers, security scanners, and orchestration engines.

Understanding how to design a Docker-style container platform from the ground up is one of the most valuable exercises a senior or staff engineer can undertake. It forces you to confront real challenges in process isolation, filesystem management, network namespace handling, image layering, and distributed coordination. This guide walks through every major subsystem of a container platform, providing architectural blueprints, C# code samples, data models, API specifications, performance benchmarks, and cost analyses. Whether you are building a proprietary platform for your organization, contributing to an open-source project, or preparing for a senior-level system design interview, this guide gives you the depth and breadth you need.

We will examine the Linux kernel primitives that make containers possible — namespaces, cgroups, and UnionFS — before building up through the Docker daemon architecture, image building pipelines, registry implementations, networking models, volume management, multi-host orchestration, security scanning, resource governance, logging pipelines, multi-stage builds, and container-as-a-service abstractions. By the end of this guide, you will have a complete mental model of every component in a production container platform and the engineering trade-offs involved at each layer.

2. The Container Revolution

Before containers became mainstream, deployment workflows relied on virtual machines. A typical VM-based deployment involved provisioning an entire operating system image, installing dependencies, configuring networking, and deploying the application binary. This process was slow, resource-intensive, and prone to environment drift. Virtual machines provide strong isolation at the hardware level through hypervisors, but this isolation comes at the cost of significant overhead — each VM runs a full guest operating system, consumes dedicated memory, and takes minutes to boot.

Containers changed this equation dramatically. A container shares the host operating system kernel, which means there is no guest OS overhead. Container startup times are measured in milliseconds rather than minutes. Container images are typically measured in megabytes rather than gigabytes. The density of workloads on a single host increases by an order of magnitude. A server that could run 10-15 VMs might comfortably run 100 or more containers, depending on workload characteristics. This density improvement translates directly into cost savings and operational efficiency.

The Linux kernel primitives that enable containers are threefold: namespaces provide isolated views of system resources (PID, network, mount, UTS, IPC, user), cgroups enforce resource limits (CPU, memory, I/O, network bandwidth), and UnionFS (such as OverlayFS) enables efficient layered filesystems where image layers are stacked and only differences are stored. Together, these primitives allow a container to appear as an independent system to its processes while actually sharing the host kernel. Docker was the first platform to package these primitives into a developer-friendly toolchain, but the underlying technologies predate Docker by years. LXC, OpenVZ, and Solaris Zones all used similar concepts before Docker popularized the container abstraction.

The cultural shift was equally important. Docker introduced the concept of the Dockerfile — a declarative, reproducible build specification. This single innovation made environments shareable and version-controlled. The Docker Hub registry created an ecosystem of pre-built images. The combination of reproducible builds and a centralized registry created the foundation for the entire cloud-native ecosystem, including Kubernetes, Helm, Istio, and the broader CNCF landscape.

3. Docker Architecture Deep Dive

Docker follows a client-server architecture with three primary components: the Docker CLI (client), the Docker daemon (dockerd), and the container runtime (containerd and runc). When a user executes docker run, the CLI constructs a REST API request and sends it to the daemon. The daemon interprets the request, pulls images if necessary, creates container configurations, and delegates container lifecycle management to containerd, which in turn uses runc to create and run containers based on OCI specifications.

The daemon acts as the central orchestrator on a single host. It manages images, containers, networks, volumes, and builds. It exposes a REST API over a Unix socket by default, though TCP and TLS-secured endpoints are also supported. The daemon maintains an in-memory state cache backed by a persistent store on disk, typically at /var/lib/docker. The daemon handles image layer resolution, content-addressable storage, and garbage collection of unused layers and containers.

The separation between dockerd and containerd is architecturally significant. Docker made the decision to split the daemon into a higher-level management component (dockerd) and a lower-level runtime component (containerd) to enable alternative runtimes and reduce the blast radius of daemon restarts. When dockerd crashes or is restarted, running containers continue to operate because containerd manages their lifecycle independently. This design also allows Kubernetes to use containerd directly, bypassing the Docker daemon entirely — which is exactly what happened with the "dockershim" deprecation in Kubernetes 1.24.

The image store uses a content-addressable model. Every image layer is identified by its SHA256 content hash, which guarantees integrity and deduplication. When multiple images share a common base layer, the layer is stored only once on disk. The manifest describes the ordered list of layers and the image configuration, which includes environment variables, entrypoint commands, labels, and build history. This content-addressable approach also enables efficient image pulls: the daemon checks which layers are already present locally and only downloads the missing ones.

4. Requirements & Functional Scope

Before designing the platform, we must define clear functional and non-functional requirements. The platform must support the full container lifecycle: image building, image storage, container creation, container execution, container monitoring, container termination, and container cleanup. It must support both single-host and multi-host deployments. It must provide networking primitives for inter-container communication, storage abstractions for persistent data, and resource governance for fair multi-tenancy.

Functional Requirements

• Build container images from Dockerfiles with multi-stage support
• Push and pull images to and from a distributed registry
• Create, start, stop, restart, and remove containers
• Attach volumes for persistent and shared storage
• Configure bridging, overlay, and host networking modes
• Set CPU, memory, I/O, and network resource limits
• Compose multi-container applications with dependency ordering
• Orchestrate containers across a cluster of hosts
• Scan images for vulnerabilities and enforce admission policies
• Stream container logs and collect runtime metrics

Non-Functional Requirements

• Container startup latency under 500ms (P99)
• Image pull time under 5 seconds for common images (warm cache)
• Support for 10,000+ concurrent containers per host
• High availability with no single point of failure in the cluster
• Audit logging for all administrative operations
• Secret management with encryption at rest and in transit
• API backward compatibility for 2 major versions
• Sub-second health check response time

5. Capacity Estimation & Sizing

Capacity planning for a container platform requires estimating resource consumption across every layer of the stack. At the host level, each container consumes a base overhead for the runtime process, the snapshot filesystem, and the network namespace. A minimal Alpine-based container might consume 2-5 MB of memory beyond the application process. A Java-based container with a 256 MB heap will consume roughly 350-400 MB total when accounting for JVM overhead, metaspace, and native memory.

For a cluster-level capacity estimate, assume a 100-node cluster of 64-core, 256 GB RAM machines. After reserving resources for the host OS, containerd, and Docker daemon, each node has approximately 60 cores and 250 GB available for workloads. With an average container consuming 0.5 CPU cores and 1 GB RAM, each node could theoretically host 120 containers. In practice, a target utilization of 70-80% leaves room for burst workloads and system overhead, yielding approximately 85-100 containers per node. Across 100 nodes, this provides capacity for 8,500 to 10,000 concurrent containers.

For the image registry, estimate storage growth based on the number of unique images, layers, and retention policies. A typical CI/CD pipeline produces 10-50 unique images per day. Each image might contain 5-15 layers averaging 20 MB each, for a total of 100-300 MB per image. With a 90-day retention policy and 50 images per day, storage requirements reach approximately 1.35 TB per 90-day window. With layer deduplication, actual storage is typically 40-60% less, yielding 540-810 GB for active images.

6. Data Model Design

The data model is the backbone of the platform. Every entity — image, container, volume, network, build — must be tracked with full metadata, state transitions, and relationships. The data model must support efficient queries (list all running containers for a given image), state transitions (container goes from created to running to stopped), and garbage collection (remove all containers and images not referenced by any active workload).

public class ContainerEntity
{
    public string Id { get; set; }                    // SHA256 truncated to 12 chars
    public string Name { get; set; }                  // User-friendly name
    public string ImageId { get; set; }               // Reference to image manifest
    public string ImageReference { get; set; }        // nginx:1.25-alpine
    public ContainerState State { get; set; }         // Created, Running, Paused, Stopped, Removed
    public ContainerStatus Status { get; set; }       // Running, Exited, Dead, Restarting
    public int ExitCode { get; set; }                 // Process exit code
    public DateTime CreatedAt { get; set; }
    public DateTime? StartedAt { get; set; }
    public DateTime? FinishedAt { get; set; }
    public int RestartCount { get; set; }
    public RestartPolicy RestartPolicy { get; set; }  // No, OnFailure, Always
    public string HostId { get; set; }                // Which node it runs on
    public ContainerConfig Config { get; set; }
    public HostConfig HostConfig { get; set; }
    public NetworkSettings NetworkSettings { get; set; }
    public List<MountPoint> Mounts { get; set; }
    public Dictionary<string, string> Labels { get; set; }
}

public class ImageEntity
{
    public string Id { get; set; }                    // Full SHA256 digest
    public string Repository { get; set; }            // library/nginx
    public List<string> Tags { get; set; }           // ["1.25-alpine", "latest"]
    public List<ImageLayer> Layers { get; set; }     // Ordered bottom to top
    public ImageConfig Config { get; set; }
    public long TotalSizeBytes { get; set; }
    public DateTime CreatedAt { get; set; }
    public DateTime? LastPullAt { get; set; }
    public string Architecture { get; set; }          // amd64, arm64
    public string Os { get; set; }                    // linux, windows
}

public class ImageLayer
{
    public string Digest { get; set; }                // sha256:abc123...
    public long SizeBytes { get; set; }
    public string DiffID { get; set; }                // Uncompressed content hash
    public bool IsCompressed { get; set; }
    public string MediaType { get; set; }             // application/vnd.oci.image.layer.v1.tar+gzip
}

public enum ContainerState
{
    Created,
    Running,
    Paused,
    Stopped,
    Removed
}

public class VolumeEntity
{
    public string Id { get; set; }
    public string Name { get; set; }
    public string Driver { get; set; }                // local, nfs, aws-ebs
    public string Mountpoint { get; set; }            // /var/lib/docker/volumes/mydata/_data
    public Dictionary<string, string> Labels { get; set; }
    public DateTime CreatedAt { get; set; }
    public long SizeBytes { get; set; }
}

public class NetworkEntity
{
    public string Id { get; set; }
    public string Name { get; set; }
    public NetworkDriver Driver { get; set; }         // bridge, overlay, host, macvlan
    public string Subnet { get; set; }
    public string Gateway { get; set; }
    public string IpRange { get; set; }
    public bool IsInternal { get; set; }
    public List<string> ConnectedContainers { get; set; }
    public DateTime CreatedAt { get; set; }
}

public class BuildEntity
{
    public string Id { get; set; }
    public string DockerfilePath { get; set; }
    public string ContextPath { get; set; }
    public string Tag { get; set; }
    public BuildStatus Status { get; set; }           // Queued, Building, Success, Failed
    public List<BuildStep> Steps { get; set; }
    public string ResultImageId { get; set; }
    public DateTime StartedAt { get; set; }
    public DateTime? CompletedAt { get; set; }
    public TimeSpan? Duration { get; set; }
}

The data model uses content-addressable identifiers (SHA256 hashes) for images and layers, which provides built-in integrity verification and deduplication. Containers use truncated SHA256 IDs for brevity while maintaining sufficient uniqueness. State transitions follow a strict lifecycle: Created → Running → Stopped → Removed. The RestartPolicy field controls automatic recovery behavior, which is essential for production workloads. Labels provide a flexible key-value metadata system that enables filtering, grouping, and policy enforcement without schema changes.

7. High-Level Architecture

The platform architecture must balance simplicity with extensibility. A single-host architecture is sufficient for development environments, but production deployments require multi-host orchestration with high availability, fault tolerance, and workload scheduling. The architecture comprises five major subsystems: the API gateway, the control plane, the data plane, the image service, and the observability stack.

The API gateway handles authentication, rate limiting, TLS termination, and request routing. It enforces RBAC policies and integrates with identity providers through OIDC. The control plane manages the desired state of the cluster through an etcd-backed metadata store. The scheduler assigns workloads to nodes based on resource availability, affinity rules, and constraint expressions. The data plane runs on each node and is responsible for actual container lifecycle management. The image service provides a distributed, content-addressable image registry with vulnerability scanning and garbage collection. The observability stack collects metrics, logs, and traces from all components.

This separation of concerns ensures that each subsystem can be scaled, upgraded, and failure-isolated independently. The control plane can be replicated for high availability. Data plane nodes can be added or removed without disrupting the control plane. The registry can be scaled independently of the compute nodes. This architecture is the same fundamental pattern used by Docker Swarm, Kubernetes, and most production container platforms.

8. API Design

The container platform API follows RESTful conventions with JSON request and response bodies. The API is versioned through URL path prefixes (/v1.43/) and supports content negotiation for backward compatibility. All mutating operations are idempotent where possible, and all read operations support pagination, filtering, and field selection. Authentication is handled through bearer tokens, and all requests are audit-logged.

Core API Endpoints

C# API Client Implementation

public class ContainerPlatformClient
{
    private readonly HttpClient _http;
    private readonly string _baseUrl;

    public ContainerPlatformClient(string baseUrl, string apiToken)
    {
        _baseUrl = baseUrl.TrimEnd('/');
        _http = new HttpClient { BaseAddress = new Uri(_baseUrl) };
        _http.DefaultRequestHeaders.Authorization =
            new AuthenticationHeaderValue("Bearer", apiToken);
        _http.DefaultRequestHeaders.Accept.Add(
            new MediaTypeWithQualityHeaderValue("application/json"));
    }

    public async Task<ContainerCreateResponse> CreateContainerAsync(
        ContainerCreateRequest request)
    {
        var json = JsonSerializer.Serialize(request, new JsonSerializerOptions
        {
            PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
            DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull
        });
        var content = new StringContent(json, Encoding.UTF8, "application/json");
        var response = await _http.PostAsync("/v1.43/containers/create", content);
        response.EnsureSuccessStatusCode();
        var body = await response.Content.ReadAsStringAsync();
        return JsonSerializer.Deserialize<ContainerCreateResponse>(body);
    }

    public async Task<ContainerInfo> InspectContainerAsync(string containerId)
    {
        var response = await _http.GetAsync($"/v1.43/containers/{containerId}/json");
        response.EnsureSuccessStatusCode();
        var body = await response.Content.ReadAsStringAsync();
        return JsonSerializer.Deserialize<ContainerInfo>(body);
    }

    public async Task StartContainerAsync(string containerId)
    {
        var response = await _http.PostAsync(
            $"/v1.43/containers/{containerId}/start", null);
        response.EnsureSuccessStatusCode();
    }

    public async Task StopContainerAsync(string containerId, int timeoutSeconds = 30)
    {
        var response = await _http.PostAsync(
            $"/v1.43/containers/{containerId}/stop?t={timeoutSeconds}", null);
        response.EnsureSuccessStatusCode();
    }

    public async Task<IReadOnlyList<ContainerSummary>> ListContainersAsync(
        bool all = false, string? labelFilter = null)
    {
        var query = $"all={all.ToString().ToLower()}";
        if (!string.IsNullOrEmpty(labelFilter))
            query += $"&filters={Uri.EscapeDataString($"{{\"label\":[\"{labelFilter}\"]}}")}";
        var response = await _http.GetAsync($"/v1.43/containers/json?{query}");
        response.EnsureSuccessStatusCode();
        var body = await response.Content.ReadAsStringAsync();
        return JsonSerializer.Deserialize<List<ContainerSummary>>(body)
            ?? Array.Empty<ContainerSummary>();
    }

    public async Task<Stream> PullImageAsync(string image, string tag = "latest")
    {
        var response = await _http.PostAsync(
            $"/v1.43/images/create?fromImage={Uri.EscapeDataString(image)}&tag={tag}",
            null);
        response.EnsureSuccessStatusCode();
        return await response.Content.ReadAsStreamAsync();
    }

    public async Task<string> ExecInContainerAsync(string containerId, string[] command)
    {
        var execConfig = new { AttachStdout = true, AttachStderr = true, Cmd = command };
        var json = JsonSerializer.Serialize(execConfig);
        var content = new StringContent(json, Encoding.UTF8, "application/json");
        var resp = await _http.PostAsync(
            $"/v1.43/containers/{containerId}/exec", content);
        resp.EnsureSuccessStatusCode();
        var execResult = JsonSerializer.Deserialize<ExecCreateResponse>(
            await resp.Content.ReadAsStringAsync());

        var startConfig = new { Detach = false };
        var startJson = JsonSerializer.Serialize(startConfig);
        var startContent = new StringContent(startJson, Encoding.UTF8, "application/json");
        var startResp = await _http.PostAsync(
            $"/v1.43/exec/{execResult.Id}/start", startContent);
        startResp.EnsureSuccessStatusCode();
        return await startResp.Content.ReadAsStringAsync();
    }
}

9. Container Runtime (containerd / runc)

The container runtime is the lowest-level component responsible for actually creating and running container processes. Modern container platforms use a two-tier runtime architecture: containerd serves as the high-level runtime managing the container lifecycle, image distribution, storage, and networking, while runc serves as the low-level OCI-compliant runtime that creates individual container processes using Linux kernel primitives.

When containerd receives a request to create a container, it performs the following steps: it resolves the image, pulls any missing layers, creates a filesystem snapshot by stacking image layers with a writable top layer, generates an OCI runtime bundle (a directory containing a config.json specification and the root filesystem), and then invokes runc with the bundle path. Runc reads the OCI specification, creates the necessary namespaces, sets up cgroups, configures the network, mounts the filesystem, and finally executes the container's entrypoint process using the clone() system call with the appropriate namespace flags.

public class ContainerRuntimeService
{
    private readonly IContentStore _contentStore;
    private readonly ISnapshotService _snapshotService;
    private readonly INamespaceManager _namespaceManager;
    private readonly IProcessMonitor _processMonitor;

    public async Task<ContainerRuntimeResult> CreateAndRunContainerAsync(
        ContainerSpec spec)
    {
        // Step 1: Create namespace for isolation
        var nsId = await _namespaceManager.CreateNamespaceAsync(
            $"container-{spec.ContainerId}");

        // Step 2: Resolve and prepare image layers
        var manifest = await _contentStore.GetManifestAsync(spec.ImageRef);
        var snapshotId = await _snapshotService.PrepareAsync(
            spec.ContainerId, manifest.Layers);

        // Step 3: Generate OCI runtime bundle
        var bundlePath = await CreateOciBundleAsync(spec, snapshotId);

        // Step 4: Configure cgroups for resource limits
        var cgroupPath = await SetupCgroupsAsync(
            spec.ContainerId, spec.Resources);

        // Step 5: Configure network namespace with veth pair
        var netConfig = await ConfigureNetworkingAsync(
            spec.ContainerId, spec.NetworkConfig);

        // Step 6: Create process monitor for lifecycle tracking
        var monitor = _processMonitor.CreateTracker(spec.ContainerId);

        // Step 7: Invoke runc to create and start the container process
        var pid = await InvokeRuncAsync(bundlePath, nsId, cgroupPath);

        // Step 8: Track the process
        await monitor.TrackProcessAsync(pid);

        return new ContainerRuntimeResult
        {
            ContainerId = spec.ContainerId,
            Pid = pid,
            CgroupPath = cgroupPath,
            SnapshotId = snapshotId,
            StartedAt = DateTimeOffset.UtcNow
        };
    }

    private async Task<string> CreateOciBundleAsync(
        ContainerSpec spec, string snapshotId)
    {
        var bundlePath = Path.Combine("/run/containerd", spec.ContainerId);
        Directory.CreateDirectory(bundlePath);

        var ociConfig = new OciRuntimeConfig
        {
            Process = new OciProcess
            {
                Terminal = spec.Interactive,
                User = spec.User ?? "root",
                Args = spec.Entrypoint.Concat(spec.Cmd).ToArray(),
                Env = spec.Env?.Select(e => $"{e.Key}={e.Value}").ToArray()
                    ?? Array.Empty<string>(),
                Cwd = spec.WorkingDir ?? "/"
            },
            Root = new OciRoot
            {
                Path = snapshotId,
                Readonly = spec.ReadonlyRootFs
            },
            Linux = new OciLinux
            {
                CgroupsPath = $"/docker/{spec.ContainerId}",
                Namespaces = new[]
                {
                    new OciNamespace { Type = "pid" },
                    new OciNamespace { Type = "network" },
                    new OciNamespace { Type = "ipc" },
                    new OciNamespace { Type = "uts" },
                    new OciNamespace { Type = "mount" }
                },
                Resources = MapResources(spec.Resources)
            }
        };

        var json = JsonSerializer.Serialize(ociConfig,
            new JsonSerializerOptions { PropertyNamingPolicy = null });
        await File.WriteAllTextAsync(
            Path.Combine(bundlePath, "config.json"), json);
        return bundlePath;
    }
}

runc Namespace Configuration

Each namespace type provides a specific isolation boundary. The PID namespace gives the container its own process ID numbering, so the container's PID 1 is mapped to a different host PID. The network namespace provides a separate network stack with its own interfaces, routing tables, and firewall rules. The mount namespace isolates the filesystem view. The UTS namespace isolates hostname and domain name. The IPC namespace isolates inter-process communication resources like semaphores and message queues. The user namespace maps container root to an unprivileged host user, providing defense-in-depth against kernel exploits.

10. Image Building & Dockerfile

Image building is the process of transforming a Dockerfile and a build context into a container image. Each instruction in a Dockerfile produces a new image layer, and the final image is a stack of these layers. Understanding the layer model is critical for optimizing build performance, managing storage, and debugging build failures. The Dockerfile instruction set includes FROM (base image), RUN (execute commands), COPY (copy files), ADD (copy with auto-extraction), WORKDIR (set working directory), ENV (set environment variables), EXPOSE (document ports), CMD (default command), ENTRYPOINT (main command), and ARG (build-time variables).

A well-structured Dockerfile follows several best practices: minimize the number of layers by combining related RUN commands, place frequently changing instructions (like COPY of application code) late in the Dockerfile to maximize layer cache hits, use official base images from Docker Hub to reduce attack surface, use specific tags rather than latest for reproducibility, and add a .dockerignore file to exclude unnecessary files from the build context.

public class DockerfileParser
{
    public Dockerfile ParseDockerfile(string content)
    {
        var instructions = new List<DockerfileInstruction>();
        var lines = content.Split('\n');
        string? continuationLine = null;

        foreach (var rawLine in lines)
        {
            var line = rawLine.Trim();
            if (string.IsNullOrEmpty(line) || line.StartsWith('#'))
                continue;

            if (continuationLine != null)
            {
                continuationLine += line.TrimEnd('\\');
                if (!line.EndsWith('\\'))
                {
                    instructions.Add(ParseInstruction(continuationLine));
                    continuationLine = null;
                }
                continue;
            }

            if (line.EndsWith('\\'))
            {
                continuationLine = line.TrimEnd('\\');
            }
            else
            {
                instructions.Add(ParseInstruction(line));
            }
        }

        return new Dockerfile
        {
            Instructions = instructions,
            BaseImage = instructions.OfType<FromInstruction>().FirstOrDefault()?.Image,
            EstimatedLayers = instructions.Count(i => i.CreatesLayer)
        };
    }

    public DockerfileInstruction ParseInstruction(string line)
    {
        var parts = line.Split(new[] { ' ', '\t' }, 2);
        var keyword = parts[0].ToUpperInvariant();
        var arguments = parts.Length > 1 ? parts[1] : string.Empty;

        return keyword switch
        {
            "FROM" => new FromInstruction { Image = arguments.Split(' ')[0],
                StageName = arguments.Contains("AS ")
                    ? arguments.Split(" AS ")[1].Trim() : null },
            "RUN" => new RunInstruction { Command = arguments, CreatesLayer = true },
            "COPY" => new CopyInstruction { Arguments = arguments, CreatesLayer = true },
            "ADD" => new AddInstruction { Arguments = arguments, CreatesLayer = true },
            "WORKDIR" => new WorkdirInstruction { Path = arguments, CreatesLayer = true },
            "ENV" => new EnvInstruction { Arguments = arguments, CreatesLayer = true },
            "ARG" => new ArgInstruction { Arguments = arguments },
            "EXPOSE" => new ExposeInstruction { Ports = arguments },
            "CMD" => new CmdInstruction { Arguments = arguments },
            "ENTRYPOINT" => new EntrypointInstruction { Arguments = arguments },
            "LABEL" => new LabelInstruction { Arguments = arguments },
            "HEALTHCHECK" => new HealthcheckInstruction { Arguments = arguments },
            _ => new UnknownInstruction { Keyword = keyword, Arguments = arguments }
        };
    }
}

Optimized Dockerfile for a .NET Application

FROM mcr.microsoft.com/dotnet/aspnet:8.0-alpine AS base
WORKDIR /app
EXPOSE 8080

FROM mcr.microsoft.com/dotnet/sdk:8.0-alpine AS build
WORKDIR /src
COPY ["MyApp.csproj", "."]
RUN dotnet restore
COPY . .
RUN dotnet publish -c Release -o /app/publish --no-restore

FROM base AS final
WORKDIR /app
COPY --from=build /app/publish .
ENTRYPOINT ["dotnet", "MyApp.dll"]

11. Layer Caching & Optimization

Docker's layer caching mechanism is one of the most important concepts for build performance. When Docker builds an image, it processes each Dockerfile instruction sequentially. For each instruction, Docker checks if a matching layer already exists in the local cache. If a cache hit is found, Docker reuses the cached layer instead of re-executing the instruction. If a cache miss occurs, all subsequent instructions must also be re-executed because their input context has changed.

The cache invalidation rules are straightforward: COPY and ADD instructions invalidate the cache based on file content checksums. RUN instructions invalidate based on the exact command string. Environment variables and arguments from ARG and ENV affect the cache. The key optimization strategy is to order Dockerfile instructions so that stable, rarely-changing steps come first (base image, system dependencies) and frequently-changing steps come last (application code, configuration).

Layer Size Comparison

C# Build Cache Analyzer

public class BuildCacheAnalyzer
{
    public CacheAnalysisResult AnalyzeCacheEfficiency(Dockerfile dockerfile)
    {
        var result = new CacheAnalysisResult();
        int cumulativeCost = 0;

        foreach (var instruction in dockerfile.Instructions)
        {
            var cost = EstimateBuildCost(instruction);
            var cacheability = AssessCacheability(instruction);

            result.Steps.Add(new CacheStepAnalysis
            {
                Instruction = instruction.ToString(),
                EstimatedCostSeconds = cost,
                Cacheability = cacheability,
                CumulativeCostSeconds = cumulativeCost,
                Recommendation = GenerateRecommendation(instruction, cost)
            });

            cumulativeCost += cost;
        }

        result.TotalEstimatedBuildTimeSeconds = cumulativeCost;
        result.OptimizationPotential = CalculateOptimizationPotential(result.Steps);
        return result;
    }

    private int EstimateBuildCost(DockerfileInstruction instruction)
    {
        return instruction switch
        {
            RunInstruction r when r.Command.Contains("apt-get") => 30,
            RunInstruction r when r.Command.Contains("npm install") => 45,
            RunInstruction r when r.Command.Contains("dotnet") => 60,
            CopyInstruction _ => 2,
            FromInstruction _ => 1,
            _ => 3
        };
    }

    private CacheabilityLevel AssessCacheability(DockerfileInstruction instruction)
    {
        return instruction switch
        {
            FromInstruction _ => CacheabilityLevel.High,
            RunInstruction r when !r.Command.Contains("$") =>
                CacheabilityLevel.High,
            RunInstruction _ => CacheabilityLevel.Medium,
            CopyInstruction _ => CacheabilityLevel.High,
            EnvInstruction _ => CacheabilityLevel.High,
            _ => CacheabilityLevel.Low
        };
    }

    private string GenerateRecommendation(
        DockerfileInstruction instruction, int cost)
    {
        if (cost > 30 && instruction is CopyInstruction)
            return "Consider splitting COPY to separate dependency files from source";
        if (instruction is RunInstruction r &&
            r.Command.Contains("apt-get update") &&
            !r.Command.Contains("apt-get install"))
            return "Combine apt-get update and install in single RUN for cache";
        return "No optimization needed";
    }
}

Docker BuildKit, the modern build engine, provides additional cache strategies beyond the default layer cache. The --cache-from flag allows importing cache from remote registries, enabling CI/CD pipelines to share cache across builds. The --mount=type=cache directive in Dockerfile allows persisting directories (like package manager caches) across builds without baking them into image layers. These advanced caching mechanisms can reduce build times by 50-80% for large projects with extensive dependency trees.

12. Docker Registry (Harbor)

A Docker registry is a stateless, scalable server-side application that stores and distributes container images. Docker Hub is the default public registry, but production deployments require private registries with access control, vulnerability scanning, replication, and audit logging. Harbor is the most widely adopted open-source registry for enterprise deployments, providing these features out of the box.

The registry follows the OCI Distribution Specification, which defines the HTTP API for pushing and pulling images and artifacts. Images are stored as content-addressable blobs, organized by repository and tagged with manifests. The registry uses a pull-through cache for proxied upstream registries and implements garbage collection to remove unreferenced blobs. Storage backends include local filesystem, S3-compatible object storage, Azure Blob Storage, Google Cloud Storage, and OpenStack Swift.

Registry Architecture Components

C# Registry Client for Image Push

public class RegistryClient
{
    private readonly HttpClient _http;
    private readonly string _registryUrl;
    private readonly string _namespace;

    public async Task<PushResult> PushImageAsync(
        string repository, string tag, ImageManifest manifest)
    {
        // Step 1: Get auth token from the registry
        var token = await ObtainAuthTokenAsync(repository, "push");

        // Step 2: Check which layers the registry already has (avoid re-upload)
        var existingLayers = new List<string>();
        foreach (var digest in manifest.Layers.Select(l => l.Digest))
        {
            var exists = await CheckBlobExistsAsync(repository, digest, token);
            if (exists) existingLayers.Add(digest);
        }

        // Step 3: Upload only missing layers
        var missingLayers = manifest.Layers
            .Where(l => !existingLayers.Contains(l.Digest));
        foreach (var layer in missingLayers)
        {
            await UploadBlobAsync(repository, layer, token);
        }

        // Step 4: Upload config blob
        await UploadConfigBlobAsync(repository, manifest.Configuration, token);

        // Step 5: Upload manifest (finalizes the push)
        var manifestJson = JsonSerializer.Serialize(manifest);
        var content = new StringContent(manifestJson, Encoding.UTF8,
            manifest.MediaType);
        var response = await _http.PutAsync(
            $"/v2/{_namespace}/{repository}/manifests/{tag}", content);
        response.EnsureSuccessStatusCode();

        return new PushResult
        {
            Tag = tag,
            ManifestDigest = ComputeSha256(manifestJson),
            LayersUploaded = missingLayers.Count(),
            LayersReused = existingLayers.Count,
            TotalLayers = manifest.Layers.Count
        };
    }

    private async Task<string> ObtainAuthTokenAsync(
        string repository, string action)
    {
        var scope = $"repository:{_namespace}/{repository}:{action}";
        var response = await _http.GetAsync(
            $"/service/token?service=harbor-registry&scope={scope}");
        response.EnsureSuccessStatusCode();
        var json = await response.Content.ReadAsStringAsync();
        var tokenResponse = JsonSerializer.Deserialize<TokenResponse>(json);
        return tokenResponse.Token;
    }
}

13. Networking Model

Docker networking is implemented through libnetwork, which provides a pluggable driver-based networking framework. The default network drivers are bridge, host, overlay, macvlan, and none. The bridge driver creates a private internal network on the host, and containers connected to this bridge can communicate with each other using IP addresses or DNS names. The host driver removes network isolation entirely, sharing the host's network namespace with the container. The overlay driver enables multi-host networking for Swarm or Kubernetes clusters by creating VXLAN tunnels between hosts.

Each container gets its own network namespace with a virtual Ethernet (veth) pair connecting it to the host's bridge. The veth pair acts as a virtual cable: one end is inside the container's namespace (named eth0), and the other end is attached to the bridge on the host. Traffic between containers on the same bridge is switched at Layer 2. Traffic to external networks is routed through the host's IP forwarding and NAT rules (iptables or nftables).

For multi-host networking, the overlay driver encapsulates container traffic in VXLAN packets, routing them between host machines through the underlay network. The overlay network creates a virtual Layer 2 network that spans multiple hosts, allowing containers on different physical machines to communicate as if they were on the same local network. The control plane uses the Serf gossip protocol for membership and failure detection, and the data plane uses UDP encapsulation for packet transport.

C# Network Manager Implementation

public class NetworkManager
{
    private readonly IIptablesManager _iptables;
    private readonly IVethManager _vethManager;
    private readonly IDnsManager _dnsManager;
    private readonly INetworkStore _store;

    public async Task<NetworkAttachment> AttachContainerToNetworkAsync(
        string containerId, string networkId)
    {
        var network = await _store.GetNetworkAsync(networkId);
        var subnet = IPAddress.Parse(network.Subnet);
        var containerIp = await AllocateIpAddressAsync(networkId);
        var vethName = $"veth_{containerId[..8]}";

        // Create veth pair: one end in container namespace, other on bridge
        var veth = await _vethManager.CreateVethPairAsync(
            hostEnd: vethName,
            containerEnd: "eth0",
            containerId: containerId);

        // Attach host end to the bridge
        await AttachToBridgeAsync(vethName, network.BridgeName);

        // Configure IP inside the container namespace
        await ConfigureContainerNetworkAsync(
            containerId: containerId,
            interfaceName: "eth0",
            ipAddress: containerIp,
            subnet: subnet,
            gateway: IPAddress.Parse(network.Gateway));

        // Add DNS resolution entry
        await _dnsManager.RegisterContainerAsync(
            containerId, containerIp, network.Name);

        // Configure iptables for NAT and port mapping
        await ConfigureNtpAndFirewallAsync(containerId, network, containerIp);

        return new NetworkAttachment
        {
            NetworkId = networkId,
            ContainerId = containerId,
            IpAddress = containerIp,
            MacAddress = veth.MacAddress,
            Gateway = IPAddress.Parse(network.Gateway),
            DnsServers = network.DnsServers
        };
    }

    public async Task<PortMapping> PublishPortAsync(
        string containerId, int containerPort,
        int hostPort, Protocol protocol)
    {
        var container = await GetContainerNetworkConfigAsync(containerId);
        var chain = protocol == Protocol.Tcp ? "DOCKER" : "DOCKER_UDP";

        // Add DNAT rule: host:port -> container:port
        await _iptables.AddRuleAsync(new IptablesRule
        {
            Chain = chain,
            Match = $"-p {protocol} --dport {hostPort}",
            Target = "DNAT",
            JumpArgs = $"--to-destination {container.IpAddress}:{containerPort}"
        });

        // Add filter rule to allow forwarded traffic
        await _iptables.AddRuleAsync(new IptablesRule
        {
            Chain = "DOCKER-USER",
            Match = $"-d {container.IpAddress} -p {protocol} --dport {containerPort}",
            Target = "ACCEPT"
        });

        return new PortMapping
        {
            HostPort = hostPort,
            ContainerPort = containerPort,
            Protocol = protocol,
            HostIp = "0.0.0.0"
        };
    }
}

14. Volume & Storage Management

Container filesystems are ephemeral by design — the writable layer is destroyed when a container is removed. Persistent data requires volumes, which are managed storage objects that exist independently of container lifecycles. Docker supports several volume types: named volumes (managed by Docker on the host filesystem), bind mounts (direct mapping of host paths), tmpfs mounts (in-memory filesystems), and plugin-based volumes (NFS, iSCSI, cloud storage).

Named volumes are the recommended approach for most persistent data. They are managed by Docker's volume driver and stored in /var/lib/docker/volumes/ on the host. Bind mounts are useful for development workflows where you want to mount source code from the host into the container. The key difference is that named volumes are managed by Docker and their content can be initialized from an image, while bind mounts expose the host filesystem directly to the container.

public class VolumeManager
{
    private readonly IVolumeDriverRegistry _driverRegistry;
    private readonly IFileSystemChecker _fsChecker;
    private readonly IMetricsCollector _metrics;

    public async Task<Volume> CreateVolumeAsync(VolumeCreateRequest request)
    {
        var driver = _driverRegistry.GetDriver(request.Driver ?? "local");

        // Validate driver capabilities
        var capabilities = await driver.GetCapabilitiesAsync();
        if (request.AccessMode == VolumeAccessMode.ReadOnly &&
            !capabilities.SupportsReadOnly)
            throw new NotSupportedException(
                $"Driver {request.Driver} does not support read-only access");

        // Create the volume through the appropriate driver
        var volume = await driver.CreateAsync(new DriverCreateRequest
        {
            Name = request.Name ?? GenerateVolumeName(),
            Labels = request.Labels,
            Options = request.Options
        });

        // For local driver, ensure the mountpoint directory exists and
        // has correct permissions
        if (request.Driver == "local" || request.Driver == null)
        {
            Directory.CreateDirectory(volume.Mountpoint);
            if (request.Uid != null || request.Gid != null)
            {
                await SetDirectoryOwnershipAsync(
                    volume.Mountpoint, request.Uid ?? 0, request.Gid ?? 0);
            }
            if (request.Permissions != null)
            {
                await SetDirectoryPermissionsAsync(
                    volume.Mountpoint, request.Permissions);
            }
        }

        // If an initialization image is provided, copy its data into the volume
        if (!string.IsNullOrEmpty(request.InitImage))
        {
            await InitializeVolumeFromImageAsync(volume, request.InitImage);
        }

        // Track volume metrics
        await _metrics.RecordVolumeCreatedAsync(volume);

        return volume;
    }

    public async Task<string> MountVolumeAsync(
        string volumeId, string containerId, MountOptions options)
    {
        var volume = await GetVolumeAsync(volumeId);
        var driver = _driverRegistry.GetDriver(volume.Driver);

        // Get the mount specification from the driver
        var mountSpec = await driver.MountAsync(volume, new DriverMountRequest
        {
            ContainerId = containerId,
            ReadOnly = options.ReadOnly,
            PropagationMode = options.Propagation ?? "rprivate"
        });

        // Calculate disk usage before mount for monitoring
        var diskUsage = await _fsChecker.GetDirectorySizeAsync(
            volume.Mountpoint);

        return mountSpec.Source;
    }
}

15. Docker Compose

Docker Compose is a tool for defining and running multi-container applications using a declarative YAML configuration file. A Compose file specifies services (containers), networks (connectivity), volumes (storage), and configs (secrets). The Compose CLI translates this declarative specification into a series of Docker API calls that create and start the defined resources in the correct order, respecting dependency declarations.

Compose version 3 (the modern specification) introduced deep integration with Docker Swarm for multi-host deployments. The deploy section allows specifying replica counts, resource constraints, update policies, restart policies, and placement constraints. This makes Compose files portable between local development (using docker compose up) and production Swarm deployments (using docker stack deploy).

version: "3.9"
services:
  web:
    build: ./web
    ports:
      - "8080:8080"
    environment:
      - DATABASE_URL=postgres://db:5432/myapp
      - REDIS_URL=redis://cache:6379
    depends_on:
      db:
        condition: service_healthy
      cache:
        condition: service_started
    deploy:
      replicas: 3
      resources:
        limits:
          cpus: "1.0"
          memory: 512M
        reservations:
          cpus: "0.25"
          memory: 128M
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
      timeout: 5s
      retries: 3

  db:
    image: postgres:16-alpine
    volumes:
      - db-data:/var/lib/postgresql/data
    environment:
      POSTGRES_DB: myapp
      POSTGRES_PASSWORD_FILE: /run/secrets/db-password
    secrets:
      - db-password
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 5s
      retries: 5

  cache:
    image: redis:7-alpine
    command: redis-server --maxmemory 256mb --maxmemory-policy allkeys-lru
    volumes:
      - cache-data:/data

volumes:
  db-data:
  cache-data:

networks:
  default:
    driver: bridge
    ipam:
      config:
        - subnet: 172.28.0.0/16

secrets:
  db-password:
    file: ./secrets/db_password.txt

C# Compose File Parser

public class ComposeFileParser
{
    public ComposeDefinition Parse(string yamlContent)
    {
        var deserializer = new DeserializerBuilder()
            .WithNamingConvention(UnderscoredNamingConvention.Instance)
            .Build();
        var root = deserializer.Deserialize<Dictionary<object, object>>(yamlContent);

        var definition = new ComposeDefinition();

        if (root.ContainsKey("services"))
        {
            var services = (Dictionary<object, object>)root["services"];
            foreach (var kvp in services)
            {
                var serviceName = kvp.Key.ToString()!;
                var serviceConfig = (Dictionary<object, object>)kvp.Value;
                definition.Services.Add(ParseService(serviceName, serviceConfig));
            }
        }

        if (root.ContainsKey("volumes"))
        {
            var volumes = (Dictionary<object, object>)root["volumes"];
            foreach (var kvp in volumes)
            {
                definition.Volumes.Add(new VolumeDefinition
                {
                    Name = kvp.Key.ToString()!,
                    Driver = kvp.Value?.ToString() ?? "local"
                });
            }
        }

        if (root.ContainsKey("networks"))
        {
            var networks = (Dictionary<object, object>)root["networks"];
            foreach (var kvp in networks)
            {
                definition.Networks.Add(ParseNetwork(
                    kvp.Key.ToString()!, kvp.Value));
            }
        }

        return definition;
    }

    private ServiceDefinition ParseService(
        string name, Dictionary<object, object> config)
    {
        var service = new ServiceDefinition { Name = name };

        if (config.ContainsKey("image"))
            service.Image = config["image"]?.ToString();
        if (config.ContainsKey("build"))
            service.Build = config["build"]?.ToString();
        if (config.ContainsKey("command"))
            service.Command = ParseStringOrList(config["command"]);
        if (config.ContainsKey("environment"))
            service.Environment = ParseEnvironment(config["environment"]);

        if (config.ContainsKey("depends_on"))
        {
            var deps = config["depends_on"];
            if (deps is Dictionary<object, object> depMap)
            {
                service.DependsOn = depMap.Select(d =>
                    new ServiceDependency
                    {
                        Service = d.Key.ToString()!,
                        Condition = d.Value?.ToString() ?? "service_started"
                    }).ToList();
            }
        }

        if (config.ContainsKey("deploy"))
        {
            service.Deploy = ParseDeploy(
                (Dictionary<object, object>)config["deploy"]);
        }

        return service;
    }
}

16. Docker Swarm

Docker Swarm is Docker's native orchestration solution for clustering multiple Docker hosts into a single virtual host. Swarm mode is built into the Docker daemon and requires no additional software. A Swarm cluster consists of manager nodes (which maintain cluster state through the Raft consensus algorithm) and worker nodes (which run container workloads). The manager nodes use the Raft protocol to replicate state across a quorum, ensuring high availability and consistency even if individual managers fail.

Swarm provides service-level abstractions beyond individual containers. A service definition specifies the desired number of replicas, update strategy, rollback configuration, resource constraints, and placement constraints. The Swarm scheduler assigns tasks (container instances) to nodes based on resource availability, node labels, and constraint expressions. If a node fails, Swarm automatically reschedules its tasks to healthy nodes, maintaining the desired replica count.

public class SwarmOrchestrator
{
    private readonly IRaftConsensus _raft;
    private readonly INodeManager _nodeManager;
    private readonly ITaskScheduler _scheduler;

    public async Task<ServiceDeploymentResult> DeployServiceAsync(
        ServiceSpec spec)
    {
        // Validate the service specification
        ValidateServiceSpec(spec);

        // Assign the service a unique ID and commit to Raft log
        var serviceId = GenerateServiceId();
        var serviceRecord = new ServiceRecord
        {
            Id = serviceId,
            Name = spec.Name,
            Spec = spec,
            Version = 1,
            State = ServiceState.Pending,
            CreatedAt = DateTimeOffset.UtcNow
        };

        await _raft.AppendEntryAsync(new RaftEntry
        {
            Type = EntryType.ServiceCreate,
            Service = serviceRecord
        });

        // Wait for Raft quorum confirmation
        await _raft.WaitForQuorumAsync();

        // Generate tasks based on replica count
        var tasks = new List<TaskAssignment>();
        for (int i = 0; i < spec.Replicas; i++)
        {
            var task = new TaskAssignment
            {
                Id = GenerateTaskId(serviceId, i),
                ServiceId = serviceId,
                Slot = i,
                DesiredState = TaskDesiredState.Running,
                Spec = MapToTaskSpec(spec)
            };
            tasks.Add(task);
        }

        // Schedule tasks to available nodes
        var scheduleResult = await _scheduler.ScheduleTasksAsync(
            tasks, spec.Placement);

        // Dispatch tasks to node agents
        foreach (var assignment in scheduleResult.Assignments)
        {
            await DispatchToNodeAsync(assignment.NodeId, assignment.Task);
        }

        return new ServiceDeploymentResult
        {
            ServiceId = serviceId,
            TasksCreated = tasks.Count,
            NodesAssigned = scheduleResult.Assignments
                .Select(a => a.NodeId).Distinct().Count()
        };
    }

    public async Task<ServiceUpdateResult> UpdateServiceAsync(
        string serviceId, ServiceUpdate update)
    {
        var current = await _raft.GetServiceAsync(serviceId);

        // Apply rolling update strategy
        var strategy = update.UpdateConfig ?? new UpdateConfig
        {
            Parallelism = 1,
            Delay = TimeSpan.FromSeconds(10),
            FailureAction = UpdateFailureAction.Rollback,
            Order = UpdateOrder.StopFirst
        };

        var updatePlan = await CreateUpdatePlanAsync(
            current, update, strategy);

        foreach (var batch in updatePlan.Batches)
        {
            // Stop old tasks
            foreach (var oldTask in batch.ToStop)
            {
                await StopTaskAsync(oldTask.Id, strategy.Delay);
            }

            // Start new tasks
            foreach (var newTask in batch.ToStart)
            {
                var node = await _scheduler.SelectNodeAsync(
                    current.Spec.Placement);
                await DispatchToNodeAsync(node.Id, newTask);
            }

            // Wait for health check confirmation
            await WaitForHealthyAsync(batch.ToStart, TimeSpan.FromSeconds(30));
        }

        return new ServiceUpdateResult
        {
            ServiceId = serviceId,
            NewVersion = current.Version + 1,
            TasksUpdated = updatePlan.TotalTasks
        };
    }
}

17. Security Scanning (Trivy)

Container image security is a critical concern. Images frequently contain known vulnerabilities in base images, system packages, and application dependencies. Trivy is the most widely adopted open-source vulnerability scanner for container images. It scans the image filesystem and cross-references installed packages against vulnerability databases from NVD (National Vulnerability Database), OS-specific advisories (Debian, Alpine, Red Hat, Ubuntu), and language-specific advisory databases (npm, NuGet, pip, Maven).

A production-grade security scanning pipeline integrates with the image registry to automatically scan every image on push, enforce admission policies that prevent deployment of images with critical vulnerabilities, and provide dashboards for tracking vulnerability trends over time. The scanner must handle multi-architecture images, embedded signatures, and SBOM (Software Bill of Materials) generation for compliance reporting.

Vulnerability Severity Distribution (Typical .NET Image)

public class SecurityScannerService
{
    private readonly ITrivyClient _trivyClient;
    private readonly IRegistryClient _registry;
    private readonly IAdmissionController _admission;
    private readonly IVulnerabilityStore _vulnStore;

    public async Task<ScanResult> ScanImageAsync(
        string imageRef, ScanPolicy policy)
    {
        // Pull image metadata and layers
        var manifest = await _registry.GetManifestAsync(imageRef);

        // Run Trivy scan
        var trivyResult = await _trivyClient.ScanAsync(new TrivyScanRequest
        {
            Target = imageRef,
            ScanType = ScanType.Image,
            Severity = new[] { Severity.Critical, Severity.High, Severity.Medium },
            IgnoreUnfixed = policy.IgnoreUnfixed,
            Timeout = TimeSpan.FromMinutes(5)
        });

        // Parse and classify findings
        var findings = trivyResult.Results
            .SelectMany(r => r.Vulnerabilities ?? Array.Empty<Vulnerability>())
            .Select(v => new SecurityFinding
            {
                VulnerabilityId = v.VulnerabilityID,
                PkgName = v.PkgName,
                InstalledVersion = v.InstalledVersion,
                FixedVersion = v.FixedVersion,
                Severity = Enum.Parse<FindingSeverity>(v.Severity),
                Title = v.Title,
                Description = v.Description,
                References = v.References,
                CvssScore = v.CvssScore
            })
            .ToList();

        // Check against admission policy
        var violations = new List<PolicyViolation>();
        if (policy.MaxCritical > 0)
        {
            var criticalCount = findings.Count(f =>
                f.Severity == FindingSeverity.Critical);
            if (criticalCount > policy.MaxCritical)
                violations.Add(new PolicyViolation
                {
                    Rule = "MAX_CRITICAL",
                    Actual = criticalCount,
                    Threshold = policy.MaxCritical
                });
        }

        var hasBlockingViolations = violations.Any(v => v.Severity == PolicySeverity.Block);
        await _admission.SetImagePolicyAsync(imageRef, !hasBlockingViolations);

        // Store results for historical tracking
        await _vulnStore.SaveScanResultAsync(new StoredScanResult
        {
            ImageReference = imageRef,
            ScanTime = DateTimeOffset.UtcNow,
            TotalFindings = findings.Count,
            CriticalCount = findings.Count(f => f.Severity == FindingSeverity.Critical),
            HighCount = findings.Count(f => f.Severity == FindingSeverity.High),
            PassedPolicy = !hasBlockingViolations,
            Findings = findings
        });

        return new ScanResult
        {
            ImageReference = imageRef,
            Findings = findings,
            Violations = violations,
            PassesPolicy = !hasBlockingViolations,
            ScanDuration = trivyResult.Duration
        };
    }
}

18. Resource Limits & cgroups

Control Groups (cgroups) are a Linux kernel feature that limits, accounts for, and isolates the resource usage of process groups. Docker uses cgroups to enforce resource constraints on containers, ensuring that a single container cannot monopolize host resources. Cgroups version 2 (unified hierarchy) provides more consistent resource control than cgroups v1, with better support for memory protection, I/O weight management, and CPU scheduling.

The key resource limits that Docker exposes are CPU shares (relative weight for CPU time), CPU quota (hard limit on CPU time per period), CPU pinning (affinity to specific CPU cores), memory limit (hard cap on RAM usage), memory swap limit (total memory + swap), memory reservation (soft guarantee), memory kernel limit (kernel memory allocation), I/O weight (relative I/O priority), and I/O bandwidth limits (bytes per second for read/write).

cgroups v2 Resource Limits

public class CgroupManager
{
    private readonly string _cgroupRoot = "/sys/fs/cgroup";

    public async Task ConfigureContainerResourcesAsync(
        string containerId, ContainerResources resources)
    {
        var cgroupPath = Path.Combine(_cgroupRoot, $"docker/{containerId}");
        Directory.CreateDirectory(cgroupPath);

        // Set CPU limits
        if (resources.CpuLimit.HasValue)
        {
            // cpu.max: "quota period" format, e.g., "200000 100000" = 2 cores
            var periodUs = 100000; // 100ms period
            var quotaUs = (long)(resources.CpuLimit.Value * periodUs);
            await File.WriteAllTextAsync(
                Path.Combine(cgroupPath, "cpu.max"),
                $"{quotaUs} {periodUs}");
        }

        // Set CPU weight (shares equivalent in cgroups v2)
        if (resources.CpuShares.HasValue)
        {
            // cpu.weight range: 1-10000, default 100
            var weight = MapSharesToWeight(resources.CpuShares.Value);
            await File.WriteAllTextAsync(
                Path.Combine(cgroupPath, "cpu.weight"),
                weight.ToString());
        }

        // Set CPU affinity
        if (resources.CpuSetCpus != null)
        {
            await File.WriteAllTextAsync(
                Path.Combine(cgroupPath, "cpuset.cpus"),
                resources.CpuSetCpus);
        }

        // Set memory limit
        if (resources.MemoryLimit.HasValue)
        {
            var bytes = resources.MemoryLimit.Value.ToBytes();
            await File.WriteAllTextAsync(
                Path.Combine(cgroupPath, "memory.max"),
                bytes.ToString());
        }

        // Set memory low (reservation/soft limit)
        if (resources.MemoryReservation.HasValue)
        {
            var bytes = resources.MemoryReservation.Value.ToBytes();
            await File.WriteAllTextAsync(
                Path.Combine(cgroupPath, "memory.low"),
                bytes.ToString());
        }

        // Set memory swap limit
        if (resources.MemorySwapLimit.HasValue)
        {
            var memBytes = resources.MemoryLimit?.ToBytes() ?? 0;
            var swapBytes = resources.MemorySwapLimit.Value.ToBytes() - memBytes;
            await File.WriteAllTextAsync(
                Path.Combine(cgroupPath, "memory.swap.max"),
                Math.Max(0, swapBytes).ToString());
        }

        // Set PID limit
        if (resources.PidsLimit.HasValue)
        {
            await File.WriteAllTextAsync(
                Path.Combine(cgroupPath, "pids.max"),
                resources.PidsLimit.Value.ToString());
        }

        // Configure I/O limits
        if (resources.IoReadBps.HasValue || resources.IoWriteBps.HasValue)
        {
            var deviceId = await FindBlkDeviceAsync();
            if (resources.IoReadBps.HasValue)
            {
                var bytesPerSecond = resources.IoReadBps.Value.ToBytes();
                await File.WriteAllTextAsync(
                    Path.Combine(cgroupPath, $"io.max"),
                    $"rbps={deviceId}:{bytesPerSecond}");
            }
            if (resources.IoWriteBps.HasValue)
            {
                var bytesPerSecond = resources.IoWriteBps.Value.ToBytes();
                await File.WriteAllTextAsync(
                    Path.Combine(cgroupPath, $"io.max"),
                    $"wbps={deviceId}:{bytesPerSecond}");
            }
        }
    }

    private int MapSharesToWeight(int cpuShares)
    {
        // Docker cpu-shares (2-262144) maps to cgroups v2 cpu.weight (1-10000)
        // Default shares=1024 maps to weight=100
        return Math.Clamp((int)(cpuShares * 10000.0 / 262144.0), 1, 10000);
    }

    public async Task<ResourceUsage> GetUsageAsync(string containerId)
    {
        var cgroupPath = Path.Combine(_cgroupRoot, $"docker/{containerId}");

        var cpuMax = await File.ReadAllTextAsync(
            Path.Combine(cgroupPath, "cpu.stat"));
        var memCurrent = await File.ReadAllTextAsync(
            Path.Combine(cgroupPath, "memory.current"));
        var pidsCurrent = await File.ReadAllTextAsync(
            Path.Combine(cgroupPath, "pids.current"));

        return new ResourceUsage
        {
            CpuUsageMicros = ParseCpuStat(cpuMax, "usage_usec"),
            MemoryUsageBytes = long.Parse(memCurrent.Trim()),
            PidCount = int.Parse(pidsCurrent.Trim())
        };
    }
}

19. Container Logging & Monitoring

Container logging and monitoring are essential for operating containerized workloads in production. Docker supports multiple logging drivers that capture container stdout/stderr output and route it to different destinations. The default json-file driver writes logs to a JSON file on the host, while the fluentd, syslog, journald, and awslogs drivers forward logs to centralized logging systems. For monitoring, containers expose metrics through cAdvisor (Container Advisor), which collects CPU, memory, network, and filesystem usage data at the container level.

A production observability stack typically combines three pillars: logs (what happened), metrics (what is happening), and traces (the request flow through services). For containers, logs are collected using a daemon-level log shipper (Fluentd, Filebeat, or Promtail) running on each host, metrics are collected using Prometheus with cAdvisor and node exporters, and traces are collected using OpenTelemetry with Jaeger or Tempo as the backend.

Container Metrics Schema

public class ContainerMetricsCollector
{
    private readonly ICAdvisorClient _cadvisor;
    private readonly ILogger<ContainerMetricsCollector> _logger;
    private readonly ConcurrentDictionary<string, ContainerMetricsSnapshot>
        _previousSnapshots = new();

    public async Task<ContainerMetrics> CollectMetricsAsync(string containerId)
    {
        var stats = await _cadvisor.GetContainerStatsAsync(containerId);
        var spec = await _cadvisor.GetContainerSpecAsync(containerId);

        var previous = _previousSnapshots.GetValueOrDefault(containerId);

        var metrics = new ContainerMetrics
        {
            ContainerId = containerId,
            Timestamp = DateTimeOffset.UtcNow,
            Cpu = new CpuMetrics
            {
                UsageTotalNanos = stats.Cpu.Usage.Total,
                UsageDeltaNanos = previous != null
                    ? stats.Cpu.Usage.Total - previous.CpuUsageTotal
                    : 0,
                UsagePercent = previous != null
                    ? CalculateCpuPercent(
                        stats.Cpu.Usage.Total - previous.CpuUsageTotal,
                        stats.Timestamp - previous.Timestamp)
                    : 0,
                ThrottledPeriods = stats.Cpu.Cfs.ThrottledPeriods,
                ThrottledTimeNanos = stats.Cpu.Cfs.ThrottledTime
            },
            Memory = new MemoryMetrics
            {
                UsageBytes = stats.Memory.Usage,
                RssBytes = stats.Memory.Rss,
                CacheBytes = stats.Memory.Cache,
                LimitBytes = spec.Memory.Limit,
                UsagePercent = spec.Memory.Limit > 0
                    ? (double)stats.Memory.Usage / spec.Memory.Limit * 100
                    : 0,
                OomKillCount = stats.Memory.OomEvents
            },
            Network = new NetworkMetrics
            {
                ReceiveBytes = stats.Network.Interfaces
                    .Sum(i => i.RxBytes),
                TransmitBytes = stats.Network.Interfaces
                    .Sum(i => i.TxBytes),
                ReceivePackets = stats.Network.Interfaces
                    .Sum(i => i.RxPackets),
                TransmitPackets = stats.Network.Interfaces
                    .Sum(i => i.TxPackets),
                ReceiveErrors = stats.Network.Interfaces
                    .Sum(i => i.RxErrors),
                TransmitErrors = stats.Network.Interfaces
                    .Sum(i => i.TxErrors)
            },
            Filesystem = new FilesystemMetrics
            {
                UsageBytes = stats.Filesystem.Usage,
                LimitBytes = stats.Filesystem.Limit,
                UsagePercent = stats.Filesystem.Limit > 0
                    ? (double)stats.Filesystem.Usage /
                      stats.Filesystem.Limit * 100
                    : 0
            },
            Pids = new PidMetrics
            {
                CurrentCount = stats.Pids.Current,
                LimitCount = spec.Pids.Limit
            }
        };

        // Update snapshot for delta calculations
        _previousSnapshots[containerId] = new ContainerMetricsSnapshot
        {
            CpuUsageTotal = stats.Cpu.Usage.Total,
            Timestamp = stats.Timestamp
        };

        return metrics;
    }

    private double CalculateCpuPercent(
        long deltaNanos, TimeSpan deltaTime)
    {
        if (deltaTime.TotalNanos == 0) return 0;
        var numCpus = Environment.ProcessorCount;
        return (double)deltaNanos / (deltaTime.TotalNanos * numCpus) * 100;
    }
}

20. Multi-Stage Builds

Multi-stage builds are a Dockerfile feature that allows using multiple FROM instructions in a single Dockerfile, each starting a new build stage. The key innovation is the ability to copy artifacts from one stage to another using the COPY --from directive. This enables separating the build environment (which includes compilers, SDKs, and build tools) from the runtime environment (which only needs the compiled output and runtime dependencies). The result is dramatically smaller production images with reduced attack surface.

The security benefits of multi-stage builds are significant. Build tools like compilers, package managers, and development libraries are never present in the production image. This reduces the attack surface because these tools often have their own vulnerabilities. It also reduces image size, which improves pull times, reduces storage costs, and decreases the blast radius of any potential exploit. A typical .NET application might have a build stage of 700 MB (containing the .NET SDK and all build dependencies) and a runtime stage of 200 MB (containing only the .NET runtime and compiled output).

Multi-Stage Build Size Comparison

public class MultiStageBuildOptimizer
{
    public OptimizationReport AnalyzeDockerfile(Dockerfile dockerfile)
    {
        var report = new OptimizationReport();
        var stages = dockerfile.GetBuildStages();

        foreach (var stage in stages)
        {
            var stageAnalysis = new StageAnalysis
            {
                StageName = stage.Name,
                BaseImage = stage.BaseImage,
                InstructionCount = stage.Instructions.Count,
                EstimatedSizeMb = EstimateStageSize(stage)
            };

            // Check for common optimization opportunities
            if (stage == stages.Last())
            {
                // Analyze final stage for size optimization
                var hasBuildTools = stage.Instructions.Any(i =>
                    i is RunInstruction r &&
                    (r.Command.Contains("apt-get") ||
                     r.Command.Contains("apk add") ||
                     r.Command.Contains("yum install")));

                if (hasBuildTools)
                {
                    report.Recommendations.Add(new OptimizationRecommendation
                    {
                        Severity = RecommendationSeverity.High,
                        Category = "Multi-Stage Separation",
                        Description = "Final stage contains build tools. " +
                            "Move build steps to an earlier stage.",
                        EstimatedSavingMb = 200
                    });
                }
            }

            // Check for layer optimization within a stage
            var runInstructions = stage.Instructions
                .OfType<RunInstruction>().ToList();
            if (runInstructions.Count > 1)
            {
                report.Recommendations.Add(new OptimizationRecommendation
                {
                    Severity = RecommendationSeverity.Medium,
                    Category = "Layer Consolidation",
                    Description = $"Stage '{stage.Name}' has " +
                        $"{runInstructions.Count} RUN instructions that could " +
                        $"be combined.",
                    EstimatedSavingMb = runInstructions.Count * 5
                });
            }

            report.Stages.Add(stageAnalysis);
        }

        report.TotalEstimatedSizeMb =
            report.Stages.Sum(s => s.EstimatedSizeMb);
        report.OptimizationPotentialMb =
            report.Recommendations.Sum(r => r.EstimatedSavingMb);

        return report;
    }

    private int EstimateStageSize(BuildStage stage)
    {
        int sizeMb = 0;
        foreach (var instruction in stage.Instructions)
        {
            sizeMb += instruction switch
            {
                FromInstruction f => EstimateBaseImageSize(f.Image),
                RunInstruction _ => 30,
                CopyInstruction c => EstimateCopySize(c),
                _ => 1
            };
        }
        return sizeMb;
    }
}

21. Container-as-a-Service (CaaS)

Container-as-a-Service (CaaS) is a cloud computing model that provides a managed platform for deploying and managing containers. Unlike Infrastructure-as-a-Service (IaaS), where you manage the underlying VMs, or Platform-as-a-Service (PaaS), where you deploy application code directly, CaaS gives you control over container orchestration while abstracting away the underlying infrastructure management. Examples include AWS Fargate, Google Cloud Run, Azure Container Instances, and platform-specific implementations built on Kubernetes.

Building a CaaS platform on top of a container platform requires several additional abstractions: deployment templates (declarative specifications for how to run a service), auto-scaling policies (horizontal and vertical scaling based on metrics), service discovery (automatic DNS registration and load balancing), TLS termination (automatic certificate provisioning and renewal), traffic splitting (canary deployments and A/B testing), and developer experience tooling (CLI, dashboard, CI/CD integration).

public class CaaSDeploymentService
{
    private readonly IContainerPlatform _platform;
    private readonly IDeploymentStore _store;
    private readonly IServiceDiscovery _discovery;
    private readonly ICertificateManager _certs;
    private readonly IAutoScaler _autoScaler;

    public async Task<DeploymentResult> DeployServiceAsync(
        CaaSServiceSpec spec)
    {
        // Step 1: Validate and resolve the container image
        var imageInfo = await _platform.ResolveImageAsync(spec.Image);
        if (imageInfo == null)
            throw new ImageNotFoundException(
                $"Image {spec.Image} not found in registry");

        // Step 2: Generate deployment configuration
        var deployment = new Deployment
        {
            Id = GenerateDeploymentId(),
            ServiceName = spec.Name,
            Image = spec.Image,
            Replicas = spec.InitialReplicas,
            Resources = spec.Resources,
            EnvironmentVariables = spec.Env,
            Secrets = spec.Secrets,
            HealthCheckPath = spec.HealthCheckPath,
            Port = spec.Port,
            CreatedAt = DateTimeOffset.UtcNow
        };

        // Step 3: Provision TLS certificate
        if (spec.EnableTls)
        {
            deployment.Certificate = await _certs.ProvisionAsync(
                spec.Hostname, spec.Domain);
        }

        // Step 4: Create the underlying containers
        var containers = await CreateContainerBatchAsync(
            deployment, spec.InitialReplicas);

        // Step 5: Register with service discovery
        await _discovery.RegisterAsync(new ServiceRegistration
        {
            Name = spec.Name,
            Endpoints = containers.Select(c =>
                new ServiceEndpoint
                {
                    Host = c.IpAddress,
                    Port = spec.Port,
                    Healthy = true
                }).ToList(),
            HealthCheckPath = spec.HealthCheckPath,
            HealthCheckInterval = TimeSpan.FromSeconds(15)
        });

        // Step 6: Configure auto-scaling
        if (spec.AutoScaling != null)
        {
            await _autoScaler.ConfigureAsync(new AutoScalingPolicy
            {
                DeploymentId = deployment.Id,
                MinReplicas = spec.AutoScaling.MinReplicas,
                MaxReplicas = spec.AutoScaling.MaxReplicas,
                TargetCpuPercent = spec.AutoScaling.TargetCpuPercent,
                TargetMemoryPercent = spec.AutoScaling.TargetMemoryPercent,
                ScaleUpCooldown = TimeSpan.FromMinutes(1),
                ScaleDownCooldown = TimeSpan.FromMinutes(5)
            });
        }

        // Step 7: Store deployment record
        await _store.SaveDeploymentAsync(deployment);

        return new DeploymentResult
        {
            DeploymentId = deployment.Id,
            ServiceUrl = $"https://{spec.Hostname}",
            Status = DeploymentStatus.Running,
            ContainersCreated = containers.Count
        };
    }
}

CaaS Platform Comparison

22. Docker Desktop & Dev Environments

Docker Desktop is the local development companion that provides a complete Docker environment on macOS, Windows, and Linux. On macOS and Windows, Docker Desktop runs a Linux VM (using HyperKit, QEMU, or WSL2) to provide the Linux kernel needed for containers. On Linux, Docker Desktop runs natively. Docker Desktop includes the Docker Engine, Docker CLI, Docker Compose, Docker Content Trust, Kubernetes, and a graphical dashboard for managing containers, images, volumes, and networks.

The development workflow with Docker Desktop centers on the Dockerfile and docker-compose.yml as the single source of truth for the development environment. New developers can clone a repository and run docker compose up to get a fully configured environment with all dependencies, databases, message brokers, and application services running and connected. This eliminates the "works on my machine" problem and dramatically reduces onboarding time.

Docker Dev Environments extend this concept further by providing pre-configured, shareable development environments that include IDE settings, extensions, and tools. These environments are defined as Docker Compose configurations and can be version-controlled alongside the application code. Dev Environments use feature branches for parallel development, allowing developers to work on multiple features simultaneously without environment conflicts.

Docker Desktop Architecture

23. Performance Optimization

Container performance optimization operates at multiple levels: the host OS, the container runtime, the image build process, the container runtime resource allocation, and the application code itself. At the host level, using a modern Linux kernel (5.15+) with cgroups v2, OverlayFS, and BPF (Berkeley Packet Filter) provides significant performance improvements over older configurations. The container runtime can be optimized by choosing the appropriate snapshotter (overlayfs for most workloads, stargz for lazy loading of large images), configuring content store replication for multi-host environments, and tuning garbage collection intervals.

Performance Benchmarks

public class PerformanceOptimizer
{
    private readonly IHostConfig _hostConfig;
    private readonly IRuntimeConfig _runtimeConfig;

    public async Task<OptimizationReport> RunPerformanceAuditAsync(
        HostPerformanceProfile profile)
    {
        var report = new OptimizationReport();

        // Analyze kernel version and features
        var kernelVersion = await _hostConfig.GetKernelVersionAsync();
        if (kernelVersion < new Version(5, 15))
        {
            report.Issues.Add(new PerformanceIssue
            {
                Category = "Kernel",
                Severity = IssueSeverity.High,
                Description = $"Kernel {kernelVersion} is below recommended " +
                    "5.15+. Upgrade for cgroups v2 and OverlayFS improvements.",
                EstimatedImpact = "15-30% container startup improvement"
            });
        }

        // Check cgroups version
        var cgroupVersion = await _hostConfig.GetCgroupVersionAsync();
        if (cgroupVersion != CgroupVersion.V2)
        {
            report.Issues.Add(new PerformanceIssue
            {
                Category = "Cgroups",
                Severity = IssueSeverity.Medium,
                Description = "Running cgroups v1. Upgrade to unified cgroups v2 " +
                    "for better resource control and performance.",
                EstimatedImpact = "10-20% memory management improvement"
            });
        }

        // Analyze storage driver
        var storageDriver = await _hostConfig.GetStorageDriverAsync();
        if (storageDriver != "overlay2")
        {
            report.Issues.Add(new PerformanceIssue
            {
                Category = "Storage",
                Severity = IssueSeverity.High,
                Description = $"Using {storageDriver} driver. overlay2 is " +
                    "recommended for best performance.",
                EstimatedImpact = "20-50% filesystem operation improvement"
            });
        }

        // Check container image layer efficiency
        var images = await _hostConfig.GetInstalledImagesAsync();
        foreach (var image in images)
        {
            if (image.Layers.Count > 20)
            {
                report.Issues.Add(new PerformanceIssue
                {
                    Category = "Image Optimization",
                    Severity = IssueSeverity.Medium,
                    Description = $"Image {image.Repository}:{image.Tag} has " +
                        $"{image.Layers.Count} layers (recommended < 20).",
                    EstimatedImpact = "Reduced layer metadata overhead"
                });
            }
        }

        // Analyze I/O scheduler
        var ioScheduler = await _hostConfig.GetIoSchedulerAsync();
        if (ioScheduler != "mq-deadline" && ioScheduler != "none")
        {
            report.Issues.Add(new PerformanceIssue
            {
                Category = "I/O",
                Severity = IssueSeverity.Low,
                Description = $"I/O scheduler is '{ioScheduler}'. " +
                    "mq-deadline or none recommended for SSDs.",
                EstimatedImpact = "5-15% I/O latency improvement"
            });
        }

        report.Score = CalculateHealthScore(report.Issues);
        return report;
    }
}

24. Cost Estimation

Understanding the cost structure of a container platform is essential for capacity planning and budget allocation. The primary cost drivers are compute resources (CPU and memory for container hosts), storage (image registry, container writable layers, persistent volumes), network (inter-node traffic, external bandwidth, load balancers), and operational overhead (tooling, monitoring, security, and personnel).

Cost Breakdown for a 100-Node Container Platform

On-premises costs are significantly lower in ongoing operational expenditure but require higher upfront capital investment for hardware procurement. A 100-node cluster with high-spec servers might require $500,000-$800,000 in initial hardware costs, amortized over a 3-5 year lifecycle. Cloud deployments offer greater flexibility for scaling up and down, pay-as-you-go pricing, and elimination of hardware management overhead. The break-even point typically occurs at 18-24 months for consistent workloads, after which on-prem becomes more cost-effective.

Cost optimization strategies include using spot instances for non-critical workloads (50-70% savings), right-sizing containers based on actual resource utilization (20-30% savings), implementing automatic scaling to reduce idle capacity, using Graviton/ARM instances (20% cheaper with comparable performance), and consolidating small workloads to improve container density per node.

25. Testing Strategy

Testing a container platform requires a comprehensive strategy that covers unit testing of individual components, integration testing of the container lifecycle, end-to-end testing of the full stack, and chaos engineering to validate resilience under failure conditions. The testing pyramid applies: fast unit tests form the base, integration tests form the middle, and end-to-end tests form the narrow top.

Unit tests should cover the Dockerfile parser, the image layer resolver, the cgroup configuration logic, the network namespace setup, and the API request handling. Integration tests should verify the complete container lifecycle: create, start, exec, stop, remove, and validate filesystem isolation, network connectivity, and resource limits. End-to-end tests should exercise the full platform including the registry, the scheduler, the monitoring stack, and the security scanner. Chaos engineering tests should simulate node failures, network partitions, and storage degradation to validate the platform's self-healing capabilities.

public class ContainerLifecycleTests
{
    [Fact]
    public async Task CreateAndStartContainer_WithResourceLimits_EnforcesCpuAndMemory()
    {
        // Arrange
        var client = new ContainerPlatformClient("http://localhost:2375", "test-token");
        var createRequest = new ContainerCreateRequest
        {
            Image = "alpine:3.18",
            Cmd = new[] { "sh", "-c", "dd if=/dev/zero of=/dev/null bs=1M" },
            HostConfig = new HostConfig
            {
                Resources = new ResourceRequirements
                {
                    CpuQuota = 50000,  // 50% of one CPU
                    CpuPeriod = 100000,
                    Memory = 128 * 1024 * 1024,  // 128 MB
                    MemorySwap = 128 * 1024 * 1024
                }
            }
        };

        // Act
        var createResponse = await client.CreateContainerAsync(createRequest);
        await client.StartContainerAsync(createResponse.Id);

        // Wait for container to start and stabilize
        await Task.Delay(TimeSpan.FromSeconds(5));

        // Assert
        var inspect = await client.InspectContainerAsync(createResponse.Id);
        Assert.Equal(ContainerStatus.Running, inspect.State.Status);

        // Verify resource limits through cgroup files
        var cgroupPath = $"/sys/fs/cgroup/docker/{createResponse.Id}";
        var cpuMax = await File.ReadAllTextAsync($"{cgroupPath}/cpu.max");
        Assert.StartsWith("50000", cpuMax);

        var memMax = await File.ReadAllTextAsync($"{cgroupPath}/memory.max");
        Assert.Equal((128 * 1024 * 1024).ToString(), memMax.Trim());

        // Cleanup
        await client.StopContainerAsync(createResponse.Id, 5);
        await client.RemoveContainerAsync(createResponse.Id, force: true);
    }

    [Fact]
    public async Task ContainerNetworking_TwoContainersOnSameBridge_CanCommunicate()
    {
        var client = new ContainerPlatformClient("http://localhost:2375", "test-token");

        // Create a custom bridge network
        var network = await client.CreateNetworkAsync(new NetworkCreateRequest
        {
            Name = "test-net",
            Driver = "bridge"
        });

        // Start two containers on the same network
        var container1 = await CreateAndStartOnNetworkAsync(
            client, network.Id, "alpine:3.18",
            new[] { "sh", "-c", "apk add --no-cache socat && " +
                "socat TCP-LISTEN:8080,reuseaddr,fork EXEC:'/bin/cat'" });

        var container2 = await CreateAndStartOnNetworkAsync(
            client, network.Id, "alpine:3.18",
            new[] { "sh", "-c", "echo hello | nc container1 8080" });

        await Task.Delay(TimeSpan.FromSeconds(5));

        // Exec into container2 and check if data was received
        var logs = await client.GetContainerLogsAsync(container2.Id);

        // Assert communication happened
        Assert.Contains("hello", logs);

        // Cleanup
        await client.StopAndRemoveAsync(container1.Id);
        await client.StopAndRemoveAsync(container2.Id);
        await client.RemoveNetworkAsync(network.Id);
    }

    [Fact]
    public async Task ImageBuild_MultiStageBuild_ProducesSmallerImage()
    {
        var client = new ContainerPlatformClient("http://localhost:2375", "test-token");

        // Build a multi-stage image
        var dockerfile = @"
            FROM mcr.microsoft.com/dotnet/sdk:8.0-alpine AS build
            WORKDIR /src
            COPY . .
            RUN dotnet publish -c Release -o /app/publish

            FROM mcr.microsoft.com/dotnet/aspnet:8.0-alpine
            COPY --from=build /app/publish .
            ENTRYPOINT [""dotnet"", ""MyApp.dll""]";

        var buildContext = CreateTarArchive(dockerfile, "Dockerfile");
        var buildResult = await client.BuildImageAsync(
            buildContext, "test-app:multi-stage");

        // Assert image was built successfully
        Assert.NotNull(buildResult.ImageId);

        // Verify the final image doesn't contain SDK
        var inspect = await client.InspectImageAsync(buildResult.ImageId);
        Assert.Contains("aspnet", inspect.Config.Image);

        // Cleanup
        await client.RemoveImageAsync("test-app:multi-stage", force: true);
    }

    [Fact]
    public async Task VolumePersistence_DataSurvivesContainerRestart()
    {
        var client = new ContainerPlatformClient("http://localhost:2375", "test-token");

        // Create a named volume
        var volume = await client.CreateVolumeAsync(new VolumeCreateRequest
        {
            Name = "test-data-vol"
        });

        // Start a container, write data to the volume, stop it
        var container1 = await client.CreateContainerAsync(
            new ContainerCreateRequest
            {
                Image = "alpine:3.18",
                Cmd = new[] { "sh", "-c", "echo persistent-data > /data/file.txt" },
                HostConfig = new HostConfig
                {
                    Binds = new[] { "test-data-vol:/data" }
                }
            });
        await client.StartContainerAsync(container1.Id);
        await Task.Delay(2000);
        await client.StopContainerAsync(container1.Id, 5);
        await client.RemoveContainerAsync(container1.Id);

        // Start a new container, read the data
        var container2 = await client.CreateContainerAsync(
            new ContainerCreateRequest
            {
                Image = "alpine:3.18",
                Cmd = new[] { "sh", "-c", "cat /data/file.txt" },
                HostConfig = new HostConfig
                {
                    Binds = new[] { "test-data-vol:/data" }
                }
            });
        await client.StartContainerAsync(container2.Id);
        await Task.Delay(2000);

        var logs = await client.GetContainerLogsAsync(container2.Id);
        Assert.Contains("persistent-data", logs);

        // Cleanup
        await client.StopAndRemoveAsync(container2.Id);
        await client.RemoveVolumeAsync("test-data-vol");
    }
}

Test Metrics & Coverage Targets

26. Interview Q&A

Q1: How does a container differ from a virtual machine at the kernel level?

Answer: A container shares the host OS kernel and uses Linux namespaces (PID, network, mount, UTS, IPC, user) for isolation and cgroups for resource limiting. A virtual machine runs an entirely separate guest OS kernel on a hypervisor (Type 1: bare-metal like KVM, or Type 2: hosted like VirtualBox). Containers are process-level isolation; VMs are hardware-level isolation. This is why containers start in milliseconds (just process creation + namespace setup) while VMs take 30+ seconds (full OS boot). The trade-off is that containers share the kernel attack surface with other containers and the host, while VMs provide stronger isolation boundaries.

Q2: Explain the Docker image layer model and how layer caching works.

Answer: Each Dockerfile instruction creates a read-only image layer identified by its SHA256 content hash. Layers are stacked using UnionFS (OverlayFS), where the bottom layer is the base image and each subsequent layer applies changes on top. Layer caching means Docker reuses a cached layer if the instruction and its inputs haven't changed. The cache invalidation cascades forward — once a layer is invalidated, all subsequent layers must be rebuilt. This is why Dockerfile ordering matters: place stable steps (FROM, system packages) first, and volatile steps (COPY source code) last. BuildKit adds advanced caching with --cache-from for remote cache import and --mount=type=cache for persistent caches.

Q3: How would you design a container networking solution that supports 10,000+ containers?

Answer: At scale, the default bridge network with NAT becomes a bottleneck. I would use an overlay network with VXLAN encapsulation for multi-host communication, combined with BGP-based routing (like Calico or Cilium) for efficient pod-to-pod traffic without encapsulation overhead. For service discovery, I would integrate a DNS-based system (like CoreDNS) with a health-aware load balancer (like Envoy). I would implement network policies using iptables or eBPF to enforce microsegmentation. For high-throughput workloads, I would use SR-IOV (Single Root I/O Virtualization) to bypass the vswitch and provide near-native network performance. The control plane would use the gossip protocol for membership management and the data plane would use hardware offloading where available.

Q4: What is the significance of the containerd/runc split, and why did Kubernetes deprecate dockershim?

Answer: The split between containerd (high-level runtime) and runc (low-level OCI runtime) follows the Unix philosophy of small, composable tools. containerd manages the container lifecycle, image distribution, and storage, while runc handles the actual process creation using kernel primitives. This separation allows swapping runc for alternative runtimes like kata-containers (lightweight VMs) or gVisor (user-space kernel). Kubernetes deprecated dockershim because the Docker daemon (dockerd) included functionality that Kubernetes didn't need — it was an orchestration layer (Docker Swarm) on top of the container runtime. By connecting directly to containerd, Kubernetes eliminates unnecessary indirection, reduces memory overhead (~1GB per node), and simplifies the dependency chain. The Docker image format and runtime behavior remain fully compatible.

Q5: How do you handle secrets in a container platform?

Answer: Secrets should never be baked into images (environment variables in Dockerfiles are visible in docker inspect and image history). Instead, I use a layered approach: Docker secrets (available as files in /run/secrets/) for Swarm, Kubernetes Secrets with etcd encryption at rest for K8s, and an external secrets manager (HashiCorp Vault, AWS Secrets Manager) for production. The secrets are mounted as tmpfs volumes (in-memory, never written to disk) and referenced as files or environment variables at runtime. For CI/CD, secrets are injected through the pipeline's secret management, not stored in repository variables. RBAC policies restrict which containers can access which secrets, and audit logging tracks every secret access.

Q6: Describe your approach to container image vulnerability scanning in a CI/CD pipeline.

Answer: I implement scanning at three points in the pipeline: (1) during the build stage, scanning the Dockerfile and base image for known vulnerabilities before building, (2) immediately after the build, scanning the resulting image for OS package and application dependency vulnerabilities, and (3) periodically rescanning deployed images as new vulnerabilities are discovered. The scanner (Trivy) cross-references installed packages against NVD, OS-specific advisories, and language-specific databases. The pipeline enforces quality gates: Critical vulnerabilities block deployment, High vulnerabilities trigger alerts with a 72-hour SLA, Medium and Low are tracked for remediation. Results feed into an SBOM (Software Bill of Materials) for compliance reporting. False positives are managed through a suppress file reviewed by the security team.

Q7: How do you optimize container startup time for latency-sensitive services?

Answer: Container startup time is dominated by image pulling and application initialization. For image optimization: use minimal base images (Alpine, distroless), pre-pull images on target nodes using DaemonSets, use lazy-loading with eStargz or Nydus for large images, and leverage multi-stage builds to minimize layer count. For application optimization: use ahead-of-time (AOT) compilation for .NET (NativeAOT) or GraalVM native-image for Java, implement health check endpoints early in the startup sequence, use init containers for one-time setup, implement connection pooling to avoid cold connection overhead, and use sidecar patterns for shared dependencies. At the infrastructure level: use local SSD storage, enable the overlay snapshotter, configure the container runtime with warm caches, and pin containers to dedicated nodes for cache warmth.

Q8: Explain the CAP theorem as it applies to container orchestration.

Answer: Container orchestration clusters must make CAP trade-offs. Docker Swarm favors Consistency and Partition tolerance (CP) — the Raft-based control plane requires a quorum of managers to make scheduling decisions, meaning the cluster is unavailable during leader election. Kubernetes also uses etcd (CP), but its architecture allows greater operational resilience because the kubelets continue running existing workloads even when the API server is unreachable. The data plane (running containers) continues operating independently of the control plane — this is an important architectural property. For the registry, S3 provides AP (eventual consistency for new objects), while etcd provides CP. The key insight is that the control plane should be CP (you want consistent scheduling decisions), while the data plane should be AP (containers should keep running even during control plane disruptions).