1. Introduction - Why Payment Systems Are Hard

Designing a payment processing system is one of the most challenging problems in software engineering. Unlike most distributed systems, payment processing demands absolute correctness. A lost tweet is annoying; a lost payment is catastrophic. When Stripe, Square, or Adyen process billions of dollars per day, there is zero tolerance for data loss, duplicate charges, or inconsistent state. The system must be simultaneously highly available, strongly consistent, and globally scalable - a set of requirements that are often at odds with one another in distributed systems theory.

A payment processing system sits at the intersection of multiple complex domains: financial regulations, banking networks, fraud prevention, cryptography, distributed systems, and real-time event processing. Every design decision carries implications not just for performance, but for legal compliance, financial accuracy, and customer trust. A single bug can result in millions of dollars in incorrect charges, regulatory fines, or the loss of a banking partner.

In this guide, we will design a complete payment processing system from the ground up - similar in scope to what Stripe, Adyen, or PayPal's core infrastructure provides. We will cover everything from low-level tokenization of card data, through multi-party settlement and ledger design, to webhook delivery guarantees and fraud detection pipelines. The system we design will handle card payments, bank transfers, subscription billing, multi-currency transactions, refunds, disputes, and reconciliation - all while maintaining PCI DSS compliance and financial accuracy.

Why This Problem Matters

The global digital payments market processes over 9 trillion dollars annually and is growing at roughly 15 percent per year. Every e-commerce transaction, subscription service, marketplace payout, and peer-to-peer transfer flows through some form of payment processing infrastructure. The systems that handle this volume must operate with the reliability characteristics of critical infrastructure - think power grids, not typical web applications.

The unique challenges of payment systems include:

• Financial correctness is non-negotiable: Every cent must be accounted for. There is no eventually consistent when it comes to money - the system must guarantee that every transaction is recorded exactly once with absolute precision.
• External system dependencies: Payment systems must integrate with banks, card networks (Visa, Mastercard), payment processors, and fraud detection services - all of which have their own latency characteristics, availability guarantees, and failure modes.
• Regulatory compliance: PCI DSS, PSD2, SOX, GDPR, and dozens of other regulations impose strict requirements on how cardholder data is stored, processed, and transmitted.
• Multi-party settlement: A single card payment involves the merchant, the payment facilitator, the acquiring bank, the card network, and the issuing bank - each needing accurate financial records.
• Global operations: Currencies, payment methods, banking regulations, and fraud patterns vary dramatically by region. A system designed for the US must also work seamlessly in Japan, Brazil, and Nigeria.

Lessons from Production Systems

Having worked with and studied payment infrastructure at scale, several hard-won lessons stand out. First, never underestimate the complexity of idempotency. Network partitions, retries, and partial failures mean that every payment operation must be safely retryable without creating duplicates. Second, your ledger is sacred - it must be immutable, append-only, and designed to survive catastrophic failures. Third, reconciliation is not optional - it is a continuous, real-time process that catches discrepancies before they compound. Fourth, the most dangerous bugs in payment systems are the ones that appear to work correctly under normal conditions but fail silently during edge cases - partial authorizations, network timeouts mid-transaction, or race conditions in concurrent payment attempts.

Key Design Principles

The entire system is built around five non-negotiable principles. First, correctness over performance - we will always choose the slower, correct path over the faster, risky one. Second, defense in depth - multiple layers of validation, encryption, and consistency checks ensure that a failure at any single layer does not compromise the system. Third, auditability - every action must be traceable, every state change must be recorded, and the complete history must be reconstructable. Fourth, graceful degradation - when external dependencies fail, the system must degrade gracefully rather than failing completely. Fifth, financial completeness - every dollar must be accounted for at every stage, from the initial authorization through settlement and payout.

2. Functional and Non-Functional Requirements

Functional Requirements

Before designing any system, we must precisely define what it needs to do. For a payment processing platform, the functional requirements span several categories that cover every aspect of money movement and financial record-keeping.

Core Payment Operations

• Charge Processing: Accept payment details, tokenize sensitive data, route to the appropriate payment processor, handle authorization, and capture funds. Support one-time payments as well as authorized-but-not-captured flows (common in hospitality and car rental).
• Payment Methods: Store and manage customer payment methods - credit/debit cards, bank accounts (ACH), digital wallets (Apple Pay, Google Pay), and local payment methods (iDEAL, SEPA, Boleto). Each method has different authorization and settlement characteristics.
• Refunds and Partial Refunds: Support full and partial refunds against original transactions, with proper ledger entries and settlement adjustments.
• Disputes and Chargebacks: Handle the full lifecycle of payment disputes - from receiving a chargeback notification, through evidence submission, to final resolution. This involves strict timelines and specific response formats dictated by card networks.
• Subscriptions and Recurring: Support recurring billing with configurable intervals, trial periods, proration, plan changes, and dunning (automatic retry of failed payments).

Financial Operations

• Multi-Currency: Process payments in 135+ currencies with real-time or daily foreign exchange rates. Support currency conversion at the payment level and at the settlement level.
• Payouts and Settlement: Aggregate merchant transactions and disburse funds on configurable schedules (daily, weekly, monthly), accounting for refunds, chargebacks, and fees.
• Ledger and Double-Entry Bookkeeping: Maintain a complete, immutable audit trail of every financial event with proper double-entry accounting where every debit has a corresponding credit.
• Reconciliation: Automatically match internal records against processor reports, bank statements, and network settlement files to detect discrepancies.

Integration and Developer Experience

• REST API: Comprehensive API covering all payment operations with consistent design, pagination, filtering, and sorting.
• Webhooks: Real-time event delivery for payment state changes with guaranteed delivery semantics.
• Sandboxes and Test Mode: Full simulation environment for development and testing, including test card numbers for various scenarios (success, decline, 3DS, fraud).
• SDKs: Client-side libraries for tokenization (JS, iOS, Android) that keep sensitive data off merchant servers.

Non-Functional Requirements

3. Capacity Estimation and Scale

Understanding the scale of a payment processing system is critical for making appropriate technology choices. Let us work through the numbers for a Stripe-scale platform processing hundreds of millions of transactions daily.

Transaction Volume

Stripe processes approximately 500 million transactions per day. Let us use this as our baseline and design for growth:

• Daily transactions: 500 million
• Average transactions per second (TPS): 500M / 86,400 = approximately 5,800 TPS average
• Peak TPS (5x average): approximately 30,000 TPS during Black Friday or major sales events
• Peak burst TPS: approximately 100,000+ TPS for flash sales (must be handled via queuing and rate limiting)

Data Volume Estimates

Network and Bandwidth

• Inbound (API requests): 500M requests times 2 KB average equals approximately 1 TB per day, approximately 12 MB/s average, approximately 120 MB/s peak
• Outbound (API responses plus webhooks): approximately 1 TB per day, approximately 12 MB/s average
• Internal (inter-service): approximately 5 TB per day of internal service-to-service communication
• Total daily network transfer: approximately 7 TB across all tiers

Cache and Hot Data

• Payment method tokens: approximately 50M unique active payment methods times 500 bytes equals approximately 25 GB (fits in a few Redis clusters)
• Fraud rules and models: approximately 1 GB of model parameters, updated hourly
• FX rates: approximately 50 KB of rate data, updated every 15-60 seconds
• Merchant configuration: approximately 10 GB total, heavily read, rarely written

Query Patterns

Understanding access patterns is essential for choosing the right storage technologies:

• Point lookups by payment ID: approximately 40% of all reads - must be sub-millisecond
• Lookups by customer ID (recent payments): approximately 25% of all reads
• Merchant dashboard queries (date range plus filters): approximately 15% of reads, can tolerate 100-200ms
• Reconciliation batch queries: approximately 10% of reads, scan-heavy, can run on replicas
• Analytics and reporting queries: approximately 10% of reads, complex aggregations, run on data warehouse

Storage Technology Choices

Given the scale and access patterns, the recommended storage stack is:

• Primary OLTP: PostgreSQL with Citus extension for horizontal sharding. The strong ACID guarantees are essential for ledger operations, while Citus provides the horizontal scaling needed for 500M+ rows.
• Cache layer: Redis Cluster for hot data (payment methods, fraud models, FX rates, idempotency keys). Typically 3-5 shards with replicas.
• Event streaming: Apache Kafka for event sourcing, webhook delivery, and inter-service communication. Retain events for 7+ days for replay capability.
• Data warehouse: ClickHouse or BigQuery for analytics, reporting, and reconciliation queries that scan large date ranges.
• Object storage: S3 or GCS for long-term archival of settlement files, dispute evidence, and compliance documents.

4. Data Model and Entity Design

The data model of a payment system is its most critical design element. Get it wrong, and you will spend years fighting inconsistencies. Get it right, and the rest of the system becomes dramatically simpler. The core principle is that the data model must make illegal states unrepresentable - if a payment cannot logically be in a certain state, the schema should prevent it.

Core Entities

C#
public enum PaymentStatus
{
    Pending,
    Authorized,
    Captured,
    PartiallyCaptured,
    Settled,
    Refunded,
    PartiallyRefunded,
    Disputed,
    WonDispute,
    LostDispute,
    Cancelled,
    Failed
}

public enum PaymentMethodType
{
    Card,
    BankAccount,
    DigitalWallet,
    LocalPaymentMethod
}

public class PaymentIntent
{
    public string Id { get; set; }                    // pi_xxxxxxxxxxxx (prefixed, collision-resistant)
    public string MerchantId { get; set; }            // References the merchant account
    public string CustomerId { get; set; }            // Optional, for returning customers
    public long Amount { get; set; }                  // In smallest currency unit (cents)
    public string Currency { get; set; }              // ISO 4217 uppercase (USD, EUR, JPY)
    public PaymentStatus Status { get; set; }
    public string PaymentMethodId { get; set; }       // Tokenized payment method reference
    public string IdempotencyKey { get; set; }        // Client-provided, unique per operation
    public Dictionary<string, string> Metadata { get; set; }
    public PaymentCapabilities Capabilities { get; set; }
    public DateTime CreatedAt { get; set; }
    public DateTime? CapturedAt { get; set; }
    public DateTime? SettledAt { get; set; }
    public long AmountReceived { get; set; }
    public long AmountRefunded { get; set; }
    public List<Charge> Charges { get; set; }
    public List<Refund> Refunds { get; set; }
    public string ClientSecret { get; set; }
    public string NextAction { get; set; }
}

public class Charge
{
    public string Id { get; set; }                     // ch_xxxxxxxxxxxx
    public string PaymentIntentId { get; set; }
    public long Amount { get; set; }
    public long AmountCaptured { get; set; }
    public string Currency { get; set; }
    public PaymentMethodType PaymentMethodType { get; set; }
    public string ProcessorTransactionId { get; set; }
    public AuthorizationResult Authorization { get; set; }
    public bool IsLive { get; set; }
    public DateTime CreatedAt { get; set; }
    public List<Fee> Fees { get; set; }
    public Outcome Outcome { get; set; }
}

public class AuthorizationResult
{
    public string AuthorizationCode { get; set; }
    public string ResponseCode { get; set; }
    public string DeclineReason { get; set; }
    public string NetworkTransactionId { get; set; }
    public string AvsResult { get; set; }
    public string CvvResult { get; set; }
    public string ProcessorResponseId { get; set; }
    public RiskAssessment Risk { get; set; }
    public DateTime AuthorizedAt { get; set; }
    public DateTime AuthorizationExpiry { get; set; }
}

public class RiskAssessment
{
    public decimal Score { get; set; }                 // 0.0 - 99.9
    public RiskLevel Level { get; set; }
    public List<string> TriggeredRules { get; set; }
    public FraudCheckResult FraudCheck { get; set; }
    public bool RequiresThreeDS { get; set; }
}

public enum RiskLevel
{
    Low,
    Medium,
    High,
    VeryHigh
}

Refund and Dispute Entities

C#
public class Refund
{
    public string Id { get; set; }                     // re_xxxxxxxxxxxx
    public string PaymentIntentId { get; set; }
    public string ChargeId { get; set; }
    public long Amount { get; set; }
    public string Currency { get; set; }
    public RefundStatus Status { get; set; }
    public string Reason { get; set; }
    public string ProcessorRefundId { get; set; }
    public DateTime CreatedAt { get; set; }
    public DateTime? CompletedAt { get; set; }
    public Dictionary<string, string> Metadata { get; set; }
}

public enum RefundStatus
{
    Pending,
    Succeeded,
    Failed,
    Cancelled
}

public class Dispute
{
    public string Id { get; set; }                     // dp_xxxxxxxxxxxx
    public string PaymentIntentId { get; set; }
    public string ChargeId { get; set; }
    public long Amount { get; set; }
    public string Currency { get; set; }
    public DisputeReason Reason { get; set; }
    public DisputeStatus Status { get; set; }
    public DateTime CreatedAt { get; set; }
    public DateTime EvidenceDueBy { get; set; }
    public Evidence Evidence { get; set; }
    public DateTime? ResolvedAt { get; set; }
    public string Resolution { get; set; }
}

public enum DisputeStatus
{
    NeedsResponse,
    UnderReview,
    Won,
    Lost,
    WarningNeedsResponse,
    WarningUnderReview
}

public class MerchantAccount
{
    public string Id { get; set; }                     // acct_xxxxxxxxxxxx
    public string BusinessName { get; set; }
    public string Country { get; set; }
    public string DefaultCurrency { get; set; }
    public MerchantStatus Status { get; set; }
    public PayoutSchedule PayoutSchedule { get; set; }
    public FeeStructure Fees { get; set; }
    public VerificationStatus Verification { get; set; }
    public List<string> SupportedCurrencies { get; set; }
    public Capabilities EnabledCapabilities { get; set; }
    public DateTime CreatedAt { get; set; }
    public BusinessProfile BusinessProfile { get; set; }
    public RiskSettings RiskSettings { get; set; }
}

Payment Method Storage

Payment methods require special handling due to PCI requirements. We never store raw card numbers - instead, we use tokenized references that point to encrypted data in the PCI vault.

C#
public class PaymentMethod
{
    public string Id { get; set; }                     // pm_xxxxxxxxxxxx
    public string CustomerId { get; set; }
    public PaymentMethodType Type { get; set; }
    public BillingDetails BillingDetails { get; set; }
    public CardDetails Card { get; set; }
    public BankAccountDetails BankAccount { get; set; }
    public DateTime CreatedAt { get; set; }
    public bool IsDefault { get; set; }
    public DateTime? LastUsedAt { get; set; }
    public int Fingerprint { get; set; }
}

public class CardDetails
{
    public string Brand { get; set; }                  // visa, mastercard, amex
    public string Funding { get; set; }                // credit, debit, prepaid
    public string Country { get; set; }
    public int ExpMonth { get; set; }
    public int ExpYear { get; set; }
    public string Last4 { get; set; }                  // Last 4 digits (safe to store)
    public string Fingerprint { get; set; }
    public string IssuerName { get; set; }
    public List<string> SupportedNetworks { get; set; }
    public ThreeDSecureStatus ThreeDSecure { get; set; }
    public string NetworkToken { get; set; }
}

public class BillingDetails
{
    public string Name { get; set; }
    public string Email { get; set; }
    public string Phone { get; set; }
    public Address Address { get; set; }
}

Schema Design Principles

• Immutable event log: Core entities (PaymentIntent, Charge, Refund) are append-only. State changes create new records rather than updating existing ones, providing a complete audit trail.
• Prefixed IDs: Every entity type has a unique prefix (pi_, ch_, re_, dp_, pm_, acct_) that makes it immediately identifiable and prevents cross-type ID collisions.
• Amounts in smallest currency units: All monetary amounts are stored as longs (cents, not dollars) to avoid floating-point precision issues. This is critical - .99 is stored as 1999.
• Separation of concerns: The payment flow (PaymentIntent), the financial record (Charge), the accounting record (Ledger), and the customer-facing object (PaymentMethod) are distinct entities with clear boundaries.
• Soft deletes: Financial records are never physically deleted. Instead, they are marked with a deleted_at timestamp and excluded from normal queries. This preserves the complete audit trail for compliance and debugging.

5. High-Level Architecture

The architecture of a payment processing system follows a layered approach with clear separation of concerns. Each layer has specific responsibilities and communicates with adjacent layers through well-defined interfaces. The system is designed as a set of microservices, each owning its data and communicating asynchronously where possible.

Layer Responsibilities

Multi-Region Deployment

For a payment system requiring 99.999% availability, multi-region active-active deployment is mandatory. The architecture uses a geo-distributed database cluster with automatic failover:

• US-East (Primary): Serves approximately 50% of global traffic, hosts primary database write leader
• EU-West: Serves approximately 30% of global traffic, hosts EU merchant data, acts as disaster recovery for US
• AP-Southeast: Serves approximately 20% of global traffic, hosts APAC merchant data
• Database replication: Synchronous replication within region (3 replicas), asynchronous cross-region with bounded staleness (less than 500ms lag)
• Failover: Automated with DNS-based traffic shifting, targeting less than 30 second RTO

6. Payment Flow - Tokenization, Authorization, and Capture

The payment flow is the heart of the system. Every design decision in this flow directly impacts security, reliability, and user experience. We will trace a payment from the moment a customer enters their card details through to the funds being captured and settled.

Complete Payment Lifecycle

Step 1: Tokenization

Tokenization converts sensitive card data into a non-reversible token that can be safely stored and transmitted. This is the first and most critical step in PCI compliance - it ensures that raw card data never touches your servers. The client-side SDK intercepts card data and sends it directly to the token vault, which returns a token that references the encrypted card data without ever exposing the full card number.

C#
public class TokenizationService
{
    private readonly ITokenVault _vault;
    private readonly IEncryptionService _encryption;

    public async Task<TokenizationResult> TokenizeCardAsync(
        CardTokenizationRequest request)
    {
        if (!LuhnAlgorithm.IsValid(request.CardNumber))
            return TokenizationResult.Failed("Invalid card number");

        var cardInfo = CardDetector.Detect(request.CardNumber);
        var encryptedTrackData = await _encryption.EncryptAsync(
            request.CardNumber,
            keyVersion: await _vault.GetCurrentKeyVersionAsync());

        var fingerprint = FingerprintGenerator.Generate(
            request.CardNumber, request.ExpMonth, request.ExpYear);

        var token = new TokenizedCard
        {
            Id = GeneratePrefixedId("pm_"),
            Fingerprint = fingerprint,
            EncryptedData = encryptedTrackData,
            Last4 = request.CardNumber[^4..],
            Brand = cardInfo.Brand,
            ExpMonth = request.ExpMonth,
            ExpYear = request.ExpYear,
            Funding = cardInfo.Funding,
            Country = cardInfo.Country,
            CreatedAt = DateTime.UtcNow
        };

        await _vault.StoreTokenAsync(token);
        var networkToken = await TryGetNetworkTokenAsync(cardInfo, token);
        return TokenizationResult.Success(token.Id, networkToken);
    }
}

public class LuhnAlgorithm
{
    public static bool IsValid(string cardNumber)
    {
        int sum = 0;
        bool alternate = false;
        for (int i = cardNumber.Length - 1; i >= 0; i--)
        {
            int digit = int.Parse(cardNumber[i].ToString());
            if (alternate) { digit *= 2; if (digit > 9) digit -= 9; }
            sum += digit;
            alternate = !alternate;
        }
        return sum % 10 == 0;
    }
}

Step 2: Authorization

Authorization verifies that the card is valid and has sufficient funds, placing a hold on the requested amount. This is a synchronous operation completing in 1-3 seconds.

C#
public class AuthorizationService
{
    private readonly IProcessorRouter _processorRouter;
    private readonly IFraudEngine _fraudEngine;
    private readonly IFXService _fxService;

    public async Task<AuthorizationOutcome> AuthorizeAsync(
        PaymentIntent paymentIntent)
    {
        var riskAssessment = await _fraudEngine.AssessRiskAsync(
            new RiskContext
        {
            PaymentIntentId = paymentIntent.Id,
            Amount = paymentIntent.Amount,
            Currency = paymentIntent.Currency,
            MerchantId = paymentIntent.MerchantId,
            CustomerId = paymentIntent.CustomerId,
            PaymentMethodFingerprint = paymentIntent.PaymentMethodId,
            DeviceFingerprint = paymentIntent.Metadata
                .GetValueOrDefault("device_fp"),
            IpAddress = paymentIntent.Metadata.GetValueOrDefault("ip_address")
        });

        if (riskAssessment.Level == RiskLevel.VeryHigh)
            return AuthorizationOutcome.Declined("Blocked by fraud screening");

        var (chargeCurrency, chargeAmount, fxRate) =
            await ResolveCurrencyAsync(paymentIntent);

        var processor = await _processorRouter.SelectProcessorAsync(
            paymentIntent.MerchantId, paymentIntent.PaymentMethodId,
            chargeCurrency, riskAssessment);

        var authRequest = new AuthorizationRequest
        {
            Amount = chargeAmount,
            Currency = chargeCurrency,
            PaymentToken = paymentIntent.PaymentMethodId,
            IdempotencyKey = $"auth_{paymentIntent.Id}",
            MerchantId = processor.MerchantIdentifier,
            Metadata = paymentIntent.Metadata,
            ThreeDSecure = riskAssessment.RequiresThreeDS
                ? "required" : "optional",
            RiskData = riskAssessment.ToProcessorFormat()
        };

        var authResponse = await processor.AuthorizeAsync(authRequest);

        if (authResponse.IsApproved)
            return AuthorizationOutcome.Approved(
                authResponse.AuthorizationCode,
                authResponse.NetworkTransactionId, fxRate);

        if (authResponse.IsRequiresThreeDS)
            return AuthorizationOutcome.Requires3DS(
                authResponse.RedirectUrl);

        return AuthorizationOutcome.Declined(authResponse.DeclineReason);
    }
}

Step 3: Capture

Capture converts an authorization into an actual fund transfer. The capture window varies by card network - Visa typically allows 7 days, Mastercard allows 30 days for certain merchant categories.

C#
public class CaptureService
{
    private readonly IPaymentRepository _payments;
    private readonly IProcessorRouter _processorRouter;
    private readonly ILedgerService _ledger;
    private readonly IEventPublisher _events;

    public async Task<CaptureResult> CaptureAsync(
        string paymentIntentId, long? amountToCapture = null)
    {
        var payment = await _payments.GetForUpdateAsync(paymentIntentId);
        if (payment == null)
            throw new PaymentNotFoundException(paymentIntentId);

        if (payment.Status != PaymentStatus.Authorized)
            throw new InvalidStateTransitionException(
                payment.Status, PaymentStatus.Captured);

        var authorizedAmount = payment.Charges.Last().Amount;
        var remainingAmount = authorizedAmount - payment.AmountCaptured;
        var captureAmount = amountToCapture ?? remainingAmount;

        if (captureAmount > remainingAmount)
            throw new CaptureAmountExceededException(
                captureAmount, remainingAmount);

        var processor = _processorRouter.GetProcessor(
            payment.Charges.Last().ProcessorTransactionId);
        var captureResult = await processor.CaptureAsync(
            transactionId: payment.Charges.Last().ProcessorTransactionId,
            amount: captureAmount);

        if (captureResult.IsSuccessful)
        {
            payment.AmountCaptured += captureAmount;
            payment.Status = payment.AmountCaptured == authorizedAmount
                ? PaymentStatus.Captured
                : PaymentStatus.PartiallyCaptured;
            payment.CapturedAt ??= DateTime.UtcNow;
            await _payments.UpdateAsync(payment);

            await _ledger.PostCaptureAsync(new CaptureLedgerEntry
            {
                PaymentIntentId = paymentIntentId,
                Amount = captureAmount,
                Currency = payment.Currency,
                MerchantId = payment.MerchantId,
                ProcessorFee = captureResult.Fee,
                NetworkFee = captureResult.NetworkFee
            });

            await _events.PublishAsync(new PaymentCapturedEvent
            {
                PaymentIntentId = paymentIntentId,
                AmountCaptured = captureAmount
            });

            return CaptureResult.Success(captureAmount);
        }
        return CaptureResult.Failed(captureResult.ErrorMessage);
    }
}

7. Payment Gateway Integration

A production payment system never relies on a single payment processor. The gateway integration layer provides a unified abstraction over multiple underlying processors (Stripe, Adyen, Worldpay, etc.), enabling intelligent routing, failover, and cost optimization. This adapter pattern is essential for reliability - when one processor has an outage, traffic automatically shifts to another.

Processor Router Architecture

C#
public interface IPaymentProcessor
{
    string Name { get; }
    Task<AuthorizationResponse> AuthorizeAsync(
        AuthorizationRequest request);
    Task<CaptureResponse> CaptureAsync(
        string transactionId, long amount);
    Task<RefundResponse> RefundAsync(
        string transactionId, long amount, string reason);
    Task<ProcessorStatus> GetStatusAsync(string transactionId);
    bool SupportsCurrency(string currency);
    ProcessorCapabilities Capabilities { get; }
}

public class ProcessorRouter
{
    private readonly IEnumerable<IPaymentProcessor> _processors;
    private readonly ICircuitBreakerFactory _circuitBreakers;
    private readonly IRouteSelector _routeSelector;

    public async Task<IPaymentProcessor> SelectProcessorAsync(
        string merchantId, string paymentMethodId,
        string currency, RiskAssessment risk)
    {
        var context = new RoutingContext
        {
            MerchantId = merchantId,
            Currency = currency,
            CardBrand = await GetCardBrandAsync(paymentMethodId),
            RiskLevel = risk.Level,
            Amount = risk.Amount
        };

        var candidates = await _routeSelector
            .GetRankedProcessorsAsync(context);

        var available = candidates
            .Where(p => _circuitBreakers.Get(p.Name).State
                != CircuitState.Open)
            .Where(p => p.SupportsCurrency(currency))
            .Where(p => p.Capabilities.MeetsRequirements(context))
            .ToList();

        if (available.Count == 0)
            throw new NoAvailableProcessorException(currency);

        return new FaultTolerantProcessorSelection(
            available, _circuitBreakers);
    }
}

public class FaultTolerantProcessorSelection : IPaymentProcessor
{
    private readonly List<IPaymentProcessor> _candidates;
    private readonly ICircuitBreakerFactory _circuitBreakers;

    public async Task<AuthorizationResponse> AuthorizeAsync(
        AuthorizationRequest request)
    {
        for (int i = 0; i < _candidates.Count; i++)
        {
            var processor = _candidates[i];
            var cb = _circuitBreakers.Get(processor.Name);
            try
            {
                using var _ = cb.AcquireSlot();
                var response = await processor
                    .AuthorizeAsync(request);
                cb.RecordSuccess();
                return response;
            }
            catch (Exception ex) when (IsTransient(ex))
            {
                cb.RecordFailure();
                await _routeSelector
                    .RecordFailureAsync(processor.Name, request, ex);
            }
        }
        throw new AllProcessorsFailedException(
            _candidates.Select(p => p.Name).ToList());
    }
}

public class CircuitBreaker
{
    private int _failureCount = 0;
    private DateTime _lastFailureTime = DateTime.MinValue;
    private CircuitState _state = CircuitState.Closed;
    private readonly int _failureThreshold = 5;
    private readonly TimeSpan _openDuration =
        TimeSpan.FromMinutes(1);

    public CircuitState State
    {
        get
        {
            if (_state == CircuitState.Open &&
                DateTime.UtcNow - _lastFailureTime > _openDuration)
                _state = CircuitState.HalfOpen;
            return _state;
        }
    }

    public void RecordFailure()
    {
        _failureCount++;
        _lastFailureTime = DateTime.UtcNow;
        if (_failureCount >= _failureThreshold)
            _state = CircuitState.Open;
    }

    public void RecordSuccess()
    {
        _failureCount = 0;
        _state = CircuitState.Closed;
    }
}

8. Idempotency and Exactly-Once Processing

In a distributed payment system, at-least-once delivery is the norm - networks timeout, servers crash mid-request, and clients retry. Without idempotency, these retries create duplicate charges, which are catastrophic for customer trust and financial accuracy. Idempotency ensures that performing the same operation multiple times produces the same result as performing it once.

Idempotency Key Architecture

C#
public class IdempotencyMiddleware
{
    private readonly RequestDelegate _next;
    private readonly IDistributedCache _cache;
    private readonly IDbContextFactory<PaymentDbContext> _dbFactory;

    public async Task InvokeAsync(HttpContext context)
    {
        var idempotencyKey =
            context.Request.Headers["Idempotency-Key"]
                .FirstOrDefault();

        if (string.IsNullOrEmpty(idempotencyKey))
        {
            if (context.Request.Method != "GET")
            {
                context.Response.StatusCode = 400;
                await context.Response.WriteAsync(
                    "Idempotency-Key header required");
                return;
            }
            await _next(context);
            return;
        }

        if (idempotencyKey.Length > 255 ||
            !IsValidKeyFormat(idempotencyKey))
        {
            context.Response.StatusCode = 400;
            await context.Response.WriteAsync(
                "Invalid Idempotency-Key format");
            return;
        }

        var cachedResult = await _cache.GetStringAsync(
            $"idem:{idempotencyKey}");
        if (cachedResult != null)
        {
            await WriteCachedResponse(context, cachedResult);
            return;
        }

        await using var db =
            await _dbFactory.CreateDbContextAsync();
        await using var transaction =
            await db.Database.BeginTransactionAsync();

        try
        {
            var existing = await db.IdempotencyKeys
                .FirstOrDefaultAsync(
                    k => k.Key == idempotencyKey);

            if (existing == null)
            {
                var newKey = new IdempotencyKeyRecord
                {
                    Key = idempotencyKey,
                    Status = IdempotencyStatus.InProgress,
                    CreatedAt = DateTime.UtcNow,
                    ExpiresAt = DateTime.UtcNow.AddDays(24),
                    RequestHash = HashRequest(context.Request)
                };
                db.IdempotencyKeys.Add(newKey);
                await db.SaveChangesAsync();
                await transaction.CommitAsync();

                var responseBody =
                    await CaptureResponseAsync(context, _next);

                await StoreResultAsync(idempotencyKey,
                    responseBody, IdempotencyStatus.Completed);
                await _cache.SetStringAsync(
                    $"idem:{idempotencyKey}", responseBody,
                    new DistributedCacheEntryOptions
                    {
                        AbsoluteExpirationRelativeToNow =
                            TimeSpan.FromHours(24)
                    });
            }
            else if (existing.Status ==
                IdempotencyStatus.Completed)
            {
                await transaction.RollbackAsync();
                await WriteCachedResponse(
                    context, existing.ResponseBody);
            }
            else
            {
                await transaction.RollbackAsync();
                context.Response.StatusCode = 409;
                await context.Response.WriteAsJsonAsync(
                    new { error = new
                    {
                        message = "Request in progress",
                        type = "conflict"
                    }});
            }
        }
        catch
        {
            await transaction.RollbackAsync();
            throw;
        }
    }
}

public class IdempotencyKeyRecord
{
    public string Key { get; set; }
    public IdempotencyStatus Status { get; set; }
    public string RequestHash { get; set; }
    public string ResponseBody { get; set; }
    public string PaymentIntentId { get; set; }
    public DateTime CreatedAt { get; set; }
    public DateTime? CompletedAt { get; set; }
    public DateTime ExpiresAt { get; set; }
}

public enum IdempotencyStatus
{
    InProgress,
    Completed,
    Failed
}

9. Ledger and Double-Entry Bookkeeping

The ledger is the financial backbone of the payment system. It must be an immutable, append-only record of every financial event, following the principles of double-entry bookkeeping. Every financial transaction must have equal debits and credits - if money moves from one account, it must arrive somewhere else. The ledger must be independently auditable and must survive any single infrastructure failure.

Double-Entry Ledger Design

C#
public class LedgerEntry
{
    public long Id { get; set; }
    public string EntryId { get; set; }
    public string TransactionId { get; set; }
    public LedgerEntryType Type { get; set; }
    public AccountId DebitAccount { get; set; }
    public AccountId CreditAccount { get; set; }
    public long Amount { get; set; }
    public string Currency { get; set; }
    public string PaymentIntentId { get; set; }
    public string MerchantId { get; set; }
    public DateTime PostedAt { get; set; }
    public DateTime? ReconciledAt { get; set; }
    public string PreviousEntryHash { get; set; }
    public string EntryHash { get; set; }
}

public class AccountId
{
    public string AccountNumber { get; set; }
    public AccountType Type { get; set; }
    public string Currency { get; set; }
}

public enum AccountType
{
    Assets, Liabilities, Revenue, Expenses, Equity
}

public enum LedgerEntryType
{
    PaymentAuthorization, PaymentCapture, Refund,
    Chargeback, ChargebackReversal, Payout,
    PayoutReversal, PlatformFee, ProcessingFee,
    NetworkFee, Reserve, ReserveRelease,
    CurrencyConversion
}

Ledger Posting for Payment Capture

C#
public class LedgerService
{
    private readonly ILedgerStore _store;
    private readonly IEventPublisher _events;

    public async Task<LedgerResult> PostCaptureAsync(
        CaptureLedgerEntry entry)
    {
        var txnId = $"txn_{entry.PaymentIntentId}" +
            $"_{DateTime.UtcNow.Ticks}";

        var entries = new List<LedgerEntry>
        {
            new LedgerEntry
            {
                EntryId = GenerateULID(),
                TransactionId = txnId,
                Type = LedgerEntryType.PaymentCapture,
                DebitAccount = new AccountId
                {
                    AccountNumber =
                        $"processor:stripe:receivable:" +
                        $"{entry.Currency}",
                    Type = AccountType.Assets,
                    Currency = entry.Currency
                },
                CreditAccount = new AccountId
                {
                    AccountNumber =
                        $"merchant:{entry.MerchantId}:" +
                        $"available:{entry.Currency}",
                    Type = AccountType.Liabilities,
                    Currency = entry.Currency
                },
                Amount = entry.Amount - entry.ProcessorFee
                    - entry.NetworkFee,
                Currency = entry.Currency,
                PaymentIntentId = entry.PaymentIntentId,
                MerchantId = entry.MerchantId,
                PostedAt = DateTime.UtcNow
            },
            new LedgerEntry
            {
                EntryId = GenerateULID(),
                TransactionId = txnId,
                Type = LedgerEntryType.ProcessingFee,
                DebitAccount = new AccountId
                {
                    AccountNumber =
                        $"merchant:{entry.MerchantId}:" +
                        $"available:{entry.Currency}",
                    Type = AccountType.Liabilities,
                    Currency = entry.Currency
                },
                CreditAccount = new AccountId
                {
                    AccountNumber =
                        $"revenue:processing_fees:" +
                        $"{entry.Currency}",
                    Type = AccountType.Revenue,
                    Currency = entry.Currency
                },
                Amount = entry.ProcessorFee,
                Currency = entry.Currency,
                PaymentIntentId = entry.PaymentIntentId,
                MerchantId = entry.MerchantId,
                PostedAt = DateTime.UtcNow
            },
            new LedgerEntry
            {
                EntryId = GenerateULID(),
                TransactionId = txnId,
                Type = LedgerEntryType.NetworkFee,
                DebitAccount = new AccountId
                {
                    AccountNumber =
                        $"merchant:{entry.MerchantId}:" +
                        $"available:{entry.Currency}",
                    Type = AccountType.Liabilities,
                    Currency = entry.Currency
                },
                CreditAccount = new AccountId
                {
                    AccountNumber =
                        $"revenue:network_fees:" +
                        $"{entry.Currency}",
                    Type = AccountType.Revenue,
                    Currency = entry.Currency
                },
                Amount = entry.NetworkFee,
                Currency = entry.Currency,
                PaymentIntentId = entry.PaymentIntentId,
                MerchantId = entry.MerchantId,
                PostedAt = DateTime.UtcNow
            }
        };

        VerifyDoubleEntryBalance(entries);

        var previousHash =
            await _store.GetLatestHashAsync(entry.MerchantId);
        foreach (var e in entries)
        {
            e.PreviousEntryHash = previousHash;
            e.EntryHash = ComputeEntryHash(e);
            previousHash = e.EntryHash;
        }

        await _store.AppendEntriesAsync(entries);

        await _events.PublishAsync(new LedgerPostedEvent
        {
            TransactionId = txnId,
            Entries = entries.Select(e => e.EntryId).ToList()
        });

        return LedgerResult.Success(txnId, entries.Count);
    }

    private void VerifyDoubleEntryBalance(
        List<LedgerEntry> entries)
    {
        var debits = entries
            .GroupBy(e => e.Currency)
            .ToDictionary(g => g.Key,
                g => g.Sum(e => e.Amount));
        var credits = entries
            .GroupBy(e => e.Currency)
            .ToDictionary(g => g.Key,
                g => g.Sum(e => e.Amount));

        foreach (var currency in debits.Keys)
        {
            if (debits[currency] != credits[currency])
                throw new LedgerBalanceException(
                    $"Violation for {currency}");
        }
    }
}

Ledger Account Hierarchy

10. Reconciliation System

Reconciliation is the process of comparing internal records against external sources to ensure they match. In a payment system, this means matching our ledger entries against processor settlement reports, bank statements, and card network settlement files. Discrepancies must be detected quickly - ideally in real-time - and investigated before they compound into larger financial issues.

Reconciliation Pipeline Architecture

C#
public class ReconciliationService
{
    private readonly ILedgerService _ledger;
    private readonly IProcessorReportFetcher _processorReports;
    private readonly IReconciliationStore _store;
    private readonly IAlertService _alerts;

    public async Task<ReconciliationResult> ReconcileAsync(
        string processorName, DateOnly date,
        CancellationToken ct = default)
    {
        var result = new ReconciliationResult
        {
            Processor = processorName,
            Date = date,
            StartedAt = DateTime.UtcNow
        };

        var processorRecords = await _processorReports
            .FetchSettlementAsync(processorName, date);

        var internalRecords = await _ledger
            .GetSettlementRecordsAsync(
                processorName,
                date.StartOfDay(),
                date.EndOfDay());

        var internalByTxnId = internalRecords
            .ToDictionary(
                r => r.ProcessorTransactionId, r => r);

        var matchedIds = new HashSet<string>();

        foreach (var procRec in processorRecords)
        {
            if (internalByTxnId.TryGetValue(
                procRec.TransactionId, out var intRec))
            {
                var match = CompareRecords(procRec, intRec);
                if (match.IsExactMatch)
                {
                    matchedIds.Add(procRec.TransactionId);
                    result.ExactMatches++;
                    continue;
                }

                result.Discrepancies.Add(new Discrepancy
                {
                    Type = match.AmountMismatch
                        ? DiscrepancyType.AmountMismatch
                        : DiscrepancyType.StatusMismatch,
                    ProcessorRecord = procRec,
                    InternalRecord = intRec,
                    Difference =
                        procRec.Amount - intRec.Amount,
                    Severity = CalculateSeverity(match)
                });
            }
            else
            {
                result.Discrepancies.Add(new Discrepancy
                {
                    Type =
                        DiscrepancyType.MissingInternal,
                    ProcessorRecord = procRec,
                    Severity = DiscrepancySeverity.High
                });
            }
        }

        foreach (var kvp in internalByTxnId)
        {
            if (!matchedIds.Contains(kvp.Key))
            {
                result.Discrepancies.Add(new Discrepancy
                {
                    Type =
                        DiscrepancyType.MissingProcessor,
                    InternalRecord = kvp.Value,
                    Severity = DiscrepancySeverity.Medium
                });
            }
        }

        var autoResolved =
            await AutoResolveDiscrepanciesAsync(
                result.Discrepancies);
        result.AutoResolved = autoResolved;

        var critical = result.Discrepancies
            .Where(d => d.Severity ==
                DiscrepancySeverity.Critical)
            .ToList();

        if (critical.Any())
        {
            await _alerts.SendCriticalAlertAsync(
                new ReconciliationAlert
            {
                Processor = processorName,
                Date = date,
                CriticalCount = critical.Count,
                TotalDifference = critical
                    .Sum(d => Math.Abs(d.Difference))
            });
        }

        result.CompletedAt = DateTime.UtcNow;
        await _store.SaveResultAsync(result);
        return result;
    }
}

11. Fraud Detection and Risk Scoring

Fraud detection is a continuous arms race. Payment systems must balance two competing objectives: approving legitimate transactions (minimizing false declines) and blocking fraudulent ones (minimizing false positives). A false decline costs you the sale and potentially the customer; a false positive lets fraud through. The best systems achieve both goals by combining rule-based checks, machine learning models, and behavioral analysis.

Fraud Detection Pipeline

C#
public class FraudDetectionEngine
{
    private readonly IRuleEngine _rules;
    private readonly IMLModelClient _mlModel;
    private readonly IGraphAnalyzer _graphAnalyzer;
    private readonly IVelocityTracker _velocityTracker;
    private readonly IIPIntelligence _ipIntel;

    public async Task<RiskAssessment> AssessRiskAsync(
        RiskContext context)
    {
        var signals = new List<RiskSignal>();

        // Rule-based checks (less than 5ms)
        var ruleResults =
            await _rules.EvaluateAsync(context);
        signals.AddRange(ruleResults);

        // Velocity checks (less than 10ms)
        var velocity =
            await _velocityTracker.CheckVelocityAsync(
                new VelocityQuery
        {
            CustomerId = context.CustomerId,
            CardFingerprint =
                context.PaymentMethodFingerprint,
            MerchantId = context.MerchantId,
            IpAddress = context.IpAddress,
            TimeWindows = new[] { "1h", "24h", "7d", "30d" }
        });
        signals.AddRange(velocity.ToRiskSignals());

        // IP intelligence (less than 20ms)
        var ipAnalysis =
            await _ipIntel.AnalyzeAsync(context.IpAddress);
        if (ipAnalysis.IsProxy || ipAnalysis.IsVPN)
            signals.Add(new RiskSignal(
                "network:proxy_detected", 0.15m));
        if (ipAnalysis.Country != context.ExpectedCountry)
            signals.Add(new RiskSignal(
                "geo:mismatch", 0.20m));

        // ML model scoring (less than 50ms)
        var mlFeatures = BuildFeatureVector(
            context, signals, velocity, ipAnalysis);
        var mlScore =
            await _mlModel.PredictAsync(mlFeatures);

        // Graph analysis (less than 100ms, async)
        var graphScore =
            await _graphAnalyzer.AnalyzeAsync(
                new GraphQuery
        {
            CardFingerprint =
                context.PaymentMethodFingerprint,
            DeviceFingerprint =
                context.DeviceFingerprint,
            MerchantId = context.MerchantId,
            IpSubnet = ipAnalysis.Subnet
        });

        // Ensemble scoring
        var finalScore = ComputeEnsembleScore(
            ruleResults, mlScore, graphScore,
            velocity.ToRiskSignals(),
            new[] { ipAnalysis });

        var riskLevel = finalScore switch
        {
            < 0.20m => RiskLevel.Low,
            < 0.50m => RiskLevel.Medium,
            < 0.80m => RiskLevel.High,
            _ => RiskLevel.VeryHigh
        };

        return new RiskAssessment
        {
            Score = finalScore,
            Level = riskLevel,
            TriggeredRules = ruleResults
                .Where(r => r.Triggered)
                .Select(r => r.RuleId).ToList(),
            RequiresThreeDS =
                riskLevel == RiskLevel.Medium ||
                riskLevel == RiskLevel.High,
            FraudCheck = new FraudCheckResult
            {
                MLScore = mlScore,
                GraphScore = graphScore,
                VelocityFlags = velocity.Flags,
                IPAnalysis = ipAnalysis
            }
        };
    }
}

Key Fraud Signals

12. PCI DSS Compliance and Security

PCI DSS (Payment Card Industry Data Security Standard) is a mandatory compliance framework for any entity that stores, processes, or transmits cardholder data. For a payment processing platform, PCI DSS Level 1 compliance is not optional - it is a prerequisite for doing business with card networks.

PCI DSS Requirements for Payment Systems

Token Vault Architecture

C#
public class SecureTokenVault
{
    private readonly IKeyManagementService _kms;
    private readonly IEncryptionEngine _encryption;
    private readonly IAuditLogger _auditLog;

    public async Task<TokenizedCard> StoreCardAsync(
        RawCardData cardData)
    {
        var keyVersion =
            await _kms.GetCurrentKeyVersionAsync();

        var encryptedPan = await _encryption.EncryptAsync(
            plaintext: cardData.CardNumber,
            keyVersion: keyVersion,
            associatedData: Encoding.UTF8.GetBytes(
                cardData.MerchantId));

        var fingerprint =
            await ComputeFingerprintAsync(
                cardData.CardNumber);

        var token = new TokenizedCard
        {
            Id = GenerateSecureId("pm_"),
            Fingerprint = fingerprint,
            EncryptedPan = encryptedPan,
            KeyVersion = keyVersion,
            Last4 = cardData.CardNumber[^4..],
            ExpMonth = cardData.ExpMonth,
            ExpYear = cardData.ExpYear,
            CreatedAt = DateTime.UtcNow
        };

        await _vaultRepository.InsertAsync(token);

        await _auditLog.LogAsync(new AuditEntry
        {
            Action = "card_tokenized",
            ResourceId = token.Id,
            MerchantId = cardData.MerchantId,
            Timestamp = DateTime.UtcNow,
            Details = $"Card tokenized: ****{token.Last4}"
        });

        return token;
    }

    public async Task RotateKeysAsync()
    {
        var newKeyVersion =
            await _kms.RotateKeyAsync();

        await _backgroundJobs.EnqueueAsync(
            new KeyRotationJob
        {
            NewKeyVersion = newKeyVersion,
            StartedAt = DateTime.UtcNow,
            BatchSize = 10000
        });

        await _auditLog.LogAsync(new AuditEntry
        {
            Action = "key_rotation_initiated",
            Details = $"New key: {newKeyVersion}",
            Timestamp = DateTime.UtcNow
        });
    }

    private async Task<int> ComputeFingerprintAsync(
        string pan)
    {
        var hmacKey =
            await _kms.GetFingerprintKeyAsync();
        var hmac = new HMACSHA256(hmacKey);
        var hash = hmac.ComputeHash(
            Encoding.UTF8.GetBytes(pan));
        return BitConverter.ToInt32(hash, 0);
    }
}

13. Subscription and Recurring Billing

Subscription billing adds significant complexity to a payment system. Unlike one-time payments, subscriptions require managing time-based state machines, handling plan changes with proration, dealing with failed payment retries (dunning), and supporting complex billing models (usage-based, tiered, per-seat). The subscription engine must be reliable enough to handle millions of recurring charges without missing a beat.

Subscription State Machine

C#
public class Subscription
{
    public string Id { get; set; }
    public string CustomerId { get; set; }
    public string MerchantId { get; set; }
    public string PriceId { get; set; }
    public SubscriptionStatus Status { get; set; }
    public BillingCycle Cycle { get; set; }
    public DateTime CurrentPeriodStart { get; set; }
    public DateTime CurrentPeriodEnd { get; set; }
    public DateTime? TrialEnd { get; set; }
    public DateTime? CanceledAt { get; set; }
    public DateTime? CancelAt { get; set; }
    public string DefaultPaymentMethodId { get; set; }
    public int FailedPaymentAttempts { get; set; }
    public DateTime? NextRetryAt { get; set; }
    public SubscriptionItem[] Items { get; set; }
    public Dictionary<string, string> Metadata { get; set; }
}

public class SubscriptionBillingEngine
{
    private readonly ISubscriptionRepository _subscriptions;
    private readonly IPaymentService _payments;
    private readonly IInvoiceService _invoices;
    private readonly IProrationEngine _proration;
    private readonly IDunningManager _dunning;

    public async Task<BillingResult>
        ProcessRecurringBillingAsync()
    {
        var dueSubscriptions =
            await _subscriptions.GetDueForBillingAsync(
                before: DateTime.UtcNow,
                status: new[] { SubscriptionStatus.Active },
                limit: 10000);

        var results = new List<BillingResult>();
        var batches = dueSubscriptions.Chunk(100);
        var batchTasks = batches.Select(
            batch => ProcessBatchAsync(batch));
        var batchResults =
            await Task.WhenAll(batchTasks);

        return BillingResult.Aggregate(batchResults);
    }

    private async Task<BillingResult> ProcessBatchAsync(
        Subscription[] batch)
    {
        var result = new BillingResult();

        foreach (var subscription in batch)
        {
            try
            {
                var invoice =
                    await _invoices
                        .CreateForSubscriptionAsync(
                            subscription);

                var paymentResult =
                    await _payments.ChargeAsync(
                        new ChargeRequest
                {
                    Amount = invoice.AmountDue,
                    Currency = invoice.Currency,
                    CustomerId = subscription.CustomerId,
                    PaymentMethodId =
                        subscription.DefaultPaymentMethodId,
                    IdempotencyKey =
                        $"sub_{subscription.Id}_" +
                        $"{subscription.CurrentPeriodEnd:yyyyMMdd}",
                    Metadata = new Dictionary<string, string>
                    {
                        ["subscription_id"] =
                            subscription.Id,
                        ["invoice_id"] = invoice.Id
                    }
                });

                if (paymentResult.IsSuccessful)
                {
                    await AdvanceSubscriptionAsync(
                        subscription);
                    await _invoices.MarkPaidAsync(
                        invoice,
                        paymentResult.PaymentIntentId);
                    result.Successful++;
                }
                else
                {
                    await _dunning
                        .HandleFailedPaymentAsync(
                            subscription,
                            paymentResult, invoice);
                    result.Failed++;
                }
            }
            catch (Exception ex)
            {
                result.Errors.Add(
                    new BillingError(
                        subscription.Id, ex.Message));
            }
        }

        return result;
    }

    public async Task<PlanChangeResult> ChangePlanAsync(
        string subscriptionId,
        string newPriceId,
        ProrationBehavior proration)
    {
        var subscription =
            await _subscriptions.GetByIdAsync(subscriptionId);
        var newPrice =
            await _prices.GetByIdAsync(newPriceId);

        var prorationAmount = proration switch
        {
            ProrationBehavior.CreateProrations =>
                await _proration.CalculateCreditAsync(
                    subscription, DateTime.UtcNow),
            ProrationBehavior.None => 0,
            _ => throw new ArgumentOutOfRangeException(
                nameof(proration))
        };

        subscription.PriceId = newPriceId;
        subscription.Items =
            BuildItemsFromPrice(newPrice);
        await _subscriptions.UpdateAsync(subscription);

        if (prorationAmount < 0)
        {
            await _invoices.CreateCreditAsync(
                subscription,
                Math.Abs(prorationAmount));
        }
        else if (prorationAmount > 0)
        {
            await _payments.ChargeAsync(
                new ChargeRequest
            {
                Amount = prorationAmount,
                Currency = subscription.Currency,
                CustomerId = subscription.CustomerId,
                PaymentMethodId =
                    subscription.DefaultPaymentMethodId,
                IdempotencyKey =
                    $"proration_{subscriptionId}_" +
                    $"{DateTime.UtcNow.Ticks}"
            });
        }

        return PlanChangeResult.Success(
            subscription, prorationAmount);
    }
}

public enum SubscriptionStatus
{
    Active, PastDue, Unpaid, Canceled,
    Paused, Trialing
}

public enum ProrationBehavior
{
    CreateProrations,
    None
}

Dunning (Failed Payment Recovery)

Dunning is the automated process of retrying failed recurring payments and communicating with customers about payment issues. A well-designed dunning system maximizes revenue recovery while avoiding customer harassment. The retry schedule is carefully tuned based on the failure reason - insufficient funds may resolve quickly, while a lost card requires customer action.

14. Multi-Currency and Foreign Exchange

Supporting multi-currency payments is essential for a global payment platform. The system must handle 135+ currencies, each with different formatting rules (JPY has no decimal places, BHD has 3), different minimum transaction amounts, and different settlement requirements. Foreign exchange adds another layer of complexity - rates change constantly, and the system must handle both real-time conversion at payment time and batch conversion at settlement time.

C#
public class CurrencyService
{
    private readonly IFXRateProvider _rateProvider;
    private readonly ICurrencyConfig _config;

    private static readonly HashSet<string>
        ZeroDecimalCurrencies = new()
    {
        "JPY", "KRW", "VND", "CLP", "ISK",
        "UGX", "RWF", "VUV", "XAF", "XOF",
        "XPF", "BIF", "DJF", "GNF", "KMF",
        "MGA", "PYG", "TWD"
    };

    private static readonly HashSet<string>
        ThreeDecimalCurrencies = new()
    {
        "BHD", "KWD", "OMR", "JOD", "LYD",
        "TND", "IQD"
    };

    public long ToSmallestUnit(
        decimal amount, string currency)
    {
        var cur = currency.ToUpperInvariant();
        return cur switch
        {
            _ when ZeroDecimalCurrencies.Contains(cur)
                => (long)Math.Round(amount),
            _ when ThreeDecimalCurrencies.Contains(cur)
                => (long)Math.Round(amount * 1000),
            _ => (long)Math.Round(amount * 100)
        };
    }

    public decimal FromSmallestUnit(
        long amount, string currency)
    {
        var cur = currency.ToUpperInvariant();
        return cur switch
        {
            _ when ZeroDecimalCurrencies.Contains(cur)
                => amount,
            _ when ThreeDecimalCurrencies.Contains(cur)
                => amount / 1000m,
            _ => amount / 100m
        };
    }

    public async Task<ConversionResult> ConvertAsync(
        long amount, string fromCurrency,
        string toCurrency,
        ConversionType type = ConversionType.Payment)
    {
        if (fromCurrency == toCurrency)
            return ConversionResult.NoConversion(amount);

        var rate = type switch
        {
            ConversionType.Payment =>
                await _rateProvider
                    .GetPaymentRateAsync(
                        fromCurrency, toCurrency),
            ConversionType.Settlement =>
                await _rateProvider
                    .GetSettlementRateAsync(
                        fromCurrency, toCurrency),
            ConversionType.Display =>
                await _rateProvider
                    .GetDisplayRateAsync(
                        fromCurrency, toCurrency),
            _ => throw new ArgumentOutOfRangeException(
                nameof(type))
        };

        var convertedAmount =
            (long)Math.Round(amount * rate.Factor);

        var minAmount = _config
            .GetMinimumChargeAmount(toCurrency);
        if (convertedAmount < minAmount)
            throw new AmountBelowMinimumException(
                toCurrency, convertedAmount, minAmount);

        return ConversionResult.Converted(
            originalAmount: amount,
            convertedAmount: convertedAmount,
            rate: rate.Value,
            rateTimestamp: rate.Timestamp,
            markup: rate.Markup);
    }
}

public class FXRateProvider : IFXRateProvider
{
    private readonly IMarketDataClient _marketData;
    private readonly IDistributedCache _cache;

    public async Task<FXRate> GetPaymentRateAsync(
        string from, string to)
    {
        var cacheKey = $"fx:payment:{from}:{to}";
        var cached =
            await _cache.GetAsync<FXRate>(cacheKey);
        if (cached != null) return cached;

        var rates = await Task.WhenAll(
            _marketData.GetRateAsync(from, to, "reuters"),
            _marketData.GetRateAsync(from, to, "ecb"),
            _marketData.GetRateAsync(from, to, "bloomberg")
        );

        var medianRate = rates
            .OrderBy(r => r.Value).ToArray()[1];

        var markup = 0.01m;
        var finalRate = new FXRate
        {
            BaseCurrency = from,
            QuoteCurrency = to,
            Value = medianRate.Value * (1 + markup),
            Markup = markup,
            Timestamp = DateTime.UtcNow,
            Source = "composite"
        };

        await _cache.SetAsync(cacheKey, finalRate,
            TimeSpan.FromSeconds(60));
        return finalRate;
    }
}

15. Refund and Dispute Management

Refunds and disputes are an inevitable part of any payment system. A well-designed refund system must handle partial refunds, full refunds, and the complex dispute (chargeback) process mandated by card networks. The dispute process involves strict timelines, specific evidence formats, and significant financial implications - losing a dispute means losing both the money and paying additional fees.

Refund and Dispute Lifecycle

C#
public class DisputeManagementService
{
    private readonly IDisputeRepository _disputes;
    private readonly IPaymentRepository _payments;
    private readonly IProcessorRouter _processors;
    private readonly ILedgerService _ledger;
    private readonly IEventPublisher _events;
    private readonly IAlertService _alerts;

    public async Task<DisputeResponse>
        HandleIncomingChargebackAsync(
            ChargebackNotification notification)
    {
        var payment = await _payments
            .GetByProcessorTransactionIdAsync(
                notification.TransactionId);
        if (payment == null)
            return DisputeResponse
                .Failed("Unknown transaction");

        var dispute = new Dispute
        {
            Id = GeneratePrefixedId("dp_"),
            PaymentIntentId = payment.Id,
            ChargeId = payment.Charges.Last().Id,
            Amount = notification.DisputedAmount,
            Currency = notification.Currency,
            Reason = MapDisputeReason(notification.Reason),
            Status = DisputeStatus.NeedsResponse,
            CreatedAt = DateTime.UtcNow,
            EvidenceDueBy = notification.DueByDate,
            ProcessorDisputeId = notification.DisputeId
        };

        await _disputes.CreateAsync(dispute);

        await _ledger.PostDisputeAsync(
            new DisputeLedgerEntry
        {
            DisputeId = dispute.Id,
            PaymentIntentId = payment.Id,
            MerchantId = payment.MerchantId,
            Amount = dispute.Amount,
            Currency = dispute.Currency,
            Type = LedgerEntryType.Chargeback
        });

        await _events.PublishAsync(
            new DisputeCreatedEvent
        {
            DisputeId = dispute.Id,
            PaymentIntentId = payment.Id,
            Amount = dispute.Amount,
            Reason = dispute.Reason,
            EvidenceDueBy = dispute.EvidenceDueBy
        });

        await _alerts.ScheduleEvidenceReminderAsync(
            dispute);

        return DisputeResponse.Success(dispute);
    }

    public async Task<EvidenceSubmissionResult>
        SubmitEvidenceAsync(
            string disputeId, Evidence evidence)
    {
        var dispute =
            await _disputes.GetByIdAsync(disputeId);

        if (DateTime.UtcNow > dispute.EvidenceDueBy)
            throw new EvidenceDeadlinePassedException(
                dispute.EvidenceDueBy);

        var validation =
            ValidateEvidence(evidence, dispute.Reason);
        if (!validation.IsValid)
            return EvidenceSubmissionResult.Incomplete(
                validation.MissingFields);

        var formattedEvidence =
            FormatEvidenceForNetwork(
                evidence, dispute.Reason);

        var processor = _processors
            .GetProcessorForDispute(
                dispute.ProcessorDisputeId);
        var submission =
            await processor.SubmitEvidenceAsync(
                disputeId: dispute.ProcessorDisputeId,
                evidence: formattedEvidence);

        dispute.Status =
            DisputeStatus.UnderReview;
        await _disputes.UpdateAsync(dispute);

        return EvidenceSubmissionResult.Success(
            submission.ReferenceId);
    }

    public async Task<DisputeResolutionResult>
        ResolveDisputeAsync(
            string disputeId, DisputeOutcome outcome)
    {
        var dispute =
            await _disputes.GetByIdAsync(disputeId);
        var payment = await _payments
            .GetByIdAsync(dispute.PaymentIntentId);

        switch (outcome)
        {
            case DisputeOutcome.Won:
                dispute.Status = DisputeStatus.Won;
                dispute.ResolvedAt = DateTime.UtcNow;
                await _ledger
                    .PostDisputeReversalAsync(
                        new DisputeReversalEntry
                {
                    DisputeId = disputeId,
                    PaymentIntentId = payment.Id,
                    MerchantId = payment.MerchantId,
                    Amount = dispute.Amount,
                    Currency = dispute.Currency,
                    Type =
                        LedgerEntryType.ChargebackReversal
                });
                break;

            case DisputeOutcome.Lost:
                dispute.Status = DisputeStatus.Lost;
                dispute.ResolvedAt = DateTime.UtcNow;
                var fee = 1500;
                await _ledger.PostFeeAsync(new FeeEntry
                {
                    PaymentIntentId = payment.Id,
                    MerchantId = payment.MerchantId,
                    Amount = fee,
                    Currency = dispute.Currency,
                    FeeType = "chargeback_fee"
                });
                break;
        }

        await _disputes.UpdateAsync(dispute);
        return DisputeResolutionResult.Success(dispute);
    }
}

16. Webhooks and Event Delivery

Webhooks are the primary mechanism by which payment systems notify merchants about payment events in real-time. When a payment succeeds, a refund is processed, or a dispute is filed, the merchant system needs to know immediately. Webhook delivery must be reliable - a missed webhook means a merchant does not update their order status, leading to a poor customer experience. The system must guarantee at-least-once delivery with ordered events per resource.

Webhook Delivery Architecture

C#
public class WebhookDeliveryService
{
    private readonly IWebhookEndpointStore _endpoints;
    private readonly ISignatureService _signatures;
    private readonly IHttpClientFactory _httpClientFactory;
    private readonly IEventStore _eventStore;
    private readonly IDeliveryLogStore _deliveryLog;
    private readonly ILogger<WebhookDeliveryService> _logger;

    private const int MaxRetries = 6;
    private static readonly TimeSpan[] RetryDelays =
        new[]
    {
        TimeSpan.FromSeconds(1),
        TimeSpan.FromSeconds(5),
        TimeSpan.FromSeconds(30),
        TimeSpan.FromMinutes(5),
        TimeSpan.FromMinutes(30),
        TimeSpan.FromHours(2)
    };

    public async Task DeliverEventAsync(
        WebhookEvent webhookEvent)
    {
        var endpoints = await _endpoints
            .GetSubscribedEndpointsAsync(
                webhookEvent.MerchantId,
                webhookEvent.EventType);

        var deliveryTasks = endpoints.Select(
            endpoint => DeliverToEndpointAsync(
                webhookEvent, endpoint, 0));

        await Task.WhenAll(deliveryTasks);
    }

    private async Task DeliverToEndpointAsync(
        WebhookEvent webhookEvent,
        WebhookEndpoint endpoint,
        int attempt)
    {
        var payload = JsonSerializer.Serialize(new
        {
            id = webhookEvent.Id,
            type = webhookEvent.EventType,
            created = webhookEvent.CreatedAt
                .ToUnixTimeSeconds(),
            data = new
            {
                @object = webhookEvent.Data
            }
        });

        var signature = _signatures.ComputeSignature(
            payload: payload,
            secret: endpoint.SigningSecret,
            timestamp: webhookEvent.CreatedAt);

        if (!await CheckRateLimitAsync(endpoint.Id))
        {
            await RequeueForLaterAsync(
                webhookEvent, endpoint,
                attempt, RetryDelays[attempt]);
            return;
        }

        try
        {
            var client =
                _httpClientFactory.CreateClient();
            client.Timeout = TimeSpan.FromSeconds(30);

            var request = new HttpRequestMessage(
                HttpMethod.Post, endpoint.Url)
            {
                Content = new StringContent(
                    payload, Encoding.UTF8,
                    "application/json")
            };
            request.Headers.Add(
                "Webhook-Id", webhookEvent.Id);
            request.Headers.Add(
                "Webhook-Timestamp",
                webhookEvent.CreatedAt
                    .ToUnixTimeSeconds().ToString());
            request.Headers.Add(
                "Webhook-Signature",
                $"v1,{signature}");

            var response = await client
                .SendAsync(request);

            await _deliveryLog.LogAsync(
                new DeliveryAttempt
            {
                WebhookEventId = webhookEvent.Id,
                EndpointId = endpoint.Id,
                Attempt = attempt,
                ResponseCode =
                    (int)response.StatusCode,
                Success =
                    response.IsSuccessStatusCode,
                Timestamp = DateTime.UtcNow
            });

            if (response.IsSuccessStatusCode)
                return;

            if (IsNonRetryable(response.StatusCode))
            {
                await DisableEndpointAsync(
                    endpoint.Id,
                    $"Non-retryable: " +
                    $"{response.StatusCode}");
                return;
            }
        }
        catch (Exception ex)
        {
            _logger.LogError(ex,
                "Webhook delivery failed");
        }

        if (attempt < MaxRetries - 1)
        {
            await RequeueForLaterAsync(
                webhookEvent, endpoint,
                attempt + 1,
                RetryDelays[attempt + 1]);
        }
        else
        {
            await MoveToDeadLetterAsync(
                webhookEvent, endpoint);
        }
    }

    private bool IsNonRetryable(
        HttpStatusCode statusCode)
    {
        return statusCode is
            HttpStatusCode.BadRequest or
            HttpStatusCode.Unauthorized or
            HttpStatusCode.Forbidden or
            HttpStatusCode.Gone;
    }
}

public class WebhookEvent
{
    public string Id { get; set; }
    public string MerchantId { get; set; }
    public string EventType { get; set; }
    public DateTime CreatedAt { get; set; }
    public object Data { get; set; }
}

public class WebhookEndpoint
{
    public string Id { get; set; }
    public string MerchantId { get; set; }
    public string Url { get; set; }
    public string[] EnabledEvents { get; set; }
    public string SigningSecret { get; set; }
    public EndpointStatus Status { get; set; }
}

17. Payment Method Management

Modern payment platforms must support a wide variety of payment methods beyond just credit cards. Different regions have strong preferences - iDEAL in the Netherlands, Boleto in Brazil, UPI in India, Alipay in China. Each method has different authorization flows, settlement timelines, and refund capabilities. The payment method management system provides a unified abstraction while respecting each method unique characteristics.

Payment Method Abstraction

C#
public interface IPaymentMethodHandler
{
    PaymentMethodType SupportedType { get; }
    bool SupportsCurrency(string currency);
    bool SupportsRecurring { get; }
    bool SupportsRefund { get; }
    bool SupportsPartialCapture { get; }
    PaymentMethodCapabilities Capabilities { get; }
    Task<TokenizationResult> TokenizeAsync(
        TokenizationRequest request);
    Task<AuthorizationResult> AuthorizeAsync(
        AuthorizationContext context);
    Task<ConfirmationResult> ConfirmAsync(
        ConfirmationContext context);
    PaymentFlowType FlowType { get; }
}

public class CardPaymentMethodHandler :
    IPaymentMethodHandler
{
    public PaymentMethodType SupportedType =>
        PaymentMethodType.Card;
    public bool SupportsRecurring => true;
    public bool SupportsRefund => true;
    public bool SupportsPartialCapture => true;
    public PaymentFlowType FlowType =>
        PaymentFlowType.Synchronous;

    public PaymentMethodCapabilities Capabilities =>
        new()
    {
        MaxAmount = 999999999,
        SupportedCurrencies = new[]
        {
            "USD", "EUR", "GBP", "JPY"
        },
        Requires3DSecure = true,
        AverageAuthorizationTime =
            TimeSpan.FromSeconds(2)
    };

    public async Task<AuthorizationResult> AuthorizeAsync(
        AuthorizationContext context)
    {
        if (context.Requires3DS)
        {
            return AuthorizationResult
                .RequiresThreeDSecure(
                    threeDSecureUrl:
                        await Generate3DSUrlAsync(context),
                    pareq:
                        await GeneratePaReqAsync(context));
        }
        return await ProcessCardAuthorizationAsync(
            context);
    }
}

public class IDEALHandler : IPaymentMethodHandler
{
    public PaymentMethodType SupportedType =>
        PaymentMethodType.LocalPaymentMethod;
    public bool SupportsRecurring => false;
    public bool SupportsRefund => true;
    public bool SupportsPartialCapture => false;
    public PaymentFlowType FlowType =>
        PaymentFlowType.Redirect;

    public async Task<AuthorizationResult> AuthorizeAsync(
        AuthorizationContext context)
    {
        var session =
            await _idealClient.CreatePaymentAsync(
                new IdealPaymentRequest
        {
            Amount = context.Amount,
            Currency = "EUR",
            Description = context.Description,
            ReturnUrl = context.ReturnUrl,
            MerchantOrderId =
                context.PaymentIntentId
        });

        return AuthorizationResult.RequiresRedirect(
            redirectUrl: session.IssuerUrl,
            sessionId: session.Id);
    }
}

public class ACHBankTransferHandler :
    IPaymentMethodHandler
{
    public PaymentMethodType SupportedType =>
        PaymentMethodType.BankAccount;
    public bool SupportsRecurring => true;
    public bool SupportsRefund => true;
    public bool SupportsPartialCapture => false;
    public PaymentFlowType FlowType =>
        PaymentFlowType.Asynchronous;

    public async Task<AuthorizationResult> AuthorizeAsync(
        AuthorizationContext context)
    {
        var achSubmission =
            await _achClient.InitiatePaymentAsync(
                new ACHPaymentRequest
        {
            Amount = context.Amount,
            RoutingNumber =
                context.BankAccount.RoutingNumber,
            AccountNumber =
                context.BankAccount.AccountNumber,
            AccountType =
                context.BankAccount.AccountType,
            CompanyName = context.MerchantName,
            TraceNumber = GenerateTraceNumber()
        });

        return AuthorizationResult
            .PendingSettlement(
                estimatedSettlementDate:
                    DateTime.UtcNow.AddDays(2),
                traceNumber:
                    achSubmission.TraceNumber);
    }
}

Payment Method Comparison

18. Reliability and Failure Modes

Payment systems face a unique set of failure modes that do not exist in typical web applications. The consequences of failures are measured in dollars and customer trust, not just page views. Every failure mode must be anticipated, and the system must handle each gracefully without losing data or double-charging customers.

Critical Failure Modes

Saga Pattern for Payment Operations

C#
public class PaymentSaga
{
    private readonly IPaymentRepository _payments;
    private readonly ILedgerService _ledger;
    private readonly IEventPublisher _events;
    private readonly ISagaLog _sagaLog;

    public async Task ExecutePaymentAsync(
        PaymentIntent payment)
    {
        var saga = new SagaLog("payment", payment.Id);

        // Step 1: Create pending ledger entry
        try
        {
            await _ledger.PostPendingAsync(
                payment);
            await saga.RecordStep(
                "ledger_pending", SagaStatus.Completed);
        }
        catch (Exception ex)
        {
            await saga.RecordStep(
                "ledger_pending",
                SagaStatus.Failed, ex.Message);
            throw;
        }

        // Step 2: Authorize
        try
        {
            var authResult =
                await AuthorizePaymentAsync(payment);
            await saga.RecordStep(
                "authorize", SagaStatus.Completed,
                authResult);

            if (!authResult.IsSuccess)
            {
                await _ledger
                    .ReversePendingAsync(payment);
                await saga.RecordStep(
                    "ledger_reverse",
                    SagaStatus.Completed);
                return;
            }
        }
        catch (Exception ex)
        {
            await saga.RecordStep(
                "authorize",
                SagaStatus.Failed, ex.Message);

            // Check if authorization actually went through
            var status = await PollAuthorizationStatus(
                payment.Id);
            if (status == AuthorizationStatus.Approved)
            {
                // Authorization succeeded but we did
                // not get the response - continue
                await saga.RecordStep(
                    "authorize_recovery",
                    SagaStatus.Completed);
            }
            else
            {
                await _ledger
                    .ReversePendingAsync(payment);
                throw;
            }
        }

        // Step 3: Capture
        try
        {
            var captureResult =
                await CapturePaymentAsync(payment);
            await saga.RecordStep(
                "capture", SagaStatus.Completed);

            await _ledger.PostCaptureAsync(payment);
            await saga.RecordStep(
                "ledger_capture", SagaStatus.Completed);
        }
        catch (Exception ex)
        {
            await saga.RecordStep(
                "capture",
                SagaStatus.Failed, ex.Message);

            // Authorization is still valid, retry capture
            await RetryCaptureLaterAsync(payment);
        }
    }
}

Graceful Degradation Strategy

• Fraud engine down: Fall back to rule-only checks. If rules are also unavailable, route to manual review queue rather than blocking all payments.
• Primary processor down: Automatic failover to secondary processor via circuit breaker pattern. No user-visible impact.
• Database degraded: Serve reads from replicas, queue writes to durable buffer for replay when primary recovers.
• Redis down: Bypass cache layer, fall through to database queries. Performance degrades but correctness is maintained.
• Webhook delivery failing: Queue events for later delivery, maintain merchant polling API as fallback.

19. Cost Estimation and Infrastructure

Understanding the infrastructure cost of a payment processing system is essential for business planning and architectural decision-making. At Stripe-scale, the costs are significant but the revenue per transaction makes the unit economics highly favorable. Let us break down the infrastructure costs for a platform processing 500 million transactions per month.

Compute Costs

Storage and Data Costs

Operational and Security Costs

Total Infrastructure Cost Summary

20. Interview Q and A

The following questions and answers cover the most common system design interview topics for payment processing systems at the Staff and Principal Engineer level. These are designed to demonstrate deep understanding of both the technical and business aspects of payment infrastructure.

Q1: How would you handle a situation where the payment processor responds with an ambiguous result (timeout after the authorization may have gone through)?

Q2: How do you ensure that your ledger stays in balance even during partial failures?

Q3: Design the payout system for a marketplace where sellers need to receive funds after each transaction.

Q4: How would you design the system to handle 10x traffic spikes during flash sales?

Q5: Explain how you would prevent double-charging a customer during a network partition.

Q6: How do you handle PCI DSS compliance while still providing a good developer experience?

Q7: Describe how you would build the reconciliation system to catch discrepancies in real-time.

Q8: How would you design the multi-currency settlement system?