Introduction: Why Event Sourcing

Event sourcing is a persistence pattern where state changes are stored as an immutable sequence of events rather than as the current state of an entity. Instead of updating a row in a database table to reflect the current balance of a bank account, you append every deposit, withdrawal, fee assessment, and interest calculation as discrete events. The current balance is then derived by replaying every event that has ever occurred for that account. This fundamental shift from state-oriented to event-oriented thinking unlocks capabilities that are difficult or impossible to achieve with traditional CRUD-based persistence.

The decision to adopt event sourcing typically follows from a set of architectural pressures. First, auditability: in regulated domains such as finance, healthcare, and insurance, you must prove not only what the current state is but also how that state came to be. With event sourcing, every state change is automatically recorded with its full context, including the command that triggered it, the timestamp, and the actor who issued it. Second, temporal queries: you often need to answer questions like "what did this customer's profile look like six months ago?" In a CRUD system, this requires either point-in-time recovery snapshots or a complex auditing table. With event sourcing, you simply replay events up to the desired point. Third, business intelligence: the raw event stream is a goldmine for analytics. Every user action, every system decision, and every external integration is captured as first-class data, enabling sophisticated analysis that would be impossible if only the current state were stored.

Event sourcing also pairs naturally with the Command Query Responsibility Segregation (CQRS) pattern. CQRS separates the model that handles commands (writes) from the model that handles queries (reads). In a traditional system, a single data model is used for both commands and queries, often leading to compromises. For writes, you need consistency guarantees, referential integrity, and transactional boundaries. For reads, you need flexible querying, denormalized views, and high throughput. These two sets of requirements pull in opposite directions. CQRS acknowledges this tension by splitting the model, and event sourcing provides the perfect persistence mechanism for the write side: an append-only event store that captures every command that changed state. The read side is then populated by projections that consume events and build materialized views optimized for specific query patterns.

The adoption of event sourcing has grown significantly in the last decade. Major companies including Microsoft, Amazon, Uber, and Goldman Sachs have built event-sourced systems at scale. Martin Fowler's seminal 2005 article introduced the pattern to a wide audience, and Greg Young's work on CQRS and event sourcing shaped the modern architectural landscape. Today, the pattern is supported by dedicated databases like EventStoreDB, mature frameworks like Axon Framework in Java and Marten in .NET, and cloud services like AWS EventBridge and Azure Event Hubs that can serve as event stores.

However, event sourcing is not a silver bullet. It introduces operational complexity, a learning curve for developers, and challenges around event schema evolution, replay performance, and eventual consistency. This guide is written for senior engineers and architects who need to understand the trade-offs and make informed decisions. We will explore the core concepts, the detailed architecture, the practical implementation strategies, and the common pitfalls. By the end, you should be able to design an event-sourced system that is scalable, maintainable, and appropriate for your domain.

What Problem Does Event Sourcing Solve?

Traditional CRUD systems store the current state, which means historical state is lost. If you need to know what happened, you typically build an audit log as a separate concern, and that audit log is often incomplete, inconsistent with the primary data, and queried through ad-hoc mechanisms. Event sourcing makes the history the primary data. The current state is a derived view, not the source of truth. This inversion of responsibility solves the auditability problem natively. It also solves the temporal query problem: because every past state can be reconstructed by replaying events, you can answer any question about any point in time without needing separate versioning infrastructure.

Another problem event sourcing addresses is the impedance mismatch between domain events and database rows. Domain-driven design encourages modeling the business as a series of events: "OrderPlaced," "PaymentReceived," "ItemShipped." In a CRUD system, these events are ephemeral—they trigger a state change and are then discarded. In event sourcing, they become the permanent record. This aligns the persistence model with the domain model, making the code easier to understand and evolve.

When Should You NOT Use Event Sourcing?

Event sourcing is inappropriate for many systems. If your domain does not require auditability, temporal queries, or complex event-driven workflows, the overhead is unlikely to be justified. Simple CRUD applications, content management systems, and reporting dashboards are usually better served by traditional persistence. Even within domains that benefit from event sourcing, it makes sense to apply it selectively to bounded contexts where the value is highest, rather than as a blanket architectural decision.

Event sourcing also introduces latency. Writes must be appended to an event store and then projected to read models. This means the system is eventually consistent by default. If your application requires strong consistency for every read operation, you will need to blend event sourcing with other mechanisms, such as synchronous projections or a CQRS command-side that doubles as a read model. The complexity of managing eventual consistency at scale should not be underestimated.

Core Concepts — Events, Aggregates, Projections, Snapshots

Before diving into implementation details, it is essential to establish a shared vocabulary. Event sourcing and CQRS introduce several concepts that differ from traditional state-oriented thinking. We will define each one, explain its role, and show how they relate to each other.

Domain Events

A domain event is a record of something that happened in the domain that is significant to the business. Events are named in the past tense: "OrderSubmitted," "PaymentDeclined," "InventoryReserved." Each event carries the data necessary to understand what happened and to reconstruct the aggregate's state. Events are immutable. Once committed to the event store, they should never be changed or deleted. If a mistake occurs, the proper response is to append a compensating event, not to modify the original event. This immutability is the foundation of the audit trail.

Events should be fine-grained enough to capture meaningful state transitions but coarse enough to avoid overwhelming the system with trivial events. A good heuristic is that each event should correspond to a business operation that a domain expert would recognize. For example, "CustomerMoved" is a meaningful business event; "FieldUpdated" is a technical implementation detail that leaks into the event stream and should be avoided.

Aggregates

An aggregate is a cluster of domain objects that can be treated as a single unit for the purpose of data changes. Each aggregate has a root entity (the aggregate root) and a boundary that defines what is inside. In event sourcing, the aggregate is responsible for producing events in response to commands and for rebuilding its state by replaying events. The aggregate root enforces invariants. For example, an "Order" aggregate might contain line items, a shipping address, and a payment status. When a command to "AddItem" arrives, the aggregate checks invariants (is the order still open? is the item in stock?), emits an "ItemAddedToOrder" event, and adds the item to its internal state.

The aggregate boundary is critical for consistency. All events for a single aggregate instance are stored in a single stream. The event store guarantees that events within a stream are ordered and that concurrent writes are handled via optimistic concurrency. This means that the aggregate is the unit of transactional consistency. If two commands attempt to modify the same aggregate concurrently, one will fail and must be retried. This is how event sourcing maintains consistency without distributed locks.

Projections

A projection is a subscriber to one or more event streams that builds a read-optimized view of the data. Projections are the read side in CQRS. They consume events in order and update denormalized tables, search indexes, caches, or any other read store. A projection for the "order list" screen might listen for "OrderSubmitted," "OrderShipped," and "OrderCancelled" events, maintaining a table with one row per order that includes all the fields needed for the UI. This table is derived entirely from the event stream and can be rebuilt at any time by replaying all events from the beginning.

Projections can be live (continuously updated as events are appended) or batch (rebuilt periodically). Production systems typically use live projections for common queries and batch projections for analytical workloads. The same event stream can feed multiple projections optimized for different use cases: one for the customer-facing website, one for the admin dashboard, one for the reporting system.

Snapshots

When an aggregate has a long event stream (tens of thousands of events or more), replaying the entire stream to rebuild state becomes expensive. Snapshots provide a shortcut. A snapshot is a serialized copy of the aggregate's state at a specific point in the stream. When loading the aggregate, you retrieve the most recent snapshot and then replay only the events that occurred after the snapshot was taken. This reduces the replay cost from O(n) to O(number of events since last snapshot).

Snapshots can be taken at regular intervals (every 100 events, every 1000 events) or triggered by a specific event type. The snapshot must include the stream version so that you know which events to skip during replay. Snapshots add complexity: they must be stored separately, updated atomically with the event stream, and invalidated if the event schema changes. Many production systems start without snapshots and add them only when replay performance becomes a bottleneck.

Event Streams

An event stream is an ordered sequence of events belonging to a single aggregate instance. The stream is identified by a unique key (typically the aggregate ID). Each event in the stream has a monotonically increasing version number or a global sequence number. The stream is append-only: new events are added to the end, and existing events are never modified. This property makes concurrent writes easy to handle: you check the current stream version, and if another writer has appended since you last read, your write fails and you must retry.

This diagram shows the flow from command to event store to projections. The aggregate root processes commands and emits events. Events are stored in the event store and also forwarded to projections. Snapshots are loaded from the snapshot store to accelerate aggregate rebuild.

CQRS Read/Write Model Separation

Command Query Responsibility Segregation is an architectural pattern that separates the operations that mutate state (commands) from the operations that read state (queries). This separation can be applied at various levels: at the object level, at the service level, or at the database level. In the context of event sourcing, CQRS is almost always applied at the database and service level, meaning the write side uses an event store and the read side uses one or more materialized views.

Command Side

The command side is responsible for validating inputs, enforcing business rules, and appending events to the event store. It exposes a command API that accepts commands like PlaceOrder, ShipOrder, or CancelOrder. Each command is processed by an aggregate root that checks invariants and emits events. The command side does not return any data to the caller beyond a confirmation that the command was accepted or rejected. This is a critical distinction: in a traditional CRUD API, a POST request typically returns the created resource. In CQRS, the command handler returns only a status code and possibly the new aggregate version. The caller must query the read side to see the effects of the command.

Commands are named in the imperative mood: "PlaceOrder," not "OrderPlaced." This distinguishes them from events, which are named in the past tense. A command represents an intent; an event represents a fact. Commands can fail (e.g., the order is already closed), but events, once appended, are immutable facts that cannot fail.

Query Side

The query side exposes data through query models optimized for specific use cases. These models are materialized views built by projections. A query model for the order list screen might be a table in a relational database with columns for order ID, customer name, total, status, and date. This table is updated by projections that listen to events from the order aggregate. The query side is read-only from the client's perspective: clients send queries and receive responses, but they never directly modify the query models. All mutations go through the command side and flow to the query side via events.

Because the query side is derived from events, it is naturally eventually consistent. There is a delay between when an event is appended and when the projection updates the read model. This delay is typically on the order of milliseconds, but under heavy load it can grow. Applications must be designed to tolerate this eventual consistency. For example, after placing an order, the UI might show a "submission received" confirmation and then poll the query side until the order appears in the list.

Benefits of Separation

The separation of read and write models enables independent scaling. If the application has a high read-to-write ratio, the read side can be scaled horizontally without affecting the write side. Multiple read models can be maintained for different purposes without complicating the write model. The write model focuses on consistency and business rules; the read models focus on query performance and data denormalization. This separation also improves security: the command side can enforce fine-grained authorization policies without impacting read performance.

Challenges of Separation

The main challenge is managing eventual consistency. If an admin deletes a user account, a rogue projection might serve stale data for a brief period. Application developers must be aware of this and design the UI accordingly. Another challenge is operational complexity: instead of one database, you now have an event store plus one or more query databases. Each component must be monitored, backed up, and scaled independently. The eventual consistency also complicates testing: tests must account for the asynchronous nature of the data flow.

System Architecture Overview

An event-sourced CQRS system comprises several distinct components that work together to process commands, store events, and serve queries. This section presents a high-level architecture diagram and explains each component's role.

The architecture is layered into five main areas. The client layer includes the user interface and the API gateway that routes requests to the appropriate services. The command side handles all mutations: it validates commands, loads the aggregate root, applies business logic, and appends events. The event store is the system of record, persisting every event in an append-only log. The query side listens to events via subscriptions and builds materialized views optimized for reading. The infrastructure layer provides the message bus, event queue, and monitoring that glue everything together.

API Gateway and Command Routing

The API gateway receives incoming HTTP requests and routes them based on the request type. Write requests (POST, PUT, DELETE) are routed to the command handler; read requests (GET) are routed to the query service. The gateway also handles authentication, rate limiting, and request validation. In a microservices deployment, each bounded context might have its own gateway, or a single gateway might route to multiple backend services.

Command Handlers and Aggregate Roots

Command handlers are stateless services that coordinate the processing of a command. They load the aggregate root from the event store (or from a cache/snapshot), invoke the appropriate method on the aggregate, and append the resulting events. The command handler is also responsible for managing the transactional boundary: if the event store write fails due to a concurrency conflict, the handler retries the entire operation. This load-process-append cycle is the core pattern of event-sourced command processing.

Event Store and Subscriptions

The event store is a specialized database designed for append-only event streams. It guarantees durability, ordering within a stream, and optimistic concurrency control. Modern event stores also support subscriptions: persistent, position-tracked subscriptions that deliver events to subscribers in order. These subscriptions are the backbone of the projection system. Each projection subscribes to one or more event types and processes them sequentially.

Projection Engine

The projection engine runs one or more projection instances, each consuming events from a subscription and updating a read model. Projections can be stateless (computing a result on the fly) or stateful (storing intermediate results in a database). The projection engine handles retries, idempotency, and error logging. In a well-designed system, projections are idempotent: processing the same event twice produces the same result. This is critical for recovery scenarios where events might be replayed.

Event Store Design — Append-Only Log and Partitioning

The event store is the heart of an event-sourced system. It must provide durability, ordering guarantees, high write throughput, and efficient read access for replay. Designing an event store requires decisions about storage format, partitioning strategy, indexing, and concurrency control.

Append-Only Log Structure

At its simplest, an event store is an append-only log. New events are always written to the end of the log. This sequential write pattern is extremely efficient on modern hardware, especially on SSDs. The append-only nature also simplifies concurrency: writers do not block each other as long as they are writing to different streams. Within a single stream, writes are serialized by the optimistic concurrency check: the writer provides the expected version, and the store rejects the write if the stream has advanced.

The event store should expose a minimal API: AppendToStream(streamId, expectedVersion, events) and ReadStream(streamId, fromVersion, maxCount). Some implementations also support global reads across all streams, which is useful for rebuilding projections or creating global subscriptions.

Partitioning and Sharding

As the system grows, a single event store node becomes a bottleneck. Partitioning distributes streams across multiple nodes based on a partition key. The stream ID is a natural partition key: all events for the same stream go to the same partition. This ensures ordering within a stream is maintained at the partition level. Cross-stream ordering is not guaranteed, which is acceptable because events in different streams are independent.

Partitioning strategies include hash-based partitioning (using a hash of the stream ID modulo the number of partitions), range-based partitioning (streams with IDs in a certain range go to a certain partition), and directory-based partitioning (each partition is a separate database or table). The choice depends on the scale and the query patterns. Hash-based partitioning provides good distribution but makes global queries expensive because they require scanning all partitions. Range-based partitioning preserves locality for related streams but can create hot spots if some ranges are more active than others.

Stream Metadata and Indexing

Each event includes metadata such as the event type, timestamp, stream ID, version number, and the correlation ID that ties related events together across streams. The event body contains the domain-specific data. Indexing strategies typically focus on the event type, stream ID, and timestamp. Some event stores also support custom metadata indexes that allow filtering events by arbitrary properties.

A common indexing approach is to maintain a global event position (a monotonically increasing sequence number) as an additional index. This enables global subscriptions that start from a specific position and read events in the order they were committed, regardless of the stream they belong to. This is useful for projections that need to process events from multiple stream types in commit order.

C#
public interface IEventStore
{
    Task AppendToStreamAsync(
        string streamId,
        long expectedVersion,
        IReadOnlyList<IDomainEvent> events,
        CancellationToken ct = default);

    Task<IReadOnlyList<IDomainEvent>> ReadStreamAsync(
        string streamId,
        long fromVersion = 0,
        int maxCount = int.MaxValue,
        CancellationToken ct = default);

    Task<long> GetStreamVersionAsync(
        string streamId,
        CancellationToken ct = default);

    Task SubscribeToAllAsync(
        long? checkpoint,
        Func<IDomainEvent, Task> handler,
        CancellationToken ct = default);
}
    

Concurrency Control

Optimistic concurrency is the standard approach for event stores. When appending to a stream, the caller provides the expectedVersion. If the stream's current version matches, the write succeeds and the stream version is incremented. If it does not match, the write fails with a concurrency exception, and the caller must retry the operation. This mechanism prevents lost updates without requiring distributed locks.

For the initial write to a new stream, the expected version is typically -1 or 0, depending on the convention. Some event stores also support idempotent writes: if the same event ID has already been appended, the write is a no-op rather than a failure. This is useful for at-least-once delivery scenarios where the same command might be dispatched multiple times.

When two writers append to the same stream concurrently with the same expected version, only one succeeds. The other must retry with the updated version, ensuring no events are lost.

Storage Backend Considerations

The event store can be built on top of a relational database (PostgreSQL, MySQL), a dedicated event store database (EventStoreDB), or a distributed log (Apache Kafka, Amazon Kinesis). Relational databases provide strong consistency, SQL querying, and mature tooling. PostgreSQL is a popular choice for custom event stores because of its native JSONB support, transactional guarantees, and LISTEN/NOTIFY for real-time subscriptions. Kafka is ideal for high-throughput scenarios where global ordering matters, but it introduces complexity around schema management and stream semantics. EventStoreDB is purpose-built for event sourcing and provides the best developer experience for the pattern, but it is a smaller ecosystem.

Aggregate Root and Domain Events

The aggregate root is the central concept on the write side of an event-sourced system. It encapsulates business logic, enforces invariants, and emits domain events. Designing the aggregate root correctly is critical to the success of the system.

Aggregate Root Pattern

The aggregate root is the entry point for all operations on the aggregate. External code interacts only with the root; internal entities are accessed through the root. The root maintains a list of uncommitted events that are generated during command processing. These events are held in memory until they are appended to the event store. After appending, the events are cleared. This pattern ensures that events are only emitted when the aggregate state is consistent.

C#
public abstract class AggregateRoot
{
    private readonly List<IDomainEvent> _uncommittedEvents = new();
    
    public string Id { get; protected set; }
    public long Version { get; private set; }
    
    public IReadOnlyList<IDomainEvent> GetUncommittedEvents() =>
        _uncommittedEvents.AsReadOnly();
    
    public void ClearUncommittedEvents() => _uncommittedEvents.Clear();
    
    protected void ApplyEvent(IDomainEvent @event)
    {
        ((dynamic)this).Apply((dynamic)@event);
        _uncommittedEvents.Add(@event);
        Version++;
    }
    
    public void LoadFromHistory(IReadOnlyList<IDomainEvent> history)
    {
        foreach (var @event in history)
        {
            ((dynamic)this).Apply((dynamic)@event);
            Version++;
        }
    }
}
    

Order Aggregate Example

Consider an order management system. The Order aggregate root manages line items, the order status, and the total amount. When a PlaceOrder command is processed, the aggregate checks that the order does not already exist, validates the line items, and emits an OrderPlaced event. When an AddItem command is processed, it checks that the order is still open and emits an ItemAddedToOrder event.

C#
public class Order : AggregateRoot
{
    private readonly List<OrderItem> _items = new();
    private OrderStatus _status;
    private string _customerId;
    
    public Order() { }
    
    public static Order Place(string orderId, string customerId, 
        IReadOnlyList<OrderItem> items)
    {
        var order = new Order();
        order.ApplyEvent(new OrderPlaced
        {
            OrderId = orderId,
            CustomerId = customerId,
            Items = items,
            PlacedAt = DateTime.UtcNow
        });
        return order;
    }
    
    public void AddItem(OrderItem item)
    {
        if (_status != OrderStatus.Open)
            throw new InvalidOperationException("Order is not open.");
            
        ApplyEvent(new ItemAddedToOrder
        {
            OrderId = Id,
            Item = item,
            AddedAt = DateTime.UtcNow
        });
    }
    
    public void RemoveItem(string sku)
    {
        if (_status != OrderStatus.Open)
            throw new InvalidOperationException("Order is not open.");
            
        var existing = _items.FirstOrDefault(i => i.Sku == sku);
        if (existing == null)
            throw new InvalidOperationException("Item not found.");
            
        ApplyEvent(new ItemRemovedFromOrder
        {
            OrderId = Id,
            Sku = sku,
            RemovedAt = DateTime.UtcNow
        });
    }
    
    public void Submit()
    {
        if (_status != OrderStatus.Open)
            throw new InvalidOperationException("Order is not open.");
        if (_items.Count == 0)
            throw new InvalidOperationException("Cannot submit empty order.");
            
        ApplyEvent(new OrderSubmitted
        {
            OrderId = Id,
            SubmittedAt = DateTime.UtcNow
        });
    }
    
    private void Apply(OrderPlaced e)
    {
        Id = e.OrderId;
        _customerId = e.CustomerId;
        _items.AddRange(e.Items);
        _status = OrderStatus.Open;
    }
    
    private void Apply(ItemAddedToOrder e)
    {
        _items.Add(e.Item);
    }
    
    private void Apply(ItemRemovedFromOrder e)
    {
        _items.RemoveAll(i => i.Sku == e.Sku);
    }
    
    private void Apply(OrderSubmitted e)
    {
        _status = OrderStatus.Submitted;
    }
}
    

Domain Event Definitions

Domain events are simple data classes with no behavior. They contain the data needed to reconstruct state and the metadata that identifies the context of the event. Events should be immutable and should define a clear contract that can evolve over time.

C#
public interface IDomainEvent
{
    string EventId { get; }
    DateTime Timestamp { get; }
}

public class OrderPlaced : IDomainEvent
{
    public string EventId { get; set; } = Guid.NewGuid().ToString();
    public DateTime Timestamp { get; set; } = DateTime.UtcNow;
    public string OrderId { get; set; }
    public string CustomerId { get; set; }
    public IReadOnlyList<OrderItem> Items { get; set; }
}

public class ItemAddedToOrder : IDomainEvent
{
    public string EventId { get; set; } = Guid.NewGuid().ToString();
    public DateTime Timestamp { get; set; } = DateTime.UtcNow;
    public string OrderId { get; set; }
    public OrderItem Item { get; set; }
}

public class OrderSubmitted : IDomainEvent
{
    public string EventId { get; set; } = Guid.NewGuid().ToString();
    public DateTime Timestamp { get; set; } = DateTime.UtcNow;
    public string OrderId { get; set; }
}

public class OrderItem
{
    public string Sku { get; set; }
    public string Name { get; set; }
    public int Quantity { get; set; }
    public decimal UnitPrice { get; set; }
}
    

Invariants and Consistency

The aggregate root must enforce all invariants. An invariant is a business rule that must always be true. Examples include "an order cannot be submitted without at least one item" and "a closed order cannot accept new items." The aggregate root checks these invariants before emitting events. Because the aggregate is the unit of consistency, and because each aggregate has its own event stream, invariants can be enforced without distributed transactions.

However, some invariants span multiple aggregates. For example, "a customer cannot exceed their credit limit" involves both the customer aggregate and the order aggregate. In event sourcing, cross-aggregate invariants are handled through eventual consistency patterns like sagas or process managers, not through synchronous checks. This is a fundamental trade-off: event sourcing gives up strong consistency across aggregates in exchange for the benefits of the event-driven model.

Event Schema Evolution and Versioning

Event schema evolution is one of the most challenging aspects of event sourcing. Because events are immutable and stored permanently, you cannot simply alter the schema of an existing event type. Instead, you must evolve the schema over time while maintaining the ability to read old events. This section covers strategies for versioning events and migrating event schemas.

Why Schema Evolution Matters

In a CRUD system, schema changes are handled by database migrations: you alter the table structure, backfill data, and move on. In event sourcing, the events are the source of truth, and they exist in potentially millions of historical records. If you change the definition of OrderPlaced to include a new field, all the old OrderPlaced events that lack that field must still be deserializable. Moreover, projections that consume events must handle multiple versions of the same event type.

Versioning Strategies

The most common approach is to include a version number in the event schema. Each event type is tagged with a version number, and the serialized event includes this version. When deserializing, the code reads the version and uses the appropriate handler. There are several strategies for managing different versions:

Upcasting: Old events are transformed into the latest version at read time. An upcaster is a function that takes an event of version N and returns an event of version N+1. This keeps the application logic simple because it only deals with the latest version. The downside is that upcasters must be maintained for every schema version, and they can become a performance bottleneck if there are many versions. Upcasting is typically done lazily when events are read from the store, not eagerly when they are committed.

Forward-only migration: Each event carries a version, and the application handles all versions. Code paths branch based on the version number. This is simpler than upcasting because there is no transformation step, but it clutters the code with version checks and makes the logic harder to follow. This approach is viable only when the number of versions is small (typically 2-3).

Separate event types: Each version is a completely separate event type with its own class name. For example, OrderPlacedV1, OrderPlacedV2. The event handlers (projections, aggregate `Apply` methods) implement separate methods for each version. This is the cleanest approach from a type-safety perspective, but it leads to many event classes and requires careful naming conventions.

C#
public interface IUpcaster
{
    IDomainEvent Upcast(IDomainEvent sourceEvent);
}

public class OrderPlacedV1Upcaster : IUpcaster
{
    public bool CanUpcast(IDomainEvent e) => 
        e is OrderPlacedV1;
    
    public IDomainEvent Upcast(IDomainEvent sourceEvent)
    {
        var v1 = (OrderPlacedV1)sourceEvent;
        return new OrderPlacedV2
        {
            OrderId = v1.OrderId,
            CustomerId = v1.CustomerId,
            Items = v1.Items,
            ShippingAddress = null, // added in V2, default for old events
            PlacedAt = v1.PlacedAt
        };
    }
}

public class EventStoreWithUpcasting : IEventStore
{
    private readonly IEventStore _inner;
    private readonly IReadOnlyList<IUpcaster> _upcasters;
    
    public async Task<IReadOnlyList<IDomainEvent>> ReadStreamAsync(
        string streamId, long fromVersion = 0, int maxCount = int.MaxValue, 
        CancellationToken ct = default)
    {
        var events = await _inner.ReadStreamAsync(streamId, fromVersion, maxCount, ct);
        return events.Select(e =>
        {
            var current = e;
            foreach (var upcaster in _upcasters)
            {
                if (upcaster.CanUpcast(current))
                    current = upcaster.Upcast(current);
            }
            return current;
        }).ToList();
    }
}
    

Serialization Considerations

JSON is the most common serialization format for events, but it has weaknesses for schema evolution. JSON does not distinguish between a missing field and a field set to null. When a new field is added, old events will not have it, and the deserializer must handle the missing field gracefully. Using a schema-aware format like JSON Schema, Avro, or Protocol Buffers can help enforce these rules. Avro is particularly well-suited because it supports schema resolution: the reader can declare a different schema than the writer, and the Avro library handles field-level mapping automatically.

Backward and Forward Compatibility

Backward compatibility means that new code can read old events. This is essential: you cannot change the past. Forward compatibility means that old code can read new events, which is relevant during rolling deployments where different versions of the service are running simultaneously. The safest approach is to design events with forward compatibility in mind from day one: use optional fields for everything, avoid required fields, and ignore unknown fields during deserialization. Most JSON libraries support this pattern natively.

Events flow through upcasters before reaching the application. Old versions are transparently upgraded to the current version, so the application code always works with the latest schema.

Projections and Materialized Views

Projections transform the raw event stream into query-optimized data structures. They are the bridge between the event-sourced write side and the CQRS read side. A projection subscribes to events, processes them in order, and updates a materialized view. The materialized view can be a relational table, a document store collection, a search index, or any other data store optimized for the queries it serves.

Projection Types

There are two main categories of projections: stateful and stateless. Stateless projections recompute the result on every request by scanning events in memory. They are simple to implement and guarantee consistency (because they always read directly from the event store), but they do not scale to large event streams. Stateful projections maintain an intermediate store that they update incrementally as events arrive. They are more complex but can handle millions of events efficiently.

Projections can also be categorized by their trigger mechanism. Push-based projections receive events via subscriptions and update the read model immediately. Pull-based projections poll the event store on a schedule and process new events in batches. Push-based projections are suitable for low-latency requirements; pull-based projections are simpler to implement and more resilient to failures because they can always catch up from the last checkpoint.

Implementing a Projection

A projection typically implements a handler for each event type it cares about. The handler receives the event and updates the read model. The projection must track its position in the event stream (a checkpoint) so that it can resume from where it left off after a crash. Checkpoints are stored in a durable store, typically the same database as the read model, to ensure transactional consistency between the checkpoint and the read model updates.

C#
public class OrderListProjection : IProjection
{
    private readonly IReadModelStore _store;
    
    public OrderListProjection(IReadModelStore store)
    {
        _store = store;
    }
    
    public async Task HandleAsync(IDomainEvent @event, CancellationToken ct)
    {
        switch (@event)
        {
            case OrderPlaced e:
                await _store.InsertAsync(new OrderListEntry
                {
                    OrderId = e.OrderId,
                    CustomerId = e.CustomerId,
                    Total = e.Items.Sum(i => i.Quantity * i.UnitPrice),
                    Status = "Open",
                    PlacedAt = e.PlacedAt,
                    ItemCount = e.Items.Count
                }, ct);
                break;
                
            case OrderSubmitted e:
                await _store.UpdateAsync(e.OrderId,
                    entry => { entry.Status = "Submitted"; }, ct);
                break;
                
            case OrderShipped e:
                await _store.UpdateAsync(e.OrderId,
                    entry => { entry.Status = "Shipped"; }, ct);
                break;
                
            case OrderCancelled e:
                await _store.UpdateAsync(e.OrderId,
                    entry => { entry.Status = "Cancelled"; }, ct);
                break;
        }
    }
    
    public string Name => "OrderListProjection";
}
    

Idempotency

Projections must be idempotent because events can be delivered more than once. If a projection crashes after updating the read model but before saving the checkpoint, the same event will be delivered again when the projection restarts. The projection must handle this gracefully. One approach is to use upsert operations (insert or update) so that processing the same event twice has no ill effect. Another approach is to store the event ID alongside the read model data and skip events that have already been processed.

Rebuilding Projections

A key feature of event-sourced systems is the ability to rebuild projections from scratch. When the read model schema changes, or when a bug is discovered in a projection, you can drop the read model and replay all events from the beginning. This is why projections should be pure functions of the event stream: given the same sequence of events, they should always produce the same result. Any external dependency (like calling an external API during projection) breaks this purity and makes rebuilding unreliable.

C#
public class ProjectionRebuilder
{
    private readonly IEventStore _eventStore;
    private readonly IProjection _projection;
    
    public async Task RebuildAsync(CancellationToken ct)
    {
        await _projection.ResetAsync(ct);
        
        long position = 0;
        const int batchSize = 1000;
        
        while (!ct.IsCancellationRequested)
        {
            var events = await _eventStore.ReadAllEventsAsync(
                position, batchSize, ct);
            
            if (events.Count == 0)
                break;
            
            foreach (var @event in events)
            {
                await _projection.HandleAsync(@event, ct);
            }
            
            position += events.Count;
            await _projection.SaveCheckpointAsync(position, ct);
        }
    }
}
    

Snapshot Strategy for Long-Lived Aggregates

Snapshots are a performance optimization for aggregates that accumulate many events over time. Without snapshots, loading an aggregate requires replaying its entire event stream from version 1 to the current version. For aggregates with thousands or tens of thousands of events, this replay can take hundreds of milliseconds or even seconds, making the system too slow for interactive use.

How Snapshots Work

A snapshot captures the state of an aggregate at a specific version. When loading the aggregate, the system first retrieves the most recent snapshot. It then replays only the events that were appended after the snapshot was taken. This reduces the replay cost dramatically. For example, if an aggregate has 10,000 events and a snapshot is taken every 100 events, the worst-case replay is 100 events instead of 10,000.

Snapshot Frequency

Choosing the snapshot frequency requires balancing two costs: the cost of taking a snapshot (writing the serialized state) and the cost of replaying events when a snapshot is not available. A common heuristic is to take a snapshot every N events, where N is determined by profiling. For most systems, N = 100 or N = 1000 works well. Some systems use adaptive strategies: take snapshots more frequently for aggregates that are loaded often and less frequently for aggregates that are rarely accessed.

Implementing Snapshots in C#

C#
public interface ISnapshotStore
{
    Task<Snapshot> GetSnapshotAsync(string aggregateId, CancellationToken ct);
    Task SaveSnapshotAsync(string aggregateId, Snapshot snapshot, CancellationToken ct);
}

public class Snapshot
{
    public string AggregateId { get; set; }
    public long Version { get; set; }
    public string SerializedState { get; set; }
    public DateTime TakenAt { get; set; }
}

public class SnapshotRepository
{
    private readonly IEventStore _eventStore;
    private readonly ISnapshotStore _snapshotStore;
    private readonly int _snapshotInterval;
    
    public async Task<T> LoadAggregateAsync<T>(string id) where T : AggregateRoot, new()
    {
        var snapshot = await _snapshotStore.GetSnapshotAsync(id, default);
        T aggregate;
        long fromVersion;
        
        if (snapshot != null)
        {
            aggregate = Deserialize<T>(snapshot.SerializedState);
            fromVersion = snapshot.Version + 1;
        }
        else
        {
            aggregate = new T();
            fromVersion = 0;
        }
        
        var events = await _eventStore.ReadStreamAsync(id, fromVersion);
        aggregate.LoadFromHistory(events);
        
        return aggregate;
    }
    
    public async Task SaveAggregateAsync<T>(T aggregate) where T : AggregateRoot
    {
        await _eventStore.AppendToStreamAsync(
            aggregate.Id,
            aggregate.Version - aggregate.GetUncommittedEvents().Count,
            aggregate.GetUncommittedEvents());
        
        aggregate.ClearUncommittedEvents();
        
        if (aggregate.Version % _snapshotInterval == 0)
        {
            var snapshot = new Snapshot
            {
                AggregateId = aggregate.Id,
                Version = aggregate.Version,
                SerializedState = Serialize(aggregate),
                TakenAt = DateTime.UtcNow
            };
            
            await _snapshotStore.SaveSnapshotAsync(aggregate.Id, snapshot, default);
        }
    }
}
    

Serialization of Snapshots

Snapshots must be serialized to a format that can be stored and deserialized later. JSON is a common choice, but it has limitations: the class structure must be compatible, and renaming fields or changing types breaks deserialization. Protocol Buffers and MessagePack are more compact and support schema evolution better. Some systems use the same serialization format for snapshots and events to minimize complexity.

Snapshot Invalidation

When the event schema changes significantly, existing snapshots may become incompatible with the current code. The system must handle this gracefully. One approach is to discard all snapshots after a schema migration and let them be recreated naturally as events are processed. Another approach is to version the snapshot format and maintain upcasters similar to event upcasters. The simplest approach is to check whether the snapshot format matches the expected version and fall back to full replay if it does not.

Saga and Process Manager Patterns

In a distributed system, a single business operation often spans multiple aggregates and services. For example, placing an order might involve reserving inventory, charging the customer's credit card, and scheduling shipment. These operations must be coordinated, and failures must be handled gracefully. The saga and process manager patterns provide the coordination logic for multi-step, long-running transactions in event-sourced systems.

Saga Pattern

A saga is a sequence of local transactions that are either completed successfully or compensated on failure. In event sourcing, each step of the saga is a command that produces events. If a later step fails, the saga emits compensating events to undo the earlier steps. For example, if the credit card charge fails after inventory was reserved, the saga emits an InventoryReservationCancelled event to free the reserved stock.

Sagas come in two flavors: choreography-based and orchestration-based. In choreography-based sagas, each service listens for events and decides what to do next. The coordination is decentralized but can be hard to track. In orchestration-based sagas, a central coordinator (the process manager) tells each service what to do and handles failures centrally.

Process Manager Pattern

A process manager is an event-sourced component that orchestrates a multi-step workflow. It is itself an aggregate: it has its own event stream, its own state, and it emits events in response to external events. The process manager listens for events from other aggregates, evaluates the current state of the workflow, and issues commands to other aggregates. Because the process manager is event-sourced, it is crash-resilient: if the service restarts, the process manager replays its event stream and picks up where it left off.

C#
public class OrderFulfillmentSaga : AggregateRoot
{
    private string _orderId;
    private bool _paymentReceived;
    private bool _inventoryReserved;
    private bool _shipmentScheduled;
    private SagaStatus _status;
    
    public void Start(string orderId)
    {
        if (_status != SagaStatus.None)
            throw new InvalidOperationException("Saga already started.");
            
        ApplyEvent(new OrderFulfillmentStarted { OrderId = orderId });
    }
    
    public void HandlePaymentReceived()
    {
        if (_status != SagaStatus.AwaitingPayment)
            throw new InvalidOperationException("Not awaiting payment.");
            
        ApplyEvent(new PaymentConfirmed { OrderId = _orderId });
        
        if (_inventoryReserved)
        {
            ApplyEvent(new InitiateShipment { OrderId = _orderId });
        }
    }
    
    public void HandleInventoryReserved()
    {
        if (_status != SagaStatus.AwaitingInventory)
            throw new InvalidOperationException("Not awaiting inventory.");
            
        ApplyEvent(new InventoryReserved { OrderId = _orderId });
        
        if (_paymentReceived)
        {
            ApplyEvent(new InitiateShipment { OrderId = _orderId });
        }
    }
    
    public void HandleShipmentScheduled()
    {
        ApplyEvent(new OrderFulfillmentCompleted { OrderId = _orderId });
    }
    
    public void HandlePaymentFailed(string reason)
    {
        ApplyEvent(new PaymentFailed { OrderId = _orderId, Reason = reason });
        ApplyEvent(new CancelInventoryReservation { OrderId = _orderId });
        ApplyEvent(new OrderFulfillmentFailed { OrderId = _orderId, Reason = reason });
    }
    
    // Apply methods omitted for brevity
}
    

Compensating Events

When a saga fails partway through, it must undo the steps that already completed. Compensating events are the mechanism for this. For example, if inventory was reserved but payment fails, the saga emits a CompensateInventoryReservation event. The inventory projection listens for this event and frees the reserved stock. Compensating events are themselves domain events that are appended to the event store, ensuring that the compensation is also durable and auditable.

Correlation and Causation

Because sagas involve multiple events across multiple streams, it is essential to track correlation. Each event should carry a CorrelationId that ties it to the original business transaction. Events may also carry a CausationId that identifies the specific event that caused this event. This creates a traceable chain across aggregate boundaries, which is invaluable for debugging, monitoring, and auditing.

Event-Driven Integration Between Bounded Contexts

In a microservices architecture, different bounded contexts need to communicate without tight coupling. Event-driven integration uses the event store as the backbone for inter-service communication. When a service emits events, other services subscribe to those events and react accordingly. This section explores the patterns and pitfalls of event-driven integration in an event-sourced system.

Context Mapping with Shared Events

Each bounded context maintains its own event store and its own set of aggregates. When a context needs to notify other contexts about something that happened, it publishes integration events. These are distinct from domain events: integration events are part of the public contract between contexts, while domain events are internal to a context. Integration events are typically published to a shared message bus (like Kafka, RabbitMQ, or EventBridge) and may be stored in a shared event log for durability.

Anti-Corruption Layer

When subscribing to events from another context, the receiving context should implement an anti-corruption layer (ACL). The ACL translates the integration event into the receiving context's internal representation. This prevents the receiving context from being coupled to the publishing context's schema. For example, the "Ordering" context might publish an OrderSubmitted integration event that contains a flat list of product IDs. The "Inventory" context's ACL translates this into an InventoryReservationRequested internal event with a different structure.

Eventual Consistency Across Contexts

When context A publishes an event and context B consumes it, there is no transactional guarantee that B will see the event immediately. This is by design: bounded contexts are independently deployable and should not require distributed transactions. The trade-off is that cross-context operations are eventually consistent. Application designers must account for this at the user experience level. For example, after placing an order, the UI might show "Order accepted, processing" rather than immediately showing the inventory reservation status.

C#
public class IntegrationEventPublisher
{
    private readonly IEventStore _eventStore;
    private readonly IMessageBus _messageBus;
    
    public async Task PublishIntegrationEventAsync<T>(
        T integrationEvent, CancellationToken ct) where T : IIntegrationEvent
    {
        // Store the integration event for durability
        await _eventStore.AppendToStreamAsync(
            $"integration-{integrationEvent.EventId}",
            -1,
            new[] { integrationEvent },
            ct);
        
        // Publish to message bus for real-time delivery
        await _messageBus.PublishAsync(integrationEvent, ct);
    }
}

public class InventoryAntiCorruptionLayer
{
    private readonly ICommandHandler _inventoryHandler;
    
    public async Task HandleOrderSubmittedAsync(OrderSubmittedIntegrationEvent e)
    {
        // Translate from the Ordering context's event
        // to the Inventory context's command
        var items = e.Items.Select(i => new ProductBooking
        {
            ProductId = MapProductId(i.Sku),
            Quantity = i.Quantity,
            SourceOrderId = e.OrderId
        }).ToList();
        
        var command = new ReserveInventoryCommand
        {
            BookingId = Guid.NewGuid().ToString(),
            Items = items
        };
        
        await _inventoryHandler.HandleAsync(command);
    }
    
    private string MapProductId(string externalSku)
    {
        // Translate external SKU to internal product ID
        return _productMappingService.ToInternalId(externalSku);
    }
}
    

Idempotency and Exactly-Once Processing

Integration events may be delivered more than once due to network failures, retries, or duplicate publications. Receiving services must be idempotent. The standard approach is to track processed event IDs in a deduplication table. Before processing an event, the service checks whether it has already processed that event ID. If it has, the event is skipped. This table should be in the same database as the service's read model to ensure transactional consistency.

Rebuild and Replay Mechanisms

One of the most powerful features of event sourcing is the ability to rebuild the entire system from the event stream. Whether recovering from a disaster, fixing a bug in a projection, or deploying a new read model, the ability to replay events from the beginning is a safety net that traditional systems lack. This section covers the rebuild and replay mechanisms.

Scenarios for Rebuild

There are several scenarios where a full rebuild is necessary. The most common is fixing a bug in a projection. If a projection has been processing events incorrectly for months, the read model is corrupted. The fix is to correct the projection code and rebuild the read model from scratch by replaying all events. Another scenario is schema migration: when the read model schema changes, it is often easier to rebuild than to write complex migration scripts. Disaster recovery is another scenario: if the read database is lost, you can recreate it entirely from the event store.

Performance of Rebuild

Replaying millions of events is not instantaneous. The rebuild time depends on the number of events, the complexity of the projection logic, and the speed of the event store and the read database. For large systems, a full rebuild can take hours. Strategies to accelerate rebuilds include parallelizing projection processing (partitioning events by a key and processing each partition in parallel), using snapshot-based rebuilds (starting from the nearest snapshot), and running rebuilds on dedicated infrastructure that does not affect production traffic.

Read-Only Mode During Rebuild

During a rebuild, the read side is unavailable or serves stale data. The system must handle this gracefully. One approach is to run the rebuild on a secondary instance and swap it in when complete. Another approach is to serve queries from the event store directly during the rebuild, accepting degraded performance. A third approach is to use blue-green deployment: build the new read model alongside the old one and cut over when ready.

C#
public class FullSystemRebuilder
{
    private readonly IEventStore _eventStore;
    private readonly IReadOnlyList<IProjection> _projections;
    private readonly ILogger _logger;
    
    public async Task RebuildAllAsync(CancellationToken ct)
    {
        _logger.LogInformation("Starting full system rebuild");
        
        // Reset all projections
        foreach (var projection in _projections)
        {
            await projection.ResetAsync(ct);
            _logger.LogInformation("Reset projection: {Name}", projection.Name);
        }
        
        // Read all events in batches
        long position = 0;
        const int batchSize = 5000;
        int totalProcessed = 0;
        
        while (!ct.IsCancellationRequested)
        {
            var batch = await _eventStore.ReadAllEventsAsync(
                position, batchSize, ct);
            
            if (batch.Count == 0)
                break;
            
            foreach (var @event in batch)
            {
                foreach (var projection in _projections)
                {
                    await projection.HandleAsync(@event, ct);
                }
                totalProcessed++;
            }
            
            position += batch.Count;
            
            if (totalProcessed % 50000 == 0)
            {
                _logger.LogInformation(
                    "Rebuild progress: {Count} events processed", totalProcessed);
            }
        }
        
        // Save final checkpoints
        foreach (var projection in _projections)
        {
            await projection.SaveCheckpointAsync(position, ct);
        }
        
        _logger.LogInformation(
            "Rebuild complete: {Count} events processed", totalProcessed);
    }
}
    

Incremental Rebuild

In some cases, only a subset of events needs to be replayed. For example, if a bug affected only events of a specific type, you can rebuild by replaying only those events. This is more efficient than a full rebuild but requires that the projection logic is independent of the order with respect to other event types. In practice, most projections are not independent: the state depends on the order of all events. Incremental rebuild is most useful for projections that maintain separate view per aggregate and can be rebuilt independently.

Checkpoint Management

Each projection maintains a checkpoint that indicates how far it has progressed through the event stream. Checkpoints are stored in a durable store and updated atomically with the read model. During normal operation, the checkpoint advances as events are processed. During a rebuild, the checkpoint is reset to 0 (or to the position after the last valid snapshot). Care must be taken to handle concurrent updates to the checkpoint: if the projection is processing live events while a rebuild is running (on a separate instance), the checkpoints can conflict. Most production systems avoid this by taking projections offline during a rebuild, or by running the rebuild on a separate copy of the read model.

Consistency Models — Eventual, Strong, Causal

Event sourcing and CQRS introduce important trade-offs in consistency guarantees. Understanding the consistency models available and how to choose between them is essential for designing a system that meets business requirements without over-engineering.

Eventual Consistency

Eventual consistency is the default consistency model for event-sourced CQRS systems. When a command is processed and events are appended to the event store, there is a delay before the projections update the read models. During this window, a query might return stale data. The duration of the window depends on the projection infrastructure: with in-memory projections it can be microseconds; with Kafka-based projections it can be hundreds of milliseconds; with batch projections it can be minutes or hours.

Eventual consistency is acceptable for many use cases. A user who submits an order does not need to see the order in the "submitted orders" list instantly; a confirmation message is sufficient. However, some operations require stronger guarantees. For example, a fraud detection system that checks whether an account has been used from two different locations in the last five minutes cannot tolerate stale read models.

Strong Consistency Within the Aggregate

Within a single aggregate, event sourcing provides strong consistency. The aggregate root enforces invariants synchronously before emitting events. The event store ensures that concurrent writes to the same stream are serialized via optimistic concurrency. This means that the write side is strongly consistent within the aggregate boundary. For example, two concurrent attempts to submit the same order will result in exactly one succeeding; the other will get a concurrency exception. This is the same guarantee you would get from a database transaction on a single row.

Causal Consistency

Causal consistency bridges the gap between eventual and strong consistency. It ensures that if event A caused event B (e.g., a command that processed event A emitted event B), then any observer that sees B will also see A. In other words, causally related events are always delivered in order. Causal consistency is important for user experience: if a user sees the result of a command, they should also see the events that led to that result. Most event stores and message brokers support causal consistency through partitioning: events with the same partition key are delivered in order, and causal relationships are maintained by assigning the same partition key to causally related events.

C#
public class ConsistencyManager
{
    private readonly IEventStore _eventStore;
    private readonly IReadModel _readModel;
    
    public async Task<bool> WaitForConsistencyAsync(
        string aggregateId, 
        long expectedVersion, 
        TimeSpan timeout, 
        CancellationToken ct)
    {
        var deadline = DateTime.UtcNow.Add(timeout);
        
        while (DateTime.UtcNow < deadline)
        {
            var readModelVersion = await _readModel.GetVersionAsync(aggregateId, ct);
            
            if (readModelVersion >= expectedVersion)
                return true;
            
            await Task.Delay(50, ct);
        }
        
        return false;
    }
}

public class CommandResult
{
    public bool Success { get; set; }
    public string ErrorMessage { get; set; }
    public long AggregateVersion { get; set; }
}

public class CommandApiController
{
    public async Task<ActionResult> PlaceOrder(PlaceOrderCommand command)
    {
        var result = await _commandHandler.HandleAsync(command);
        
        if (!result.Success)
            return BadRequest(result.ErrorMessage);
        
        // Optionally wait for read model to catch up
        // This is useful when the client needs to see the result immediately
        if (command.RequireConsistentRead)
        {
            await _consistencyManager.WaitForConsistencyAsync(
                command.OrderId, 
                result.AggregateVersion,
                TimeSpan.FromSeconds(5));
        }
        
        return Ok(new { 
            orderId = command.OrderId, 
            version = result.AggregateVersion 
        });
    }
}
    

Read-Your-Writes Consistency

A common requirement is "read-your-writes": after a user performs a write, they should see the effects of that write in subsequent reads. This is a weaker guarantee than strong consistency but stronger than pure eventual consistency. It can be achieved by routing the user's reads to a projection that is guaranteed to have processed the user's last write. This is typically done by associating the user's session with a specific projection instance and ensuring that writes are processed by that instance before reads are allowed.

Consistency Ladder

Different operations within the same system may require different consistency levels. The consistency ladder pattern assigns a consistency level to each operation based on its requirements. For example, "get my recent orders" might use read-your-writes, "get all orders for admin report" might use eventual consistency, and "check for duplicate payment" might use strong consistency by reading directly from the event store. Choosing the right consistency level for each operation is a key skill in designing event-sourced systems.

Different query types use different read paths based on their consistency requirements. Strong consistency queries bypass projections and read directly from the event store, at the cost of higher latency and lower throughput.

Event Store Implementations — EventStoreDB, Marten, Custom

Choosing the right event store implementation is a critical architectural decision. This section compares the three main options: purpose-built databases (EventStoreDB), libraries that add event sourcing to existing databases (Marten), and custom implementations on top of relational databases or message brokers.

EventStoreDB

EventStoreDB is a purpose-built database for event sourcing. It was created by Greg Young, one of the pioneers of CQRS and event sourcing. EventStoreDB provides an append-only log with optimized storage, built-in subscriptions (including persistent subscriptions with checkpoint management), projection support (JavaScript-based projections that run inside the database), and HTTP/gRPC APIs. It is available as a commercial product with enterprise features (clustering, encryption, LDAP) and as a community edition. EventStoreDB excels at high write throughput and provides strong guarantees around concurrency and ordering. Its main drawback is operational complexity: running a cluster requires expertise, and the ecosystem is smaller than PostgreSQL or Kafka.

Marten

Marten is a .NET library that turns PostgreSQL into an event store and document database. It provides a clean API for appending events, loading aggregates, and building projections. Marten leverages PostgreSQL's JSONB data type for storing event data and its transactional guarantees for consistency. Projections are defined in C# and can run as synchronous (inline) or asynchronous (background). Marten also supports event metadata, multi-tenancy, and LINQ-based queries on the event stream. For .NET shops that already use PostgreSQL, Marten is often the most natural choice. It avoids the operational overhead of a separate event store while providing a mature event sourcing platform.

C#
// Marten-based event sourcing example
public class MartenOrderRepository : IOrderRepository
{
    private readonly IDocumentStore _store;
    
    public MartenOrderRepository(IDocumentStore store)
    {
        _store = store;
    }
    
    public async Task<Order> LoadAsync(string orderId)
    {
        await using var session = _store.LightweightSession();
        
        var aggregate = await session.Events.AggregateStreamAsync<Order>(orderId);
        return aggregate;
    }
    
    public async Task SaveAsync(Order order)
    {
        await using var session = _store.LightweightSession();
        
        // Append uncommitted events
        session.Events.Append(
            order.Id, 
            order.GetUncommittedEvents().ToArray());
        
        // Take snapshot if threshold reached
        if (order.Version % 100 == 0)
        {
            session.Events.AddSnapshot(order.Id, order);
        }
        
        await session.SaveChangesAsync();
        order.ClearUncommittedEvents();
    }
}

// Marten projection definition
public class OrderListProjection : 
    ViewProjection<OrderListEntry, string>
{
    public OrderListProjection()
    {
        ProjectEvent<OrderPlaced>(evt => evt.OrderId,
            (entry, evt) =>
            {
                entry.OrderId = evt.OrderId;
                entry.CustomerId = evt.CustomerId;
                entry.Total = evt.Items.Sum(i => i.Quantity * i.UnitPrice);
                entry.Status = "Open";
                entry.PlacedAt = evt.PlacedAt;
            });
        
        ProjectEvent<OrderSubmitted>(evt => evt.OrderId,
            (entry, evt) => entry.Status = "Submitted");
        
        ProjectEvent<OrderShipped>(evt => evt.OrderId,
            (entry, evt) => entry.Status = "Shipped");
    }
}
    

Custom Implementation on PostgreSQL

For teams that need full control or have specific requirements not met by existing products, building a custom event store on top of PostgreSQL (or another relational database) is a viable option. A minimal implementation requires two tables: one for events (stream_id, version, event_type, event_data, metadata, created_at) and one for streams (stream_id, version, snapshot_data). Optionally, a checkpoint table tracks projection progress. PostgreSQL's JSONB and stored procedures make this implementation surprisingly simple and performant.

SQL
CREATE TABLE IF NOT EXISTS events (
    id BIGSERIAL PRIMARY KEY,
    stream_id VARCHAR(255) NOT NULL,
    version BIGINT NOT NULL,
    event_type VARCHAR(255) NOT NULL,
    event_data JSONB NOT NULL,
    metadata JSONB NOT NULL DEFAULT '{}',
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    UNIQUE(stream_id, version)
);

CREATE INDEX idx_events_stream_id ON events (stream_id, version);
CREATE INDEX idx_events_created_at ON events (created_at);

CREATE TABLE IF NOT EXISTS snapshots (
    stream_id VARCHAR(255) PRIMARY KEY,
    version BIGINT NOT NULL,
    snapshot_data JSONB NOT NULL,
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

CREATE TABLE IF NOT EXISTS projection_checkpoints (
    projection_name VARCHAR(255) PRIMARY KEY,
    last_position BIGINT NOT NULL DEFAULT 0,
    updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
    

Kafka as Event Store

Apache Kafka is sometimes used as an event store because of its append-only log, high throughput, and partitioning. However, Kafka was designed for message streaming, not event sourcing. Key differences include: Kafka topics are append-only logs without the concept of stream-level concurrency control; Kafka does not support loading a single event stream by aggregate ID efficiently (you must scan the entire topic or use a compacted topic with key-value semantics); Kafka's consumer offset management replaces checkpoint management. While Kafka can be hacked into an event store, it is usually better to use a purpose-built event store or a relational database.

Testing Event-Sourced Systems

Testing event-sourced systems requires a different approach than testing CRUD systems. The core challenge is that state is derived from events, so tests must verify both the command processing logic (does the aggregate emit the correct events?) and the projection logic (does the projection build the correct read model?). The event stream itself provides a natural structure for test automation: given a sequence of events, the system should produce a deterministic output.

Test Structure: Given-When-Then

Event-sourced tests follow the Given-When-Then pattern. Given a set of previously stored events, when a command is executed, then a specific set of events should be emitted. This pattern maps directly to the aggregate's replay-and-emit cycle. The test replays the "given" events to build the aggregate state, executes the "when" command, and asserts that the "then" events match the expected output.

C#
[TestClass]
public class OrderAggregateTests
{
    [TestMethod]
    public void Submit_EmptyOrder_ThrowsException()
    {
        var order = new Order();
        order.LoadFromHistory(new IDomainEvent[]
        {
            new OrderPlaced
            {
                OrderId = "ORD-001",
                CustomerId = "CUST-001",
                Items = Array.Empty<OrderItem>()
            }
        });
        
        Assert.ThrowsException<InvalidOperationException>(
            () => order.Submit());
    }
    
    [TestMethod]
    public void Submit_OrderWithItems_EmitsOrderSubmitted()
    {
        var order = new Order();
        order.LoadFromHistory(new IDomainEvent[]
        {
            new OrderPlaced
            {
                OrderId = "ORD-001",
                CustomerId = "CUST-001",
                Items = new[]
                {
                    new OrderItem 
                    { 
                        Sku = "SKU-001", 
                        Quantity = 2, 
                        UnitPrice = 10.99m 
                    }
                }
            }
        });
        
        order.Submit();
        
        var events = order.GetUncommittedEvents();
        Assert.AreEqual(1, events.Count);
        
        var submitted = events[0] as OrderSubmitted;
        Assert.IsNotNull(submitted);
        Assert.AreEqual("ORD-001", submitted.OrderId);
    }
    
    [TestMethod]
    public void AddItem_ToSubmittedOrder_ThrowsException()
    {
        var order = new Order();
        order.LoadFromHistory(new IDomainEvent[]
        {
            new OrderPlaced
            {
                OrderId = "ORD-001",
                CustomerId = "CUST-001",
                Items = new[]
                {
                    new OrderItem { Sku = "SKU-001", Quantity = 1, UnitPrice = 5m }
                }
            },
            new OrderSubmitted
            {
                OrderId = "ORD-001"
            }
        });
        
        Assert.ThrowsException<InvalidOperationException>(
            () => order.AddItem(
                new OrderItem { Sku = "SKU-002", Quantity = 1, UnitPrice = 15m }));
    }
}
    

Testing Projections

Projection tests are similar: given a sequence of events, the projection should produce a specific read model state. These tests use an in-memory read model store to avoid database dependencies. The test feeds events to the projection and then asserts that the read model contains the expected data.

C#
[TestClass]
public class OrderListProjectionTests
{
    [TestMethod]
    public async Task OrderPlaced_CreatesEntry()
    {
        var store = new InMemoryReadModelStore();
        var projection = new OrderListProjection(store);
        
        await projection.HandleAsync(new OrderPlaced
        {
            OrderId = "ORD-001",
            CustomerId = "CUST-001",
            Items = new[]
            {
                new OrderItem { Sku = "A", Quantity = 2, UnitPrice = 10m }
            }
        }, default);
        
        var entry = await store.GetAsync<OrderListEntry>("ORD-001");
        Assert.IsNotNull(entry);
        Assert.AreEqual("ORD-001", entry.OrderId);
        Assert.AreEqual("Open", entry.Status);
        Assert.AreEqual(20m, entry.Total);
    }
    
    [TestMethod]
    public async Task OrderSubmitted_UpdatesStatus()
    {
        var store = new InMemoryReadModelStore();
        var projection = new OrderListProjection(store);
        
        // Given
        await projection.HandleAsync(new OrderPlaced
        {
            OrderId = "ORD-001",
            CustomerId = "CUST-001",
            Items = new[] { new OrderItem { Sku = "A", Quantity = 1, UnitPrice = 5m } }
        }, default);
        
        // When
        await projection.HandleAsync(new OrderSubmitted
        {
            OrderId = "ORD-001"
        }, default);
        
        // Then
        var entry = await store.GetAsync<OrderListEntry>("ORD-001");
        Assert.AreEqual("Submitted", entry.Status);
    }
}
    

Integration Testing

Integration tests verify the entire pipeline: command handler, event store, subscription, projection, and read model. These tests typically run against a real event store (or an in-memory implementation) and a real read database (or a test container). Integration tests are slower and more brittle than unit tests, so they should be reserved for critical paths and edge cases.

Event Store Test Doubles

In-memory implementations of the event store are invaluable for testing. They provide the same API as the real event store but store events in memory. This enables fast, deterministic tests without external dependencies. The in-memory event store should enforce the same concurrency rules as the production implementation to catch concurrency bugs early.

Migration from CRUD to Event Sourcing

Migrating an existing CRUD system to event sourcing is a significant undertaking. It requires careful planning, incremental adoption, and strategies for maintaining data consistency during the transition. This section provides a practical guide for migrating a live system without downtime.

Migration Strategies

The safest migration strategy is the strangler fig pattern: build the event-sourced system alongside the existing CRUD system, route new writes to both systems, and gradually shift reads to the new event-sourced projection. This allows you to validate the event-sourced system against the existing system before cutting over. During the migration, the CRUD system remains the primary source of truth for reads, and the event store is populated by intercepting writes to the CRUD system.

Another strategy is to migrate one aggregate type at a time. Start with a bounded context that has the highest need for auditability or temporal queries. Build the event store and projections for that aggregate, route its commands through the new pipeline, and keep the rest of the system unchanged. This incremental approach reduces risk and allows the team to gain experience with event sourcing before expanding.

Bootstrapping the Event Store

When you start using event sourcing, the event store is empty. You need to populate it with events that represent the existing state of the system. This process is called bootstrapping. For each existing entity, you determine its state and emit a single "bootstrapping" event (or a sequence of events) that captures its history. For example, for an existing order, you might emit an OrderPlaced event with the current state of the order. This gives the event store a starting point for replaying the entity's history.

Bootstrapping events are flagged with a special metadata field so that projections can handle them appropriately. Some projections might skip bootstrapping events entirely (if they are populated from the CRUD database directly), while others might process them normally. The key is to ensure that the read models are consistent after bootstrapping.

C#
public class MigrationBootstrapper
{
    private readonly IEventStore _eventStore;
    private readonly ICrudRepository _crudRepo;
    
    public async Task BootstrapOrdersAsync(CancellationToken ct)
    {
        var orders = await _crudRepo.GetAllOrdersAsync(ct);
        int count = 0;
        
        foreach (var order in orders)
        {
            var streamId = $"order-{order.Id}";
            
            // Check if stream already exists (idempotent migration)
            var version = await _eventStore.GetStreamVersionAsync(streamId, ct);
            if (version > 0)
                continue;
            
            var bootstrapEvent = new OrderPlaced
            {
                OrderId = order.Id,
                CustomerId = order.CustomerId,
                Items = order.Items.Select(i => new OrderItem
                {
                    Sku = i.Sku,
                    Name = i.Name,
                    Quantity = i.Quantity,
                    UnitPrice = i.UnitPrice
                }).ToList(),
                PlacedAt = order.CreatedAt
            };
            
            // Add migration metadata
            var metadata = new Dictionary<string, object>
            {
                ["$migrationSource"] = "CRUD-Bootstrap",
                ["$migrationVersion"] = "1.0",
                ["$bootstrappedAt"] = DateTime.UtcNow
            };
            
            await _eventStore.AppendToStreamAsync(
                streamId, -1, new[] { bootstrapEvent }, metadata, ct);
            
            count++;
        }
        
        _logger.LogInformation("Bootstrapped {Count} orders", count);
    }
}
    

Dual-Write Pattern

During migration, the system writes to both the CRUD database and the event store. This ensures that if the event store has a problem, the CRUD system can continue to serve requests. Dual writes must be handled carefully to avoid inconsistency. The simplest approach is to write to the event store synchronously in the same request as the CRUD write, using a transactional outbox pattern. If the event store write fails, the CRUD write is also rolled back. If the CRUD write succeeds but the event store write fails (because of a network issue), the outbox ensures that the event is eventually written.

Validating the Migration

During migration, you should continuously validate that the event-sourced read models match the CRUD read models. Run comparison queries that fetch data from both systems and compare the results. Any discrepancies indicate a bug in the event sourcing pipeline that must be fixed before cutting over. This validation can run as a background job that samples a percentage of queries.

Interview Q&A

This section covers common interview questions about event sourcing and CQRS at senior+ level. Each question includes a concise answer suitable for system design interviews.

Q1: What is the difference between event sourcing and CRUD?

CRUD stores the current state of an entity and overwrites it on every update, losing historical information. Event sourcing stores every state change as an immutable event in an append-only log. The current state is derived by replaying events. Event sourcing provides native auditability, temporal queries, and a complete event history. CRUD is simpler and provides stronger immediate consistency at the cost of losing history.

Q2: How do you handle concurrency in an event-sourced system?

Concurrency is handled through optimistic concurrency control at the event stream level. Each stream has a version number that increments with every appended event. When a command handler appends events to a stream, it provides the expected version. If the stream has advanced because another writer appended events, the write fails with a concurrency exception. The handler must then reload the aggregate, reapply business logic, and retry. This ensures that no events are lost and that invariants are maintained, without requiring distributed locks or transactions across streams.

Q3: How do you evolve event schemas over time?

Event schemas evolve through versioning and upcasting. Each event type carries a version number. When a new version is introduced, old events are upcast to the new version at read time. Upcasters are small functions that transform events from one version to the next. The application code works with the latest version exclusively, keeping the logic clean. For larger changes, you can add new event types alongside old ones. Forward compatibility is ensured by making all event fields optional and ignoring unknown fields during deserialization.

Q4: What is an aggregate and how do you decide the boundaries?

An aggregate is a cluster of domain objects treated as a single unit for consistency. The aggregate root is the entry point, and all invariants are enforced within the aggregate boundary. Boundaries are determined by consistency requirements: if two entities must always be consistent with each other, they belong in the same aggregate. The classic heuristic is "one aggregate per transaction." Aggregates should be as small as possible while still maintaining invariants. Model the aggregate boundary based on how the business changes state, not on how the data is structured.

Q5: How do you achieve strong consistency if needed?

Strong consistency is available within a single aggregate. The aggregate root enforces invariants synchronously, and the event store guarantees serialized writes to a stream. For cross-aggregate consistency, you have three options: using a saga with compensating actions for eventual consistency; reading directly from the event store (bypassing projections) at the cost of performance; or combining multiple aggregates into a single larger aggregate. The third option is rarely recommended because it creates large, unwieldy aggregates. Most systems accept eventual consistency for cross-aggregate scenarios.

Q6: How do snapshots work and when should you use them?

Snapshots cache the state of an aggregate at a specific version to speed up loading. Instead of replaying the entire event stream, you load the latest snapshot and replay only the events that occurred after it. Snapshots are taken at regular intervals (every N events). They should be used when aggregates accumulate thousands of events and replay time becomes a performance bottleneck. Snapshots add complexity: they must be serialized, stored, and versioned. Start without snapshots and add them when profiling shows they are needed.

Q7: How do you test an event-sourced system?

Testing follows the Given-When-Then pattern. Unit tests verify that an aggregate emits the correct events given a sequence of previous events and a command. Projection tests verify that the correct read model state is built from events. Integration tests cover the full pipeline: command handler, event store, subscription, projection, and read model. In-memory implementations of the event store and read model store are used for fast, deterministic unit tests. The event stream provides a natural test structure: tests are explicit about the state history, making them clearer than traditional CRUD tests.

Q8: How do you migrate from a CRUD system to event sourcing?

Migration uses the strangler fig pattern. First, bootstrap the event store by emitting events that represent the existing state from the CRUD database. Then implement dual writes: write to both CRUD and the event store. Run comparison queries to validate that the event-sourced read models match CRUD reads. Once validation passes, cut over by routing reads to the event-sourced projections. Finally, retire the CRUD code and database. Migrate one aggregate at a time to reduce risk. The dual-write phase should run for at least a few weeks to catch edge cases.

Q9: What are the trade-offs of using Kafka as an event store?

Kafka provides high throughput, partitioning, and durable append-only logs, which align with event sourcing requirements. However, Kafka lacks stream-level concurrency control, making it difficult to implement optimistic concurrency for aggregate streams. Loading a specific stream requires either a compacted topic (which is key-value, not event-sourced) or scanning the entire topic. Kafka's consumer offset management replaces checkpoint management, but the semantics differ. Generally, purpose-built event stores (EventStoreDB, Marten) or relational databases are better choices. Kafka is best used as the integration event bus between bounded contexts.

Q10: How do you handle long-running sagas with many steps?

Long-running sagas are modeled as process managers, which are themselves event-sourced aggregates. The process manager has its own event stream and state. It listens for events, evaluates workflow state, and issues commands to other aggregates. Because the process manager is event-sourced, it is crash-resilient: restarting it replays the event stream and resumes the workflow. State persistence means the saga can run for days or weeks. Compensation logic is built into the saga: if a step fails, the saga emits compensating events that undo earlier steps. The key is to make the saga as simple as possible and to test it thoroughly with all possible failure paths.