system-design48 min read

How to Design a Knowledge Base and Wiki Platform — A Senior+ Guide | Ayodhyya

How to Design a Knowledge Base & Wiki Platform

Building a Production-Grade Confluence/Notion — Editing, Collaboration, Search, Permissions & Beyond

Senior+ System Design Guide 12,000+ Words Architecture · APIs · Code · Cost

1. Overview & Requirements

A Knowledge Base & Wiki Platform is a collaborative workspace where teams create, organize, and share documents, wikis, databases, and project knowledge. Think Confluence, Notion, or Coda. The core promise is that any team member can find the right information in seconds, edit it collaboratively, and trust that it is up-to-date and secure.

At its heart, a wiki platform is a content management system with real-time collaboration superpowers. Unlike a traditional CMS, every page is simultaneously a document, a database row, and a collaborative canvas. The challenge is building a system that supports rich formatting, structured data, real-time editing by dozens of concurrent users, granular permissions, full-text search, and offline access — all while maintaining sub-50ms latency for keystroke rendering.

Functional Requirements

  • Document Editing: Rich-text editor with block-based architecture (paragraphs, headings, lists, tables, code blocks, images, embeds, databases).
  • Real-Time Collaboration: Multiple users editing the same page simultaneously with CRDT-based merging.
  • Page Hierarchy: Spaces, Folders, Pages, and Sub-pages with drag-and-drop reordering.
  • Templates: Reusable page templates (meeting notes, sprint retrospectives) and database templates (Kanban, spreadsheet, gallery).
  • Full-Text Search: Search across all pages, including content inside images (OCR), code blocks, and attachments.
  • Versioning: Every edit produces a version; users can view diffs and restore any previous version.
  • Comments & Annotations: Inline comments on text selections, resolved/unresolved threads, @mentions in comments.
  • Permissions: RBAC at workspace, space, folder, and page levels. Public share links with expiry.
  • Embeds: Inline images, videos (YouTube), code blocks with syntax highlighting, database views, Figma frames.
  • Table / Database Blocks: Spreadsheet-like tables with sorting, filtering, formula columns, and linked databases.
  • Kanban Board Views: Visual board views for database blocks with drag-and-drop between columns.
  • @Mentions & Notifications: Mention users or groups; deliver notifications via in-app, email, Slack, and push.
  • Export: Export pages as PDF, Markdown, or HTML.
  • Import: Import from Confluence, Google Docs, Markdown, HTML, Notion, Word.
  • API: RESTful + GraphQL API for programmatic access and third-party integrations.
  • Offline Mode: Edit pages offline; sync when connectivity resumes.
  • Knowledge Graph: Automatic backlinks between pages; graph visualization of page relationships.
  • Auto-Generated TOC: Table of contents generated from page headings.
  • Analytics: Page views, popular pages, recently edited, contributor stats.
  • Audit Log: Track all user actions (page created, edited, permissions changed, deleted).

Non-Functional Requirements

AttributeTarget
Latency (editor keystroke)< 50ms perceived, < 200ms server round-trip
Availability99.95% (SaaS), 99.99% (enterprise tier)
Throughput100K concurrent editors, 10M pages indexed
Data Durability99.999999999% (11 nines) via multi-region replication
Search Latency< 200ms (p95) for full-text queries
ConsistencyStrong for permissions; eventual for search indexing
ComplianceSOC 2 Type II, GDPR, HIPAA (enterprise), FedRAMP
Offline SupportFull read/write for 7+ days without connectivity
Design Insight

The fundamental tension in a wiki platform is between rich formatting and structured data. Notion solved this with a block-based model where every element (text, table, database, embed) is a "block" with a uniform schema. We adopt the same philosophy — it simplifies CRDT collaboration, versioning, and API design.

2. High-Level Architecture

graph TB subgraph Clients["Client Layer"] WEB["Web App (React + Blocknote)"] DESKTOP["Desktop App (Electron)"] MOBILE["Mobile App (React Native)"] API_CLIENT["API Clients (REST / GraphQL)"] end subgraph EdgeLayer["Edge Layer"] CDN["CDN (CloudFront)"] WSS["WebSocket Gateway"] LB["Load Balancer (ALB)"] end subgraph AppLayer["Application Layer"] API_GW["API Gateway (Kong)"] AUTH_SVC["Auth Service (JWT + RBAC)"] DOC_SVC["Document Service"] COLLAB_SVC["Collaboration Service (CRDT)"] SEARCH_SVC["Search Service (Elasticsearch)"] NOTIF_SVC["Notification Service"] EXPORT_SVC["Export Service (Puppeteer)"] IMPORT_SVC["Import Service"] TEMPLATE_SVC["Template Service"] ANALYTICS_SVC["Analytics Service"] AUDIT_SVC["Audit Service"] GRAPH_SVC["Knowledge Graph Service"] VERSION_SVC["Version Service"] EMBED_SVC["Embed / Asset Service"] end subgraph DataLayer["Data Layer"] PG["PostgreSQL (Metadata)"] RDS["Redis Cluster (Sessions + Cache)"] ES["Elasticsearch (Search)"] S3["S3 (Assets + Snapshots)"] RABBIT["RabbitMQ (Event Bus)"] CLICKHOUSE["ClickHouse (Analytics)"] NEO4J["Neo4j (Knowledge Graph)"] end WEB --> CDN DESKTOP --> LB MOBILE --> LB API_CLIENT --> CDN CDN --> LB WSS --> LB LB --> API_GW API_GW --> AUTH_SVC API_GW --> DOC_SVC API_GW --> SEARCH_SVC API_GW --> EXPORT_SVC API_GW --> IMPORT_SVC API_GW --> TEMPLATE_SVC API_GW --> ANALYTICS_SVC API_GW --> AUDIT_SVC DOC_SVC --> PG DOC_SVC --> S3 DOC_SVC --> RABBIT COLLAB_SVC --> RDS COLLAB_SVC --> RABBIT SEARCH_SVC --> ES SEARCH_SVC --> RABBIT NOTIF_SVC --> RABBIT NOTIF_SVC --> RDS VERSION_SVC --> PG VERSION_SVC --> S3 EMBED_SVC --> S3 ANALYTICS_SVC --> CLICKHOUSE ANALYTICS_SVC --> RABBIT AUDIT_SVC --> RABBIT AUDIT_SVC --> PG GRAPH_SVC --> NEO4J GRAPH_SVC --> RABBIT WEB --> WSS DESKTOP --> WSS MOBILE --> WSS

Key Architectural Decisions

DecisionChoiceRationale
Document FormatBlock-based JSON (CRDT)Enables granular merge, per-block permissions, API-friendly
Real-Time SyncYjs CRDT via WebSocketPeer-to-peer merging without central server; handles offline
Primary DBPostgreSQL + CitusStrong consistency for metadata; Citus for horizontal partitioning
SearchElasticsearch 8.xProven full-text search with analyzers, facets, and aggregations
Asset StorageS3 + CloudFrontDurable blob storage; CDN for low-latency delivery
Event BusRabbitMQDecouples services; reliable event delivery for indexing, notifications
Knowledge GraphNeo4jNative graph DB for backlinks and page relationship queries
AnalyticsClickHouseColumnar store for fast aggregation queries on page views
ExportPuppeteer on LambdaHeadless Chrome for pixel-perfect PDF generation

Service Communication Patterns

Services communicate through two primary patterns:

  • Synchronous (gRPC/HTTP): Used for user-facing operations that require immediate response — page load, save, search, permission check. API Gateway routes requests to the appropriate service.
  • Asynchronous (RabbitMQ events): Used for background work — search indexing, notification delivery, analytics collection, knowledge graph updates. Events are published after successful database writes and consumed by dedicated worker processes.
Why Microservices?

A wiki platform has distinct workloads: real-time collaboration (CPU-bound, stateful), search (I/O-bound, read-heavy), export (CPU-bound, bursty), and analytics (batch processing). Microservices allow independent scaling of each workload. The collaboration service needs persistent WebSocket connections; the search service needs Elasticsearch nodes; the export service needs burst compute. A monolith would force all these into a single scaling profile.

3. Document Editing — Rich Text & Block-Based Editor

The Block Model

Every page is a sequence of blocks. Each block has a unique ID, a type, content, and optional metadata. This is the fundamental unit of editing, collaboration, permissions, and versioning. The block model allows us to treat every element uniformly — whether it is a paragraph of text, an embedded video, or a full database view.

C#
public class Block
{
    public Guid Id { get; set; } = Guid.NewGuid();
    public BlockType Type { get; set; }
    public string Content { get; set; } = string.Empty;
    public Dictionary<string, object> Properties { get; set; } = new();
    public List<Block> Children { get; set; } = new();
    public Guid? ParentId { get; set; }
    public int SortOrder { get; set; }
    public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
    public DateTime UpdatedAt { get; set; } = DateTime.UtcNow;
    public Guid CreatedBy { get; set; }
}

public enum BlockType
{
    Paragraph,
    Heading1, Heading2, Heading3, Heading4, Heading5, Heading6,
    BulletedList, NumberedList, TodoList,
    Quote, Callout, Divider,
    Code,
    Image, Video, Audio, File,
    Table, Spreadsheet,
    Embed, Bookmark,
    Database, KanbanBoard, Gallery,
    Equation, TableOfContents,
    SyncedBlock, Breadcrumb
}

public class Document
{
    public Guid Id { get; set; } = Guid.NewGuid();
    public string Title { get; set; } = string.Empty;
    public Guid SpaceId { get; set; }
    public Guid? ParentPageId { get; set; }
    public List<Block> Blocks { get; set; } = new();
    public Guid OwnerId { get; set; }
    public PageStatus Status { get; set; } = PageStatus.Draft;
    public string Icon { get; set; } = "📄";
    public string CoverImageUrl { get; set; }
    public List<string> Tags { get; set; } = new();
    public Dictionary<string, string> Properties { get; set; } = new();
    public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
    public DateTime UpdatedAt { get; set; } = DateTime.UtcNow;
    public DateTime? PublishedAt { get; set; }
    public bool IsDeleted { get; set; }
    public DateTime? DeletedAt { get; set; }
}

Block Rendering Pipeline

flowchart LR A["User Input (Keystroke)"] --> B["Blocknote Editor (ProseMirror)"] B --> C["CRDT Operation (Yjs)"] C --> D["WebSocket Broadcast"] D --> E["Collaboration Service"] E --> F["Yjs Doc Persistence"] E --> G["Other Clients (Merge + Render)"] F --> H["Snapshot to PostgreSQL/S3"]

Rich Text Formatting

Inline formatting is handled at the leaf level within a block. Each block contains a list of inline segments with formatting marks. This design means that a single paragraph block can contain bold text, italic text, code spans, and links — all within a single block's content model.

C#
public class InlineSegment
{
    public string Text { get; set; } = string.Empty;
    public FormattingMarks Marks { get; set; } = new();
}

public class FormattingMarks
{
    public bool Bold { get; set; }
    public bool Italic { get; set; }
    public bool Underline { get; set; }
    public bool Strikethrough { get; set; }
    public bool Code { get; set; }
    public string? Link { get; set; }
    public string? Color { get; set; }
    public HighlightStyle? Highlight { get; set; }
}

public enum HighlightStyle
{
    Gray, Brown, Orange, Yellow, Green, Blue, Purple, Pink, Red
}

Code Block Implementation

Code blocks support syntax highlighting via Shiki (for the frontend) and store language metadata. Execution can be optionally enabled for sandboxed languages using WebAssembly runtimes (JavaScript via V8 isolates, Python via Pyodide). The code block stores both the source and execution output to support collaborative debugging.

C#
public class CodeBlockProperties
{
    public string Language { get; set; } = "plaintext";
    public string Code { get; set; } = string.Empty;
    public bool ShowLineNumbers { get; set; }
    public bool IsExecutable { get; set; }
    public string? ExecutionOutput { get; set; }
    public List<CodeExecutionResult> ExecutionHistory { get; set; } = new();
}

public class CodeExecutionResult
{
    public string Output { get; set; }
    public bool IsError { get; set; }
    public int ExitCode { get; set; }
    public TimeSpan ExecutionTime { get; set; }
    public DateTime ExecutedAt { get; set; }
}
Editor Choice

We use Blocknote (built on ProseMirror) as the editor framework. It provides a Notion-like block editor out of the box, is open-source, supports React, and can be extended with custom block types. For the CRDT layer, we integrate Yjs as a y-prosemirror binding. Alternatively, Tiptap is another excellent option with built-in collaboration via their cloud service.

4. Real-Time Collaboration — CRDT-Based

Why CRDT Over OT?

Operational Transformation (OT) requires a central server to transform operations, creating a single point of failure and requiring careful ordering. CRDTs (Conflict-free Replicated Data Types) allow each client to apply operations independently and merge them deterministically, making them ideal for offline-first collaboration. In the OT model, a server must receive and sequence every operation before broadcasting — if the server is slow, the entire collaboration stalls. With CRDTs, clients can edit independently and merge later, with guaranteed convergence.

sequenceDiagram participant A as User A (Offline) participant S as CRDT Server (Yjs) participant B as User B (Online) participant C as User C (Online) Note over A,C: All users editing Architecture Docs page A->>A: Types "Hello" locally Note right of A: Applied locally, no connection B->>S: Types "World" via WebSocket S->>C: Broadcast Y.Update (World) C->>C: Merge - "World" A->>S: Reconnects, sends Y.Update (Hello) S->>S: Merge: "Hello" + "World" = "HelloWorld" S->>B: Broadcast merged state S->>C: Broadcast merged state

CRDT Document Manager

C#
public class CrdtDocumentManager
{
    private readonly ConcurrentDictionary<Guid, ActiveDocument> _activeDocs = new();
    private readonly IConnectionPool _connectionPool;

    public async Task<YDoc> LoadDocument(Guid documentId)
    {
        if (_activeDocs.TryGetValue(documentId, out var cached))
            return cached.YDoc;

        var snapshot = await LoadSnapshot(documentId);
        var yDoc = new YDoc();

        if (snapshot != null)
        {
            var update = YUpdate.Merge(snapshot.Updates);
            yDoc.ApplyUpdate(update);
        }

        _activeDocs.TryAdd(documentId, new ActiveDocument
        {
            Id = documentId,
            YDoc = yDoc,
            LastAccessed = DateTime.UtcNow
        });

        return yDoc;
    }

    public async Task ApplyUpdate(Guid documentId, byte[] update)
    {
        var doc = await LoadDocument(documentId);
        doc.ApplyUpdate(update);
        await DebouncedPersist(documentId, doc);
    }

    public byte[] GetStateVector(Guid documentId)
    {
        var doc = _activeDocs.GetValueOrDefault(documentId);
        return doc?.YDoc.StateVector ?? Array.Empty<byte>();
    }
}

WebSocket Protocol

Message TypeDirectionPayloadDescription
sync_step1Client to ServerStateVectorInitial sync request
sync_step2Server to ClientUpdateFull state from server
updateBoth directionsUpdateIncremental CRDT update
awarenessBoth directionsAwarenessStateCursor position, selection, user info
cursorClient to ServerCursorPositionReal-time cursor broadcast

Awareness Protocol

The awareness layer tracks ephemeral state — who is online, where their cursor is, what they have selected. This state is not persisted and is broadcast to all connected clients for a document. It gives the "multiplayer" feel to the editor, showing colored cursors and name labels for each collaborator.

C#
public class AwarenessState
{
    public Guid UserId { get; set; }
    public string DisplayName { get; set; }
    public string AvatarUrl { get; set; }
    public string Color { get; set; }  // Unique cursor color per user
    public CursorPosition? Cursor { get; set; }
    public TextSelection? Selection { get; set; }
    public DateTime LastActive { get; set; }
    public UserStatus Status { get; set; }
}

public class CursorPosition
{
    public Guid BlockId { get; set; }
    public int Offset { get; set; }
    public int Length { get; set; }
}

public class CollaborationService
{
    private readonly Dictionary<Guid, HashSet<Guid>> _documentUsers = new();
    private readonly Dictionary<Guid, AwarenessState> _userStates = new();

    public void UpdateAwareness(Guid documentId, Guid userId, AwarenessState state)
    {
        _userStates[userId] = state;

        if (_documentUsers.ContainsKey(documentId))
        {
            var clients = _documentUsers[documentId];
            foreach (var clientId in clients.Where(id => id != userId))
            {
                SendAwarenessUpdate(clientId, state);
            }
        }
    }

    public void RemoveUser(Guid documentId, Guid userId)
    {
        _documentUsers.GetValueOrDefault(documentId)?.Remove(userId);
        _userStates.Remove(userId);
        BroadcastPresence(documentId, userId, UserStatus.Offline);
    }
}
CRDT Garbage Collection

Yjs CRDTs grow unbounded as edits accumulate because tombstones (deleted content) are retained for merge correctness. Implement periodic garbage collection by taking a full snapshot, resetting the CRDT state, and deleting old updates. Run GC daily during low-traffic hours. For a document with 1M+ operations, this can reduce memory by 90%+ and storage significantly.

5. Page Hierarchy & Organization

The Hierarchy Model

A wiki is organized into a tree of Spaces, Folders, Pages, and Sub-pages. Each space is an isolated workspace (e.g., "Engineering", "Product", "Marketing"). The hierarchy is stored using the adjacency list pattern with a materialized path for efficient subtree queries. This gives us the best of both worlds: simple parent-child lookups via adjacency list, and fast subtree operations via the materialized path string.

graph TB WS["Workspace: Acme Corp"] WS --> S1["Space: Engineering"] WS --> S2["Space: Product"] WS --> S3["Space: Marketing"] S1 --> F1["Folder: Architecture"] S1 --> F2["Folder: Runbooks"] S1 --> F3["Folder: Onboarding"] F1 --> P1["System Design"] F1 --> P2["API Guidelines"] F1 --> P3["Database Schema"] P1 --> SP1["Sub-page: Auth Service"] P1 --> SP2["Sub-page: Payment Service"] S2 --> P4["Product Roadmap"] S2 --> P5["Sprint Retrospective"]
C#
public class Space
{
    public Guid Id { get; set; } = Guid.NewGuid();
    public string Name { get; set; } = string.Empty;
    public string Description { get; set; } = string.Empty;
    public string Icon { get; set; } = "📁";
    public string CoverImageUrl { get; set; }
    public Guid WorkspaceId { get; set; }
    public Guid CreatedBy { get; set; }
    public SpaceVisibility Visibility { get; set; } = SpaceVisibility.Private;
    public string MaterializedPath { get; set; } = "/";
    public int Depth { get; set; }
    public int SortOrder { get; set; }
    public bool IsArchived { get; set; }
    public DateTime CreatedAt { get; set; }
    public DateTime UpdatedAt { get; set; }
}

public class PageTreeNode
{
    public Guid Id { get; set; }
    public string Title { get; set; }
    public Guid? ParentId { get; set; }
    public string MaterializedPath { get; set; }
    public int Depth { get; set; }
    public int SortOrder { get; set; }
    public int ChildCount { get; set; }
    public string Icon { get; set; }
    public bool IsPublished { get; set; }
}

Hierarchy Service

C#
public class PageHierarchyService
{
    private readonly IDbConnection _db;

    public async Task<List<PageTreeNode>> GetSubtree(Guid spaceId, Guid? parentId = null, int maxDepth = 5)
    {
        var basePath = parentId.HasValue
            ? $"/{spaceId}/{parentId}"
            : $"/{spaceId}";

        var pages = await _db.QueryAsync<PageTreeNode>(
            @"SELECT id, title, parent_id, materialized_path, depth, sort_order,
                     child_count, icon, is_published
              FROM pages
              WHERE space_id = @SpaceId
                AND materialized_path LIKE @Path
                AND depth <= @MaxDepth
                AND is_deleted = false
              ORDER BY sort_order",
            new { SpaceId = spaceId, Path = $"{basePath}%", MaxDepth = maxDepth });

        return pages.ToList();
    }

    public async Task<PageTreeNode> MovePage(Guid pageId, Guid? newParentId, int newSortOrder)
    {
        using var transaction = _db.BeginTransaction();

        var page = await _db.QuerySingleAsync<PageTreeNode>(
            "SELECT * FROM pages WHERE id = @Id FOR UPDATE",
            new { Id = pageId }, transaction);

        var newParentPath = newParentId.HasValue
            ? (await _db.QuerySingleAsync<string>(
                "SELECT materialized_path FROM pages WHERE id = @Id",
                new { Id = newParentId.Value }, transaction))
            : $"/{page.Id}";

        var oldPath = page.MaterializedPath;
        var newPath = $"{newParentPath}/{page.Id}";

        // Update materialized path for this page and ALL descendants
        await _db.ExecuteAsync(
            @"UPDATE pages
              SET materialized_path = REPLACE(materialized_path, @OldPath, @NewPath),
                  parent_id = @NewParentId,
                  sort_order = @SortOrder,
                  updated_at = NOW()
              WHERE materialized_path LIKE @OldPath || '%'",
            new { OldPath = oldPath, NewPath = newPath,
                  NewParentId = newParentId, SortOrder = newSortOrder },
            transaction);

        transaction.Commit();
        return await GetPageTreeNode(pageId);
    }
}
Materialized Path vs Alternatives

We choose Materialized Path because: (1) subtree queries are a simple LIKE prefix match, (2) moves only update affected rows (not the entire tree), (3) it naturally stores the full ancestry for breadcrumbs. The tradeoff is that deep moves require updating the path string of all descendants, but wiki pages are rarely moved frequently enough for this to matter. For extremely large spaces (100K+ pages), consider Closure Table for better query performance at the cost of additional storage.

6. Templates — Page & Database Templates

Template Architecture

Templates are first-class documents with an is_template = true flag. They contain pre-configured blocks, properties, and layout. When a user creates a page from a template, we deep-copy the block tree and CRDT state. Templates are categorized into page templates (for content) and database templates (for structured data views).

flowchart TD A["User clicks New Page"] --> B{"Choose Template?"} B -->|Yes| C["Browse Template Gallery"] C --> D["Select Template"] D --> E["Deep Copy Block Tree"] E --> F["Initialize New CRDT Doc"] F --> G["New Page Created"] B -->|No| H["Empty Page"] H --> G
C#
public class Template
{
    public Guid Id { get; set; } = Guid.NewGuid();
    public string Name { get; set; } = string.Empty;
    public string Description { get; set; }
    public TemplateType Type { get; set; }
    public string Category { get; set; }
    public string Icon { get; set; }
    public string ThumbnailUrl { get; set; }
    public byte[] BlockSnapshot { get; set; }
    public List<Block> Blocks { get; set; }
    public Dictionary<string, string> DefaultProperties { get; set; } = new();
    public Guid? SpaceId { get; set; }
    public Guid CreatedBy { get; set; }
    public bool IsPublic { get; set; }
    public int UseCount { get; set; }
    public DateTime CreatedAt { get; set; }
}

public class TemplateService
{
    private readonly IDbConnection _db;
    private readonly CrdtDocumentManager _crdtManager;

    public async Task<Document> CreateFromTemplate(Guid templateId, Guid userId, string pageTitle)
    {
        var template = await _db.QuerySingleAsync<Template>(
            "SELECT * FROM templates WHERE id = @Id", new { Id = templateId });

        var templateDoc = new YDoc();
        templateDoc.ApplyUpdate(template.BlockSnapshot);

        var newDoc = new YDoc();
        var update = templateDoc.EncodeStateAsUpdate(templateDoc.StateVector);
        newDoc.ApplyUpdate(update);
        RegenerateBlockIds(newDoc);

        var document = new Document
        {
            Title = pageTitle,
            Blocks = ConvertToBlockList(newDoc),
            OwnerId = userId,
            Properties = new Dictionary<string, string>(template.DefaultProperties)
        };

        await _db.ExecuteAsync(
            @"INSERT INTO pages (id, title, space_id, owner_id, properties, template_id)
              VALUES (@Id, @Title, @SpaceId, @OwnerId, @Properties, @TemplateId)", document);

        await _db.ExecuteAsync(
            "UPDATE templates SET use_count = use_count + 1 WHERE id = @Id",
            new { Id = templateId });

        return document;
    }
}

Built-in Templates

TemplateTypeKey Blocks
Meeting NotesPageTitle, Attendees, Agenda, Notes, Action Items, Decisions
Sprint RetrospectivePageWent Well, To Improve, Action Items, Vote Counts
Technical RFCPageSummary, Motivation, Design, Alternatives, Open Questions
Project TrackerDatabaseStatus, Priority, Assignee, Due Date columns + Kanban view
Team DirectoryDatabaseName, Role, Department, Photo, Contact columns
Knowledge Base HomePageCover image, icon, linked databases, table of contents

8. Content Versioning & History

Version Storage Strategy

Every save creates a version. We use a snapshot + delta approach: the latest state is always available as a snapshot, while historical versions are stored as Yjs CRDT updates (deltas) to save space. Every 50 versions, we create a new full snapshot to bound recovery time. This approach uses ~95% less storage than full snapshots while maintaining fast version restoration.

C#
public class PageVersion
{
    public Guid Id { get; set; } = Guid.NewGuid();
    public Guid PageId { get; set; }
    public int VersionNumber { get; set; }
    public Guid CreatedBy { get; set; }
    public string? ChangeDescription { get; set; }
    public byte[] CrdtUpdate { get; set; }
    public byte[]? FullSnapshot { get; set; }
    public bool IsSnapshot { get; set; }
    public int BlocksChanged { get; set; }
    public int WordsAdded { get; set; }
    public int WordsRemoved { get; set; }
    public DateTime CreatedAt { get; set; }
}

public class VersionService
{
    private readonly IDbConnection _db;

    public async Task<PageVersion> CreateVersion(Guid pageId, Guid userId, byte[] currentUpdate)
    {
        var lastVersion = await _db.QuerySingleAsync<PageVersion>(
            @"SELECT * FROM page_versions
              WHERE page_id = @PageId
              ORDER BY version_number DESC LIMIT 1",
            new { PageId = pageId });

        var newVersionNumber = (lastVersion?.VersionNumber ?? 0) + 1;
        var isSnapshot = newVersionNumber % 50 == 0 || lastVersion == null;

        var version = new PageVersion
        {
            PageId = pageId,
            VersionNumber = newVersionNumber,
            CreatedBy = userId,
            CrdtUpdate = currentUpdate,
            IsSnapshot = isSnapshot,
            FullSnapshot = isSnapshot ? await GetFullSnapshot(pageId) : null
        };

        await _db.ExecuteAsync(
            @"INSERT INTO page_versions
              (id, page_id, version_number, created_by, crdt_update,
               full_snapshot, is_snapshot, created_at)
              VALUES (@Id, @PageId, @VersionNumber, @CreatedBy, @CrdtUpdate,
                      @FullSnapshot, @IsSnapshot, @CreatedAt)", version);

        return version;
    }

    public async Task<Document> RestoreVersion(Guid pageId, int versionNumber)
    {
        var snapshot = await _db.QuerySingleAsync<PageVersion>(
            @"SELECT * FROM page_versions
              WHERE page_id = @PageId AND is_snapshot = true
                AND version_number <= @VersionNumber
              ORDER BY version_number DESC LIMIT 1",
            new { PageId = pageId, VersionNumber = versionNumber });

        var yDoc = new YDoc();
        yDoc.ApplyUpdate(snapshot.FullSnapshot);

        var deltas = await _db.QueryAsync<PageVersion>(
            @"SELECT * FROM page_versions
              WHERE page_id = @PageId AND version_number > @SnapVer
                AND version_number <= @TargetVer
              ORDER BY version_number",
            new { PageId = pageId, SnapVer = snapshot.VersionNumber, TargetVer = versionNumber });

        foreach (var delta in deltas)
            yDoc.ApplyUpdate(delta.CrdtUpdate);

        return ConvertToDocument(yDoc);
    }
}
Storage Optimization

Yjs CRDT updates are compact binary buffers. A typical edit (typing a word) produces a 50-200 byte update. Even with 100 edits per day on a page, a year of versions for that page would be ~5MB — very manageable. For comparison, storing full document snapshots every save would use 10-100x more storage. The snapshot-every-50 approach balances recovery speed with storage efficiency.

9. Commenting & Annotations

Comment System Design

Comments are anchored to specific blocks or text ranges within blocks. They support threads (replies), reactions (emoji), resolution, and @mentions. When a user highlights text and clicks "Comment," the selected text range is stored as an anchor, enabling other users to see exactly which text the comment refers to.

C#
public class Comment
{
    public Guid Id { get; set; } = Guid.NewGuid();
    public Guid PageId { get; set; }
    public Guid? ParentCommentId { get; set; }
    public string AnchorBlockId { get; set; }
    public TextRange? AnchorRange { get; set; }
    public AnchorType AnchorType { get; set; }
    public string Content { get; set; } = string.Empty;
    public List<CommentMention> Mentions { get; set; } = new();
    public List<CommentReaction> Reactions { get; set; } = new();
    public CommentStatus Status { get; set; } = CommentStatus.Open;
    public Guid CreatedBy { get; set; }
    public DateTime CreatedAt { get; set; }
    public DateTime? ResolvedAt { get; set; }
    public Guid? ResolvedBy { get; set; }
    public bool IsEdited { get; set; }
}

public class TextRange
{
    public int StartOffset { get; set; }
    public int EndOffset { get; set; }
    public string SelectedText { get; set; }
}

public enum AnchorType { TextSelection, Block, Page }
public enum CommentStatus { Open, Resolved, Archived }

Comment Notification Flow

flowchart TD A["User posts comment"] --> B["Comment Service"] B --> C["Store in PostgreSQL"] B --> D["Publish CommentCreated event"] D --> E["Notification Service"] E --> F{"Mentioned users?"} F -->|Yes| G["Direct notification to mentioned users"] F -->|No| H["Notify page subscribers"] G --> I["In-App + Email + Slack"] H --> I D --> L["Search Indexer"]

10. Page Permissions & Access Control (RBAC)

Permission Model

Permissions follow a hierarchical inheritance model with override capability. Permissions are checked at: Workspace, Space, Folder, and Page levels. A more specific permission always overrides a less specific one. The hierarchy is: Workspace Level (Admin, Member, Guest) → Space Level (Can Edit, Can View, No Access) → Folder Level (inherited from Space) → Page Level (Custom Override).

graph TB W["Workspace Permissions: Admin, Member, Guest"] W --> S["Space Permissions: Can Edit, Can View, No Access"] W --> SP["Space Settings: Invite Members, Manage Space"] S --> F["Folder Permissions: Inherited from Space"] S --> P["Page Permissions: Custom Override"] P --> PL["Page-Level Roles: Full Access, Can Edit, Can Comment, Can View, No Access"]
C#
public enum PermissionRole
{
    NoAccess = 0, CanView = 1, CanComment = 2,
    CanEdit = 3, CanManage = 4, FullAccess = 5
}

public class PermissionService
{
    private readonly IDbConnection _db;
    private readonly IDistributedCache _cache;

    public async Task<PermissionRole> GetEffectivePermission(
        Guid userId, ResourceType resourceType, Guid resourceId)
    {
        var cacheKey = $"perm:{userId}:{resourceType}:{resourceId}";
        var cached = await _cache.GetAsync<PermissionRole>(cacheKey);
        if (cached.HasValue) return cached.Value;

        var permissionChain = await BuildPermissionChain(userId, resourceType, resourceId);

        var effective = permissionChain
            .OrderByDescending(p => p.IsInherited ? 0 : 1)
            .ThenByDescending(p => p.ResourceType)
            .ThenByDescending(p => p.Role)
            .FirstOrDefault();

        var role = effective?.Role ?? PermissionRole.NoAccess;
        await _cache.SetAsync(cacheKey, role, TimeSpan.FromMinutes(5));
        return role;
    }

    public async Task<bool> HasPermission(Guid userId, Guid pageId, PermissionRole required)
    {
        var effective = await GetEffectivePermission(userId, ResourceType.Page, pageId);
        return effective >= required;
    }
}

Public Share Links

FeatureImplementation
Public URLUUID-based URL: /share/{uuid} — no guessable paths
Password Protectionbcrypt-hashed password stored; verified on access
Expiryexpires_at timestamp; enforced at middleware level
Download PermissionToggle to allow/disallow PDF/HTML download
View LimitsOptional max view count; link auto-deactivates
Permission Check Performance

Permission checks happen on every API request and WebSocket message. We solve this with: (1) Redis caching with 5-minute TTL, (2) Permission invalidation events published when permissions change, (3) Bulk permission loading at page-load time for the client sidebar, and (4) Deny-by-default — if no permission entry exists, access is denied (no expensive chain walk needed).

11. Embedding Content — Images, Videos, Code, Databases

Asset Upload Pipeline

flowchart LR A["User drops image"] --> B["Client: Upload to presigned URL"] B --> C["S3 Storage"] C --> D["Lambda: Process Image"] D --> E["Generate thumbnails"] D --> F["Extract metadata"] D --> G["Virus scan"] E --> C G --> H{"Clean?"} H -->|Yes| I["Mark as ready"] H -->|No| J["Quarantine + alert"]
C#
public class EmbedBlockProperties
{
    public EmbedType EmbedType { get; set; }
    public string Url { get; set; }
    public string? Title { get; set; }
    public string? Description { get; set; }
    public string? ThumbnailUrl { get; set; }
    public int? Width { get; set; }
    public int? Height { get; set; }
    public string? OembedHtml { get; set; }
    public Dictionary<string, string> Metadata { get; set; } = new();
}

public enum EmbedType
{
    Image, Video, Audio, File, CodeBlock, Bookmark,
    Database, Figma, GoogleMaps, Miro, Loom, Tweet, CustomIframe
}

public class AssetService
{
    private readonly IS3Client _s3;

    public async Task<AssetUploadResponse> CreateUploadRequest(
        Guid userId, string fileName, string contentType, long fileSize)
    {
        var allowedTypes = new HashSet<string>
        {
            "image/jpeg", "image/png", "image/gif", "image/webp", "image/svg+xml",
            "video/mp4", "video/webm", "audio/mpeg", "audio/ogg",
            "application/pdf", "application/zip"
        };

        if (!allowedTypes.Contains(contentType))
            throw new BadRequestException($"File type {contentType} not allowed");
        if (fileSize > 100 * 1024 * 1024)
            throw new BadRequestException("File too large (max 100MB)");

        var assetId = Guid.NewGuid();
        var key = $"assets/{assetId}/{SanitizeFileName(fileName)}";
        var presignedUrl = await _s3.GeneratePresignedPutUrl(key, contentType, TimeSpan.FromMinutes(15));

        return new AssetUploadResponse
        {
            AssetId = assetId,
            UploadUrl = presignedUrl,
            AssetUrl = $"https://cdn.example.com/{key}"
        };
    }
}

Database Block (Linked Database)

C#
public class DatabaseBlock
{
    public Guid Id { get; set; } = Guid.NewGuid();
    public string Name { get; set; } = string.Empty;
    public List<DatabaseColumn> Columns { get; set; } = new();
    public List<DatabaseView> Views { get; set; } = new();
    public List<DatabaseRow> Rows { get; set; } = new();
}

public class DatabaseColumn
{
    public Guid Id { get; set; }
    public string Name { get; set; }
    public ColumnType Type { get; set; }
    public ColumnConfig Config { get; set; }
    public bool IsVisible { get; set; } = true;
    public int SortOrder { get; set; }
}

public enum ColumnType
{
    Title, Text, Number, Select, MultiSelect,
    Date, Person, Checkbox, Url, Email, Phone,
    Files, Relation, Rollup, Formula,
    CreatedTime, UpdatedTime, CreatedBy, UpdatedBy
}

public class DatabaseView
{
    public Guid Id { get; set; }
    public string Name { get; set; }
    public ViewType Type { get; set; }
    public List<FilterRule> Filters { get; set; } = new();
    public List<SortRule> Sorts { get; set; } = new();
    public List<Guid> VisibleColumns { get; set; } = new();
}

public class DatabaseRow
{
    public Guid Id { get; set; }
    public Guid PageId { get; set; }
    public Dictionary<Guid, object> Cells { get; set; } = new();
    public int SortOrder { get; set; }
}

12. Table / Spreadsheet Blocks

Table Block Architecture

Table blocks are lightweight spreadsheet-like grids embedded in pages. Unlike database blocks (which are full structured data stores), table blocks are simpler — they are primarily for display and don't support formulas or relations. They support merge cells, column resize, row/column insert/delete/reorder, cell formatting, column types, and CSV import/export.

C#
public class TableBlockProperties
{
    public int Rows { get; set; }
    public int Columns { get; set; }
    public List<List<TableCell>> Cells { get; set; } = new();
    public List<TableColumnConfig> ColumnConfigs { get; set; } = new();
    public bool HasHeaderRow { get; set; } = true;
    public TableBorderStyle BorderStyle { get; set; } = TableBorderStyle.Solid;
}

public class TableCell
{
    public string Content { get; set; } = string.Empty;
    public int RowSpan { get; set; } = 1;
    public int ColSpan { get; set; } = 1;
    public bool IsMerged { get; set; }
    public string? BackgroundColor { get; set; }
    public TextAlignment Alignment { get; set; } = TextAlignment.Left;
}

public class TableColumnConfig
{
    public int Index { get; set; }
    public string? HeaderName { get; set; }
    public int Width { get; set; } = 150;
    public ColumnDataType DataType { get; set; } = ColumnDataType.Text;
    public bool IsSortable { get; set; }
}
Table vs Database Block

Use Table Block for simple data display — comparing features, showing specs, organizing text. Use Database Block when you need structured data with filtering, sorting, multiple views (kanban, gallery), relations between databases, or computed columns (formulas, rollups). Database blocks are backed by dedicated storage; table blocks are part of the page's block tree.

13. Kanban Board Views

Kanban Implementation

Kanban boards are a view type for database blocks. Cards represent rows; columns represent a Select or Status column value. Dragging a card between columns updates the underlying row's status field and broadcasts the change to all connected clients via WebSocket.

flowchart TD subgraph Kanban["Kanban Board View"] C1["To Do: Card 1, Card 2, Card 3"] C2["In Progress: Card 4, Card 5"] C3["In Review: Card 6"] C4["Done: Card 7, Card 8"] end C1 -->|Drag| C2 C2 -->|Drag| C3 C3 -->|Drag| C4
C#
public class KanbanService
{
    private readonly IDbConnection _db;
    private readonly IEventBus _eventBus;

    public async Task<KanbanBoard> GetKanbanBoard(Guid viewId, Guid userId)
    {
        var view = await _db.QuerySingleAsync<DatabaseView>(
            "SELECT * FROM database_views WHERE id = @Id", new { Id = viewId });

        var groupByColumn = await _db.QuerySingleAsync<DatabaseColumn>(
            "SELECT * FROM database_columns WHERE id = @Id",
            new { Id = view.GroupByColumnId });

        var rows = await GetFilteredRows(view);
        var columns = new List<KanbanColumn>();

        if (groupByColumn.Type == ColumnType.Select)
        {
            foreach (var option in groupByColumn.Config.Options)
            {
                var cards = rows.Where(r =>
                    r.Cells.ContainsKey(groupByColumn.Id) &&
                    r.Cells[groupByColumn.Id]?.ToString() == option.Label);

                columns.Add(new KanbanColumn
                {
                    Name = option.Label,
                    Color = option.Color,
                    Cards = cards.Select(MapToCard).ToList(),
                    CardCount = cards.Count()
                });
            }
        }

        return new KanbanBoard { ViewId = viewId, Columns = columns };
    }

    public async Task MoveCard(Guid cardRowId, string fromStatus, string toStatus, int newPosition)
    {
        using var tx = _db.BeginTransaction();
        await _db.ExecuteAsync(
            @"UPDATE database_cells SET value = @NewStatus
              WHERE row_id = @RowId AND column_id = @ColumnId",
            new { RowId = cardRowId, NewStatus = toStatus, ColumnId = GetGroupByColumnId() }, tx);
        await ReorderRows(cardRowId, newPosition, toStatus, tx);
        tx.Commit();

        await _eventBus.PublishAsync(new CardMovedEvent
        {
            RowId = cardRowId, FromStatus = fromStatus, ToStatus = toStatus
        });
    }
}

14. @Mentions & Notifications

Mention System

Users can @mention other users, groups, or pages within any text content. Mentions are parsed from the CRDT content and trigger notifications. The mention system supports autocomplete — when a user types "@", a dropdown appears with searchable users and groups.

C#
public class Mention
{
    public Guid Id { get; set; }
    public MentionType Type { get; set; }
    public Guid TargetId { get; set; }
    public string DisplayText { get; set; }
    public Guid MentionedBy { get; set; }
    public Guid PageId { get; set; }
    public string BlockId { get; set; }
    public DateTime CreatedAt { get; set; }
}

public enum MentionType { User, Group, Page, Date, Formula }

public class NotificationService
{
    private readonly IDbConnection _db;
    private readonly IPushNotificationService _push;
    private readonly IEmailService _email;
    private readonly ISlackService _slack;

    public async Task ProcessMention(Mention mention)
    {
        var notification = new Notification
        {
            Id = Guid.NewGuid(),
            Type = NotificationType.Mention,
            TargetUserId = mention.TargetId,
            TriggeredByUser = mention.MentionedBy,
            PageId = mention.PageId,
            Message = "mentioned you in a page",
            CreatedAt = DateTime.UtcNow
        };

        await _db.ExecuteAsync(
            @"INSERT INTO notifications (id, type, target_user_id, triggered_by_user,
              page_id, message, created_at, is_read)
              VALUES (@Id, @Type, @TargetUserId, @TriggeredByUser,
              @PageId, @Message, @CreatedAt, false)", notification);

        var prefs = await GetUserNotificationPreferences(mention.TargetId);
        var tasks = new List<Task>();
        if (prefs.InAppEnabled) tasks.Add(SendInAppNotification(notification));
        if (prefs.EmailEnabled) tasks.Add(_email.SendMentionEmail(notification));
        if (prefs.SlackEnabled) tasks.Add(_slack.SendMentionNotification(notification));
        if (prefs.PushEnabled) tasks.Add(_push.SendPushNotification(notification));
        await Task.WhenAll(tasks);
    }
}

Notification Channels

ChannelUse CaseDelivery SLAImplementation
In-AppReal-time bell icon, notification centerInstant (WebSocket)Redis pub/sub to WebSocket broadcast
EmailDaily digest or instant for mentions< 5 minSendGrid / SES
SlackTeam notifications in channels< 1 minSlack Web API
Microsoft TeamsEnterprise team notifications< 1 minTeams Bot Framework
Push (Mobile)Critical notifications on mobile< 2 minFCM / APNs

15. Page Export — PDF, Markdown, HTML

Export Architecture

flowchart LR A["Export Request"] --> B["Export Service"] B --> C{"Format?"} C -->|PDF| D["Puppeteer Lambda"] C -->|Markdown| E["Block-to-MD Converter"] C -->|HTML| F["Block-to-HTML Renderer"] D --> H["S3 Export Bucket"] E --> H F --> H H --> I["Presigned URL (15 min)"] I --> J["Download"]
C#
public class ExportService
{
    private readonly IS3Client _s3;
    private readonly IPuppeteerService _puppeteer;
    private readonly IBlockRenderer _renderer;

    public async Task<ExportResult> ExportPage(Guid pageId, ExportFormat format, Guid userId)
    {
        var document = await LoadDocument(pageId);
        var html = await _renderer.RenderToHtml(document);

        byte[] fileBytes = format switch
        {
            ExportFormat.Pdf => await _puppeteer.GeneratePdf(html, new PdfOptions
            {
                Format = PaperFormat.A4,
                Margin = new Margin { Top = "20mm", Bottom = "20mm",
                    Left = "15mm", Right = "15mm" },
                DisplayHeaderFooter = true,
                HeaderTemplate = $"{document.Title}",
                FooterTemplate = "Page "
            }),
            ExportFormat.Markdown => Encoding.UTF8.GetBytes(
                await _renderer.RenderToMarkdown(document)),
            ExportFormat.Html => Encoding.UTF8.GetBytes(html),
            _ => throw new NotSupportedException($"Format {format} not supported")
        };

        var key = $"exports/{userId}/{pageId}/{DateTime.UtcNow:yyyyMMddHHmmss}.{format}";
        await _s3.PutObjectAsync("exports-bucket", key, fileBytes);
        var presignedUrl = await _s3.GeneratePresignedGetUrl("exports-bucket", key,
            TimeSpan.FromMinutes(15));

        return new ExportResult { DownloadUrl = presignedUrl, FileSize = fileBytes.Length };
    }
}

16. Import from Other Tools

Supported Import Sources

SourceFormatFeatures PreservedLimitations
ConfluenceXML exportPages, attachments, hierarchy, macrosSome macros converted to basic blocks
NotionMarkdown exportPages, databases (as tables), imagesDatabase relations lost
Google DocsDOCX exportText, images, tables, headingsComments become page comments
Markdown.md filesFull Markdown syntaxCustom HTML blocks simplified
HTML.html filesWeb page contentJavaScript stripped
Word.docx filesText, images, tables, stylesComplex layouts simplified
C#
public class ImportService
{
    private readonly IServiceProvider _services;

    public async Task<ImportResult> ImportContent(
        ImportSource source, Stream fileStream, Guid userId, Guid targetSpaceId)
    {
        var handler = source switch
        {
            ImportSource.ConfluenceXml => _services.GetRequiredService<ConfluenceXmlImporter>(),
            ImportSource.Markdown => _services.GetRequiredService<MarkdownImporter>(),
            ImportSource.NotionMarkdown => _services.GetRequiredService<NotionImporter>(),
            ImportSource.Docx => _services.GetRequiredService<DocxImporter>(),
            ImportSource.Html => _services.GetRequiredService<HtmlImporter>(),
            _ => throw new NotSupportedException($"Import from {source} not supported")
        };

        var parsed = await handler.Parse(fileStream);
        var result = new ImportResult { TotalPages = parsed.Pages.Count };

        foreach (var page in parsed.Pages)
        {
            var blocks = handler.ConvertToBlocks(page);
            var document = new Document
            {
                Title = page.Title, Blocks = blocks,
                SpaceId = targetSpaceId, OwnerId = userId
            };
            await SaveDocument(document);
            result.CreatedPages.Add(document.Id);

            foreach (var attachment in page.Attachments)
            {
                var assetUrl = await UploadAttachment(attachment, document.Id);
                ReplaceAttachmentReferences(blocks, attachment.Name, assetUrl);
                result.UploadedAttachments++;
            }
        }
        return result;
    }
}

17. API for Programmatic Access

REST API Endpoints

MethodEndpointDescription
GET/api/v1/spacesList all spaces
POST/api/v1/spacesCreate a new space
GET/api/v1/spaces/:id/pagesList pages in a space
POST/api/v1/spaces/:id/pagesCreate a page in a space
GET/api/v1/pages/:idGet page details + blocks
PATCH/api/v1/pages/:idUpdate page metadata
DELETE/api/v1/pages/:idDelete (soft) a page
GET/api/v1/pages/:id/childrenList child pages
GET/api/v1/pages/:id/versionsList page version history
POST/api/v1/pages/:id/versions/:v/restoreRestore a version
GET/api/v1/pages/:id/commentsList page comments
POST/api/v1/pages/:id/commentsAdd a comment
GET/api/v1/search?q=...Full-text search
POST/api/v1/pages/:id/exportExport page (PDF/MD/HTML)
POST/api/v1/importImport from file
GET/api/v1/templatesList available templates
POST/api/v1/templates/:id/instantiateCreate page from template
GET/api/v1/databases/:id/rowsList database rows
GET/api/v1/audit-logQuery audit log
GET/api/v1/analytics/pages/:idGet page analytics

Webhook System

C#
public class Webhook
{
    public Guid Id { get; set; }
    public string Url { get; set; }
    public List<string> Events { get; set; }
    public string Secret { get; set; }
    public Guid WorkspaceId { get; set; }
    public bool IsActive { get; set; }
    public int FailureCount { get; set; }
}

public class WebhookPayload
{
    public string Event { get; set; }
    public DateTime Timestamp { get; set; }
    public Guid WorkspaceId { get; set; }
    public object Data { get; set; }
    public string Signature { get; set; }
}

18. Offline Mode

Offline Architecture

flowchart TD A["User goes offline"] --> B["IndexedDB stores: page content, metadata, preferences"] B --> C["User edits page offline"] C --> D["Changes stored locally in Yjs doc"] D --> E{"Back online?"} E -->|No| C E -->|Yes| F["Sync Engine connects to WebSocket"] F --> G["Exchange state vectors"] G --> H["Send local updates to server"] H --> I["Receive remote updates"] I --> J["CRDT merge: automatic conflict resolution"] J --> K["Server persists merged state"]
C#
public class OfflineSyncEngine
{
    private readonly IndexedDb _db;
    private readonly WebSocketClient _ws;
    private readonly YDoc _localDoc;

    public async Task Sync(Guid documentId)
    {
        var localState = await _db.GetDocumentState(documentId);
        _localDoc.ApplyUpdate(localState);

        await _ws.Connect();
        var stateVector = _localDoc.StateVector;
        await _ws.Send(new SyncStep1Message { StateVector = stateVector });

        var serverUpdate = await _ws.Receive<SyncStep2Message>();
        _localDoc.ApplyUpdate(serverUpdate.Update);

        var localUpdates = _localDoc.EncodeStateAsUpdate(stateVector);
        if (localUpdates.Length > 2)
            await _ws.Send(new UpdateMessage { Update = localUpdates });

        await _db.SaveDocumentState(documentId, _localDoc.StateVector);

        _ws.OnUpdate += async (update) =>
        {
            _localDoc.ApplyUpdate(update);
            await _db.SaveDocumentState(documentId, _localDoc.StateVector);
            NotifyEditor();
        };
    }
}
Offline Storage Budget

A typical page with 50 blocks and 5,000 words requires ~200KB in CRDT format. Even with 1,000 offline pages, storage is only ~200MB — well within IndexedDB limits. We use a LRU eviction policy that removes pages not accessed in 30 days.

19. Knowledge Graph & Backlinks

Graph Model

The knowledge graph captures relationships between pages: links, mentions, parent-child, and database relations. It is stored in Neo4j and maintained asynchronously via events. When a page is saved, the system extracts all internal links and updates the graph accordingly.

graph LR A["System Design Doc"] -->|"links_to"| B["API Guidelines"] A -->|"links_to"| C["Database Schema"] B -->|"links_to"| D["Auth Service"] C -->|"links_to"| E["Migration Guide"] D -->|"mentioned_in"| F["Sprint Retro"] C -->|"parent_of"| E
C#
public class KnowledgeGraphService
{
    private readonly IDriver _neo4j;

    public async Task UpdatePageLinks(Guid pageId, List<PageLink> links)
    {
        using var session = _neo4j.AsyncSession();
        await session.ExecuteWriteTransactionAsync(async tx =>
        {
            await tx.RunAsync(
                @"MATCH (p:Page {id: $PageId})-[r:LINKS_TO|MENTIONS]->()
                  DELETE r",
                new { PageId = pageId.ToString() });

            foreach (var link in links)
            {
                var relType = link.Type == LinkType.Hyperlink ? "LINKS_TO" : "MENTIONS";
                await tx.RunAsync(
                    $@"MATCH (source:Page {{id: $SourceId}})
                       MATCH (target:Page {{id: $TargetId}})
                       CREATE (source)-[:{relType} {{blockId: $BlockId, context: $Context}}]->(target)",
                    new
                    {
                        SourceId = pageId.ToString(),
                        TargetId = link.TargetPageId.ToString(),
                        BlockId = link.BlockId,
                        Context = link.Context
                    });
            }
        });
    }

    public async Task<BacklinkResult> GetBacklinks(Guid pageId, int limit = 50)
    {
        using var session = _neo4j.AsyncSession();
        var result = await session.RunAsync(
            @"MATCH (source:Page)-[r:LINKS_TO|MENTIONS]->(target:Page {id: $PageId})
              RETURN source.id as SourcePageId, source.title as Title,
                     type(r) as LinkType, r.context as Context
              ORDER BY source.updated_at DESC LIMIT $Limit",
            new { PageId = pageId.ToString(), Limit = limit });

        var backlinks = await result.Select(record => new Backlink
        {
            PageId = Guid.Parse(record["SourcePageId"].As<string>()),
            Title = record["Title"].As<string>(),
            LinkType = record["LinkType"].As<string>(),
            Context = record["Context"].As<string>()
        }).ToListAsync();

        return new BacklinkResult { Backlinks = backlinks, TotalCount = backlinks.Count };
    }

    public async Task<GraphVisualization> GetPageGraph(Guid pageId, int depth = 2)
    {
        using var session = _neo4j.AsyncSession();
        var result = await session.RunAsync(
            @"MATCH path = (start:Page {id: $PageId})-[*1..2]-(related:Page)
              RETURN DISTINCT
                collect(DISTINCT {id: related.id, title: related.title}) as nodes,
                collect(DISTINCT {
                  source: startNode(last(relationships(path))).id,
                  target: endNode(last(relationships(path))).id,
                  type: type(last(relationships(path)))
                }) as edges",
            new { PageId = pageId.ToString() });

        var record = await result.SingleAsync();
        return MapToGraphVisualization(record);
    }
}

20. Table of Contents Auto-Generation

When a page contains heading blocks (H1-H6), a Table of Contents block can be inserted that automatically generates a navigable outline from the headings. The TOC updates in real-time as users add or modify headings. Clicking a TOC entry scrolls the page to the corresponding heading anchor.

C#
public class TableOfContentsService
{
    public List<TocEntry> GenerateToc(List<Block> blocks)
    {
        var entries = new List<TocEntry>();
        int counter = 0;

        foreach (var block in blocks)
        {
            if (block.Type == BlockType.TableOfContents) continue;

            if (block.Type >= BlockType.Heading1 && block.Type <= BlockType.Heading6)
            {
                var level = (int)(block.Type - BlockType.Heading1);
                var text = ExtractPlainText(block);

                entries.Add(new TocEntry
                {
                    Id = $"heading-{counter++}",
                    Text = text,
                    Level = level,
                    BlockId = block.Id,
                    Anchor = GenerateAnchor(text)
                });
            }
        }
        return entries;
    }

    private string GenerateAnchor(string text)
    {
        return text.ToLower().Replace(" ", "-")
            .Replace("[^\\w-]", "").Replace("-+", "-");
    }
}

21. Analytics — Page Views & Popular Pages

Analytics Pipeline

flowchart LR A["Page View Event"] --> B["RabbitMQ"] B --> C["Analytics Consumer"] C --> D["ClickHouse"] C --> E["Redis: Real-time counters"] D --> F["Analytics Dashboard"] E --> G["Real-time view counts"]
C#
public class AnalyticsService
{
    private readonly ClickHouseConnection _clickhouse;
    private readonly IDistributedCache _cache;

    public async Task TrackPageView(Guid pageId, Guid? userId, PageViewMetadata metadata)
    {
        var evt = new PageViewEvent
        {
            PageId = pageId, UserId = userId, Timestamp = DateTime.UtcNow,
            UserAgent = metadata.UserAgent, Device = metadata.Device
        };
        _buffer.Add(evt);

        var today = DateTime.UtcNow.ToString("yyyy-MM-dd");
        await _cache.IncrementAsync($"pageviews:{pageId}:{today}");
    }

    public async Task<PageAnalytics> GetPageAnalytics(Guid pageId, DateRange range)
    {
        var result = await _clickhouse.QueryAsync<PageAnalyticsRow>(
            @"SELECT
                toDate(timestamp) as Date,
                uniqExact(user_id) as UniqueViewers,
                count() as TotalViews,
                avg(duration_ms) as AvgDurationMs
              FROM page_views
              WHERE page_id = @PageId
                AND timestamp BETWEEN @Start AND @End
              GROUP BY Date ORDER BY Date",
            new { PageId = pageId, Start = range.Start, End = range.End });

        return new PageAnalytics
        {
            DailyViews = result.ToList(),
            TotalViews = result.Sum(r => r.TotalViews)
        };
    }

    public async Task<List<PopularPage>> GetPopularPages(Guid spaceId, int limit = 10)
    {
        return (await _clickhouse.QueryAsync<PopularPage>(
            @"SELECT page_id, title, sum(views) as TotalViews,
                     uniqExact(user_id) as UniqueViewers
              FROM page_views pv JOIN pages p ON pv.page_id = p.id
              WHERE p.space_id = @SpaceId AND pv.timestamp >= now() - INTERVAL 30 DAY
              GROUP BY page_id, title ORDER BY TotalViews DESC LIMIT @Limit",
            new { SpaceId = spaceId, Limit = limit })).ToList();
    }
}

Analytics Dashboard Metrics

MetricTime RangeUse Case
Page views (total, unique)Day, Week, MonthContent popularity
Top pages in space7d, 30d, 90dContent strategy
Most active editorsWeek, MonthContributor recognition
Search queries with no resultsWeekContent gap identification
Stale pages (not updated in 90d)RollingContent freshness

22. Audit Log

The audit log captures every significant action in the system for compliance and forensic analysis. Events are immutable and stored in an append-only table. Every event includes the user, action, resource, metadata (before/after values), IP address, and timestamp.

C#
public class AuditEvent
{
    public Guid Id { get; set; } = Guid.NewGuid();
    public Guid WorkspaceId { get; set; }
    public Guid? UserId { get; set; }
    public string? UserEmail { get; set; }
    public AuditAction Action { get; set; }
    public ResourceType ResourceType { get; set; }
    public Guid ResourceId { get; set; }
    public string ResourceName { get; set; }
    public Dictionary<string, object> Metadata { get; set; } = new();
    public string? IpAddress { get; set; }
    public DateTime Timestamp { get; set; } = DateTime.UtcNow;
}

public enum AuditAction
{
    PageCreated, PageUpdated, PageDeleted, PageRestored,
    PagePublished, PageArchived, PageMoved, PageExported,
    PermissionGranted, PermissionRevoked, PermissionChanged,
    SpaceCreated, SpaceUpdated, SpaceArchived,
    UserJoined, UserRemoved, RoleChanged,
    LoginSuccess, LoginFailed, PasswordChanged,
    ApiKeyCreated, ApiKeyRevoked,
    WorkspaceSettingsChanged, DataExported, DataDeleted
}

public class AuditService
{
    private readonly IDbConnection _db;

    public async Task Log(AuditEvent auditEvent)
    {
        await _db.ExecuteAsync(
            @"INSERT INTO audit_log (id, workspace_id, user_id, action,
              resource_type, resource_id, metadata, ip_address, timestamp)
              VALUES (@Id, @WorkspaceId, @UserId, @Action,
              @ResourceType, @ResourceId, @Metadata, @IpAddress, @Timestamp)",
            auditEvent);
    }

    public async Task<AuditLogResult> QueryAuditLog(Guid workspaceId, AuditLogQuery query)
    {
        var sql = @"SELECT * FROM audit_log
                    WHERE workspace_id = @WorkspaceId
                      AND timestamp BETWEEN @Start AND @End";

        if (query.Actions?.Any() == true) sql += " AND action = ANY(@Actions)";
        if (query.UserId.HasValue) sql += " AND user_id = @UserId";
        sql += " ORDER BY timestamp DESC LIMIT @Limit OFFSET @Offset";

        var events = await _db.QueryAsync<AuditEvent>(sql, query);
        return new AuditLogResult { Events = events.ToList() };
    }
}

23. Integrations — Slack, Jira & More

Integration Architecture

flowchart TD subgraph WikiPlatform["Wiki Platform"] A["Integration Service"] B["Webhook Dispatcher"] C["OAuth Manager"] end subgraph External["External Services"] D["Slack"] E["Jira"] F["GitHub"] G["Google Drive"] H["Figma"] I["Zapier"] end A --> B A --> C B --> D B --> E B --> F C --> G C --> H C --> I

Slack Integration

C#
public class SlackIntegration
{
    private readonly ISlackClient _slack;
    private readonly ISearchService _search;

    public async Task HandleSlashCommand(SlashCommand command)
    {
        switch (command.Command)
        {
            case "/wiki search":
                var results = await _search.SearchAsync(
                    command.Text, command.UserId, new SearchFilters { PageSize = 5 });
                var blocks = results.Items.Select(r => new SlackBlock
                {
                    Type = "section",
                    Text = new SlackText
                    {
                        Type = "mrkdwn",
                        Text = $"*{r.Title}*\n{r.Excerpt}\n<{r.Url}|View Page>"
                    }
                }).ToList();
                await _slack.SendResponse(command.ResponseUrl, new SlackResponse
                {
                    ResponseType = "ephemeral", Blocks = blocks
                });
                break;

            case "/wiki create":
                var page = await CreatePageFromSlack(command.Text, command.UserId);
                await _slack.SendEphemeral(command.ResponseUrl,
                    $"Page created: {page.Url}");
                break;
        }
    }
}

Jira Integration

FeatureDescription
Issue EmbedsPaste Jira URL to auto-embed issue card with status, assignee, priority
Bidirectional SyncUpdate Jira issue status from wiki page, or vice versa via webhook
Sprint ReportsAuto-generate sprint retrospective pages from Jira sprint data
Link Issues to PagesAssociate wiki pages with Jira issues; show links in both systems

24. Monitoring & Observability

Monitoring Stack

LayerToolMetric
InfrastructurePrometheus + GrafanaCPU, memory, disk, network
ApplicationOpenTelemetryRequest latency, error rate, throughput
Distributed TracingJaegerRequest flow across services
Log AggregationELK StackStructured logs with correlation IDs
Error TrackingSentryException grouping, breadcrumbs
UptimeChecklyHTTP checks from multiple regions
Real-Time CollabCustom dashboardConnected users, CRDT merge latency

Key SLIs/SLOs

YAML
service_level_objectives:
  - name: api_availability
    sli: successful_requests / total_requests
    target: 99.95%
    window: 30d
  - name: api_latency_p95
    sli: histogram_bucket{le="0.5"} / total_requests
    target: 99%
    window: 30d
  - name: search_latency_p95
    sli: elasticsearch_request_duration{le="0.2"} / total_searches
    target: 99%
    window: 30d
  - name: collab_sync_latency
    sli: crdt_merge_duration{le="0.5"} / total_merges
    target: 99.5%
    window: 7d

25. Security

Security Architecture

flowchart TD A["Client Request"] --> B["WAF (AWS WAF)"] B --> C["Rate Limiter"] C --> D["Authentication (JWT + Refresh Token)"] D --> E["Authorization (RBAC Check)"] E --> F["Input Validation"] F --> G["Application Logic"] G --> H["Encryption at Rest (AES-256)"] H --> I["Database"]

Security Measures

LayerMeasure
TransportTLS 1.3 enforced; HSTS with 1-year max-age; certificate pinning for mobile
AuthenticationJWT with short-lived access tokens (15 min) + refresh tokens (30 days); MFA via TOTP/SMS
Session ManagementRedis-backed sessions with configurable timeout; concurrent session limits
Data at RestAES-256-GCM encryption for database fields; S3 SSE-KMS for assets
SecretsAWS Secrets Manager / HashiCorp Vault; no secrets in code or env vars
Input ValidationServer-side schema validation for all inputs; HTML sanitization via DOMPurify
XSS PreventionCSP headers; DOMPurify for user content; no inline scripts
CSRF PreventionSameSite cookies; CSRF tokens for state-changing requests
Rate LimitingPer-user and per-IP rate limits; exponential backoff for repeated failures
Vulnerability ScanningSnyk for dependencies; Trivy for containers; weekly SAST
Penetration TestingAnnual third-party pentest; bug bounty program via HackerOne

Content Security Service

C#
public class ContentSecurityService
{
    public string SanitizeUserContent(string html)
    {
        var sanitizer = new HtmlSanitizer(new SanitizerOptions
        {
            AllowedTags = new HashSet<string>
            {
                "p", "br", "strong", "em", "u", "s", "a", "code", "pre",
                "h1", "h2", "h3", "h4", "h5", "h6",
                "ul", "ol", "li", "blockquote",
                "table", "thead", "tbody", "tr", "th", "td",
                "img", "video", "audio", "source", "iframe",
                "div", "span", "hr", "sup", "sub"
            },
            AllowedAttributes = new Dictionary<string, HashSet<string>>
            {
                ["a"] = new() { "href", "title", "target", "rel" },
                ["img"] = new() { "src", "alt", "width", "height" },
                ["iframe"] = new() { "src", "width", "height", "frameborder" },
                ["code"] = new() { "class" },
                ["span"] = new() { "class", "style", "data-type", "data-id" }
            },
            AllowedSchemes = new HashSet<string> { "https", "mailto", "tel" }
        });

        return sanitizer.Sanitize(html);
    }
}

26. Compliance — SOC 2, GDPR

SOC 2 Type II

TCS PrincipleImplementation
SecurityEncryption at rest/transit, RBAC, MFA, WAF, intrusion detection
Availability99.95% uptime SLA, multi-AZ deployment, auto-scaling, failover
Processing IntegrityCRDT merge correctness, idempotent operations, data validation
ConfidentialityAES-256 encryption, key rotation, data classification, access logging
PrivacyData minimization, consent management, data retention policies

GDPR Compliance

GDPR RightImplementation
Right of Access (Art. 15)Self-serve data export: all user data downloadable as JSON/CSV
Right to Erasure (Art. 17)Delete Account flow: anonymizes content, removes PII, 30-day grace period
Data Portability (Art. 20)Full export in machine-readable format (JSON with block structure)
Consent (Art. 7)Cookie consent banner; granular consent for analytics and third-party sharing
Data Processing AgreementStandard DPA with sub-processors; SCCs for international transfers
Breach Notification72-hour notification process; automated detection + incident response playbook
C#
public class GdprService
{
    private readonly IDbConnection _db;

    public async Task<DataExportResult> ExportUserData(Guid userId)
    {
        var user = await _db.QuerySingleAsync<User>(
            "SELECT * FROM users WHERE id = @Id", new { Id = userId });
        var pages = await _db.QueryAsync<Document>(
            "SELECT * FROM pages WHERE owner_id = @Id OR created_by = @Id",
            new { Id = userId });
        var comments = await _db.QueryAsync<Comment>(
            "SELECT * FROM comments WHERE created_by = @Id", new { Id = userId });

        var export = new UserDataExport
        {
            User = new UserExport { Email = user.Email, Name = user.Name },
            Pages = pages.Select(p => new PageExport { Id = p.Id, Title = p.Title }).ToList(),
            Comments = comments.Select(c => new CommentExport { Id = c.Id, Content = c.Content }).ToList(),
            ExportedAt = DateTime.UtcNow
        };

        var json = JsonSerializer.Serialize(export, new JsonSerializerOptions { WriteIndented = true });
        var key = $"gdpr-exports/{userId}/{DateTime.UtcNow:yyyyMMdd}.json";
        return new DataExportResult { DownloadUrl = await GetPresignedUrl(key) };
    }

    public async Task AnonymizeUserData(Guid userId)
    {
        using var tx = _db.BeginTransaction();
        await _db.ExecuteAsync(
            @"UPDATE users SET name = 'Deleted User',
              email = CONCAT('deleted_', @UserId, '@anonymized.local'),
              avatar_url = NULL, is_deleted = true, deleted_at = NOW()
              WHERE id = @UserId", new { UserId = userId }, tx);

        await _db.ExecuteAsync(
            @"UPDATE pages SET owner_id = '00000000-0000-0000-0000-000000000001'
              WHERE owner_id = @UserId AND is_deleted = false",
            new { UserId = userId }, tx);

        await _db.ExecuteAsync(
            @"UPDATE comments SET content = '[Removed for privacy]',
              created_by = '00000000-0000-0000-0000-000000000001'
              WHERE created_by = @UserId", new { UserId = userId }, tx);

        tx.Commit();
    }
}

27. Cost Estimation

Infrastructure Cost Breakdown (Monthly, 100K Users, 10M Pages)

ServiceSpecificationMonthly Cost
Application Servers (ECS/EKS)8 x c6i.xlarge (4 vCPU, 8GB)$1,400
WebSocket Servers4 x c6i.large (2 vCPU, 4GB)$500
PostgreSQL (RDS)db.r6g.2xlarge Multi-AZ + Read Replicas$2,800
Redis (ElastiCache)cache.r6g.xlarge cluster (3 nodes)$1,200
Elasticsearch3 x m6g.xlarge.search (64GB index)$2,100
Neo4j (Graph DB)2 x db.r6g.xlarge$1,800
ClickHouse (Analytics)Self-managed 3-node cluster$900
S3 (Assets + Snapshots)5TB storage + transfers$250
CloudFront (CDN)10TB monthly transfer$850
RabbitMQ (MQ)Amazon MQ, 2-node cluster$350
Lambda (Export)100K invocations/month$50
Monitoring (Grafana Cloud)Logs + metrics + traces$500
Email (SES)500K emails/month$200
WAF + DNS + CertsAWS WAF + Route 53 + ACM$120
Total Infrastructure~$13,070

Cost Optimization Strategies

  • Reserved Instances: 1-year RI for databases saves 30-40% (~$3,000/month)
  • S3 Intelligent-Tiering: Automatically moves infrequently accessed snapshots to cheaper storage
  • Spot Instances: Export Lambda and background workers use spot for 60% savings
  • CRDT Compression: Compress Yjs updates with zstd before storage (90%+ ratio)
  • Search Sharding: Index only recently active pages in hot tier; cold pages in cheaper storage
  • Connection Pooling: PgBouncer for PostgreSQL reduces RDS connection overhead
Unit Economics

At $13K/month infrastructure cost serving 100K users, the cost per user is ~$0.13/month. With SaaS pricing of $10/user/month (Team plan), gross margin is ~98%. At early stage (1K users), infrastructure costs ~$3K/month, requiring ~50 paying users at $10/month to break even.

28. API Design (Detailed)

Authentication

HTTP
POST /api/v1/auth/login
Content-Type: application/json

{
    "email": "user@example.com",
    "password": "secure_password",
    "mfa_code": "123456"
}

# Response
{
    "access_token": "eyJhbGciOiJSUzI1NiIs...",
    "refresh_token": "dGhpcyBpcyBhIHJlZnJl...",
    "expires_in": 900,
    "token_type": "Bearer",
    "user": {
        "id": "550e8400-e29b-41d4-a716-446655440000",
        "email": "user@example.com",
        "name": "Jane Doe"
    }
}

# Subsequent requests
GET /api/v1/pages/550e8400-e29b-41d4-a716-446655440000
Authorization: Bearer eyJhbGciOiJSUzI1NiIs...

Create Page

HTTP
POST /api/v1/spaces/660e8400-e29b-41d4-a716-446655440001/pages
Authorization: Bearer {token}
Content-Type: application/json

{
    "title": "System Architecture Overview",
    "icon": "🏗",
    "parent_page_id": null,
    "template_id": "770e8400-e29b-41d4-a716-446655440002",
    "initial_blocks": [
        {"type": "heading1", "content": "System Architecture"},
        {"type": "paragraph", "content": "This document describes..."},
        {"type": "code", "properties": {"language": "csharp", "code": "public class Service { }"}}
    ],
    "properties": {"Status": "Draft", "Owner": "Jane Doe"}
}

# Response 201 Created
{
    "id": "880e8400-e29b-41d4-a716-446655440003",
    "title": "System Architecture Overview",
    "url": "https://wiki.example.com/spaces/engineering/system-architecture-overview",
    "version": 1,
    "created_at": "2025-01-10T14:30:00Z",
    "collaboration_url": "wss://wiki.example.com/ws/doc/880e8400..."
}

GraphQL Schema

GraphQL
type Query {
    space(id: ID!): Space
    spaces(first: Int, after: String): SpaceConnection!
    page(id: ID!): Page
    search(query: String!, filters: SearchFilters): SearchResult!
    myRecentPages(first: Int): [Page!]!
}

type Mutation {
    createPage(input: CreatePageInput!): Page!
    updatePage(id: ID!, input: UpdatePageInput!): Page!
    deletePage(id: ID!): Boolean!
    movePage(id: ID!, targetSpaceId: ID, parentId: ID): Page!
    addComment(input: AddCommentInput!): Comment!
    exportPage(id: ID!, format: ExportFormat!): ExportResult!
}

type Subscription {
    pageUpdated(pageId: ID!): PageUpdate!
    commentAdded(pageId: ID!): Comment!
    mentionReceived: Mention!
}

type Page {
    id: ID!
    title: String!
    space: Space!
    parent: Page
    children(first: Int): PageConnection!
    blocks: [Block!]!
    comments(first: Int): CommentConnection!
    backlinks(first: Int): [Backlink!]!
    versions(first: Int): [PageVersion!]!
    analytics: PageAnalytics!
    createdAt: DateTime!
    updatedAt: DateTime!
}

29. Testing Strategy

Test Pyramid

graph TB E2E["E2E Tests (Playwright) - 50 tests - 15 min"] INT["Integration Tests (xUnit + TestContainers) - 200 tests - 8 min"] UNIT["Unit Tests (xUnit + Moq) - 1500+ tests - 2 min"] E2E --> INT --> UNIT

Test Categories

CategoryScopeToolsCount
Unit TestsBlock rendering, CRDT ops, permission logic, search indexingxUnit, Moq, FluentAssertions1,500+
Integration TestsAPI endpoints, database operations, Elasticsearch queriesxUnit, TestContainers200
CRDT ConvergenceVerify concurrent edits merge correctlyxUnit, Yjs test utils100
E2E TestsUser flows: create, edit, share, search, exportPlaywright50
Load Tests1K concurrent editors, 10K search QPSk6, Locust20
Security TestsOWASP Top 10, permission bypassOWASP ZAP50

CRDT Convergence Test

C#
public class CrdtConvergenceTests
{
    [Fact]
    public async Task Concurrent_Edits_Merge_Correctly()
    {
        // Arrange: Create three separate Yjs documents
        var doc1 = new YDoc();
        var doc2 = new YDoc();
        var doc3 = new YDoc();

        var text1 = doc1.GetText("blocks");
        var text2 = doc2.GetText("blocks");
        var text3 = doc3.GetText("blocks");

        // Act: Each user makes independent edits
        text1.Insert(0, "Hello ");
        text2.Insert(0, "World ");
        text3.Insert(0, "From ");

        // Sync doc1 and doc2
        var update12 = doc1.EncodeStateAsUpdate(doc2.StateVector);
        doc2.ApplyUpdate(update12);
        var update21 = doc2.EncodeStateAsUpdate(doc1.StateVector);
        doc1.ApplyUpdate(update21);

        // Sync with doc3
        var update13 = doc1.EncodeStateAsUpdate(doc3.StateVector);
        doc3.ApplyUpdate(update13);
        var update31 = doc3.EncodeStateAsUpdate(doc1.StateVector);
        doc1.ApplyUpdate(update31);

        // Assert: All documents converge to the same state
        var final1 = text1.ToString();
        var final2 = text2.ToString();
        var final3 = text3.ToString();

        Assert.Equal(final1, final2);
        Assert.Equal(final2, final3);
        Assert.Contains("Hello", final1);
        Assert.Contains("World", final1);
        Assert.Contains("From", final1);
    }

    [Theory]
    [InlineData(5, 10)]   // 5 users, 10 edits each
    [InlineData(10, 50)]  // 10 users, 50 edits each
    [InlineData(20, 100)] // 20 users, 100 edits each
    public async Task Stress_Test_Convergence(int userCount, int editsPerUser)
    {
        var documents = Enumerable.Range(0, userCount)
            .Select(_ => new YDoc())
            .ToList();

        // Simulate concurrent edits
        var tasks = documents.Select((doc, i) => Task.Run(() =>
        {
            var text = doc.GetText("blocks");
            for (int j = 0; j < editsPerUser; j++)
            {
                text.Insert(j * 5, $"U{i}E{j} ");
            }
        }));

        await Task.WhenAll(tasks);

        // Sync all documents pairwise
        for (int i = 0; i < documents.Count; i++)
        {
            for (int j = i + 1; j < documents.Count; j++)
            {
                var update = documents[i].EncodeStateAsUpdate(documents[j].StateVector);
                documents[j].ApplyUpdate(update);
                var reverseUpdate = documents[j].EncodeStateAsUpdate(documents[i].StateVector);
                documents[i].ApplyUpdate(reverseUpdate);
            }
        }

        // Verify all documents converge
        var reference = documents[0].GetText("blocks").ToString();
        foreach (var doc in documents.Skip(1))
        {
            Assert.Equal(reference, doc.GetText("blocks").ToString());
        }
    }
}
CI/CD Pipeline

Tests run on every PR: unit tests (<2 min), integration tests (<8 min), CRDT convergence tests (<3 min). E2E tests run nightly and on release branches. Load tests run weekly against a staging environment. Security scans run on every merge to main. Total CI time for a typical PR is under 15 minutes.

30. Interview Q&A

Q: How would you handle real-time collaboration when two users edit the same paragraph simultaneously?

A: We use Yjs CRDTs for conflict-free merging. Each character insertion/deletion is a CRDT operation with a unique timestamp and client ID. When two users type in the same paragraph, the operations are automatically merged based on their Lamport timestamps — both users' edits appear in a deterministic, consistent order. The WebSocket server broadcasts operations to all connected clients, who merge them locally. If a user is offline, their operations are buffered locally and merged when they reconnect.

Q: How do you ensure search results respect page permissions without making permission checks slow?

A: We use a two-pronged approach. First, the visible_to field in Elasticsearch stores the list of user/group IDs that can access each page. This is updated whenever permissions change (via async event). At search time, we add a terms filter on visible_to containing the requesting user's ID. Second, we cache permission sets in Redis with a 5-minute TTL, and use Elasticsearch's built-in filter context (which is cached) for fast repeated queries. For very large tenants, we use a permission inheritance bitmap — a bitfield where each bit represents access to a space/page, enabling O(1) permission checks.

Q: How would you design the block-based editor to support both simple text and complex database views?

A: Every element is a block with a uniform schema: {id, type, content, properties, children}. Simple text blocks (paragraphs, headings) use the content field with inline segments. Complex blocks (tables, databases, embeds) store their data in the properties dictionary as structured JSON. The editor framework (Blocknote/ProseMirror) renders each block type with a custom React component. This uniformity means CRDT collaboration, versioning, permissions, and API access all work identically regardless of block type.

Q: How do you handle offline editing and conflict resolution when the user reconnects?

A: Yjs CRDTs solve this naturally. When offline, edits are applied to a local Yjs document stored in IndexedDB. When the user reconnects, the sync engine exchanges state vectors with the server — the server sends all updates the client missed, and the client sends all updates the server missed. Yjs merges everything deterministically. There are no "conflicts" in the traditional sense — all operations from all users are merged into a consistent state. The only edge case is structural conflicts (e.g., two users move the same block to different parents), which we handle by keeping the last-write-wins for metadata while preserving both content changes.

Q: How would you scale the system to support 1 million concurrent editors across 10 million pages?

A: Three scaling strategies: (1) Horizontal scaling of WebSocket servers — use sticky sessions or a Redis-backed pub/sub to route messages. Each WebSocket server handles ~10K connections. With 100 servers, we handle 1M concurrent connections. (2) Document-level sharding — route each document to a specific collaboration server based on document ID hash. This ensures all editors of the same document connect to the same server, minimizing cross-server message passing. (3) Cold page management — pages not edited in 24+ hours are evicted from the collaboration server's memory. They are loaded on-demand when someone opens them. This means only ~1% of pages (100K) need to be in-memory at any time.

Q: How do you ensure the knowledge graph stays accurate as pages are created, edited, and deleted?

A: The knowledge graph is updated asynchronously via events. When a page is saved, a PageLinksUpdated event is published containing the page's current set of internal links. The graph service consumes this event, removes all outgoing edges from the page, and creates new edges based on the current links. When a page is deleted, a PageDeleted event triggers removal of all edges involving that page. Since graph updates are eventually consistent (typically <1 second lag), the graph may briefly show stale backlinks — but this is acceptable for the use case (graph visualization and backlink discovery).

Q: How would you implement access control that works at workspace, space, folder, and page levels with inheritance?

A: Each resource (workspace, space, folder, page) has an optional explicit permission entry. At check time, we walk up the hierarchy from page to workspace, collecting all permission entries for the user. The most specific non-inherited permission wins (page overrides folder, folder overrides space). We cache the effective permission in Redis with a 5-minute TTL, invalidated immediately when any permission in the chain changes. For bulk operations (e.g., loading the sidebar tree), we pre-load all permissions for the user's spaces in a single query and build an in-memory permission map.

Knowledge Base & Wiki Platform Design — Senior+ Guide | Ayodhyya