How to Design a Knowledge Base & Wiki Platform
Building a Production-Grade Confluence/Notion — Editing, Collaboration, Search, Permissions & Beyond
1. Overview & Requirements
A Knowledge Base & Wiki Platform is a collaborative workspace where teams create, organize, and share documents, wikis, databases, and project knowledge. Think Confluence, Notion, or Coda. The core promise is that any team member can find the right information in seconds, edit it collaboratively, and trust that it is up-to-date and secure.
At its heart, a wiki platform is a content management system with real-time collaboration superpowers. Unlike a traditional CMS, every page is simultaneously a document, a database row, and a collaborative canvas. The challenge is building a system that supports rich formatting, structured data, real-time editing by dozens of concurrent users, granular permissions, full-text search, and offline access — all while maintaining sub-50ms latency for keystroke rendering.
Functional Requirements
- Document Editing: Rich-text editor with block-based architecture (paragraphs, headings, lists, tables, code blocks, images, embeds, databases).
- Real-Time Collaboration: Multiple users editing the same page simultaneously with CRDT-based merging.
- Page Hierarchy: Spaces, Folders, Pages, and Sub-pages with drag-and-drop reordering.
- Templates: Reusable page templates (meeting notes, sprint retrospectives) and database templates (Kanban, spreadsheet, gallery).
- Full-Text Search: Search across all pages, including content inside images (OCR), code blocks, and attachments.
- Versioning: Every edit produces a version; users can view diffs and restore any previous version.
- Comments & Annotations: Inline comments on text selections, resolved/unresolved threads, @mentions in comments.
- Permissions: RBAC at workspace, space, folder, and page levels. Public share links with expiry.
- Embeds: Inline images, videos (YouTube), code blocks with syntax highlighting, database views, Figma frames.
- Table / Database Blocks: Spreadsheet-like tables with sorting, filtering, formula columns, and linked databases.
- Kanban Board Views: Visual board views for database blocks with drag-and-drop between columns.
- @Mentions & Notifications: Mention users or groups; deliver notifications via in-app, email, Slack, and push.
- Export: Export pages as PDF, Markdown, or HTML.
- Import: Import from Confluence, Google Docs, Markdown, HTML, Notion, Word.
- API: RESTful + GraphQL API for programmatic access and third-party integrations.
- Offline Mode: Edit pages offline; sync when connectivity resumes.
- Knowledge Graph: Automatic backlinks between pages; graph visualization of page relationships.
- Auto-Generated TOC: Table of contents generated from page headings.
- Analytics: Page views, popular pages, recently edited, contributor stats.
- Audit Log: Track all user actions (page created, edited, permissions changed, deleted).
Non-Functional Requirements
| Attribute | Target |
|---|---|
| Latency (editor keystroke) | < 50ms perceived, < 200ms server round-trip |
| Availability | 99.95% (SaaS), 99.99% (enterprise tier) |
| Throughput | 100K concurrent editors, 10M pages indexed |
| Data Durability | 99.999999999% (11 nines) via multi-region replication |
| Search Latency | < 200ms (p95) for full-text queries |
| Consistency | Strong for permissions; eventual for search indexing |
| Compliance | SOC 2 Type II, GDPR, HIPAA (enterprise), FedRAMP |
| Offline Support | Full read/write for 7+ days without connectivity |
The fundamental tension in a wiki platform is between rich formatting and structured data. Notion solved this with a block-based model where every element (text, table, database, embed) is a "block" with a uniform schema. We adopt the same philosophy — it simplifies CRDT collaboration, versioning, and API design.
2. High-Level Architecture
Key Architectural Decisions
| Decision | Choice | Rationale |
|---|---|---|
| Document Format | Block-based JSON (CRDT) | Enables granular merge, per-block permissions, API-friendly |
| Real-Time Sync | Yjs CRDT via WebSocket | Peer-to-peer merging without central server; handles offline |
| Primary DB | PostgreSQL + Citus | Strong consistency for metadata; Citus for horizontal partitioning |
| Search | Elasticsearch 8.x | Proven full-text search with analyzers, facets, and aggregations |
| Asset Storage | S3 + CloudFront | Durable blob storage; CDN for low-latency delivery |
| Event Bus | RabbitMQ | Decouples services; reliable event delivery for indexing, notifications |
| Knowledge Graph | Neo4j | Native graph DB for backlinks and page relationship queries |
| Analytics | ClickHouse | Columnar store for fast aggregation queries on page views |
| Export | Puppeteer on Lambda | Headless Chrome for pixel-perfect PDF generation |
Service Communication Patterns
Services communicate through two primary patterns:
- Synchronous (gRPC/HTTP): Used for user-facing operations that require immediate response — page load, save, search, permission check. API Gateway routes requests to the appropriate service.
- Asynchronous (RabbitMQ events): Used for background work — search indexing, notification delivery, analytics collection, knowledge graph updates. Events are published after successful database writes and consumed by dedicated worker processes.
A wiki platform has distinct workloads: real-time collaboration (CPU-bound, stateful), search (I/O-bound, read-heavy), export (CPU-bound, bursty), and analytics (batch processing). Microservices allow independent scaling of each workload. The collaboration service needs persistent WebSocket connections; the search service needs Elasticsearch nodes; the export service needs burst compute. A monolith would force all these into a single scaling profile.
3. Document Editing — Rich Text & Block-Based Editor
The Block Model
Every page is a sequence of blocks. Each block has a unique ID, a type, content, and optional metadata. This is the fundamental unit of editing, collaboration, permissions, and versioning. The block model allows us to treat every element uniformly — whether it is a paragraph of text, an embedded video, or a full database view.
C#
public class Block
{
public Guid Id { get; set; } = Guid.NewGuid();
public BlockType Type { get; set; }
public string Content { get; set; } = string.Empty;
public Dictionary<string, object> Properties { get; set; } = new();
public List<Block> Children { get; set; } = new();
public Guid? ParentId { get; set; }
public int SortOrder { get; set; }
public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
public DateTime UpdatedAt { get; set; } = DateTime.UtcNow;
public Guid CreatedBy { get; set; }
}
public enum BlockType
{
Paragraph,
Heading1, Heading2, Heading3, Heading4, Heading5, Heading6,
BulletedList, NumberedList, TodoList,
Quote, Callout, Divider,
Code,
Image, Video, Audio, File,
Table, Spreadsheet,
Embed, Bookmark,
Database, KanbanBoard, Gallery,
Equation, TableOfContents,
SyncedBlock, Breadcrumb
}
public class Document
{
public Guid Id { get; set; } = Guid.NewGuid();
public string Title { get; set; } = string.Empty;
public Guid SpaceId { get; set; }
public Guid? ParentPageId { get; set; }
public List<Block> Blocks { get; set; } = new();
public Guid OwnerId { get; set; }
public PageStatus Status { get; set; } = PageStatus.Draft;
public string Icon { get; set; } = "📄";
public string CoverImageUrl { get; set; }
public List<string> Tags { get; set; } = new();
public Dictionary<string, string> Properties { get; set; } = new();
public DateTime CreatedAt { get; set; } = DateTime.UtcNow;
public DateTime UpdatedAt { get; set; } = DateTime.UtcNow;
public DateTime? PublishedAt { get; set; }
public bool IsDeleted { get; set; }
public DateTime? DeletedAt { get; set; }
}
Block Rendering Pipeline
Rich Text Formatting
Inline formatting is handled at the leaf level within a block. Each block contains a list of inline segments with formatting marks. This design means that a single paragraph block can contain bold text, italic text, code spans, and links — all within a single block's content model.
C#
public class InlineSegment
{
public string Text { get; set; } = string.Empty;
public FormattingMarks Marks { get; set; } = new();
}
public class FormattingMarks
{
public bool Bold { get; set; }
public bool Italic { get; set; }
public bool Underline { get; set; }
public bool Strikethrough { get; set; }
public bool Code { get; set; }
public string? Link { get; set; }
public string? Color { get; set; }
public HighlightStyle? Highlight { get; set; }
}
public enum HighlightStyle
{
Gray, Brown, Orange, Yellow, Green, Blue, Purple, Pink, Red
}
Code Block Implementation
Code blocks support syntax highlighting via Shiki (for the frontend) and store language metadata. Execution can be optionally enabled for sandboxed languages using WebAssembly runtimes (JavaScript via V8 isolates, Python via Pyodide). The code block stores both the source and execution output to support collaborative debugging.
C#
public class CodeBlockProperties
{
public string Language { get; set; } = "plaintext";
public string Code { get; set; } = string.Empty;
public bool ShowLineNumbers { get; set; }
public bool IsExecutable { get; set; }
public string? ExecutionOutput { get; set; }
public List<CodeExecutionResult> ExecutionHistory { get; set; } = new();
}
public class CodeExecutionResult
{
public string Output { get; set; }
public bool IsError { get; set; }
public int ExitCode { get; set; }
public TimeSpan ExecutionTime { get; set; }
public DateTime ExecutedAt { get; set; }
}
We use Blocknote (built on ProseMirror) as the editor framework. It provides a Notion-like block editor out of the box, is open-source, supports React, and can be extended with custom block types. For the CRDT layer, we integrate Yjs as a y-prosemirror binding. Alternatively, Tiptap is another excellent option with built-in collaboration via their cloud service.
4. Real-Time Collaboration — CRDT-Based
Why CRDT Over OT?
Operational Transformation (OT) requires a central server to transform operations, creating a single point of failure and requiring careful ordering. CRDTs (Conflict-free Replicated Data Types) allow each client to apply operations independently and merge them deterministically, making them ideal for offline-first collaboration. In the OT model, a server must receive and sequence every operation before broadcasting — if the server is slow, the entire collaboration stalls. With CRDTs, clients can edit independently and merge later, with guaranteed convergence.
CRDT Document Manager
C#
public class CrdtDocumentManager
{
private readonly ConcurrentDictionary<Guid, ActiveDocument> _activeDocs = new();
private readonly IConnectionPool _connectionPool;
public async Task<YDoc> LoadDocument(Guid documentId)
{
if (_activeDocs.TryGetValue(documentId, out var cached))
return cached.YDoc;
var snapshot = await LoadSnapshot(documentId);
var yDoc = new YDoc();
if (snapshot != null)
{
var update = YUpdate.Merge(snapshot.Updates);
yDoc.ApplyUpdate(update);
}
_activeDocs.TryAdd(documentId, new ActiveDocument
{
Id = documentId,
YDoc = yDoc,
LastAccessed = DateTime.UtcNow
});
return yDoc;
}
public async Task ApplyUpdate(Guid documentId, byte[] update)
{
var doc = await LoadDocument(documentId);
doc.ApplyUpdate(update);
await DebouncedPersist(documentId, doc);
}
public byte[] GetStateVector(Guid documentId)
{
var doc = _activeDocs.GetValueOrDefault(documentId);
return doc?.YDoc.StateVector ?? Array.Empty<byte>();
}
}
WebSocket Protocol
| Message Type | Direction | Payload | Description |
|---|---|---|---|
sync_step1 | Client to Server | StateVector | Initial sync request |
sync_step2 | Server to Client | Update | Full state from server |
update | Both directions | Update | Incremental CRDT update |
awareness | Both directions | AwarenessState | Cursor position, selection, user info |
cursor | Client to Server | CursorPosition | Real-time cursor broadcast |
Awareness Protocol
The awareness layer tracks ephemeral state — who is online, where their cursor is, what they have selected. This state is not persisted and is broadcast to all connected clients for a document. It gives the "multiplayer" feel to the editor, showing colored cursors and name labels for each collaborator.
C#
public class AwarenessState
{
public Guid UserId { get; set; }
public string DisplayName { get; set; }
public string AvatarUrl { get; set; }
public string Color { get; set; } // Unique cursor color per user
public CursorPosition? Cursor { get; set; }
public TextSelection? Selection { get; set; }
public DateTime LastActive { get; set; }
public UserStatus Status { get; set; }
}
public class CursorPosition
{
public Guid BlockId { get; set; }
public int Offset { get; set; }
public int Length { get; set; }
}
public class CollaborationService
{
private readonly Dictionary<Guid, HashSet<Guid>> _documentUsers = new();
private readonly Dictionary<Guid, AwarenessState> _userStates = new();
public void UpdateAwareness(Guid documentId, Guid userId, AwarenessState state)
{
_userStates[userId] = state;
if (_documentUsers.ContainsKey(documentId))
{
var clients = _documentUsers[documentId];
foreach (var clientId in clients.Where(id => id != userId))
{
SendAwarenessUpdate(clientId, state);
}
}
}
public void RemoveUser(Guid documentId, Guid userId)
{
_documentUsers.GetValueOrDefault(documentId)?.Remove(userId);
_userStates.Remove(userId);
BroadcastPresence(documentId, userId, UserStatus.Offline);
}
}
Yjs CRDTs grow unbounded as edits accumulate because tombstones (deleted content) are retained for merge correctness. Implement periodic garbage collection by taking a full snapshot, resetting the CRDT state, and deleting old updates. Run GC daily during low-traffic hours. For a document with 1M+ operations, this can reduce memory by 90%+ and storage significantly.
5. Page Hierarchy & Organization
The Hierarchy Model
A wiki is organized into a tree of Spaces, Folders, Pages, and Sub-pages. Each space is an isolated workspace (e.g., "Engineering", "Product", "Marketing"). The hierarchy is stored using the adjacency list pattern with a materialized path for efficient subtree queries. This gives us the best of both worlds: simple parent-child lookups via adjacency list, and fast subtree operations via the materialized path string.
C#
public class Space
{
public Guid Id { get; set; } = Guid.NewGuid();
public string Name { get; set; } = string.Empty;
public string Description { get; set; } = string.Empty;
public string Icon { get; set; } = "📁";
public string CoverImageUrl { get; set; }
public Guid WorkspaceId { get; set; }
public Guid CreatedBy { get; set; }
public SpaceVisibility Visibility { get; set; } = SpaceVisibility.Private;
public string MaterializedPath { get; set; } = "/";
public int Depth { get; set; }
public int SortOrder { get; set; }
public bool IsArchived { get; set; }
public DateTime CreatedAt { get; set; }
public DateTime UpdatedAt { get; set; }
}
public class PageTreeNode
{
public Guid Id { get; set; }
public string Title { get; set; }
public Guid? ParentId { get; set; }
public string MaterializedPath { get; set; }
public int Depth { get; set; }
public int SortOrder { get; set; }
public int ChildCount { get; set; }
public string Icon { get; set; }
public bool IsPublished { get; set; }
}
Hierarchy Service
C#
public class PageHierarchyService
{
private readonly IDbConnection _db;
public async Task<List<PageTreeNode>> GetSubtree(Guid spaceId, Guid? parentId = null, int maxDepth = 5)
{
var basePath = parentId.HasValue
? $"/{spaceId}/{parentId}"
: $"/{spaceId}";
var pages = await _db.QueryAsync<PageTreeNode>(
@"SELECT id, title, parent_id, materialized_path, depth, sort_order,
child_count, icon, is_published
FROM pages
WHERE space_id = @SpaceId
AND materialized_path LIKE @Path
AND depth <= @MaxDepth
AND is_deleted = false
ORDER BY sort_order",
new { SpaceId = spaceId, Path = $"{basePath}%", MaxDepth = maxDepth });
return pages.ToList();
}
public async Task<PageTreeNode> MovePage(Guid pageId, Guid? newParentId, int newSortOrder)
{
using var transaction = _db.BeginTransaction();
var page = await _db.QuerySingleAsync<PageTreeNode>(
"SELECT * FROM pages WHERE id = @Id FOR UPDATE",
new { Id = pageId }, transaction);
var newParentPath = newParentId.HasValue
? (await _db.QuerySingleAsync<string>(
"SELECT materialized_path FROM pages WHERE id = @Id",
new { Id = newParentId.Value }, transaction))
: $"/{page.Id}";
var oldPath = page.MaterializedPath;
var newPath = $"{newParentPath}/{page.Id}";
// Update materialized path for this page and ALL descendants
await _db.ExecuteAsync(
@"UPDATE pages
SET materialized_path = REPLACE(materialized_path, @OldPath, @NewPath),
parent_id = @NewParentId,
sort_order = @SortOrder,
updated_at = NOW()
WHERE materialized_path LIKE @OldPath || '%'",
new { OldPath = oldPath, NewPath = newPath,
NewParentId = newParentId, SortOrder = newSortOrder },
transaction);
transaction.Commit();
return await GetPageTreeNode(pageId);
}
}
We choose Materialized Path because: (1) subtree queries are a simple LIKE prefix match, (2) moves only update affected rows (not the entire tree), (3) it naturally stores the full ancestry for breadcrumbs. The tradeoff is that deep moves require updating the path string of all descendants, but wiki pages are rarely moved frequently enough for this to matter. For extremely large spaces (100K+ pages), consider Closure Table for better query performance at the cost of additional storage.
6. Templates — Page & Database Templates
Template Architecture
Templates are first-class documents with an is_template = true flag. They contain pre-configured blocks, properties, and layout. When a user creates a page from a template, we deep-copy the block tree and CRDT state. Templates are categorized into page templates (for content) and database templates (for structured data views).
C#
public class Template
{
public Guid Id { get; set; } = Guid.NewGuid();
public string Name { get; set; } = string.Empty;
public string Description { get; set; }
public TemplateType Type { get; set; }
public string Category { get; set; }
public string Icon { get; set; }
public string ThumbnailUrl { get; set; }
public byte[] BlockSnapshot { get; set; }
public List<Block> Blocks { get; set; }
public Dictionary<string, string> DefaultProperties { get; set; } = new();
public Guid? SpaceId { get; set; }
public Guid CreatedBy { get; set; }
public bool IsPublic { get; set; }
public int UseCount { get; set; }
public DateTime CreatedAt { get; set; }
}
public class TemplateService
{
private readonly IDbConnection _db;
private readonly CrdtDocumentManager _crdtManager;
public async Task<Document> CreateFromTemplate(Guid templateId, Guid userId, string pageTitle)
{
var template = await _db.QuerySingleAsync<Template>(
"SELECT * FROM templates WHERE id = @Id", new { Id = templateId });
var templateDoc = new YDoc();
templateDoc.ApplyUpdate(template.BlockSnapshot);
var newDoc = new YDoc();
var update = templateDoc.EncodeStateAsUpdate(templateDoc.StateVector);
newDoc.ApplyUpdate(update);
RegenerateBlockIds(newDoc);
var document = new Document
{
Title = pageTitle,
Blocks = ConvertToBlockList(newDoc),
OwnerId = userId,
Properties = new Dictionary<string, string>(template.DefaultProperties)
};
await _db.ExecuteAsync(
@"INSERT INTO pages (id, title, space_id, owner_id, properties, template_id)
VALUES (@Id, @Title, @SpaceId, @OwnerId, @Properties, @TemplateId)", document);
await _db.ExecuteAsync(
"UPDATE templates SET use_count = use_count + 1 WHERE id = @Id",
new { Id = templateId });
return document;
}
}
Built-in Templates
| Template | Type | Key Blocks |
|---|---|---|
| Meeting Notes | Page | Title, Attendees, Agenda, Notes, Action Items, Decisions |
| Sprint Retrospective | Page | Went Well, To Improve, Action Items, Vote Counts |
| Technical RFC | Page | Summary, Motivation, Design, Alternatives, Open Questions |
| Project Tracker | Database | Status, Priority, Assignee, Due Date columns + Kanban view |
| Team Directory | Database | Name, Role, Department, Photo, Contact columns |
| Knowledge Base Home | Page | Cover image, icon, linked databases, table of contents |
7. Full-Text Search — Elasticsearch
Search Architecture
Elasticsearch Index Mapping
JSON
{
"mappings": {
"properties": {
"page_id": { "type": "keyword" },
"space_id": { "type": "keyword" },
"title": {
"type": "text",
"analyzer": "english",
"fields": {
"keyword": { "type": "keyword" },
"autocomplete": {
"type": "text",
"analyzer": "edge_ngram",
"search_analyzer": "standard"
}
}
},
"content": { "type": "text", "analyzer": "english" },
"headings": { "type": "text", "analyzer": "english", "boost": 2.0 },
"code_content": { "type": "text", "analyzer": "whitespace" },
"tags": { "type": "keyword" },
"owner_id": { "type": "keyword" },
"visible_to": { "type": "keyword" },
"created_at": { "type": "date" },
"updated_at": { "type": "date" },
"view_count": { "type": "integer" }
}
}
}
Search Service Implementation
C#
public class SearchService
{
private readonly ElasticClient _elastic;
private readonly IPermissionService _permissions;
public async Task<SearchResult> SearchAsync(string query, Guid userId, SearchFilters filters)
{
var accessibleIds = await _permissions.GetAccessiblePageIds(userId);
var searchRequest = new SearchRequest
{
Query = new BoolQuery
{
Must = new List<QueryContainer>
{
new MultiMatchQuery
{
Query = query,
Fields = new[] {
"title^5", "title.autocomplete^3",
"headings^3", "content", "code_content", "tags^2"
},
Type = TextQueryType.BestFields,
Fuzziness = Fuzziness.Auto
}
},
Filter = new List<QueryContainer>
{
new TermsQuery { Field = "page_id",
Terms = accessibleIds.Select(id => id.ToString()) },
new TermQuery { Field = "is_deleted", Value = false }
}
},
Highlight = new Highlight
{
Fields = new Dictionary<string, HighlightField>
{
["content"] = new HighlightField { NumberOfFragments = 3, FragmentSize = 200 },
["title"] = new HighlightField()
}
},
Aggregations = new AggregationContainer
{
Spaces = new TermsAggregation("space_id") { Field = "space_id", Size = 20 },
Owners = new TermsAggregation("owner_id") { Field = "owner_id", Size = 10 }
},
Size = filters.PageSize,
From = filters.Offset
};
var response = await _elastic.SearchAsync<SearchDocument>(searchRequest);
return new SearchResult
{
TotalHits = response.Total,
Items = response.Hits.Select(MapToSearchResultItem).ToList(),
Facets = MapFacets(response.Aggregations),
TookMs = response.TookMilliseconds
};
}
}
Indexing Pipeline
- Event Capture: When a page is saved, a
PageUpdatedevent is published to RabbitMQ with the page ID and changed block IDs. - Indexer Worker: Consumes events, fetches the full document, extracts text from all blocks (including OCR for images, text extraction from code blocks), and indexes into Elasticsearch.
- Permission-Aware: The
visible_tofield stores the list of user/group IDs that can access the page, enabling fast permission filtering at search time. - Debouncing: Rapid edits within a 30-second window are debounced to avoid excessive re-indexing.
- Full Reindex: A nightly job performs a full reindex to catch any inconsistencies, using Elasticsearch alias swap for zero-downtime reindexing.
Results are ranked using a combination of: (1) text relevance (BM25), (2) recency (pages updated recently rank higher), (3) popularity (view count, edit frequency), (4) title match boost (5x), and (5) user context (pages the user recently viewed get a personalization boost). This multi-signal ranking ensures the most relevant results appear first.
8. Content Versioning & History
Version Storage Strategy
Every save creates a version. We use a snapshot + delta approach: the latest state is always available as a snapshot, while historical versions are stored as Yjs CRDT updates (deltas) to save space. Every 50 versions, we create a new full snapshot to bound recovery time. This approach uses ~95% less storage than full snapshots while maintaining fast version restoration.
C#
public class PageVersion
{
public Guid Id { get; set; } = Guid.NewGuid();
public Guid PageId { get; set; }
public int VersionNumber { get; set; }
public Guid CreatedBy { get; set; }
public string? ChangeDescription { get; set; }
public byte[] CrdtUpdate { get; set; }
public byte[]? FullSnapshot { get; set; }
public bool IsSnapshot { get; set; }
public int BlocksChanged { get; set; }
public int WordsAdded { get; set; }
public int WordsRemoved { get; set; }
public DateTime CreatedAt { get; set; }
}
public class VersionService
{
private readonly IDbConnection _db;
public async Task<PageVersion> CreateVersion(Guid pageId, Guid userId, byte[] currentUpdate)
{
var lastVersion = await _db.QuerySingleAsync<PageVersion>(
@"SELECT * FROM page_versions
WHERE page_id = @PageId
ORDER BY version_number DESC LIMIT 1",
new { PageId = pageId });
var newVersionNumber = (lastVersion?.VersionNumber ?? 0) + 1;
var isSnapshot = newVersionNumber % 50 == 0 || lastVersion == null;
var version = new PageVersion
{
PageId = pageId,
VersionNumber = newVersionNumber,
CreatedBy = userId,
CrdtUpdate = currentUpdate,
IsSnapshot = isSnapshot,
FullSnapshot = isSnapshot ? await GetFullSnapshot(pageId) : null
};
await _db.ExecuteAsync(
@"INSERT INTO page_versions
(id, page_id, version_number, created_by, crdt_update,
full_snapshot, is_snapshot, created_at)
VALUES (@Id, @PageId, @VersionNumber, @CreatedBy, @CrdtUpdate,
@FullSnapshot, @IsSnapshot, @CreatedAt)", version);
return version;
}
public async Task<Document> RestoreVersion(Guid pageId, int versionNumber)
{
var snapshot = await _db.QuerySingleAsync<PageVersion>(
@"SELECT * FROM page_versions
WHERE page_id = @PageId AND is_snapshot = true
AND version_number <= @VersionNumber
ORDER BY version_number DESC LIMIT 1",
new { PageId = pageId, VersionNumber = versionNumber });
var yDoc = new YDoc();
yDoc.ApplyUpdate(snapshot.FullSnapshot);
var deltas = await _db.QueryAsync<PageVersion>(
@"SELECT * FROM page_versions
WHERE page_id = @PageId AND version_number > @SnapVer
AND version_number <= @TargetVer
ORDER BY version_number",
new { PageId = pageId, SnapVer = snapshot.VersionNumber, TargetVer = versionNumber });
foreach (var delta in deltas)
yDoc.ApplyUpdate(delta.CrdtUpdate);
return ConvertToDocument(yDoc);
}
}
Yjs CRDT updates are compact binary buffers. A typical edit (typing a word) produces a 50-200 byte update. Even with 100 edits per day on a page, a year of versions for that page would be ~5MB — very manageable. For comparison, storing full document snapshots every save would use 10-100x more storage. The snapshot-every-50 approach balances recovery speed with storage efficiency.
10. Page Permissions & Access Control (RBAC)
Permission Model
Permissions follow a hierarchical inheritance model with override capability. Permissions are checked at: Workspace, Space, Folder, and Page levels. A more specific permission always overrides a less specific one. The hierarchy is: Workspace Level (Admin, Member, Guest) → Space Level (Can Edit, Can View, No Access) → Folder Level (inherited from Space) → Page Level (Custom Override).
C#
public enum PermissionRole
{
NoAccess = 0, CanView = 1, CanComment = 2,
CanEdit = 3, CanManage = 4, FullAccess = 5
}
public class PermissionService
{
private readonly IDbConnection _db;
private readonly IDistributedCache _cache;
public async Task<PermissionRole> GetEffectivePermission(
Guid userId, ResourceType resourceType, Guid resourceId)
{
var cacheKey = $"perm:{userId}:{resourceType}:{resourceId}";
var cached = await _cache.GetAsync<PermissionRole>(cacheKey);
if (cached.HasValue) return cached.Value;
var permissionChain = await BuildPermissionChain(userId, resourceType, resourceId);
var effective = permissionChain
.OrderByDescending(p => p.IsInherited ? 0 : 1)
.ThenByDescending(p => p.ResourceType)
.ThenByDescending(p => p.Role)
.FirstOrDefault();
var role = effective?.Role ?? PermissionRole.NoAccess;
await _cache.SetAsync(cacheKey, role, TimeSpan.FromMinutes(5));
return role;
}
public async Task<bool> HasPermission(Guid userId, Guid pageId, PermissionRole required)
{
var effective = await GetEffectivePermission(userId, ResourceType.Page, pageId);
return effective >= required;
}
}
Public Share Links
| Feature | Implementation |
|---|---|
| Public URL | UUID-based URL: /share/{uuid} — no guessable paths |
| Password Protection | bcrypt-hashed password stored; verified on access |
| Expiry | expires_at timestamp; enforced at middleware level |
| Download Permission | Toggle to allow/disallow PDF/HTML download |
| View Limits | Optional max view count; link auto-deactivates |
Permission checks happen on every API request and WebSocket message. We solve this with: (1) Redis caching with 5-minute TTL, (2) Permission invalidation events published when permissions change, (3) Bulk permission loading at page-load time for the client sidebar, and (4) Deny-by-default — if no permission entry exists, access is denied (no expensive chain walk needed).
11. Embedding Content — Images, Videos, Code, Databases
Asset Upload Pipeline
C#
public class EmbedBlockProperties
{
public EmbedType EmbedType { get; set; }
public string Url { get; set; }
public string? Title { get; set; }
public string? Description { get; set; }
public string? ThumbnailUrl { get; set; }
public int? Width { get; set; }
public int? Height { get; set; }
public string? OembedHtml { get; set; }
public Dictionary<string, string> Metadata { get; set; } = new();
}
public enum EmbedType
{
Image, Video, Audio, File, CodeBlock, Bookmark,
Database, Figma, GoogleMaps, Miro, Loom, Tweet, CustomIframe
}
public class AssetService
{
private readonly IS3Client _s3;
public async Task<AssetUploadResponse> CreateUploadRequest(
Guid userId, string fileName, string contentType, long fileSize)
{
var allowedTypes = new HashSet<string>
{
"image/jpeg", "image/png", "image/gif", "image/webp", "image/svg+xml",
"video/mp4", "video/webm", "audio/mpeg", "audio/ogg",
"application/pdf", "application/zip"
};
if (!allowedTypes.Contains(contentType))
throw new BadRequestException($"File type {contentType} not allowed");
if (fileSize > 100 * 1024 * 1024)
throw new BadRequestException("File too large (max 100MB)");
var assetId = Guid.NewGuid();
var key = $"assets/{assetId}/{SanitizeFileName(fileName)}";
var presignedUrl = await _s3.GeneratePresignedPutUrl(key, contentType, TimeSpan.FromMinutes(15));
return new AssetUploadResponse
{
AssetId = assetId,
UploadUrl = presignedUrl,
AssetUrl = $"https://cdn.example.com/{key}"
};
}
}
Database Block (Linked Database)
C#
public class DatabaseBlock
{
public Guid Id { get; set; } = Guid.NewGuid();
public string Name { get; set; } = string.Empty;
public List<DatabaseColumn> Columns { get; set; } = new();
public List<DatabaseView> Views { get; set; } = new();
public List<DatabaseRow> Rows { get; set; } = new();
}
public class DatabaseColumn
{
public Guid Id { get; set; }
public string Name { get; set; }
public ColumnType Type { get; set; }
public ColumnConfig Config { get; set; }
public bool IsVisible { get; set; } = true;
public int SortOrder { get; set; }
}
public enum ColumnType
{
Title, Text, Number, Select, MultiSelect,
Date, Person, Checkbox, Url, Email, Phone,
Files, Relation, Rollup, Formula,
CreatedTime, UpdatedTime, CreatedBy, UpdatedBy
}
public class DatabaseView
{
public Guid Id { get; set; }
public string Name { get; set; }
public ViewType Type { get; set; }
public List<FilterRule> Filters { get; set; } = new();
public List<SortRule> Sorts { get; set; } = new();
public List<Guid> VisibleColumns { get; set; } = new();
}
public class DatabaseRow
{
public Guid Id { get; set; }
public Guid PageId { get; set; }
public Dictionary<Guid, object> Cells { get; set; } = new();
public int SortOrder { get; set; }
}
12. Table / Spreadsheet Blocks
Table Block Architecture
Table blocks are lightweight spreadsheet-like grids embedded in pages. Unlike database blocks (which are full structured data stores), table blocks are simpler — they are primarily for display and don't support formulas or relations. They support merge cells, column resize, row/column insert/delete/reorder, cell formatting, column types, and CSV import/export.
C#
public class TableBlockProperties
{
public int Rows { get; set; }
public int Columns { get; set; }
public List<List<TableCell>> Cells { get; set; } = new();
public List<TableColumnConfig> ColumnConfigs { get; set; } = new();
public bool HasHeaderRow { get; set; } = true;
public TableBorderStyle BorderStyle { get; set; } = TableBorderStyle.Solid;
}
public class TableCell
{
public string Content { get; set; } = string.Empty;
public int RowSpan { get; set; } = 1;
public int ColSpan { get; set; } = 1;
public bool IsMerged { get; set; }
public string? BackgroundColor { get; set; }
public TextAlignment Alignment { get; set; } = TextAlignment.Left;
}
public class TableColumnConfig
{
public int Index { get; set; }
public string? HeaderName { get; set; }
public int Width { get; set; } = 150;
public ColumnDataType DataType { get; set; } = ColumnDataType.Text;
public bool IsSortable { get; set; }
}
Use Table Block for simple data display — comparing features, showing specs, organizing text. Use Database Block when you need structured data with filtering, sorting, multiple views (kanban, gallery), relations between databases, or computed columns (formulas, rollups). Database blocks are backed by dedicated storage; table blocks are part of the page's block tree.
13. Kanban Board Views
Kanban Implementation
Kanban boards are a view type for database blocks. Cards represent rows; columns represent a Select or Status column value. Dragging a card between columns updates the underlying row's status field and broadcasts the change to all connected clients via WebSocket.
C#
public class KanbanService
{
private readonly IDbConnection _db;
private readonly IEventBus _eventBus;
public async Task<KanbanBoard> GetKanbanBoard(Guid viewId, Guid userId)
{
var view = await _db.QuerySingleAsync<DatabaseView>(
"SELECT * FROM database_views WHERE id = @Id", new { Id = viewId });
var groupByColumn = await _db.QuerySingleAsync<DatabaseColumn>(
"SELECT * FROM database_columns WHERE id = @Id",
new { Id = view.GroupByColumnId });
var rows = await GetFilteredRows(view);
var columns = new List<KanbanColumn>();
if (groupByColumn.Type == ColumnType.Select)
{
foreach (var option in groupByColumn.Config.Options)
{
var cards = rows.Where(r =>
r.Cells.ContainsKey(groupByColumn.Id) &&
r.Cells[groupByColumn.Id]?.ToString() == option.Label);
columns.Add(new KanbanColumn
{
Name = option.Label,
Color = option.Color,
Cards = cards.Select(MapToCard).ToList(),
CardCount = cards.Count()
});
}
}
return new KanbanBoard { ViewId = viewId, Columns = columns };
}
public async Task MoveCard(Guid cardRowId, string fromStatus, string toStatus, int newPosition)
{
using var tx = _db.BeginTransaction();
await _db.ExecuteAsync(
@"UPDATE database_cells SET value = @NewStatus
WHERE row_id = @RowId AND column_id = @ColumnId",
new { RowId = cardRowId, NewStatus = toStatus, ColumnId = GetGroupByColumnId() }, tx);
await ReorderRows(cardRowId, newPosition, toStatus, tx);
tx.Commit();
await _eventBus.PublishAsync(new CardMovedEvent
{
RowId = cardRowId, FromStatus = fromStatus, ToStatus = toStatus
});
}
}
14. @Mentions & Notifications
Mention System
Users can @mention other users, groups, or pages within any text content. Mentions are parsed from the CRDT content and trigger notifications. The mention system supports autocomplete — when a user types "@", a dropdown appears with searchable users and groups.
C#
public class Mention
{
public Guid Id { get; set; }
public MentionType Type { get; set; }
public Guid TargetId { get; set; }
public string DisplayText { get; set; }
public Guid MentionedBy { get; set; }
public Guid PageId { get; set; }
public string BlockId { get; set; }
public DateTime CreatedAt { get; set; }
}
public enum MentionType { User, Group, Page, Date, Formula }
public class NotificationService
{
private readonly IDbConnection _db;
private readonly IPushNotificationService _push;
private readonly IEmailService _email;
private readonly ISlackService _slack;
public async Task ProcessMention(Mention mention)
{
var notification = new Notification
{
Id = Guid.NewGuid(),
Type = NotificationType.Mention,
TargetUserId = mention.TargetId,
TriggeredByUser = mention.MentionedBy,
PageId = mention.PageId,
Message = "mentioned you in a page",
CreatedAt = DateTime.UtcNow
};
await _db.ExecuteAsync(
@"INSERT INTO notifications (id, type, target_user_id, triggered_by_user,
page_id, message, created_at, is_read)
VALUES (@Id, @Type, @TargetUserId, @TriggeredByUser,
@PageId, @Message, @CreatedAt, false)", notification);
var prefs = await GetUserNotificationPreferences(mention.TargetId);
var tasks = new List<Task>();
if (prefs.InAppEnabled) tasks.Add(SendInAppNotification(notification));
if (prefs.EmailEnabled) tasks.Add(_email.SendMentionEmail(notification));
if (prefs.SlackEnabled) tasks.Add(_slack.SendMentionNotification(notification));
if (prefs.PushEnabled) tasks.Add(_push.SendPushNotification(notification));
await Task.WhenAll(tasks);
}
}
Notification Channels
| Channel | Use Case | Delivery SLA | Implementation |
|---|---|---|---|
| In-App | Real-time bell icon, notification center | Instant (WebSocket) | Redis pub/sub to WebSocket broadcast |
| Daily digest or instant for mentions | < 5 min | SendGrid / SES | |
| Slack | Team notifications in channels | < 1 min | Slack Web API |
| Microsoft Teams | Enterprise team notifications | < 1 min | Teams Bot Framework |
| Push (Mobile) | Critical notifications on mobile | < 2 min | FCM / APNs |
15. Page Export — PDF, Markdown, HTML
Export Architecture
C#
public class ExportService
{
private readonly IS3Client _s3;
private readonly IPuppeteerService _puppeteer;
private readonly IBlockRenderer _renderer;
public async Task<ExportResult> ExportPage(Guid pageId, ExportFormat format, Guid userId)
{
var document = await LoadDocument(pageId);
var html = await _renderer.RenderToHtml(document);
byte[] fileBytes = format switch
{
ExportFormat.Pdf => await _puppeteer.GeneratePdf(html, new PdfOptions
{
Format = PaperFormat.A4,
Margin = new Margin { Top = "20mm", Bottom = "20mm",
Left = "15mm", Right = "15mm" },
DisplayHeaderFooter = true,
HeaderTemplate = $"{document.Title}",
FooterTemplate = "Page "
}),
ExportFormat.Markdown => Encoding.UTF8.GetBytes(
await _renderer.RenderToMarkdown(document)),
ExportFormat.Html => Encoding.UTF8.GetBytes(html),
_ => throw new NotSupportedException($"Format {format} not supported")
};
var key = $"exports/{userId}/{pageId}/{DateTime.UtcNow:yyyyMMddHHmmss}.{format}";
await _s3.PutObjectAsync("exports-bucket", key, fileBytes);
var presignedUrl = await _s3.GeneratePresignedGetUrl("exports-bucket", key,
TimeSpan.FromMinutes(15));
return new ExportResult { DownloadUrl = presignedUrl, FileSize = fileBytes.Length };
}
}
16. Import from Other Tools
Supported Import Sources
| Source | Format | Features Preserved | Limitations |
|---|---|---|---|
| Confluence | XML export | Pages, attachments, hierarchy, macros | Some macros converted to basic blocks |
| Notion | Markdown export | Pages, databases (as tables), images | Database relations lost |
| Google Docs | DOCX export | Text, images, tables, headings | Comments become page comments |
| Markdown | .md files | Full Markdown syntax | Custom HTML blocks simplified |
| HTML | .html files | Web page content | JavaScript stripped |
| Word | .docx files | Text, images, tables, styles | Complex layouts simplified |
C#
public class ImportService
{
private readonly IServiceProvider _services;
public async Task<ImportResult> ImportContent(
ImportSource source, Stream fileStream, Guid userId, Guid targetSpaceId)
{
var handler = source switch
{
ImportSource.ConfluenceXml => _services.GetRequiredService<ConfluenceXmlImporter>(),
ImportSource.Markdown => _services.GetRequiredService<MarkdownImporter>(),
ImportSource.NotionMarkdown => _services.GetRequiredService<NotionImporter>(),
ImportSource.Docx => _services.GetRequiredService<DocxImporter>(),
ImportSource.Html => _services.GetRequiredService<HtmlImporter>(),
_ => throw new NotSupportedException($"Import from {source} not supported")
};
var parsed = await handler.Parse(fileStream);
var result = new ImportResult { TotalPages = parsed.Pages.Count };
foreach (var page in parsed.Pages)
{
var blocks = handler.ConvertToBlocks(page);
var document = new Document
{
Title = page.Title, Blocks = blocks,
SpaceId = targetSpaceId, OwnerId = userId
};
await SaveDocument(document);
result.CreatedPages.Add(document.Id);
foreach (var attachment in page.Attachments)
{
var assetUrl = await UploadAttachment(attachment, document.Id);
ReplaceAttachmentReferences(blocks, attachment.Name, assetUrl);
result.UploadedAttachments++;
}
}
return result;
}
}
17. API for Programmatic Access
REST API Endpoints
| Method | Endpoint | Description |
|---|---|---|
| GET | /api/v1/spaces | List all spaces |
| POST | /api/v1/spaces | Create a new space |
| GET | /api/v1/spaces/:id/pages | List pages in a space |
| POST | /api/v1/spaces/:id/pages | Create a page in a space |
| GET | /api/v1/pages/:id | Get page details + blocks |
| PATCH | /api/v1/pages/:id | Update page metadata |
| DELETE | /api/v1/pages/:id | Delete (soft) a page |
| GET | /api/v1/pages/:id/children | List child pages |
| GET | /api/v1/pages/:id/versions | List page version history |
| POST | /api/v1/pages/:id/versions/:v/restore | Restore a version |
| GET | /api/v1/pages/:id/comments | List page comments |
| POST | /api/v1/pages/:id/comments | Add a comment |
| GET | /api/v1/search?q=... | Full-text search |
| POST | /api/v1/pages/:id/export | Export page (PDF/MD/HTML) |
| POST | /api/v1/import | Import from file |
| GET | /api/v1/templates | List available templates |
| POST | /api/v1/templates/:id/instantiate | Create page from template |
| GET | /api/v1/databases/:id/rows | List database rows |
| GET | /api/v1/audit-log | Query audit log |
| GET | /api/v1/analytics/pages/:id | Get page analytics |
Webhook System
C#
public class Webhook
{
public Guid Id { get; set; }
public string Url { get; set; }
public List<string> Events { get; set; }
public string Secret { get; set; }
public Guid WorkspaceId { get; set; }
public bool IsActive { get; set; }
public int FailureCount { get; set; }
}
public class WebhookPayload
{
public string Event { get; set; }
public DateTime Timestamp { get; set; }
public Guid WorkspaceId { get; set; }
public object Data { get; set; }
public string Signature { get; set; }
}
18. Offline Mode
Offline Architecture
C#
public class OfflineSyncEngine
{
private readonly IndexedDb _db;
private readonly WebSocketClient _ws;
private readonly YDoc _localDoc;
public async Task Sync(Guid documentId)
{
var localState = await _db.GetDocumentState(documentId);
_localDoc.ApplyUpdate(localState);
await _ws.Connect();
var stateVector = _localDoc.StateVector;
await _ws.Send(new SyncStep1Message { StateVector = stateVector });
var serverUpdate = await _ws.Receive<SyncStep2Message>();
_localDoc.ApplyUpdate(serverUpdate.Update);
var localUpdates = _localDoc.EncodeStateAsUpdate(stateVector);
if (localUpdates.Length > 2)
await _ws.Send(new UpdateMessage { Update = localUpdates });
await _db.SaveDocumentState(documentId, _localDoc.StateVector);
_ws.OnUpdate += async (update) =>
{
_localDoc.ApplyUpdate(update);
await _db.SaveDocumentState(documentId, _localDoc.StateVector);
NotifyEditor();
};
}
}
A typical page with 50 blocks and 5,000 words requires ~200KB in CRDT format. Even with 1,000 offline pages, storage is only ~200MB — well within IndexedDB limits. We use a LRU eviction policy that removes pages not accessed in 30 days.
19. Knowledge Graph & Backlinks
Graph Model
The knowledge graph captures relationships between pages: links, mentions, parent-child, and database relations. It is stored in Neo4j and maintained asynchronously via events. When a page is saved, the system extracts all internal links and updates the graph accordingly.
C#
public class KnowledgeGraphService
{
private readonly IDriver _neo4j;
public async Task UpdatePageLinks(Guid pageId, List<PageLink> links)
{
using var session = _neo4j.AsyncSession();
await session.ExecuteWriteTransactionAsync(async tx =>
{
await tx.RunAsync(
@"MATCH (p:Page {id: $PageId})-[r:LINKS_TO|MENTIONS]->()
DELETE r",
new { PageId = pageId.ToString() });
foreach (var link in links)
{
var relType = link.Type == LinkType.Hyperlink ? "LINKS_TO" : "MENTIONS";
await tx.RunAsync(
$@"MATCH (source:Page {{id: $SourceId}})
MATCH (target:Page {{id: $TargetId}})
CREATE (source)-[:{relType} {{blockId: $BlockId, context: $Context}}]->(target)",
new
{
SourceId = pageId.ToString(),
TargetId = link.TargetPageId.ToString(),
BlockId = link.BlockId,
Context = link.Context
});
}
});
}
public async Task<BacklinkResult> GetBacklinks(Guid pageId, int limit = 50)
{
using var session = _neo4j.AsyncSession();
var result = await session.RunAsync(
@"MATCH (source:Page)-[r:LINKS_TO|MENTIONS]->(target:Page {id: $PageId})
RETURN source.id as SourcePageId, source.title as Title,
type(r) as LinkType, r.context as Context
ORDER BY source.updated_at DESC LIMIT $Limit",
new { PageId = pageId.ToString(), Limit = limit });
var backlinks = await result.Select(record => new Backlink
{
PageId = Guid.Parse(record["SourcePageId"].As<string>()),
Title = record["Title"].As<string>(),
LinkType = record["LinkType"].As<string>(),
Context = record["Context"].As<string>()
}).ToListAsync();
return new BacklinkResult { Backlinks = backlinks, TotalCount = backlinks.Count };
}
public async Task<GraphVisualization> GetPageGraph(Guid pageId, int depth = 2)
{
using var session = _neo4j.AsyncSession();
var result = await session.RunAsync(
@"MATCH path = (start:Page {id: $PageId})-[*1..2]-(related:Page)
RETURN DISTINCT
collect(DISTINCT {id: related.id, title: related.title}) as nodes,
collect(DISTINCT {
source: startNode(last(relationships(path))).id,
target: endNode(last(relationships(path))).id,
type: type(last(relationships(path)))
}) as edges",
new { PageId = pageId.ToString() });
var record = await result.SingleAsync();
return MapToGraphVisualization(record);
}
}
20. Table of Contents Auto-Generation
When a page contains heading blocks (H1-H6), a Table of Contents block can be inserted that automatically generates a navigable outline from the headings. The TOC updates in real-time as users add or modify headings. Clicking a TOC entry scrolls the page to the corresponding heading anchor.
C#
public class TableOfContentsService
{
public List<TocEntry> GenerateToc(List<Block> blocks)
{
var entries = new List<TocEntry>();
int counter = 0;
foreach (var block in blocks)
{
if (block.Type == BlockType.TableOfContents) continue;
if (block.Type >= BlockType.Heading1 && block.Type <= BlockType.Heading6)
{
var level = (int)(block.Type - BlockType.Heading1);
var text = ExtractPlainText(block);
entries.Add(new TocEntry
{
Id = $"heading-{counter++}",
Text = text,
Level = level,
BlockId = block.Id,
Anchor = GenerateAnchor(text)
});
}
}
return entries;
}
private string GenerateAnchor(string text)
{
return text.ToLower().Replace(" ", "-")
.Replace("[^\\w-]", "").Replace("-+", "-");
}
}
21. Analytics — Page Views & Popular Pages
Analytics Pipeline
C#
public class AnalyticsService
{
private readonly ClickHouseConnection _clickhouse;
private readonly IDistributedCache _cache;
public async Task TrackPageView(Guid pageId, Guid? userId, PageViewMetadata metadata)
{
var evt = new PageViewEvent
{
PageId = pageId, UserId = userId, Timestamp = DateTime.UtcNow,
UserAgent = metadata.UserAgent, Device = metadata.Device
};
_buffer.Add(evt);
var today = DateTime.UtcNow.ToString("yyyy-MM-dd");
await _cache.IncrementAsync($"pageviews:{pageId}:{today}");
}
public async Task<PageAnalytics> GetPageAnalytics(Guid pageId, DateRange range)
{
var result = await _clickhouse.QueryAsync<PageAnalyticsRow>(
@"SELECT
toDate(timestamp) as Date,
uniqExact(user_id) as UniqueViewers,
count() as TotalViews,
avg(duration_ms) as AvgDurationMs
FROM page_views
WHERE page_id = @PageId
AND timestamp BETWEEN @Start AND @End
GROUP BY Date ORDER BY Date",
new { PageId = pageId, Start = range.Start, End = range.End });
return new PageAnalytics
{
DailyViews = result.ToList(),
TotalViews = result.Sum(r => r.TotalViews)
};
}
public async Task<List<PopularPage>> GetPopularPages(Guid spaceId, int limit = 10)
{
return (await _clickhouse.QueryAsync<PopularPage>(
@"SELECT page_id, title, sum(views) as TotalViews,
uniqExact(user_id) as UniqueViewers
FROM page_views pv JOIN pages p ON pv.page_id = p.id
WHERE p.space_id = @SpaceId AND pv.timestamp >= now() - INTERVAL 30 DAY
GROUP BY page_id, title ORDER BY TotalViews DESC LIMIT @Limit",
new { SpaceId = spaceId, Limit = limit })).ToList();
}
}
Analytics Dashboard Metrics
| Metric | Time Range | Use Case |
|---|---|---|
| Page views (total, unique) | Day, Week, Month | Content popularity |
| Top pages in space | 7d, 30d, 90d | Content strategy |
| Most active editors | Week, Month | Contributor recognition |
| Search queries with no results | Week | Content gap identification |
| Stale pages (not updated in 90d) | Rolling | Content freshness |
22. Audit Log
The audit log captures every significant action in the system for compliance and forensic analysis. Events are immutable and stored in an append-only table. Every event includes the user, action, resource, metadata (before/after values), IP address, and timestamp.
C#
public class AuditEvent
{
public Guid Id { get; set; } = Guid.NewGuid();
public Guid WorkspaceId { get; set; }
public Guid? UserId { get; set; }
public string? UserEmail { get; set; }
public AuditAction Action { get; set; }
public ResourceType ResourceType { get; set; }
public Guid ResourceId { get; set; }
public string ResourceName { get; set; }
public Dictionary<string, object> Metadata { get; set; } = new();
public string? IpAddress { get; set; }
public DateTime Timestamp { get; set; } = DateTime.UtcNow;
}
public enum AuditAction
{
PageCreated, PageUpdated, PageDeleted, PageRestored,
PagePublished, PageArchived, PageMoved, PageExported,
PermissionGranted, PermissionRevoked, PermissionChanged,
SpaceCreated, SpaceUpdated, SpaceArchived,
UserJoined, UserRemoved, RoleChanged,
LoginSuccess, LoginFailed, PasswordChanged,
ApiKeyCreated, ApiKeyRevoked,
WorkspaceSettingsChanged, DataExported, DataDeleted
}
public class AuditService
{
private readonly IDbConnection _db;
public async Task Log(AuditEvent auditEvent)
{
await _db.ExecuteAsync(
@"INSERT INTO audit_log (id, workspace_id, user_id, action,
resource_type, resource_id, metadata, ip_address, timestamp)
VALUES (@Id, @WorkspaceId, @UserId, @Action,
@ResourceType, @ResourceId, @Metadata, @IpAddress, @Timestamp)",
auditEvent);
}
public async Task<AuditLogResult> QueryAuditLog(Guid workspaceId, AuditLogQuery query)
{
var sql = @"SELECT * FROM audit_log
WHERE workspace_id = @WorkspaceId
AND timestamp BETWEEN @Start AND @End";
if (query.Actions?.Any() == true) sql += " AND action = ANY(@Actions)";
if (query.UserId.HasValue) sql += " AND user_id = @UserId";
sql += " ORDER BY timestamp DESC LIMIT @Limit OFFSET @Offset";
var events = await _db.QueryAsync<AuditEvent>(sql, query);
return new AuditLogResult { Events = events.ToList() };
}
}
23. Integrations — Slack, Jira & More
Integration Architecture
Slack Integration
C#
public class SlackIntegration
{
private readonly ISlackClient _slack;
private readonly ISearchService _search;
public async Task HandleSlashCommand(SlashCommand command)
{
switch (command.Command)
{
case "/wiki search":
var results = await _search.SearchAsync(
command.Text, command.UserId, new SearchFilters { PageSize = 5 });
var blocks = results.Items.Select(r => new SlackBlock
{
Type = "section",
Text = new SlackText
{
Type = "mrkdwn",
Text = $"*{r.Title}*\n{r.Excerpt}\n<{r.Url}|View Page>"
}
}).ToList();
await _slack.SendResponse(command.ResponseUrl, new SlackResponse
{
ResponseType = "ephemeral", Blocks = blocks
});
break;
case "/wiki create":
var page = await CreatePageFromSlack(command.Text, command.UserId);
await _slack.SendEphemeral(command.ResponseUrl,
$"Page created: {page.Url}");
break;
}
}
}
Jira Integration
| Feature | Description |
|---|---|
| Issue Embeds | Paste Jira URL to auto-embed issue card with status, assignee, priority |
| Bidirectional Sync | Update Jira issue status from wiki page, or vice versa via webhook |
| Sprint Reports | Auto-generate sprint retrospective pages from Jira sprint data |
| Link Issues to Pages | Associate wiki pages with Jira issues; show links in both systems |
24. Monitoring & Observability
Monitoring Stack
| Layer | Tool | Metric |
|---|---|---|
| Infrastructure | Prometheus + Grafana | CPU, memory, disk, network |
| Application | OpenTelemetry | Request latency, error rate, throughput |
| Distributed Tracing | Jaeger | Request flow across services |
| Log Aggregation | ELK Stack | Structured logs with correlation IDs |
| Error Tracking | Sentry | Exception grouping, breadcrumbs |
| Uptime | Checkly | HTTP checks from multiple regions |
| Real-Time Collab | Custom dashboard | Connected users, CRDT merge latency |
Key SLIs/SLOs
YAML
service_level_objectives:
- name: api_availability
sli: successful_requests / total_requests
target: 99.95%
window: 30d
- name: api_latency_p95
sli: histogram_bucket{le="0.5"} / total_requests
target: 99%
window: 30d
- name: search_latency_p95
sli: elasticsearch_request_duration{le="0.2"} / total_searches
target: 99%
window: 30d
- name: collab_sync_latency
sli: crdt_merge_duration{le="0.5"} / total_merges
target: 99.5%
window: 7d
25. Security
Security Architecture
Security Measures
| Layer | Measure |
|---|---|
| Transport | TLS 1.3 enforced; HSTS with 1-year max-age; certificate pinning for mobile |
| Authentication | JWT with short-lived access tokens (15 min) + refresh tokens (30 days); MFA via TOTP/SMS |
| Session Management | Redis-backed sessions with configurable timeout; concurrent session limits |
| Data at Rest | AES-256-GCM encryption for database fields; S3 SSE-KMS for assets |
| Secrets | AWS Secrets Manager / HashiCorp Vault; no secrets in code or env vars |
| Input Validation | Server-side schema validation for all inputs; HTML sanitization via DOMPurify |
| XSS Prevention | CSP headers; DOMPurify for user content; no inline scripts |
| CSRF Prevention | SameSite cookies; CSRF tokens for state-changing requests |
| Rate Limiting | Per-user and per-IP rate limits; exponential backoff for repeated failures |
| Vulnerability Scanning | Snyk for dependencies; Trivy for containers; weekly SAST |
| Penetration Testing | Annual third-party pentest; bug bounty program via HackerOne |
Content Security Service
C#
public class ContentSecurityService
{
public string SanitizeUserContent(string html)
{
var sanitizer = new HtmlSanitizer(new SanitizerOptions
{
AllowedTags = new HashSet<string>
{
"p", "br", "strong", "em", "u", "s", "a", "code", "pre",
"h1", "h2", "h3", "h4", "h5", "h6",
"ul", "ol", "li", "blockquote",
"table", "thead", "tbody", "tr", "th", "td",
"img", "video", "audio", "source", "iframe",
"div", "span", "hr", "sup", "sub"
},
AllowedAttributes = new Dictionary<string, HashSet<string>>
{
["a"] = new() { "href", "title", "target", "rel" },
["img"] = new() { "src", "alt", "width", "height" },
["iframe"] = new() { "src", "width", "height", "frameborder" },
["code"] = new() { "class" },
["span"] = new() { "class", "style", "data-type", "data-id" }
},
AllowedSchemes = new HashSet<string> { "https", "mailto", "tel" }
});
return sanitizer.Sanitize(html);
}
}
26. Compliance — SOC 2, GDPR
SOC 2 Type II
| TCS Principle | Implementation |
|---|---|
| Security | Encryption at rest/transit, RBAC, MFA, WAF, intrusion detection |
| Availability | 99.95% uptime SLA, multi-AZ deployment, auto-scaling, failover |
| Processing Integrity | CRDT merge correctness, idempotent operations, data validation |
| Confidentiality | AES-256 encryption, key rotation, data classification, access logging |
| Privacy | Data minimization, consent management, data retention policies |
GDPR Compliance
| GDPR Right | Implementation |
|---|---|
| Right of Access (Art. 15) | Self-serve data export: all user data downloadable as JSON/CSV |
| Right to Erasure (Art. 17) | Delete Account flow: anonymizes content, removes PII, 30-day grace period |
| Data Portability (Art. 20) | Full export in machine-readable format (JSON with block structure) |
| Consent (Art. 7) | Cookie consent banner; granular consent for analytics and third-party sharing |
| Data Processing Agreement | Standard DPA with sub-processors; SCCs for international transfers |
| Breach Notification | 72-hour notification process; automated detection + incident response playbook |
C#
public class GdprService
{
private readonly IDbConnection _db;
public async Task<DataExportResult> ExportUserData(Guid userId)
{
var user = await _db.QuerySingleAsync<User>(
"SELECT * FROM users WHERE id = @Id", new { Id = userId });
var pages = await _db.QueryAsync<Document>(
"SELECT * FROM pages WHERE owner_id = @Id OR created_by = @Id",
new { Id = userId });
var comments = await _db.QueryAsync<Comment>(
"SELECT * FROM comments WHERE created_by = @Id", new { Id = userId });
var export = new UserDataExport
{
User = new UserExport { Email = user.Email, Name = user.Name },
Pages = pages.Select(p => new PageExport { Id = p.Id, Title = p.Title }).ToList(),
Comments = comments.Select(c => new CommentExport { Id = c.Id, Content = c.Content }).ToList(),
ExportedAt = DateTime.UtcNow
};
var json = JsonSerializer.Serialize(export, new JsonSerializerOptions { WriteIndented = true });
var key = $"gdpr-exports/{userId}/{DateTime.UtcNow:yyyyMMdd}.json";
return new DataExportResult { DownloadUrl = await GetPresignedUrl(key) };
}
public async Task AnonymizeUserData(Guid userId)
{
using var tx = _db.BeginTransaction();
await _db.ExecuteAsync(
@"UPDATE users SET name = 'Deleted User',
email = CONCAT('deleted_', @UserId, '@anonymized.local'),
avatar_url = NULL, is_deleted = true, deleted_at = NOW()
WHERE id = @UserId", new { UserId = userId }, tx);
await _db.ExecuteAsync(
@"UPDATE pages SET owner_id = '00000000-0000-0000-0000-000000000001'
WHERE owner_id = @UserId AND is_deleted = false",
new { UserId = userId }, tx);
await _db.ExecuteAsync(
@"UPDATE comments SET content = '[Removed for privacy]',
created_by = '00000000-0000-0000-0000-000000000001'
WHERE created_by = @UserId", new { UserId = userId }, tx);
tx.Commit();
}
}
27. Cost Estimation
Infrastructure Cost Breakdown (Monthly, 100K Users, 10M Pages)
| Service | Specification | Monthly Cost |
|---|---|---|
| Application Servers (ECS/EKS) | 8 x c6i.xlarge (4 vCPU, 8GB) | $1,400 |
| WebSocket Servers | 4 x c6i.large (2 vCPU, 4GB) | $500 |
| PostgreSQL (RDS) | db.r6g.2xlarge Multi-AZ + Read Replicas | $2,800 |
| Redis (ElastiCache) | cache.r6g.xlarge cluster (3 nodes) | $1,200 |
| Elasticsearch | 3 x m6g.xlarge.search (64GB index) | $2,100 |
| Neo4j (Graph DB) | 2 x db.r6g.xlarge | $1,800 |
| ClickHouse (Analytics) | Self-managed 3-node cluster | $900 |
| S3 (Assets + Snapshots) | 5TB storage + transfers | $250 |
| CloudFront (CDN) | 10TB monthly transfer | $850 |
| RabbitMQ (MQ) | Amazon MQ, 2-node cluster | $350 |
| Lambda (Export) | 100K invocations/month | $50 |
| Monitoring (Grafana Cloud) | Logs + metrics + traces | $500 |
| Email (SES) | 500K emails/month | $200 |
| WAF + DNS + Certs | AWS WAF + Route 53 + ACM | $120 |
| Total Infrastructure | ~$13,070 |
Cost Optimization Strategies
- Reserved Instances: 1-year RI for databases saves 30-40% (~$3,000/month)
- S3 Intelligent-Tiering: Automatically moves infrequently accessed snapshots to cheaper storage
- Spot Instances: Export Lambda and background workers use spot for 60% savings
- CRDT Compression: Compress Yjs updates with zstd before storage (90%+ ratio)
- Search Sharding: Index only recently active pages in hot tier; cold pages in cheaper storage
- Connection Pooling: PgBouncer for PostgreSQL reduces RDS connection overhead
At $13K/month infrastructure cost serving 100K users, the cost per user is ~$0.13/month. With SaaS pricing of $10/user/month (Team plan), gross margin is ~98%. At early stage (1K users), infrastructure costs ~$3K/month, requiring ~50 paying users at $10/month to break even.
28. API Design (Detailed)
Authentication
HTTP
POST /api/v1/auth/login
Content-Type: application/json
{
"email": "user@example.com",
"password": "secure_password",
"mfa_code": "123456"
}
# Response
{
"access_token": "eyJhbGciOiJSUzI1NiIs...",
"refresh_token": "dGhpcyBpcyBhIHJlZnJl...",
"expires_in": 900,
"token_type": "Bearer",
"user": {
"id": "550e8400-e29b-41d4-a716-446655440000",
"email": "user@example.com",
"name": "Jane Doe"
}
}
# Subsequent requests
GET /api/v1/pages/550e8400-e29b-41d4-a716-446655440000
Authorization: Bearer eyJhbGciOiJSUzI1NiIs...
Create Page
HTTP
POST /api/v1/spaces/660e8400-e29b-41d4-a716-446655440001/pages
Authorization: Bearer {token}
Content-Type: application/json
{
"title": "System Architecture Overview",
"icon": "🏗",
"parent_page_id": null,
"template_id": "770e8400-e29b-41d4-a716-446655440002",
"initial_blocks": [
{"type": "heading1", "content": "System Architecture"},
{"type": "paragraph", "content": "This document describes..."},
{"type": "code", "properties": {"language": "csharp", "code": "public class Service { }"}}
],
"properties": {"Status": "Draft", "Owner": "Jane Doe"}
}
# Response 201 Created
{
"id": "880e8400-e29b-41d4-a716-446655440003",
"title": "System Architecture Overview",
"url": "https://wiki.example.com/spaces/engineering/system-architecture-overview",
"version": 1,
"created_at": "2025-01-10T14:30:00Z",
"collaboration_url": "wss://wiki.example.com/ws/doc/880e8400..."
}
GraphQL Schema
GraphQL
type Query {
space(id: ID!): Space
spaces(first: Int, after: String): SpaceConnection!
page(id: ID!): Page
search(query: String!, filters: SearchFilters): SearchResult!
myRecentPages(first: Int): [Page!]!
}
type Mutation {
createPage(input: CreatePageInput!): Page!
updatePage(id: ID!, input: UpdatePageInput!): Page!
deletePage(id: ID!): Boolean!
movePage(id: ID!, targetSpaceId: ID, parentId: ID): Page!
addComment(input: AddCommentInput!): Comment!
exportPage(id: ID!, format: ExportFormat!): ExportResult!
}
type Subscription {
pageUpdated(pageId: ID!): PageUpdate!
commentAdded(pageId: ID!): Comment!
mentionReceived: Mention!
}
type Page {
id: ID!
title: String!
space: Space!
parent: Page
children(first: Int): PageConnection!
blocks: [Block!]!
comments(first: Int): CommentConnection!
backlinks(first: Int): [Backlink!]!
versions(first: Int): [PageVersion!]!
analytics: PageAnalytics!
createdAt: DateTime!
updatedAt: DateTime!
}
29. Testing Strategy
Test Pyramid
Test Categories
| Category | Scope | Tools | Count |
|---|---|---|---|
| Unit Tests | Block rendering, CRDT ops, permission logic, search indexing | xUnit, Moq, FluentAssertions | 1,500+ |
| Integration Tests | API endpoints, database operations, Elasticsearch queries | xUnit, TestContainers | 200 |
| CRDT Convergence | Verify concurrent edits merge correctly | xUnit, Yjs test utils | 100 |
| E2E Tests | User flows: create, edit, share, search, export | Playwright | 50 |
| Load Tests | 1K concurrent editors, 10K search QPS | k6, Locust | 20 |
| Security Tests | OWASP Top 10, permission bypass | OWASP ZAP | 50 |
CRDT Convergence Test
C#
public class CrdtConvergenceTests
{
[Fact]
public async Task Concurrent_Edits_Merge_Correctly()
{
// Arrange: Create three separate Yjs documents
var doc1 = new YDoc();
var doc2 = new YDoc();
var doc3 = new YDoc();
var text1 = doc1.GetText("blocks");
var text2 = doc2.GetText("blocks");
var text3 = doc3.GetText("blocks");
// Act: Each user makes independent edits
text1.Insert(0, "Hello ");
text2.Insert(0, "World ");
text3.Insert(0, "From ");
// Sync doc1 and doc2
var update12 = doc1.EncodeStateAsUpdate(doc2.StateVector);
doc2.ApplyUpdate(update12);
var update21 = doc2.EncodeStateAsUpdate(doc1.StateVector);
doc1.ApplyUpdate(update21);
// Sync with doc3
var update13 = doc1.EncodeStateAsUpdate(doc3.StateVector);
doc3.ApplyUpdate(update13);
var update31 = doc3.EncodeStateAsUpdate(doc1.StateVector);
doc1.ApplyUpdate(update31);
// Assert: All documents converge to the same state
var final1 = text1.ToString();
var final2 = text2.ToString();
var final3 = text3.ToString();
Assert.Equal(final1, final2);
Assert.Equal(final2, final3);
Assert.Contains("Hello", final1);
Assert.Contains("World", final1);
Assert.Contains("From", final1);
}
[Theory]
[InlineData(5, 10)] // 5 users, 10 edits each
[InlineData(10, 50)] // 10 users, 50 edits each
[InlineData(20, 100)] // 20 users, 100 edits each
public async Task Stress_Test_Convergence(int userCount, int editsPerUser)
{
var documents = Enumerable.Range(0, userCount)
.Select(_ => new YDoc())
.ToList();
// Simulate concurrent edits
var tasks = documents.Select((doc, i) => Task.Run(() =>
{
var text = doc.GetText("blocks");
for (int j = 0; j < editsPerUser; j++)
{
text.Insert(j * 5, $"U{i}E{j} ");
}
}));
await Task.WhenAll(tasks);
// Sync all documents pairwise
for (int i = 0; i < documents.Count; i++)
{
for (int j = i + 1; j < documents.Count; j++)
{
var update = documents[i].EncodeStateAsUpdate(documents[j].StateVector);
documents[j].ApplyUpdate(update);
var reverseUpdate = documents[j].EncodeStateAsUpdate(documents[i].StateVector);
documents[i].ApplyUpdate(reverseUpdate);
}
}
// Verify all documents converge
var reference = documents[0].GetText("blocks").ToString();
foreach (var doc in documents.Skip(1))
{
Assert.Equal(reference, doc.GetText("blocks").ToString());
}
}
}
Tests run on every PR: unit tests (<2 min), integration tests (<8 min), CRDT convergence tests (<3 min). E2E tests run nightly and on release branches. Load tests run weekly against a staging environment. Security scans run on every merge to main. Total CI time for a typical PR is under 15 minutes.
30. Interview Q&A
A: We use Yjs CRDTs for conflict-free merging. Each character insertion/deletion is a CRDT operation with a unique timestamp and client ID. When two users type in the same paragraph, the operations are automatically merged based on their Lamport timestamps — both users' edits appear in a deterministic, consistent order. The WebSocket server broadcasts operations to all connected clients, who merge them locally. If a user is offline, their operations are buffered locally and merged when they reconnect.
A: We use a two-pronged approach. First, the visible_to field in Elasticsearch stores the list of user/group IDs that can access each page. This is updated whenever permissions change (via async event). At search time, we add a terms filter on visible_to containing the requesting user's ID. Second, we cache permission sets in Redis with a 5-minute TTL, and use Elasticsearch's built-in filter context (which is cached) for fast repeated queries. For very large tenants, we use a permission inheritance bitmap — a bitfield where each bit represents access to a space/page, enabling O(1) permission checks.
A: Every element is a block with a uniform schema: {id, type, content, properties, children}. Simple text blocks (paragraphs, headings) use the content field with inline segments. Complex blocks (tables, databases, embeds) store their data in the properties dictionary as structured JSON. The editor framework (Blocknote/ProseMirror) renders each block type with a custom React component. This uniformity means CRDT collaboration, versioning, permissions, and API access all work identically regardless of block type.
A: Yjs CRDTs solve this naturally. When offline, edits are applied to a local Yjs document stored in IndexedDB. When the user reconnects, the sync engine exchanges state vectors with the server — the server sends all updates the client missed, and the client sends all updates the server missed. Yjs merges everything deterministically. There are no "conflicts" in the traditional sense — all operations from all users are merged into a consistent state. The only edge case is structural conflicts (e.g., two users move the same block to different parents), which we handle by keeping the last-write-wins for metadata while preserving both content changes.
A: Three scaling strategies: (1) Horizontal scaling of WebSocket servers — use sticky sessions or a Redis-backed pub/sub to route messages. Each WebSocket server handles ~10K connections. With 100 servers, we handle 1M concurrent connections. (2) Document-level sharding — route each document to a specific collaboration server based on document ID hash. This ensures all editors of the same document connect to the same server, minimizing cross-server message passing. (3) Cold page management — pages not edited in 24+ hours are evicted from the collaboration server's memory. They are loaded on-demand when someone opens them. This means only ~1% of pages (100K) need to be in-memory at any time.
A: The knowledge graph is updated asynchronously via events. When a page is saved, a PageLinksUpdated event is published containing the page's current set of internal links. The graph service consumes this event, removes all outgoing edges from the page, and creates new edges based on the current links. When a page is deleted, a PageDeleted event triggers removal of all edges involving that page. Since graph updates are eventually consistent (typically <1 second lag), the graph may briefly show stale backlinks — but this is acceptable for the use case (graph visualization and backlink discovery).
A: Each resource (workspace, space, folder, page) has an optional explicit permission entry. At check time, we walk up the hierarchy from page to workspace, collecting all permission entries for the user. The most specific non-inherited permission wins (page overrides folder, folder overrides space). We cache the effective permission in Redis with a 5-minute TTL, invalidated immediately when any permission in the chain changes. For bulk operations (e.g., loading the sidebar tree), we pre-load all permissions for the user's spaces in a single query and build an in-memory permission map.
9. Commenting & Annotations
Comment System Design
Comments are anchored to specific blocks or text ranges within blocks. They support threads (replies), reactions (emoji), resolution, and @mentions. When a user highlights text and clicks "Comment," the selected text range is stored as an anchor, enabling other users to see exactly which text the comment refers to.
Comment Notification Flow