Building advanced workflow automation, CRM automation, and multi-channel customer journeys at 150K+ customer scale
1. Introduction — What is ActiveCampaign and Why It Matters
ActiveCampaign is a comprehensive customer experience automation (CXA) platform that unifies marketing automation, email marketing, CRM, and sales automation into a single cohesive product. Founded in 2003 as an email marketing tool, it evolved into one of the most powerful marketing automation platforms serving over 150,000 customers across 170 countries. With plans starting at $49 per month and enterprise solutions scaling well into five-figure annual contracts, ActiveCampaign occupies a critical mid-market position between simple email tools like Mailchimp and enterprise platforms like HubSpot and Salesforce Marketing Cloud.
The platform's core value proposition lies in its ability to automate complex customer journeys across multiple channels — email, SMS, site messaging, social media, and CRM touchpoints — all without requiring engineering resources. A small e-commerce business can set up an abandoned cart recovery sequence, while a B2B SaaS company can orchestrate a multi-touch lead nurturing pipeline with lead scoring, deal stage automation, and sales rep assignment. This flexibility is what drives ActiveCampaign's impressive retention metrics and net revenue retention above 120%.
From a system design perspective, ActiveCampaign presents fascinating engineering challenges. The platform must process millions of events per day from site tracking scripts, evaluate complex branching automation workflows in real-time, send billions of emails annually with strict deliverability requirements, and maintain a CRM that rivals dedicated sales tools. The visual automation builder alone requires a sophisticated graph execution engine that can traverse thousands of connected nodes while maintaining idempotency and handling failure gracefully. Building such a system at scale requires deep expertise in distributed systems, event-driven architecture, state machines, and data-intensive application design.
In this comprehensive system design guide, we will deconstruct ActiveCampaign's architecture from the ground up. We will model the data layer supporting contacts, deals, automations, and campaign metadata. We will design the automation workflow engine as a distributed state machine capable of evaluating millions of active workflows concurrently. We will architect the email delivery pipeline ensuring high throughput and deliverability. We will explore the site tracking infrastructure processing billions of page-view events. We will also cover the CRM automation layer, lead scoring algorithms, and the integration platform connecting to over 900 third-party services. Each section includes architecture diagrams, data models, code implementations in C#, and capacity planning calculations suitable for senior and staff-level system design interviews.
Whether you are preparing for a system design interview at a martech company, building your own marketing automation product, or simply curious about how platforms like ActiveCampaign operate at scale, this guide provides the depth and breadth you need. We will reference real-world patterns from distributed systems literature, apply them to the specific domain of marketing automation, and provide actionable design decisions backed by concrete numbers. The goal is not merely to describe what ActiveCampaign does, but to give you the engineering knowledge to build such a system yourself.
2. Functional and Non-Functional Requirements
Before designing any system, we must clearly articulate what it needs to do and how well it needs to perform. For a marketing automation platform like ActiveCampaign, the requirements span five major functional domains: workflow automation, CRM, multi-channel communication, analytics, and integrations. Each domain contains dozens of features, but we will focus on the capabilities that define the system's architectural complexity and scaling characteristics.
Functional Requirements
Visual Automation Builder: Users must be able to create multi-step automation workflows through a drag-and-drop interface. The builder supports node types including email sends, SMS messages, wait/delay steps, conditional branching (if/else), goal tracking, A/B split testing, and external webhook calls. Workflows can have triggers (entry points) and actions (steps within the workflow). The builder must serialize the visual graph into a machine-executable format and validate it before activation.
CRM and Deal Management: The system maintains a full CRM with contacts, companies, deals, and activities. Deals move through customizable pipeline stages. Automation can create deals, update deal properties, assign tasks to sales representatives, and trigger notifications based on deal changes. The CRM must support custom fields, deal scoring, and a complete activity timeline for each contact.
Lead Scoring: The platform assigns numerical scores to contacts based on behavioral signals (email opens, website visits, form submissions) and demographic attributes (job title, company size, industry). Scoring must update in near-real-time as new events arrive. Predictive scoring uses machine learning models to predict conversion probability based on historical patterns across the customer base.
Multi-Channel Messaging: The system sends emails, SMS messages, and site-triggered messages (pop-ups, chat widgets). Email campaigns support templates, dynamic content blocks, personalization tokens, and conditional content based on contact properties. SMS automation supports shortcodes, two-way messaging, and compliance with carrier regulations. Each channel has its own delivery pipeline with provider integrations and deliverability monitoring.
Segmentation: Users create dynamic segments by combining conditions on contact properties, behavioral events, deal attributes, and automation membership. Segments update in real-time as contacts match or unmatch criteria. Segments can be used as automation triggers, email recipients, or exclusion lists.
Site and Event Tracking: A JavaScript tracking snippet installed on customer websites captures page views, form submissions, custom events, and e-commerce transactions. The system must identify anonymous visitors, merge anonymous profiles with known contacts, and feed events into the automation engine in near-real-time.
Non-Functional Requirements
| Requirement | Target | Rationale |
|---|---|---|
| Availability | 99.95% uptime | Direct revenue impact — automations run 24/7 |
| Email Delivery Latency | <30 seconds from trigger | Time-sensitive automations (abandoned carts) |
| Automation Evaluation | <5 seconds per event | Near-real-time workflow progression |
| API Response Time | p99 <200ms | Responsive UI and integration performance |
| Data Durability | Zero data loss | Contact history and automation state are critical |
| Scale | 150K+ accounts, 10B+ contacts | Multi-tenant SaaS platform |
| Throughput | 500K emails/minute peak | Campaign blasts and automation volume |
| Security | SOC 2 Type II, GDPR | Enterprise compliance requirements |
The non-functional requirements drive many architectural decisions. The 30-second email delivery SLA means we cannot batch events for more than a few seconds before evaluating automations. The 99.95% availability target requires active-active multi-region deployment with automatic failover. The need for zero data loss mandates synchronous replication for critical write paths and durable event storage. These constraints shape every component we design in subsequent sections.
3. Capacity Estimation and Back-of-Envelope Math
Capacity estimation grounds our architecture in reality. Before choosing databases, message queues, or deployment topologies, we need concrete numbers about the volume of data flowing through the system. ActiveCampaign serves approximately 150,000 paying customers managing over 10 billion contacts collectively. Let us estimate the per-second and per-day volumes for each major workload.
Contact and Account Scale
| Metric | Daily Volume | Per Second (avg) | Peak (10x) |
|---|---|---|---|
| Total accounts | 150,000 | — | — |
| Active contacts (platform-wide) | 10 billion | — | — |
| Avg contacts per account | ~66,000 | — | — |
| Contact API reads | 500M | ~5,800/s | ~58,000/s |
| Contact API writes | 200M | ~2,300/s | ~23,000/s |
| Contact property updates | 1B | ~11,500/s | ~115,000/s |
Email and Messaging Volume
ActiveCampaign sends billions of emails per year. Assuming 8 billion emails annually across all customers, that translates to roughly 22 million emails per day on average, with peak days (Black Friday, Cyber Monday) reaching 100 million or more. For SMS, the volume is lower but growing rapidly — approximately 200 million SMS messages per year across the platform.
| Metric | Daily Volume | Per Second (avg) | Peak (20x for campaigns) |
|---|---|---|---|
| Emails sent | 22M | ~255/s | ~5,100/s |
| SMS messages sent | 550K | ~6.4/s | ~128/s |
| Email opens tracked | 6M | ~70/s | ~700/s |
| Email clicks tracked | 1.5M | ~17/s | ~170/s |
| Bounce processing | 440K | ~5/s | ~50/s |
Automation and Event Processing
Automation is the most compute-intensive workload. With millions of active automations processing triggers continuously, the event processing pipeline must handle enormous throughput. Each incoming event (form submission, page visit, tag change, email interaction) may trigger evaluation across multiple automations for the same contact.
| Metric | Daily Volume | Per Second (avg) | Peak |
|---|---|---|---|
| Events processed (all types) | 5B | ~58,000/s | ~580,000/s |
| Automation evaluations triggered | 2B | ~23,000/s | ~230,000/s |
| Active automation instances | 500M | — | — |
| Workflow node executions | 8B | ~93,000/s | ~930,000/s |
| Wait/delay wakeups | 500M | ~5,800/s | ~58,000/s |
| Site tracking events | 3B | ~35,000/s | ~350,000/s |
Storage and Bandwidth
The platform stores approximately 500TB of contact data, automation state, email content, and event history. The event store alone grows by roughly 2TB per day. API bandwidth averages 10 Gbps with peaks of 50 Gbps during major campaign events. The site tracking script receives 100 billion impressions per month, requiring a globally distributed CDN with edge processing capabilities.
These numbers inform our infrastructure choices: we need a horizontally scalable event processing system capable of handling 580K events per second at peak, a workflow execution engine that can evaluate millions of automations concurrently, and a storage layer that can ingest 2TB of new data daily while supporting low-latency queries across 10 billion contacts.
4. Core Data Model Design
The data model is the foundation upon which every feature is built. A marketing automation platform requires an entity-relationship model that supports contacts, their properties, their interactions with campaigns, their movement through automation workflows, and their progression through sales pipelines. The design must balance normalization (for data integrity) with denormalization (for query performance) across multiple access patterns.
Primary Entities
| Entity | Description | Key Fields | Volume |
|---|---|---|---|
| Contact | Individual person in the system | id, account_id, email, name, properties (JSON), score, created_at | 10B+ |
| Account (Company) | Business entity | id, name, domain, industry, employee_count | 150K+ |
| Deal | Sales opportunity | id, contact_id, pipeline_id, stage, value, owner_id | 500M+ |
| Automation | Workflow definition | id, account_id, name, graph (JSON), status, trigger_config | 10M+ |
| AutomationInstance | Contact's position in automation | id, automation_id, contact_id, current_node, state, started_at | 500M+ |
| Campaign | Email/SMS campaign | id, account_id, type, subject, content, status, scheduled_at | 50M+ |
| Event | Raw interaction event | id, contact_id, type, properties, timestamp | Billions |
| Tag | Label for contact categorization | id, account_id, name | 50M+ |
| Activity | CRM activity log entry | id, contact_id, deal_id, type, description, created_at | Billions |
| Segment | Dynamic contact filter | id, account_id, name, conditions (JSON), contact_count | 5M+ |
Contact Property Model
Contact properties in ActiveCampaign are fully customizable — each account can define its own fields beyond the standard email, name, and phone. This requires a flexible schema. We model contact properties as a JSON column on the contacts table for frequently queried properties (email, first_name), with a separate contact_properties table for custom fields that enables indexed queries on user-defined attributes. The JSON approach allows unlimited extensibility without schema migrations, while the relational approach enables efficient segment queries.
public class Contact
{
public Guid Id { get; set; }
public Guid AccountId { get; set; }
public string Email { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public string PhoneNumber { get; set; }
public int Score { get; set; }
public int PredictiveScore { get; set; }
public ContactStatus Status { get; set; }
public Dictionary<string, object> CustomProperties { get; set; }
public List<string> Tags { get; set; }
public DateTime CreatedAt { get; set; }
public DateTime UpdatedAt { get; set; }
public DateTime? LastActivityAt { get; set; }
}
public enum ContactStatus
{
Subscribed,
Unsubscribed,
Bounced,
Complaint,
Deleted
}Automation Graph Model
An automation is stored as a directed acyclic graph (DAG) with nodes representing actions and edges representing the flow between them. Each node has a type (email, wait, condition, goal), configuration specific to that type, and connections to successor nodes. Branching conditions store their evaluation rules as JSON predicates. The entire graph is serialized as a JSON blob within the automation record, with node-level indexing for efficient partial updates when users modify individual steps in the builder.
The automation instance (per-contact state) tracks which node each contact is currently at, what data they have accumulated through the workflow (available as personalization variables), and when they are scheduled to wake up if in a wait state. This separation between automation definition and instance state is critical — a single automation definition may have millions of concurrent instances, each at a different position in the graph. The definition is read-heavy and rarely changes, while instances are write-heavy and constantly updated, requiring different storage strategies for each.
5. API Design and Endpoint Architecture
The API layer is the gateway through which all external interactions flow — from the frontend UI, mobile apps, third-party integrations, and the tracking snippet. ActiveCampaign's API follows REST conventions with JSON payloads and uses API key authentication for programmatic access and OAuth 2.0 for partner integrations. The API must handle 50,000+ requests per second at peak while maintaining sub-200ms p99 latency, requiring careful endpoint design, rate limiting, and caching strategies.
Core API Endpoints
| Method | Endpoint | Description | Rate Limit |
|---|---|---|---|
| GET | /api/contacts | List contacts with filtering and pagination | 5,000/min |
| POST | /api/contacts | Create a new contact | 5,000/min |
| PUT | /api/contacts/{id} | Update contact properties | 5,000/min |
| DELETE | /api/contacts/{id} | Soft-delete a contact | 1,000/min |
| GET | /api/contacts/{id}/activities | Get contact activity timeline | 5,000/min |
| POST | /api/automations | Create a new automation workflow | 1,000/min |
| PUT | /api/automations/{id} | Update automation graph | 1,000/min |
| POST | /api/automations/{id}/activate | Activate an automation | 100/min |
| POST | /api/events | Track a custom event | 10,000/min |
| POST | /api/deals | Create a new deal | 5,000/min |
| PUT | /api/deals/{id}/stage | Move deal to a pipeline stage | 5,000/min |
| GET | /api/segments/{id}/contacts | List contacts in a segment | 1,000/min |
| POST | /api/campaigns | Create an email campaign | 500/min |
| POST | /api/webhooks | Register a webhook endpoint | 100/min |
API Request Flow
Every API request flows through the API Gateway which handles SSL termination, rate limiting, and request routing. The gateway enforces per-account rate limits using a token bucket algorithm stored in Redis. Requests that modify contact data or trigger automations publish events to the event bus (Kafka or RabbitMQ), enabling asynchronous processing by downstream services without blocking the API response. This fire-and-forget pattern keeps API latency low while ensuring all side effects are eventually processed.
Webhook and Event Ingestion API
The event ingestion API is the highest-throughput endpoint, receiving tracking events from millions of websites running the ActiveCampaign tracking snippet. It must handle 350,000 events per second at peak. Events are validated, enriched with account and contact context, and immediately published to the event stream. The API returns a 202 Accepted response to indicate the event was received — actual processing happens asynchronously. This distinction between synchronous acknowledgment and asynchronous processing is fundamental to building a responsive and resilient event ingestion pipeline.
Error handling follows the standard HTTP status code conventions with structured error responses. Rate limit exceeded errors include Retry-After headers. Validation errors provide field-level detail. Authentication errors return standardized OAuth error codes. All errors are logged with correlation IDs for debugging and are surfaced in the developer dashboard for API consumers to monitor their integration health.
6. High-Level System Architecture
The ActiveCampaign platform is composed of approximately 30 microservices organized into five major subsystems: the Automation Engine, the CRM, the Messaging Pipeline, the Tracking Infrastructure, and the Analytics Platform. These subsystems communicate primarily through an event bus (Apache Kafka) with synchronous REST calls for user-facing operations that require immediate responses.
Architecture Overview
The event bus is the backbone of the system. Every state change — contact property updates, email opens, deal stage transitions, form submissions — is published as an immutable event to Kafka. The automation engine consumes these events, evaluates which automations are affected, and executes the appropriate workflow steps. The email pipeline consumes email send requests and manages delivery through multiple email service providers. The lead scoring service consumes behavioral events and recalculates contact scores. This event-driven architecture provides natural decoupling between services and enables independent scaling of producers and consumers.
The data layer uses PostgreSQL as the primary relational database, sharded by account ID for write scalability. Redis provides caching for hot contact data, automation state, and rate limit counters. Elasticsearch powers the segmentation engine and full-text search across contacts and deals. S3 stores email templates, campaign content, and attachment files. Each data store is chosen for its specific access pattern — there is no one-size-fits-all database in a system this complex.
The user-facing web application communicates with the API Gateway over HTTPS. The tracking snippet is a lightweight JavaScript library that captures browser events and sends them to the Tracking Evaluator service, which resolves anonymous visitors to known contacts using fingerprinting and cookie matching. The entire architecture is deployed across multiple AWS regions with active-active configuration for global availability. Database replication is asynchronous cross-region for performance, with synchronous replication within each region for consistency.
7. Visual Automation Builder
The visual automation builder is the defining user experience of ActiveCampaign. It provides a canvas where users drag, drop, and connect nodes to create multi-step workflow graphs. The builder must translate a visual representation into an executable workflow definition while validating correctness, preventing infinite loops, and providing a live preview of how contacts would flow through the designed journey.
Node Types
| Node Type | Category | Description | Configuration |
|---|---|---|---|
| Trigger | Entry | Defines how contacts enter the automation | Event type, filters, segment |
| Send Email | Action | Sends an email to the contact | Template, subject, content |
| Send SMS | Action | Sends an SMS message | Message template, shortcode |
| Wait | Control | Delays workflow for specified duration | Duration, unit (hours/days) |
| Condition | Branch | Evaluates a predicate, branches flow | Rules (property, score, tag) |
| Goal | Control | Jumps contact forward to this node | Goal conditions |
| A/B Split | Branch | Randomly splits contacts into paths | Split percentages |
| Update Contact | Action | Modifies contact properties or tags | Property changes |
| Add to List | Action | Adds contact to a mailing list | Target list |
| Webhook | Action | Calls an external URL | URL, method, headers |
| Create Deal | CRM | Creates a new deal in the pipeline | Pipeline, stage, value |
| Assign Task | CRM | Creates a task for a team member | Assignee, task type |
| Notify | Internal | Sends internal notification | Recipients, message |
Graph Serialization and Validation
When a user designs a workflow visually, the builder translates the canvas into a JSON graph structure. Each node has a unique ID, a type, configuration data, and a list of outgoing edge IDs. Edges connect source nodes to target nodes with optional conditions for branching edges. The serialized graph is validated before activation to ensure structural integrity: no orphan nodes, no cycles (except through explicit wait nodes that have timeout edges), all conditions are satisfiable, and all required node configurations are complete.
The validation engine performs static analysis on the graph. It detects unreachable nodes, validates that branching conditions cover all possible outcomes (or have a default path), ensures wait nodes have maximum timeout durations to prevent contacts from being stuck indefinitely, and checks that goal nodes have at least one incoming edge from a different branch. These validations prevent operational issues that would be difficult to diagnose once the automation is running with thousands of contacts.
Versioning and Live Editing
ActiveCampaign allows users to edit active automations without disrupting contacts already in the workflow. This is achieved through versioned graph snapshots. When a user saves changes to an active automation, the system creates a new version of the graph. Existing automation instances continue on the old version. New contacts entering the automation after the save are placed on the new version. When all contacts on the old version have either completed the automation or reached a checkpoint node, the old version is garbage collected. This approach ensures users can iterate on their automations without worrying about breaking running workflows.
8. Workflow Execution Engine
The workflow execution engine is the most complex and performance-critical component of the entire platform. It is responsible for evaluating automation instances in response to events, executing action nodes (sending emails, updating contacts, creating deals), managing wait timers, and advancing contacts through the workflow graph. The engine must process millions of events per second while maintaining exactly-once execution semantics for action nodes and at-least-once delivery for the overall pipeline.
State Machine Model
Each automation instance is modeled as a state machine. When an event arrives for a contact, the engine loads the instance from the state store, determines which automations the contact is enrolled in, and evaluates each one. For each automation, the engine starts at the contact's current node and processes it based on its type. Action nodes dispatch work and transition to a pending state. Wait nodes schedule a future wakeup and transition to the waiting state. Condition nodes evaluate their predicates and transition to the appropriate successor node. The state machine ensures deterministic execution and enables recovery from failures by persisting state transitions to durable storage.
Distributed Execution Architecture
The engine runs as a horizontally scalable consumer group reading from Kafka topics partitioned by contact ID. This partitioning ensures that all events for a given contact are processed by the same engine instance, preventing race conditions on automation state. Each engine instance maintains an in-memory state cache for the contacts it is processing, backed by Redis for cross-instance state access. The engine processes events in batches of 100-500 for efficiency, using the batch to amortize state load and persist operations.
public class AutomationExecutionEngine
{
private readonly IAutomationStateStore _stateStore;
private readonly IActionDispatcher _actionDispatcher;
private readonly ITriggerEvaluator _triggerEvaluator;
private readonly ILogger _logger;
public async Task ProcessEventBatchAsync(List<AutomationEvent> events)
{
var groupedByContact = events.GroupBy(e => e.ContactId);
foreach (var contactGroup in groupedByContact)
{
var contactId = contactGroup.Key;
var instances = await _stateStore.GetInstancesAsync(contactId);
foreach (var evt in contactGroup)
{
foreach (var instance in instances)
{
await ProcessEventForInstance(evt, instance);
}
}
await _stateStore.PersistInstancesAsync(instances);
}
}
private async Task ProcessEventForInstance(
AutomationEvent evt, AutomationInstance instance)
{
var node = instance.GetCurrentNode();
switch (node.Type)
{
case NodeType.Condition:
var result = _triggerEvaluator.Evaluate(
node.ConditionConfig, evt);
instance.Advance(result.BranchTarget);
break;
case NodeType.Email:
await _actionDispatcher.EnqueueAsync(new EmailAction
{
ContactId = instance.ContactId,
TemplateId = node.EmailConfig.TemplateId,
PersonalizationData = instance.ContextVariables
});
instance.Advance(node.DefaultSuccessor);
break;
case NodeType.Wait:
if (evt.Type == EventType.WaitTimeout ||
evt.Type == EventType.GoalAchieved)
{
instance.Advance(node.DefaultSuccessor);
}
break;
case NodeType.UpdateContact:
await _actionDispatcher.EnqueueAsync(new ContactUpdateAction
{
ContactId = instance.ContactId,
Updates = node.ContactUpdateConfig.Updates
});
instance.Advance(node.DefaultSuccessor);
break;
case NodeType.Goal:
if (_triggerEvaluator.EvaluateGoal(
node.GoalConfig, evt))
{
instance.JumpToNode(node.Id);
instance.Advance(node.DefaultSuccessor);
}
break;
}
}
}The engine achieves exactly-once processing for action nodes through a transactional outbox pattern. When an action is dispatched, the engine writes the action to an outbox table in the same database transaction that updates the automation state. A separate dispatcher process reads from the outbox and sends actions to downstream services (email, SMS, CRM). Once the downstream service acknowledges, the outbox entry is marked as processed. This pattern prevents both duplicate actions and lost actions in the event of engine crashes.
Wait Timer Management
Wait nodes are one of the most common node types in automation workflows. They delay contact progression for hours, days, or until a specific date. The engine manages wait timers using a distributed scheduling system backed by Redis sorted sets. When a contact enters a wait node, the engine calculates the wake-up timestamp and adds it to the sorted set. A background scheduler process polls the sorted set for expired timers and publishes wakeup events to Kafka. This approach scales to hundreds of millions of concurrent timers with minimal overhead, as Redis sorted set operations are O(log N) regardless of set size.
9. Trigger System Design
Triggers are the entry points into automation workflows. They define the conditions under which a contact enters an automation. ActiveCampaign supports four major trigger categories: event triggers (responding to specific actions), time triggers (scheduled execution), segment triggers (based on contact attributes), and API triggers (external system initiations). The trigger system must evaluate millions of trigger conditions against incoming events in real-time while avoiding duplicate entry into the same automation.
Trigger Architecture
Trigger Types and Evaluation
Event Triggers respond to specific user actions: form submissions, email opens, link clicks, tag additions, custom field updates, and e-commerce purchases. When an event arrives, the trigger evaluator looks up all automations in the account that have a trigger matching the event type. It then evaluates the trigger's filter conditions against the event data and contact properties. If all conditions are met, the contact is eligible to enter the automation.
Time Triggers execute on a schedule — daily, weekly, or monthly. They are used for recurring campaigns, periodic list cleaning, and scheduled lead scoring recalculations. The time trigger system uses a distributed cron scheduler that publishes trigger events at the scheduled times. Each time trigger is evaluated against all contacts in the target segment, creating automation instances for each matching contact.
Segment Triggers fire when a contact enters or exits a defined segment. For example, when a contact's lead score exceeds 100, they enter the "Marketing Qualified Lead" segment, triggering a sales notification automation. Segment triggers require a continuous segment evaluation pipeline that monitors contact property changes and compares them against active segment definitions.
public class TriggerEvaluator
{
private readonly ISegmentEvaluator _segmentEvaluator;
private readonly IAutomationRegistry _automationRegistry;
public async Task<List<AutomationEntry>> EvaluateTriggersAsync(
AutomationEvent incomingEvent)
{
var results = new List<AutomationEntry>();
var candidateAutomations = await _automationRegistry
.GetAutomationsByTriggerType(incomingEvent.Type);
foreach (var automation in candidateAutomations)
{
foreach (var trigger in automation.Triggers)
{
if (!trigger.MatchesEventType(incomingEvent.Type))
continue;
if (!EvaluateFilters(trigger.Filters, incomingEvent))
continue;
if (!await EvaluateSegmentConditions(
trigger.SegmentConditions, incomingEvent.ContactId))
continue;
if (await IsDuplicateEntry(
automation.Id, incomingEvent.ContactId))
continue;
results.Add(new AutomationEntry
{
AutomationId = automation.Id,
ContactId = incomingEvent.ContactId,
TriggerId = trigger.Id,
EntryPoint = trigger.EntryNodeId,
ContextData = incomingEvent.Properties
});
}
}
return results;
}
private bool EvaluateFilters(
List<TriggerFilter> filters, AutomationEvent evt)
{
return filters.All(filter =>
{
var value = filter.Source switch
{
FilterSource.EventProperty => evt.Properties
.GetValueOrDefault(filter.FieldName),
FilterSource.ContactProperty => evt.ContactProperties
.GetValueOrDefault(filter.FieldName),
_ => null
};
return filter.Operator.Evaluate(value, filter.ExpectedValue);
});
}
}API Triggers allow external systems to push contacts into automations via webhook calls. This is commonly used for integrating with custom applications, Zapier workflows, and partner platforms. API triggers validate the incoming payload against a schema, optionally upsert the contact, and then route the contact into the automation.
Deduplication
A critical concern in trigger processing is preventing duplicate automation entries. A contact should not enter the same automation twice (unless explicitly configured to allow re-entry). The deduplication engine uses a Redis-backed set keyed by (automation_id, contact_id) with a configurable TTL matching the automation's re-entry window. Before creating a new automation instance, the engine checks this set. If the contact is already present, the entry is skipped. This check is performed atomically using Redis SETNX operations to prevent race conditions in concurrent processing.
10. CRM Automation and Deal Pipelines
ActiveCampaign's CRM is not merely a contact database — it is a fully integrated sales pipeline management system with automation capabilities that rival dedicated CRM platforms. The CRM automation subsystem enables marketing and sales teams to automate deal creation, stage progression, lead assignment, task creation, and sales rep notifications. Understanding the CRM automation architecture is essential for designing a complete marketing automation platform.
Deal Pipeline Model
A pipeline represents a sales process with ordered stages. Each account can have multiple pipelines (e.g., "New Business," "Upsell," "Partner Channel"). Deals progress through stages as sales reps move them forward or automation advances them based on criteria. The pipeline model supports stage-specific automation triggers — when a deal enters a particular stage, automations can fire to send proposal emails, create follow-up tasks, or notify stakeholders.
CRM Automation Features
Deal Stage Automation: When a deal transitions between stages, the system evaluates stage-entry automations. For example, moving a deal to "Proposal Sent" can trigger an automation that attaches a proposal document, sends a personalized email to the contact, creates a follow-up task for the sales rep, and updates the deal's expected close date based on historical stage duration data.
Lead Assignment Rules: The system supports round-robin, territory-based, and score-based lead assignment. When a new deal is created or a contact reaches a lead scoring threshold, the assignment engine evaluates the assignment rules and assigns the deal to the most appropriate sales representative. Assignment rules consider the rep's current workload, territory, specialization, and historical close rates.
Task Automation: Automations can create tasks with due dates, priorities, and assignees. Tasks are surfaced in the CRM activity feed and can trigger notifications via email or Slack integration. Task automation eliminates manual follow-up tracking and ensures no lead falls through the cracks.
Activity Timeline: Every interaction with a contact is logged in the activity timeline — emails sent, pages visited, forms submitted, deals created, tasks completed, and notes added. The timeline provides a complete history that both marketing and sales teams can reference. Automation-generated activities are clearly labeled with the originating automation name for transparency.
CRM Data Consistency
The CRM must maintain strong consistency for deal state transitions. If two sales reps attempt to move the same deal simultaneously, the system uses optimistic concurrency control with version vectors. The first update succeeds; the second receives a conflict error and must re-read the current state before retrying. For automation-driven deal updates, the system acquires a distributed lock on the deal ID to serialize modifications and prevent conflicting stage transitions.
The CRM also supports custom objects — entities beyond the standard contacts, deals, and companies. An e-commerce business might define a "Product" custom object, linking deals to specific products with custom properties like SKU, category, and price. Custom objects extend the data model without requiring platform-level schema changes, stored as JSON documents with indexed properties for query performance.
11. Lead Scoring and Predictive Intelligence
Lead scoring assigns a numerical value to each contact indicating their likelihood to convert. ActiveCampaign's scoring system combines rule-based behavioral scoring with predictive machine learning models. The scoring engine must update scores in near-real-time as new events arrive, support custom scoring rules per account, and handle score decay to ensure stale engagement does not inflate lead quality signals.
Scoring Model Components
| Score Component | Weight Range | Example Factors | Update Frequency |
|---|---|---|---|
| Email Engagement | 1-10 points | Opens, clicks, replies | Real-time |
| Website Activity | 1-15 points | Pricing page, docs, blog | Real-time |
| Form Submissions | 5-25 points | Demo request, newsletter | Real-time |
| Demographic Fit | 1-20 points | Job title, company size | On property change |
| Deal Activity | 1-15 points | Deal stage progression | Real-time |
| Engagement Decay | -1 to -5/day | No activity for N days | Daily batch |
| Negative Signals | -10 to -50 | Unsubscribe, bounce | Real-time |
Rule-Based Scoring Engine
The rule-based scoring engine evaluates scoring rules against contact events and properties. Each account configures their own scoring rules through the UI, defining which events add or subtract points and by how much. The engine processes scoring events from Kafka, looks up the applicable rules for the contact's account, evaluates each rule, and updates the contact's score atomically.
public class LeadScoringService
{
private readonly IContactRepository _contactRepo;
private readonly IScoringRuleRepository _rulesRepo;
private readonly IScoreEventStore _eventStore;
private readonly IPredictiveModelService _predictiveService;
public async Task<ScoringResult> CalculateScoreAsync(
Guid contactId, ScoringEvent scoringEvent)
{
var contact = await _contactRepo.GetByIdAsync(contactId);
var rules = await _rulesRepo.GetActiveRulesAsync(contact.AccountId);
int ruleDelta = 0;
var appliedRules = new List<AppliedRule>();
foreach (var rule in rules)
{
if (rule.MatchesEvent(scoringEvent))
{
int points = rule.CalculatePoints(scoringEvent, contact);
ruleDelta += points;
appliedRules.Add(new AppliedRule
{
RuleId = rule.Id,
Points = points,
Reason = rule.Description
});
}
}
int decayDelta = await CalculateDecayAsync(contact);
int newScore = Math.Max(0, contact.Score + ruleDelta + decayDelta);
var predictiveScore = await _predictiveService
.PredictConversionProbability(contactId);
await _contactRepo.UpdateScoreAsync(
contactId, newScore, predictiveScore);
await _eventStore.AppendAsync(new ScoreChangeEvent
{
ContactId = contactId,
PreviousScore = contact.Score,
NewScore = newScore,
Delta = ruleDelta + decayDelta,
AppliedRules = appliedRules,
Timestamp = DateTime.UtcNow
});
return new ScoringResult
{
ContactId = contactId,
PreviousScore = contact.Score,
NewScore = newScore,
PredictiveScore = predictiveScore,
AppliedRules = appliedRules
};
}
private async Task<int> CalculateDecayAsync(Contact contact)
{
if (contact.LastActivityAt == null) return 0;
var daysSinceActivity = (DateTime.UtcNow
- contact.LastActivityAt.Value).Days;
if (daysSinceActivity <= 7) return 0;
int decayDays = daysSinceActivity - 7;
return -(decayDays * 2);
}
}Predictive Scoring
Predictive scoring uses machine learning models trained on historical conversion data across the ActiveCampaign customer base. The model takes as input the contact's behavioral features (engagement patterns, content interests, buying signals), demographic features (job title, company size, industry), and temporal features (recency of engagement, engagement velocity). It outputs a probability score between 0 and 100 indicating the likelihood of conversion within the next 30 days.
The model is retrained weekly using a gradient-boosted decision tree architecture optimized for tabular data. Training data includes all contacts that converted (became customers) or did not convert within a defined window, with features computed from the 90-day window before conversion. The model achieves an AUC of approximately 0.82 across the platform, with significant variation by industry and business model. For individual accounts with sufficient data (100+ conversions), the model is fine-tuned on account-specific data for improved accuracy.
Score decay is a critical component that many implementations overlook. Without decay, a contact who was highly engaged six months ago but has since gone dormant would still carry a high score, misleading sales teams. The decay system reduces scores by a configurable number of points per day of inactivity, ensuring that current engagement is weighted more heavily than historical activity. The decay rate is configurable per account, with a recommended default of 2 points per day of inactivity after a 7-day grace period.
12. Site Tracking and Event Tracking
Site tracking is the eyes and ears of the marketing automation platform. The JavaScript tracking snippet installed on customer websites captures page views, form submissions, button clicks, e-commerce transactions, and custom events in real-time. This data feeds directly into the automation engine, lead scoring system, and segmentation engine, making it one of the highest-volume data sources in the platform. The tracking infrastructure must handle 100 billion page impressions per month while identifying anonymous visitors and merging their profiles with known contacts.
Tracking Snippet Architecture
The tracking snippet is a lightweight JavaScript library (under 10KB gzipped) that loads asynchronously to avoid impacting page performance. On initialization, it generates a visitor fingerprint using browser attributes (user agent, screen resolution, timezone, installed fonts) and stores a first-party cookie for visitor identification. When a user submits a form with an email address, the anonymous visitor profile is merged with the known contact record. The snippet sends events to the tracking endpoint via the Beacon API with XHR fallback for older browsers.
public class TrackingEventProcessor
{
private readonly IContactResolver _contactResolver;
private readonly IEventPublisher _eventPublisher;
private readonly ITrackingProfileStore _profileStore;
public async Task ProcessTrackingEventAsync(TrackingEvent rawEvent)
{
var visitorId = rawEvent.VisitorId;
var contactId = await _contactResolver.ResolveAsync(
visitorId, rawEvent.Cookies);
var enrichedEvent = new EnrichedTrackingEvent
{
Id = Guid.NewGuid(),
AccountId = rawEvent.AccountId,
ContactId = contactId,
VisitorId = visitorId,
EventType = rawEvent.EventType,
Url = rawEvent.Url,
Properties = rawEvent.Properties,
Timestamp = DateTime.UtcNow,
SessionId = rawEvent.SessionId,
UserAgent = rawEvent.UserAgent,
IpAddress = rawEvent.IpAddress
};
if (contactId.HasValue)
{
enrichedEvent.ContactProperties = await _contactResolver
.GetContactPropertiesAsync(contactId.Value);
}
await _eventPublisher.PublishAsync("tracking.events", enrichedEvent);
await _profileStore.UpdateActivityAsync(
visitorId, contactId, enrichedEvent);
}
}Anonymous to Known Visitor Resolution
Visitor identification is a multi-step process. When a new visitor arrives, the snippet generates an anonymous ID and stores it in a first-party cookie. All subsequent events from that browser carry this anonymous ID. When the visitor submits a form with their email address, the system performs a profile merge: the anonymous event history is associated with the known contact record, the anonymous ID is added to a lookup table for future identification, and all automation triggers fire as if the historical events had occurred on the known contact. This merge process is critical for accurate attribution and lead scoring, as it captures the full journey from first anonymous visit to known lead.
The system also uses progressive identification techniques: if a visitor clicks a link in an email (which carries the contact ID in the URL parameters), the snippet recognizes this and links the anonymous session to the known contact without requiring a form submission. Similarly, if the visitor logs into a connected application that passes the contact ID via API, the session is linked. These techniques maximize the percentage of tracked activity that can be attributed to known contacts, improving the accuracy of lead scoring and automation triggers.
Data Volume and Retention
With 100 billion page impressions per month, the tracking system generates approximately 3.3 billion events per day. Raw events are retained in the event store for 90 days for detailed analytics, then aggregated into daily summaries for long-term retention. Aggregated data (page view counts, session durations, conversion funnels) is retained for 2 years. The event store uses a columnar storage format (Apache Parquet on S3) for efficient analytical queries, with a hot tier in Cassandra for recent events requiring low-latency access.
13. Email Campaign Engine
Email remains the primary communication channel for marketing automation, and ActiveCampaign's email engine is built for scale, deliverability, and personalization. The engine must send 22 million emails per day on average, with peak capacity of 5 million emails per minute during major campaign events. Each email must be personalized with contact-specific content, tracked for opens and clicks, and delivered with high inbox placement rates through proper authentication and reputation management.
Email Delivery Pipeline
The sending pipeline is organized into multiple stages. After a campaign or automation email is queued, the scheduler determines the optimal send time based on the contact's timezone and historical engagement patterns. The rate limiter ensures the account stays within its sending plan limits and ESP throughput constraints. The reputation engine monitors deliverability metrics (bounce rates, complaint rates, spam trap hits) and can throttle or pause sending if reputation degrades. Finally, the email is dispatched to the appropriate ESP based on volume, cost, and deliverability considerations.
Dynamic Content and Personalization
Every email sent through ActiveCampaign can contain dynamic content blocks that change based on the recipient's contact properties, behavioral history, or segment membership. The personalization engine evaluates content rules at send time, selecting the appropriate content variant for each recipient. This enables a single email campaign to deliver different content to different audience segments — for example, showing different product recommendations based on the contact's purchase history or different pricing based on their company size.
The template system uses a custom templating language with variables like {{contact.first_name}}, conditional blocks {% if contact.score > 50 %}, and content blocks that switch based on segment membership. Templates are stored as versioned documents with preview capabilities. The template editor provides a visual editing experience with live preview, while also supporting raw HTML editing for advanced users. Templates are stored in S3 with metadata in PostgreSQL for fast retrieval and version management.
Deliverability and Reputation Management
Email deliverability is arguably the most critical operational concern for an email platform. ActiveCampaign manages deliverability through several mechanisms: SPF, DKIM, and DMARC authentication for all sending domains; IP warming schedules for new sending infrastructure; bounce and complaint feedback loops with major ISPs; real-time monitoring of inbox placement rates through seed-list testing; and automatic suppression of hard bounces, unsubscribes, and complaint addresses. The reputation engine continuously monitors per-domain and per-IP metrics and can automatically shift traffic away from degraded sending infrastructure.
14. SMS and Push Notification Automation
While email remains the dominant channel, SMS and push notifications are increasingly important for time-sensitive communications. ActiveCampaign supports SMS automation through carrier-connected shortcodes and long codes, enabling two-way messaging for appointment reminders, order confirmations, shipping notifications, and conversational marketing. Push notifications extend reach to mobile app users for real-time engagement.
SMS Architecture
The SMS pipeline mirrors the email pipeline in structure but differs in compliance requirements and delivery characteristics. SMS messages must comply with TCPA regulations in the US, GDPR in Europe, and carrier-specific guidelines globally. Opt-in consent must be recorded and verifiable. Messages must include opt-out instructions. Sending frequency must respect carrier limits (typically 3 messages per day per number). The SMS service integrates with multiple aggregators (Twilio, MessageBird, Vonage) for global coverage and automatic failover.
SMS automation supports triggers like appointment reminders (scheduled N days before), order status updates (triggered by e-commerce webhooks), and two-way conversational flows where the contact's reply triggers the next step in the automation. The system handles incoming SMS messages by matching the sender's phone number to a contact record, evaluating any active SMS-triggered automations, and routing the message to the appropriate conversation thread.
Push Notification System
Push notifications require a device token registration system, segmented audience targeting, and delivery through APNs (Apple) and FCM (Google). The push notification service maintains a device registry mapping contacts to their registered devices, handling token refresh, device changes, and opt-out management. Push campaigns support rich media (images, action buttons) and deep linking to specific app screens.
The notification delivery system respects quiet hours configured by the user or inferred from the user's timezone, batches non-urgent notifications to avoid notification fatigue, and provides analytics on delivery rates, open rates, and conversion attribution. Push notifications sent from automations must be throttled to prevent overwhelming recipients who trigger multiple automations simultaneously. The system implements a per-contact notification budget that limits the total number of push notifications sent within configurable time windows.
Multi-channel orchestration enables cross-channel automation sequences — for example, sending an email, waiting 2 hours, checking if it was opened, and if not, sending an SMS follow-up. The automation engine handles the timing and channel selection, while the messaging pipelines handle the actual delivery. This unified approach ensures consistent messaging across channels and prevents conflicting communications from different automations.
15. Segmentation Engine
Segments are dynamic groupings of contacts defined by conditions on their properties, behaviors, and relationships. Unlike static lists, segments continuously evaluate their conditions against the contact database — contacts are automatically added or removed as their data changes. Segmentation powers targeted campaigns, automation triggers, exclusion lists, and analytics. The engine must evaluate segment conditions against billions of contacts efficiently, supporting complex nested conditions with AND/OR/NOT logic across contact properties, behavioral events, deal attributes, and automation membership.
Segmentation Architecture
The segment definition is parsed into a query tree that can be executed against the contact database. Simple conditions (e.g., "email contains @gmail.com") translate to SQL WHERE clauses. Complex conditions involving behavioral data ("opened an email in the last 30 days") require joining with the events table. Conditions on nested objects ("deal value > 10000") require joins with the deals table. The query engine optimizes these conditions by pushing filters down to the database level and using indexes where available.
Incremental Segment Evaluation
Re-evaluating all segment conditions for all contacts on every change would be prohibitively expensive. Instead, the system uses incremental evaluation: when a contact property changes, the system identifies all segments that reference that property and re-evaluates only those segments for the affected contact. A dependency graph maps property names to segment IDs, enabling efficient change routing. For behavioral conditions, events are consumed from Kafka and evaluated against active segments that match the event type.
public class SegmentationEngine
{
private readonly ISegmentDefinitionStore _definitionStore;
private readonly IContactQueryEngine _queryEngine;
private readonly ISegmentMembershipStore _membershipStore;
private readonly ISegmentDependencyGraph _dependencyGraph;
public async Task OnContactPropertyChangedAsync(
Guid contactId, string propertyName, object newValue)
{
var dependentSegments = _dependencyGraph
.GetSegmentsForProperty(propertyName);
foreach (var segmentId in dependentSegments)
{
var definition = await _definitionStore.GetAsync(segmentId);
var isMember = await _queryEngine.EvaluateAsync(
definition, contactId);
var wasMember = await _membershipStore
.IsMemberAsync(segmentId, contactId);
if (isMember && !wasMember)
{
await _membershipStore.AddAsync(segmentId, contactId);
await PublishSegmentEventAsync(
segmentId, contactId, SegmentEventType.Entered);
}
else if (!isMember && wasMember)
{
await _membershipStore.RemoveAsync(segmentId, contactId);
await PublishSegmentEventAsync(
segmentId, contactId, SegmentEventType.Exited);
}
}
}
}Segment Performance Optimization
For segments with millions of contacts, materialized views pre-compute the membership set and store it in Redis for O(1) membership checks. These views are refreshed on a schedule (every 5-15 minutes) or on-demand when the segment definition changes. The materialization process uses efficient set operations (intersection, union, difference) to update only the delta since the last refresh. For real-time segments, the incremental evaluation path ensures sub-second membership updates as contact data changes.
The segmentation engine also supports exclusion lists — segments that define contacts to exclude from campaigns. Exclusion evaluation happens at campaign send time, removing excluded contacts from the target audience just before dispatch. Common exclusion segments include recently emailed contacts (to prevent fatigue), unsubscribed contacts, bounced addresses, and internal test contacts. The exclusion engine is designed for high-throughput evaluation at campaign send time, processing millions of exclusion checks per minute.
16. Contact and Deal Management
At its core, ActiveCampaign is a system for managing relationships with people and the opportunities those relationships represent. The contact and deal management layer provides the CRUD operations, activity tracking, custom fields, and relationship management that underpin every other feature. This layer must support high-throughput reads and writes, complex queries across contact and deal data, and a complete audit trail of all changes.
Contact Management
Contacts are the fundamental unit of data in the system. Each contact represents a person with a unique email address within an account. The contact model supports standard fields (name, email, phone, organization) and unlimited custom fields defined by the account. Custom fields support multiple types: text, number, date, dropdown, multi-select, and boolean. The contact API supports bulk operations — importing up to 100,000 contacts per batch, bulk updating properties, and bulk adding/removing tags.
The activity timeline is the chronological log of all interactions with a contact. It combines automated activities (email sent, automation started, score changed) with manual activities (note added, call logged, deal created). The timeline is stored in an append-only event log, enabling efficient writes without requiring updates to existing records. Reads are served from a materialized view that aggregates events into a readable timeline format, refreshed in near-real-time using streaming aggregation.
Deal Management
Deals represent sales opportunities and are linked to contacts, companies, and optionally custom objects. Each deal has a pipeline, stage, monetary value, expected close date, and assigned owner. The deal API supports stage transitions with validation rules (e.g., a deal cannot skip the "Proposal" stage), value updates with historical tracking, and bulk stage movements for pipeline cleanup.
Deal forecasting uses the deal value, stage probabilities, and historical conversion rates to predict revenue. The system calculates weighted pipeline value (value multiplied by stage-specific conversion probability) and generates forecast reports by sales rep, team, and time period. Forecast accuracy improves over time as the system learns from historical deal outcomes specific to each account's sales process.
Custom Fields and Properties
The custom field system must handle the diversity of data across 150,000 accounts, each with their own schema. The implementation uses an Entity-Attribute-Value (EAV) pattern combined with JSON storage. Standard fields are stored as columns for efficient querying. Custom fields are stored in a properties JSON column with GIN indexes for key lookups, and in a separate properties table for fields that need relational queries (e.g., segments filtering on custom fields). This hybrid approach provides flexibility without sacrificing performance for common access patterns.
| Field Type | Storage | Index Support | Segment Filterable |
|---|---|---|---|
| Text | JSON + EAV | GIN + B-tree | Contains, equals, regex |
| Number | JSON + EAV | GIN + B-tree | GT, LT, equals, range |
| Date | JSON + EAV | GIN + B-tree | Before, after, range |
| Dropdown | JSON + EAV | GIN | Equals, in list |
| Multi-select | JSON array | GIN | Contains, overlap |
| Boolean | JSON + EAV | GIN | True, false |
Contact and deal data is partitioned by account ID in the database, ensuring that queries for a single account are scoped to a single partition. Cross-account queries (used for analytics and platform monitoring) run against a separate analytics replica built from the event stream, avoiding contention with the transactional workload. This separation of OLTP and OLAP workloads is essential for maintaining performance as data volume grows.
17. Reporting, Analytics, and Attribution
Reporting transforms raw data into actionable insights. ActiveCampaign provides campaign performance reports, automation analytics, deal forecasting, and multi-touch attribution modeling. The reporting infrastructure must handle analytical queries across billions of records while providing sub-second response times for dashboard views. This requires a carefully designed analytics pipeline that pre-aggregates data at multiple granularities while supporting ad-hoc drill-down queries.
Analytics Pipeline Architecture
Events flow from Kafka into a stream processor (Apache Flink or Kafka Streams) that computes real-time aggregations: emails sent, opened, and clicked per campaign; automation entries, completions, and exits per automation; deal stage transitions per pipeline. These aggregations are written to ClickHouse, a columnar OLAP database optimized for analytical queries. Simultaneously, raw events are archived to S3 as a data lake for long-term retention and ad-hoc analysis with tools like Apache Spark.
Attribution Modeling
Multi-touch attribution determines which marketing interactions deserve credit for a conversion. ActiveCampaign supports first-touch, last-touch, and linear attribution models out of the box, with custom model support for enterprise accounts. The attribution engine reconstructs the customer journey from tracking events, automation interactions, and campaign engagements, then applies the attribution model to distribute conversion credit across touchpoints.
Attribution data is computed at multiple time windows (7-day, 30-day, 60-day, 90-day lookback) and stored pre-aggregated by campaign, automation, and channel. This pre-computation enables instant dashboard rendering without expensive real-time calculations. The attribution engine also supports cross-channel attribution, correlating email opens with website conversions, SMS clicks with deal creation, and form submissions with revenue.
Dashboard and Reporting
The reporting dashboard provides real-time visibility into account performance. Key metrics include total contacts, email delivery rates, open rates, click rates, automation completion rates, deal pipeline value, and conversion rates. The dashboard API serves pre-computed metric snapshots from Redis cache, updated every 60 seconds by the stream processor. Drill-down queries bypass the cache and query ClickHouse directly for detailed breakdowns by time period, campaign, automation, or segment.
Custom report builder allows users to define reports by selecting metrics, dimensions, and filters. The report builder translates user selections into ClickHouse queries with appropriate GROUP BY and WHERE clauses. Report results are cached for 10 minutes to avoid repeated expensive queries for the same report configuration. Scheduled reports are generated as PDF exports and emailed to stakeholders on the configured schedule.
18. Integrations Platform — 900+ Connectors
ActiveCampaign's integration ecosystem with over 900 connectors is a major competitive advantage. The platform integrates with e-commerce tools (Shopify, WooCommerce), CRM systems (Salesforce, HubSpot), content management systems (WordPress), form builders (Typeform, Gravity Forms), and thousands of other services through a robust integration framework. The integration architecture must support diverse authentication methods, handle varying data formats, manage rate limits, and provide reliable bidirectional data synchronization.
Integration Architecture
| Integration Type | Authentication | Data Flow | Examples |
|---|---|---|---|
| Native | OAuth 2.0 / API Key | Bi-directional sync | Salesforce, Shopify, WordPress |
| Marketplace | OAuth 2.0 | Trigger + Action | Zapier, Make, Pabbly |
| Webhook | HMAC signature | Event push | Custom applications |
| Custom Objects | OAuth 2.0 | Schema mapping | Salesforce Custom Objects |
| API | API Token | Pull on demand | Any REST API |
Integration Framework
The integration framework provides a standardized interface for building and running connectors. Each integration implements a common interface with methods for authentication setup, data mapping, synchronization, and error handling. The framework handles OAuth token refresh, rate limit management, retry with exponential backoff, and dead letter queue management for permanently failed operations.
Native integrations with major platforms use platform-specific APIs and webhooks for real-time synchronization. For example, the Shopify integration receives order webhooks and pushes contact updates back through the Shopify Admin API. The Salesforce integration uses the Bulk API for large data transfers and the Streaming API for real-time change notifications. Each integration manages its own data mapping configuration, translating between ActiveCampaign's data model and the external platform's schema.
The webhook framework allows users to build custom integrations by registering HTTP endpoints that receive event payloads. Webhooks are dispatched with HMAC signatures for verification, retried up to 5 times with exponential backoff, and logged for debugging. Failed webhooks after all retries are moved to a dead letter queue where users can inspect and manually replay them. Webhook payloads include the event type, timestamp, and the full resource data, providing sufficient context for the receiving system to process the event.
19. Database Design and Storage Architecture
The storage architecture of a marketing automation platform must handle diverse workloads: high-throughput contact updates, complex analytical queries, append-only event logs, and real-time caching. No single database technology can optimally serve all these workloads. ActiveCampaign's data layer uses a polyglot persistence strategy, choosing the right database for each access pattern.
Data Flow Architecture
Primary Database — PostgreSQL
PostgreSQL serves as the primary OLTP database, storing contacts, deals, automations, campaigns, and all relational data. The database is sharded by account ID, with each shard containing data for approximately 5,000 accounts. Sharding is implemented at the application layer using a consistent hashing scheme, enabling elastic resharding as the customer base grows. Each shard runs as a primary-replica pair with synchronous replication within the shard for consistency and asynchronous replication to an analytics replica for read-heavy workloads.
The schema follows a normalized design for data integrity with strategic denormalization for read performance. Contact data is partitioned by creation date to manage the 10 billion+ contact volume. Old partitions are moved to cold storage after 2 years. Automation state tables use a separate tablespace on fast SSD storage due to their high write throughput. Indexes are carefully maintained — only indexes that serve specific query patterns are created, avoiding the index bloat that degrades write performance.
Event Store
The event store captures every interaction — email opens, page views, form submissions, score changes, deal transitions — as immutable events. Events are written to Kafka for real-time processing and simultaneously to S3 for archival. Elasticsearch indexes events for full-text search and ad-hoc querying. ClickHouse stores aggregated event data for analytical queries and reporting. The event store retains raw events for 90 days in Elasticsearch and forever in S3, with daily aggregates in ClickHouse for long-term analytics.
Database Capacity Planning
| Store | Data Volume | Write Throughput | Read Throughput |
|---|---|---|---|
| PostgreSQL (OLTP) | 2TB | 50K writes/s | 200K reads/s |
| Elasticsearch | 5TB (90 days) | 100K events/s | 10K queries/s |
| ClickHouse | 50TB (aggregated) | 100K events/s | 500 queries/s |
| Redis | 200GB (hot data) | 500K ops/s | 1M ops/s |
| S3 | 500TB+ (archive) | 50K writes/s | Burst reads |
The polyglot approach requires careful data synchronization. Kafka Connect provides change data capture (CDC) from PostgreSQL to downstream stores, ensuring eventual consistency across all data layers. The application layer uses the outbox pattern for reliable event publication — database writes and event publications happen in the same transaction, with a background process publishing events from the outbox table to Kafka. This pattern guarantees that every state change eventually flows to all downstream data stores without dual-write problems.
20. Caching Strategy
Caching is essential for meeting the sub-200ms API latency requirement at scale. The platform caches contact data, automation state, segment membership, rate limit counters, and session data. The caching strategy must balance freshness with performance, handle cache invalidation correctly, and survive cache failures without degrading the user experience significantly.
Cache Tiers
L1 — Application Memory Cache: Each API server maintains an in-process LRU cache for the most frequently accessed data. This tier provides sub-millisecond access for hot data like authenticated user sessions and frequently accessed contact records. Cache entries expire after 60 seconds, ensuring reasonable freshness while eliminating repeated database queries for the same data within a request window.
L2 — Redis Cluster Cache: The shared Redis cluster serves as the primary caching tier, storing contact records, automation instances, segment membership sets, and rate limit counters. Redis provides consistent latency under 1ms for point lookups and supports pub/sub for real-time cache invalidation across application servers. The cluster is deployed with sentinel-based failover, ensuring high availability for this critical infrastructure component.
L3 — CDN Edge Cache: Static assets (email templates, images, JavaScript snippets) are cached at CDN edge locations worldwide. The tracking snippet is served from CDN with aggressive caching headers, ensuring minimal load times for the millions of websites using ActiveCampaign tracking.
Cache Invalidation
Contact data cache invalidation uses a publish-subscribe pattern. When a contact is updated through the API or automation engine, the update is published to a Redis pub/sub channel. All application servers subscribed to that channel evict the stale cache entry. This approach ensures cache consistency within seconds of a write, without requiring centralized invalidation logic. For segment membership, invalidation is event-driven — when a contact enters or exits a segment, the segment membership cache is updated through the same pub/sub mechanism.
Automation state caching follows a different pattern. Since automation state is only modified by the automation execution engine (which processes events from Kafka partitioned by contact ID), each engine instance is the sole writer for its partition of contacts. This eliminates write contention and allows the engine to maintain an in-memory state cache without distributed locking. State is persisted to Redis and PostgreSQL asynchronously, with synchronous persistence only at checkpoint nodes within the workflow.
Cache Failure Handling
All cache layers are designed as read-through caches with graceful degradation. If Redis is unavailable, the application falls back to direct database queries with increased latency but no functional impairment. Rate limit counters degrade to a more lenient rate limit when Redis is down, preventing service disruption while accepting slightly higher API volumes. The monitoring system tracks cache hit rates, latency percentiles, and memory utilization, alerting when hit rates drop below thresholds that would impact API performance.
21. Multi-Region Deployment Design
ActiveCampaign serves customers across 170 countries, requiring a multi-region deployment architecture that provides low-latency access globally, compliance with data residency regulations, and high availability through geographic redundancy. The platform uses an active-active deployment model with data partitioning by account region.
Multi-Region Architecture
Each region maintains its own complete stack: application servers, PostgreSQL primary, Redis cluster, Kafka cluster, and Elasticsearch cluster. Account data is pinned to a home region based on the account's primary location. When a user accesses the platform, the global load balancer routes them to their home region for write operations. Read operations can be served from the nearest region using read replicas, reducing latency for globally distributed teams.
Cross-region Kafka replication ensures that events generated in one region are available in other regions for processing. This is critical for integrations that may receive webhooks in a different region than the account's home region, and for analytics queries that span regional boundaries. Replication lag is typically under 500ms, which is acceptable for automation processing but not for strong consistency requirements. Strongly consistent operations are always routed to the home region.
Data Residency and Compliance
GDPR and similar regulations require that customer data for EU-based contacts stays within the EU region. The data residency framework enforces this through region-scoped data policies: EU account data is stored only in the EU-West region, with no replication to US or AP regions. Analytics queries that need global data use anonymized, aggregated datasets that do not contain personally identifiable information. The framework supports per-contact data residency, allowing accounts with mixed global contacts to comply with regional regulations.
Failover and Disaster Recovery
If a region becomes unavailable, the global load balancer detects the failure through health checks and redirects traffic to the nearest healthy region. Read operations are served immediately from the failover region's read replicas. Write operations are queued in a local write-ahead log and replayed to the home region once it recovers. This active-passive failover model for writes ensures no data loss while accepting temporary write unavailability (typically under 5 minutes of recovery time). Cross-region backup replication provides disaster recovery with a 15-minute RPO (Recovery Point Objective) and 1-hour RTO (Recovery Time Objective).
22. Security, Compliance, and Data Privacy
A marketing automation platform handles sensitive personal data — email addresses, phone numbers, website browsing behavior, purchase history, and business communications. Security and compliance are not optional features but fundamental architectural requirements. ActiveCampaign maintains SOC 2 Type II certification, GDPR compliance, CCPA compliance, and ISO 27001 certification. The security architecture must protect data at rest and in transit, provide fine-grained access control, and support data subject rights under privacy regulations.
Security Architecture
Encryption: All data is encrypted at rest using AES-256 with per-account encryption keys managed through AWS KMS. Data in transit is protected with TLS 1.3. Database-level encryption uses PostgreSQL's transparent data encryption extension. Email content stored in S3 is encrypted with server-side encryption (SSE-KMS). API tokens and OAuth credentials are stored using one-way bcrypt hashing with per-account salt.
Access Control: The platform implements role-based access control (RBAC) with four standard roles: Account Admin, Manager, User, and View-Only. Custom roles allow granular permission assignment. API access is controlled through scoped API keys that can be restricted to specific endpoints and data types. Multi-factor authentication is enforced for admin accounts and available for all users.
Data Privacy: The platform implements GDPR data subject rights through automated workflows: data export (all contact data in JSON/CSV format), data deletion (permanent removal from all systems including backups within 30 days), consent management (opt-in/opt-out tracking with timestamped consent records), and data portability (machine-readable export in standard formats). The privacy dashboard provides account admins with visibility into their data processing activities and consent status.
Infrastructure Security: All production infrastructure runs in private VPCs with no direct internet access. Application servers communicate through internal load balancers. Database access is restricted to application servers through security groups. Secrets are managed through HashiCorp Vault with automatic rotation. The platform conducts annual penetration testing and maintains a bug bounty program for responsible vulnerability disclosure.
Compliance Monitoring
Automated compliance monitoring scans for configuration drift, access anomalies, and data handling violations. The system monitors API access patterns for potential data exfiltration, alerts on unusual bulk data exports, and tracks all administrative actions in an immutable audit log. Compliance reports are generated quarterly and made available to auditors through a secure portal. The security team maintains incident response playbooks for common scenarios (data breach, unauthorized access, service compromise) with defined escalation paths and communication templates.
23. Cost Estimation and Infrastructure Budget
Understanding the infrastructure cost of running a marketing automation platform at ActiveCampaign's scale is essential for capacity planning and business model validation. The cost breakdown below estimates monthly infrastructure expenses for a platform serving 150,000 accounts with 10 billion total contacts, processing 5 billion events per day, and sending 22 million emails daily.
Monthly Infrastructure Cost Breakdown
| Component | Specification | Monthly Cost (USD) |
|---|---|---|
| Compute — Application Servers | 200x c5.2xlarge (8 vCPU, 16GB) | $140,000 |
| Compute — Automation Engine | 100x c5.4xlarge (16 vCPU, 32GB) | $140,000 |
| Compute — Kafka Cluster | 30x kafka.m5.2xlarge | $75,000 |
| PostgreSQL (Primary + Replicas) | 20x r5.4xlarge + 200TB EBS | $85,000 |
| Redis Cluster | 50x r5.xlarge (200GB total) | $25,000 |
| Elasticsearch Cluster | 40x r5.2xlarge + 200TB storage | $65,000 |
| ClickHouse Cluster | 30x r5.4xlarge + 500TB | $70,000 |
| S3 Storage | 500TB + 100TB/month growth | $12,000 |
| CDN (CloudFront) | 50TB/month transfer | $8,000 |
| Email Service Providers | 22M emails/day avg, 100M peak | $200,000 |
| SMS Providers | 550K messages/day | $15,000 |
| Multi-Region Redundancy | 3 regions (3x core infra) | $350,000 |
| Monitoring and Observability | Datadog, PagerDuty, Sentry | $25,000 |
| Security and Compliance | Vault, WAF, scanning tools | $15,000 |
| DevOps and CI/CD | GitHub Enterprise, ArgoCD | $10,000 |
| Total Monthly Infrastructure Cost | ~$1,235,000 | |
| Estimated Annual Infrastructure Cost | ~$14.8M | |
Cost Per Account Analysis
With 150,000 paying accounts, the average infrastructure cost per account is approximately $8.23 per month. ActiveCampaign's lowest plan at $49/month provides healthy margins on infrastructure alone, before accounting for development, support, and sales costs. The blended gross margin on infrastructure is approximately 83%, which is strong for a SaaS platform. However, the largest accounts (enterprise with millions of contacts and high email volumes) consume 100x the infrastructure of the smallest accounts, making per-account cost distribution highly skewed.
Cost optimization opportunities include spot instances for the automation engine (which can tolerate interruptions), reserved instances for stable workloads (PostgreSQL, Redis), intelligent tiering for S3 storage, and email send batching to reduce ESP costs. The platform invests in ongoing cost optimization through FinOps practices, with monthly cost reviews and automated scaling policies that match infrastructure to actual demand rather than peak capacity.
24. Interview Questions and Answers
System design interviews for marketing automation platforms test distributed systems knowledge, data modeling skills, and ability to reason about complex stateful workflows. Below are ten interview questions with detailed answers covering the core architectural challenges of building an ActiveCampaign-like system.
Question 1: Design the Workflow Execution Engine
Q: How would you design the core execution engine that processes automation workflows for millions of contacts?
A: The engine is a distributed state machine consuming events from Kafka. Each contact's automation state is stored in Redis (hot) and PostgreSQL (durable). Events are partitioned by contact ID so each engine instance handles a consistent set of contacts, preventing race conditions. The engine processes events in batches: load the contact's automation instances, evaluate each instance against the event, execute any resulting actions through the transactional outbox pattern, and persist updated state. Wait nodes use Redis sorted sets for timer management. Exactly-once action execution is guaranteed by the outbox pattern — the action write and state update happen in the same database transaction. The engine scales horizontally by adding consumers to the Kafka consumer group.
Question 2: Handle Concurrent Modification of Automation Definitions
Q: How do you handle a user editing an active automation while contacts are currently processing through it?
A: Use versioned graph snapshots. When a user saves changes, create a new version of the automation graph. Existing automation instances reference the version they started on and continue processing against that version. New contacts entering the automation get the latest version. When all contacts on an old version complete or reach a checkpoint node, the old version is garbage collected. This is similar to how database schema migrations handle in-flight transactions. The version ID is stored in each automation instance alongside the current node reference.
Question 3: Design Lead Scoring at Scale
A: Lead scoring processes 5 billion events per day. The system uses a streaming architecture: scoring events are consumed from Kafka, scoring rules are loaded from the account's configuration, and scores are computed incrementally (delta computation, not full recalculation). The score delta is applied atomically to the contact record. Predictive scoring uses pre-computed batch models that run weekly, storing predictions in a lookup table. Score decay runs as a daily batch job that decrements scores for contacts below an inactivity threshold. The entire pipeline maintains sub-second latency from event to score update through the streaming path.
Question 4: Segment Evaluation Performance
A: Segments with complex conditions across billions of contacts cannot be re-evaluated from scratch on every change. The system uses incremental evaluation with a dependency graph: each segment condition is mapped to the contact properties it depends on. When a property changes, only segments referencing that property are re-evaluated for the affected contact. For behavioral segments, events are matched against segment conditions in the stream processor. Materialized segment membership is stored in Redis for O(1) lookups at campaign send time. Full segment recalculation runs as a background batch job every 15 minutes to catch any incremental evaluation gaps.
Question 5: Email Deliverability at Scale
A: Deliverability requires IP reputation management, authentication, and feedback loops. The system maintains multiple IP pools segmented by account reputation tier. New accounts start in a warming pool with throttled sending. The reputation engine monitors per-IP metrics (bounce rate, complaint rate, open rate) and automatically shifts traffic away from degraded IPs. SPF, DKIM, and DMARC are configured for all sending domains. Bounce and complaint feedback loops from ISPs feed into automatic suppression lists. Seed-list testing provides inbox placement monitoring. Accounts with persistent deliverability issues receive automated recommendations and may face sending restrictions.
Question 6: Multi-Tenant Data Isolation
A: Account data isolation is enforced at the application layer through tenant ID scoping on every query, and at the database layer through row-level security policies. Every API request includes the account context from the authentication token. The ORM layer automatically injects account ID filters into all queries. Cross-tenant data access is prevented by database-level row-level security as a defense-in-depth measure. Analytics queries run against pre-aggregated data that never exposes individual account details. Encryption keys are per-account, ensuring that even direct database access cannot decrypt another account's data.
Question 7: Handle Spike in Email Sends During Campaign Blasts
A: Campaign blasts can generate 100M emails in a short window. The system uses a multi-stage queuing pipeline: campaign emails are first written to a per-account send queue with rate limiting to respect the account's sending plan. A global scheduler distributes sends across the ESP connection pool, smoothing peaks to stay within ESP throughput limits. Priority queuing ensures automation emails (time-sensitive abandoned cart, triggered messages) get priority over bulk campaign sends. The system pre-warms ESP connections before major known events (Black Friday) and can spill over to secondary ESPs during extreme peaks.
Question 8: Real-Time Site Tracking Pipeline
A: The tracking snippet generates 3.3 billion events per day. Events are sent to a globally distributed tracking endpoint (edge-located) that validates, deduplicates, and buffers events. A lightweight processing layer enriches events with contact resolution (anonymous-to-known matching), geolocation, and device information. Enriched events are published to Kafka for downstream processing by the automation engine, segmentation engine, and analytics pipeline. Raw events are archived to S3. The system uses first-party cookies and probabilistic fingerprinting for visitor identification, with deterministic matching when email addresses are provided through form submissions.
Question 9: Design the Integration Framework
A: The integration framework uses an adapter pattern: each integration implements a standard interface with methods for authentication, data mapping, sync, and error handling. The framework manages OAuth token lifecycle, rate limiting per integration, retry policies with exponential backoff, and dead letter queues for failed operations. Webhook-based integrations dispatch events with HMAC verification and retry up to 5 times. The framework monitors per-integration health (success rate, latency) and can auto-disable broken integrations. Marketplace integrations run in isolated containers to prevent resource contention.
Question 10: Prevent Duplicate Email Sends
A: Duplicate prevention operates at multiple levels. First, the automation engine uses the deduplication engine (Redis SETNX) to prevent a contact from entering the same automation twice. Second, the email send pipeline maintains a send log (contact_id, campaign_id, sent_at) with a unique index — duplicate sends are rejected at the database level. Third, a global suppression list prevents sending to contacts who have unsubscribed, bounced, or complained. The combination of automation-level deduplication, send-level deduplication, and suppression list filtering ensures that a contact never receives the same email twice, even in the presence of event retries and processing failures.
Question 11: Design the Wait Timer System
A: Wait timers are managed using Redis sorted sets where the score is the Unix timestamp of the wake-up time. When a contact enters a wait node, the engine calculates the wake-up time and adds it to the sorted set (ZADD). A scheduler process polls for expired timers using ZRANGEBYSCORE, publishing wake-up events to Kafka. The engine processes wake-up events like any other event, advancing the contact past the wait node. For reliability, timer entries are also persisted to PostgreSQL as a backup. If Redis data is lost, a reconciliation process rebuilds the sorted set from PostgreSQL. The system supports 500M+ concurrent timers with O(log N) operations.
25. Full C# Implementation
The following C# implementation demonstrates the core components of the marketing automation platform: the automation engine, workflow node model, lead scoring service, trigger evaluator, and segmentation engine. This implementation covers approximately 350 lines of production-quality code with proper error handling, async patterns, and clear separation of concerns.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
namespace ActiveCampaignAutomation.Core.Models
{
public enum NodeType
{
Trigger, SendEmail, SendSms, Wait, Condition,
Goal, ABTesting, UpdateContact, AddToList,
Webhook, CreateDeal, AssignTask, Notify
}
public enum AutomationStatus { Draft, Active, Paused, Archived }
public enum InstanceState
{
Idle, Evaluating, ActionPending, Waiting,
Completed, Failed, Paused
}
public class WorkflowNode
{
public Guid Id { get; set; }
public NodeType Type { get; set; }
public string DisplayName { get; set; }
public Dictionary<string, object> Configuration { get; set; }
public List<WorkflowEdge> OutgoingEdges { get; set; }
public string DefaultSuccessorId { get; set; }
public WorkflowNode()
{
Configuration = new Dictionary<string, object>();
OutgoingEdges = new List<WorkflowEdge>();
}
}
public class WorkflowEdge
{
public string TargetNodeId { get; set; }
public EdgeCondition Condition { get; set; }
public double? SplitPercentage { get; set; }
}
public class EdgeCondition
{
public string Field { get; set; }
public string Operator { get; set; }
public string Value { get; set; }
public bool Evaluate(Dictionary<string, object> context)
{
if (!context.ContainsKey(Field)) return false;
var actual = context[Field]?.ToString() ?? "";
return Operator switch
{
"equals" => actual.Equals(Value,
StringComparison.OrdinalIgnoreCase),
"contains" => actual.Contains(Value,
StringComparison.OrdinalIgnoreCase),
"gt" => double.TryParse(actual, out var a) &&
double.TryParse(Value, out var b) && a > b,
"lt" => double.TryParse(actual, out var c) &&
double.TryParse(Value, out var d) && c < d,
"not_empty" => !string.IsNullOrWhiteSpace(actual),
_ => false
};
}
}
public class AutomationDefinition
{
public Guid Id { get; set; }
public Guid AccountId { get; set; }
public string Name { get; set; }
public AutomationStatus Status { get; set; }
public int Version { get; set; }
public Dictionary<string, WorkflowNode> Nodes { get; set; }
public List<AutomationTrigger> Triggers { get; set; }
public DateTime CreatedAt { get; set; }
public DateTime UpdatedAt { get; set; }
public AutomationDefinition()
{
Nodes = new Dictionary<string, WorkflowNode>();
Triggers = new List<AutomationTrigger>();
}
public WorkflowNode GetNode(string nodeId)
{
return Nodes.TryGetValue(nodeId, out var node) ? node : null;
}
}
public class AutomationTrigger
{
public Guid Id { get; set; }
public string EventType { get; set; }
public List<EdgeCondition> Filters { get; set; }
public string EntryNodeId { get; set; }
public bool MatchesEventType(string eventType)
{
return EventType == "*" || EventType == eventType;
}
}
public class AutomationInstance
{
public Guid Id { get; set; }
public Guid AutomationId { get; set; }
public int AutomationVersion { get; set; }
public Guid ContactId { get; set; }
public string CurrentNodeId { get; set; }
public InstanceState State { get; set; }
public Dictionary<string, object> ContextVariables { get; set; }
public DateTime StartedAt { get; set; }
public DateTime? LastProcessedAt { get; set; }
public DateTime? WakeUpAt { get; set; }
public AutomationInstance()
{
ContextVariables = new Dictionary<string, object>();
State = InstanceState.Idle;
}
public void Advance(string nextNodeId)
{
CurrentNodeId = nextNodeId;
State = InstanceState.Idle;
LastProcessedAt = DateTime.UtcNow;
}
public void SetWaiting(DateTime wakeUpTime)
{
State = InstanceState.Waiting;
WakeUpAt = wakeUpTime;
}
public void MarkCompleted()
{
State = InstanceState.Completed;
CurrentNodeId = null;
LastProcessedAt = DateTime.UtcNow;
}
}
public class AutomationEvent
{
public Guid Id { get; set; }
public Guid ContactId { get; set; }
public string Type { get; set; }
public Dictionary<string, object> Properties { get; set; }
public Dictionary<string, object> ContactProperties { get; set; }
public DateTime Timestamp { get; set; }
}
}
namespace ActiveCampaignAutomation.Core.Engine
{
using Models;
public interface IActionDispatcher
{
Task<bool> DispatchAsync(AutomationAction action);
}
public interface IAutomationStateStore
{
Task<List<AutomationInstance>> GetInstancesAsync(Guid contactId);
Task PersistInstancesAsync(List<AutomationInstance> instances);
}
public interface IAutomationDefinitionStore
{
Task<AutomationDefinition> GetAsync(Guid automationId);
}
public class AutomationAction
{
public string ActionType { get; set; }
public Guid ContactId { get; set; }
public Guid AutomationId { get; set; }
public Dictionary<string, object> Parameters { get; set; }
}
public class AutomationExecutionEngine
{
private readonly IAutomationStateStore _stateStore;
private readonly IAutomationDefinitionStore _definitionStore;
private readonly IActionDispatcher _actionDispatcher;
private readonly ITriggerEvaluator _triggerEvaluator;
private readonly IDeduplicationStore _dedupStore;
private readonly IWaitScheduler _waitScheduler;
public AutomationExecutionEngine(
IAutomationStateStore stateStore,
IAutomationDefinitionStore definitionStore,
IActionDispatcher actionDispatcher,
ITriggerEvaluator triggerEvaluator,
IDeduplicationStore dedupStore,
IWaitScheduler waitScheduler)
{
_stateStore = stateStore;
_definitionStore = definitionStore;
_actionDispatcher = actionDispatcher;
_triggerEvaluator = triggerEvaluator;
_dedupStore = dedupStore;
_waitScheduler = waitScheduler;
}
public async Task<ProcessingResult> ProcessEventBatchAsync(
List<AutomationEvent> events)
{
var result = new ProcessingResult();
var grouped = events.GroupBy(e => e.ContactId);
foreach (var contactGroup in grouped)
{
var contactId = contactGroup.Key;
var instances = await _stateStore
.GetInstancesAsync(contactId);
foreach (var evt in contactGroup)
{
var newEntries = await _triggerEvaluator
.EvaluateTriggersAsync(evt);
foreach (var entry in newEntries)
{
if (!await _dedupStore.IsDuplicateAsync(
entry.AutomationId, contactId))
{
instances.Add(CreateNewInstance(
entry, contactId));
}
}
foreach (var instance in instances
.Where(i => i.State != InstanceState.Completed
&& i.State != InstanceState.Failed))
{
try
{
await ProcessEventForInstance(evt, instance);
result.SuccessCount++;
}
catch (Exception ex)
{
result.FailureCount++;
}
}
}
await _stateStore.PersistInstancesAsync(instances);
}
return result;
}
private async Task ProcessEventForInstance(
AutomationEvent evt, AutomationInstance instance)
{
if (instance.State == InstanceState.Waiting)
{
if (instance.WakeUpAt.HasValue &&
instance.WakeUpAt.Value <= DateTime.UtcNow)
instance.State = InstanceState.Idle;
else if (evt.Type != "wait_timeout" &&
evt.Type != "goal_achieved")
return;
}
var definition = await _definitionStore
.GetAsync(instance.AutomationId);
var node = definition.GetNode(instance.CurrentNodeId);
if (node == null) { instance.MarkCompleted(); return; }
instance.State = InstanceState.Evaluating;
switch (node.Type)
{
case NodeType.SendEmail:
await HandleEmailAction(node, instance);
instance.Advance(node.DefaultSuccessorId);
break;
case NodeType.SendSms:
await HandleSmsAction(node, instance);
instance.Advance(node.DefaultSuccessorId);
break;
case NodeType.Condition:
var branchResult = _triggerEvaluator
.EvaluateCondition(node, evt, instance);
instance.Advance(branchResult);
break;
case NodeType.Wait:
var duration = TimeSpan.FromHours(
Convert.ToDouble(
node.Configuration["hours"]));
instance.SetWaiting(
DateTime.UtcNow.Add(duration));
await _waitScheduler.ScheduleAsync(
instance.Id,
DateTime.UtcNow.Add(duration));
break;
case NodeType.Goal:
var achieved = _triggerEvaluator
.EvaluateGoal(node, evt, instance);
if (achieved)
instance.Advance(node.DefaultSuccessorId);
break;
case NodeType.UpdateContact:
await HandleContactUpdate(node, instance);
instance.Advance(node.DefaultSuccessorId);
break;
case NodeType.CreateDeal:
await HandleDealCreation(node, instance);
instance.Advance(node.DefaultSuccessorId);
break;
case NodeType.Webhook:
await HandleWebhook(node, instance);
instance.Advance(node.DefaultSuccessorId);
break;
default:
instance.Advance(node.DefaultSuccessorId);
break;
}
}
private async Task HandleEmailAction(
WorkflowNode node, AutomationInstance instance)
{
await _actionDispatcher.DispatchAsync(new AutomationAction
{
ActionType = "send_email",
ContactId = instance.ContactId,
AutomationId = instance.AutomationId,
Parameters = new Dictionary<string, object>
{
["template_id"] = node.Configuration["template_id"],
["subject"] = node.Configuration["subject"],
["personalization"] = instance.ContextVariables
}
});
}
private async Task HandleSmsAction(
WorkflowNode node, AutomationInstance instance)
{
await _actionDispatcher.DispatchAsync(new AutomationAction
{
ActionType = "send_sms",
ContactId = instance.ContactId,
AutomationId = instance.AutomationId,
Parameters = new Dictionary<string, object>
{
["message"] = node.Configuration["message"],
["shortcode"] = node.Configuration["shortcode"]
}
});
}
private async Task HandleContactUpdate(
WorkflowNode node, AutomationInstance instance)
{
var updates = (Dictionary<string, object>)
node.Configuration["updates"];
await _actionDispatcher.DispatchAsync(new AutomationAction
{
ActionType = "update_contact",
ContactId = instance.ContactId,
AutomationId = instance.AutomationId,
Parameters = updates
});
}
private async Task HandleDealCreation(
WorkflowNode node, AutomationInstance instance)
{
await _actionDispatcher.DispatchAsync(new AutomationAction
{
ActionType = "create_deal",
ContactId = instance.ContactId,
AutomationId = instance.AutomationId,
Parameters = new Dictionary<string, object>
{
["pipeline_id"] = node.Configuration["pipeline_id"],
["stage"] = node.Configuration["stage"],
["title"] = node.Configuration["title"],
["value"] = node.Configuration["value"]
}
});
}
private async Task HandleWebhook(
WorkflowNode node, AutomationInstance instance)
{
await _actionDispatcher.DispatchAsync(new AutomationAction
{
ActionType = "webhook_call",
ContactId = instance.ContactId,
AutomationId = instance.AutomationId,
Parameters = new Dictionary<string, object>
{
["url"] = node.Configuration["url"],
["method"] = node.Configuration
.GetValueOrDefault("method", "POST"),
["body"] = instance.ContextVariables
}
});
}
private AutomationInstance CreateNewInstance(
AutomationEntry entry, Guid contactId)
{
return new AutomationInstance
{
Id = Guid.NewGuid(),
AutomationId = entry.AutomationId,
ContactId = contactId,
CurrentNodeId = entry.EntryPoint,
State = InstanceState.Idle,
StartedAt = DateTime.UtcNow,
ContextVariables = new Dictionary<string, object>(
entry.ContextData)
};
}
}
}
namespace ActiveCampaignAutomation.Core.Scoring
{
using Models;
public class ScoringRule
{
public Guid Id { get; set; }
public string EventType { get; set; }
public string PropertyFilter { get; set; }
public string Operator { get; set; }
public int Points { get; set; }
public string Description { get; set; }
public bool Matches(ScoringEvent evt)
{
if (EventType != "*" && EventType != evt.Type) return false;
if (string.IsNullOrEmpty(PropertyFilter)) return true;
return evt.Properties.ContainsKey(PropertyFilter);
}
}
public class LeadScoringService
{
private readonly IContactRepository _contactRepo;
private readonly IScoringRuleRepository _rulesRepo;
private readonly IScoreEventStore _eventStore;
private readonly IPredictiveModelService _predictiveService;
private readonly int _decayGraceDays = 7;
private readonly int _decayPointsPerDay = 2;
public LeadScoringService(
IContactRepository contactRepo,
IScoringRuleRepository rulesRepo,
IScoreEventStore eventStore,
IPredictiveModelService predictiveService)
{
_contactRepo = contactRepo;
_rulesRepo = rulesRepo;
_eventStore = eventStore;
_predictiveService = predictiveService;
}
public async Task<ScoringResult> ProcessEventAsync(
Guid contactId, ScoringEvent scoringEvent)
{
var contact = await _contactRepo.GetByIdAsync(contactId);
if (contact == null)
throw new ArgumentException("Contact not found");
var rules = await _rulesRepo
.GetActiveRulesAsync(contact.AccountId);
int ruleDelta = 0;
var appliedRules = new List<AppliedRule>();
foreach (var rule in rules.Where(r => r.Matches(scoringEvent)))
{
int points = rule.Points;
if (rule.Operator == "negative") points = -points;
ruleDelta += points;
appliedRules.Add(new AppliedRule
{
RuleId = rule.Id,
Points = points,
Description = rule.Description
});
}
int decayDelta = CalculateDecay(contact);
int totalDelta = ruleDelta + decayDelta;
int newScore = Math.Max(0, contact.Score + totalDelta);
var predictiveScore = await _predictiveService
.PredictScoreAsync(contactId);
await _contactRepo.UpdateScoreAsync(
contactId, newScore, predictiveScore);
await _eventStore.AppendAsync(new ScoreChangeEvent
{
ContactId = contactId,
PreviousScore = contact.Score,
NewScore = newScore,
Delta = totalDelta,
RulesApplied = appliedRules,
Timestamp = DateTime.UtcNow
});
return new ScoringResult
{
ContactId = contactId,
PreviousScore = contact.Score,
NewScore = newScore,
PredictiveScore = predictiveScore,
AppliedRules = appliedRules
};
}
private int CalculateDecay(Contact contact)
{
if (contact.LastActivityAt == null) return 0;
var daysInactive = (DateTime.UtcNow
- contact.LastActivityAt.Value).Days;
if (daysInactive <= _decayGraceDays) return 0;
return -(daysInactive - _decayGraceDays)
* _decayPointsPerDay;
}
}
}
namespace ActiveCampaignAutomation.Core.Segmentation
{
using Models;
public class SegmentCondition
{
public string Field { get; set; }
public string Operator { get; set; }
public string Value { get; set; }
public string Logic { get; set; } = "AND";
public List<SegmentCondition> Children { get; set; }
= new List<SegmentCondition>();
public bool Evaluate(Dictionary<string, object> contactProps)
{
if (Children.Any())
{
return Logic.ToUpper() == "AND"
? Children.All(c => c.Evaluate(contactProps))
: Children.Any(c => c.Evaluate(contactProps));
}
if (!contactProps.ContainsKey(Field)) return false;
var actual = contactProps[Field]?.ToString() ?? "";
return Operator switch
{
"equals" => actual.Equals(Value,
StringComparison.OrdinalIgnoreCase),
"contains" => actual.Contains(Value,
StringComparison.OrdinalIgnoreCase),
"starts_with" => actual.StartsWith(Value,
StringComparison.OrdinalIgnoreCase),
"gt" => double.TryParse(actual, out var a) &&
double.TryParse(Value, out var b) && a > b,
"lt" => double.TryParse(actual, out var c) &&
double.TryParse(Value, out var d) && c < d,
"in" => Value.Split(',').Select(v => v.Trim())
.Contains(actual, StringComparer.OrdinalIgnoreCase),
"not_empty" => !string.IsNullOrWhiteSpace(actual),
"empty" => string.IsNullOrWhiteSpace(actual),
_ => false
};
}
}
public class SegmentationEngine
{
private readonly ISegmentDefinitionStore _definitionStore;
private readonly IContactQueryEngine _queryEngine;
private readonly ISegmentMembershipStore _membershipStore;
private readonly ISegmentDependencyGraph _dependencyGraph;
public SegmentationEngine(
ISegmentDefinitionStore definitionStore,
IContactQueryEngine queryEngine,
ISegmentMembershipStore membershipStore,
ISegmentDependencyGraph dependencyGraph)
{
_definitionStore = definitionStore;
_queryEngine = queryEngine;
_membershipStore = membershipStore;
_dependencyGraph = dependencyGraph;
}
public async Task OnContactPropertyChangedAsync(
Guid contactId, string propertyName, object newValue)
{
var dependentSegments = _dependencyGraph
.GetSegmentsForProperty(propertyName);
foreach (var segmentId in dependentSegments)
{
var definition = await _definitionStore
.GetAsync(segmentId);
var isMember = await _queryEngine.EvaluateAsync(
definition, contactId);
var wasMember = await _membershipStore
.IsMemberAsync(segmentId, contactId);
if (isMember && !wasMember)
{
await _membershipStore
.AddAsync(segmentId, contactId);
await PublishSegmentEventAsync(
segmentId, contactId,
SegmentEventType.Entered);
}
else if (!isMember && wasMember)
{
await _membershipStore
.RemoveAsync(segmentId, contactId);
await PublishSegmentEventAsync(
segmentId, contactId,
SegmentEventType.Exited);
}
}
}
public async Task<List<Guid>> GetSegmentMembersAsync(
Guid segmentId, int offset, int limit)
{
return await _membershipStore.GetMembersAsync(
segmentId, offset, limit);
}
public async Task<int> GetSegmentCountAsync(Guid segmentId)
{
return await _membershipStore.GetCountAsync(segmentId);
}
}
}26. Conclusion
Designing a marketing automation platform like ActiveCampaign is one of the most challenging system design problems in the SaaS domain. It combines the complexity of a visual programming environment (the automation builder), a distributed state machine (the workflow engine), a real-time event processing pipeline (site tracking), a multi-channel messaging system (email, SMS, push), a full CRM, and a machine learning platform (predictive scoring) — all serving 150,000+ customers with 10 billion contacts on a shared multi-tenant infrastructure.
The key architectural insights from this design are: first, the automation engine is the core bottleneck and must be designed as a horizontally scalable, event-driven state machine with exactly-once action execution guarantees. Second, the polyglot persistence strategy — PostgreSQL for OLTP, Redis for caching and timers, Elasticsearch for search and segments, ClickHouse for analytics, S3 for archival — is essential because no single database can efficiently serve all the diverse access patterns. Third, the event-driven architecture using Kafka as the central nervous system provides the decoupling necessary to independently scale each subsystem. Fourth, the versioned graph snapshot pattern for live automation editing is critical for user experience and data correctness.
The cost analysis reveals that infrastructure represents approximately 4% of revenue at ActiveCampaign's scale, with the automation engine and email delivery pipeline being the largest cost centers. Multi-region deployment triples the core infrastructure cost but is essential for global availability and data residency compliance. The security architecture must protect sensitive personal data across billions of records while supporting GDPR data subject rights and SOC 2 certification requirements.
For system design interviews, the most important concepts to master are: the transactional outbox pattern for exactly-once action execution, Redis sorted sets for distributed timer management, incremental segment evaluation with dependency graphs, and the versioned snapshot approach for live workflow editing. These patterns are not unique to marketing automation — they apply broadly to any system that combines visual workflow builders with distributed execution at scale.
Building such a platform requires a multidisciplinary engineering team with expertise in distributed systems, data engineering, machine learning, frontend development (for the visual builder), and DevOps. The system touches nearly every area of computer science — graph algorithms for workflow validation, state machine theory for workflow execution, information retrieval for segmentation, machine learning for predictive scoring, and network engineering for global deployment. This breadth is what makes marketing automation platform design both challenging and deeply rewarding for senior engineers.