GetResponse Email Marketing Platform – System Design Deep Dive
Architecting a Multi-Tenant Email Delivery Platform Serving 350K+ Customers and 15M+ Emails Per Day
1. Article Overview and Introduction
GetResponse is one of the longest-running email marketing platforms in the industry, founded in 1998 in Gdańsk, Poland. Over nearly three decades, it has evolved from a simple email broadcast tool into a comprehensive marketing automation suite serving over 350,000 customers across 183 countries. The platform processes upwards of 15 million emails per day at peak, manages billions of contact records, and operates marketing automation workflows that execute millions of conditional logic branches hourly.
This article provides a rigorous system design analysis of the GetResponse platform from the perspective of senior, staff, and principal engineers. We will dissect every major subsystem — from the email ingestion pipeline and contact segmentation engine to the SMTP relay infrastructure and real-time event processing layer. Our goal is not marketing copy but architectural depth: the trade-offs, the technology choices, the failure modes, and the scalability patterns that make a platform of this magnitude functional and reliable.
The email marketing industry presents unique engineering challenges that differ fundamentally from typical web applications. Emails are asynchronous by nature, deliverability is governed by reputation systems controlled by ISPs (Internet Service Providers) that publish no formal SLAs, and the volume of data — both in terms of contact records and event streams — demands careful partitioning and tiered storage strategies. A single campaign sent to 10 million contacts generates not just 10 million SMTP transactions, but also 10 million tracking pixel loads, potentially tens of millions of click-tracking redirects, and billions of log entries across the delivery pipeline.
We will reference realistic technology choices consistent with GetResponse's known engineering stack — .NET and C# for backend services, PostgreSQL and Elasticsearch for persistence, Redis for caching and real-time state, Apache Kafka for event streaming, and a proprietary SMTP relay layer — while also discussing how these systems interact under load. Where GetResponse's internal architecture is not publicly documented in fine detail, we will reason from first principles about what the architecture must look like given the observable behavior of the platform, and we will clearly mark such inferences.
2. GetResponse Platform Overview
2.1 History and Evolution
GetResponse was founded by Simon Grabowski in 1998, making it one of the oldest email marketing SaaS platforms still in operation. The platform's evolution mirrors the broader history of email marketing technology:
- 1998–2005: Basic email broadcast and autoresponder functionality. Single-tenant architecture, likely running on early Windows Server / IIS infrastructure with SQL Server backends.
- 2006–2012: Introduction of marketing automation, landing page builder, and webinar integration. Transition toward multi-tenant SaaS architecture. Adoption of .NET framework for backend services.
- 2013–2018: Drag-and-drop email builder, advanced segmentation, A/B testing, and time travel send features. Migration toward microservices architecture with dedicated delivery infrastructure. Introduction of machine learning for send-time optimization.
- 2019–2026: Conversion funnels, AI-powered content generation, advanced automation journeys with visual builders, webhook ecosystem, and integration marketplace. Modern event-driven architecture with Kafka-based streaming.
2.2 Scale Numbers
| Metric | Value | Engineering Significance |
|---|---|---|
| Active customers | 350,000+ | Multi-tenant isolation, per-tenant resource quotas |
| Emails sent per day | 15M+ (peak) | SMTP throughput, queue depth management |
| Total contacts managed | 1B+ | Horizontal sharding, tiered storage |
| Email templates available | 500+ | Template CDN, versioned storage |
| Automation workflows active | 2M+ | Workflow engine throughput, state management |
| Integrations available | 170+ | OAuth management, webhook infrastructure |
| Data centers | 2 (EU, US) | Data residency, GDPR compliance |
| Average deliverability rate | 99.4% | Reputation management, feedback loop processing |
2.3 Core Product Capabilities
The GetResponse platform comprises several tightly integrated subsystems. The Email Marketing core handles campaign creation, contact management, and broadcast delivery. The Marketing Automation engine provides visual workflow builders with conditional branching, scoring, and event-triggered journeys. The Conversion Funnel product chains landing pages, lead magnets, email sequences, and payment processing. The Webinar platform handles live and on-demand video sessions with integrated email follow-ups. Each of these subsystems shares the underlying contact database and delivery infrastructure but has distinct compute and state management requirements.
2.4 Technology Stack Overview
| Layer | Technology | Purpose |
|---|---|---|
| Backend Services | .NET 8, C# | API layer, business logic, background workers |
| Primary Database | PostgreSQL 15 | Contact data, campaign configs, billing |
| Search Engine | Elasticsearch 8.x | Contact segmentation queries, log analytics |
| Cache Layer | Redis 7 (Cluster) | Session state, rate limiting, template cache |
| Message Broker | Apache Kafka 3.x | Event streaming, email queue, tracking events |
| Object Storage | S3-compatible (MinIO) | Email assets, attachments, template images |
| CDN | Cloudflare | Static assets, tracking pixel delivery |
| Container Orchestration | Kubernetes | Service deployment, auto-scaling |
| Monitoring | Prometheus + Grafana | Metrics, alerting, dashboards |
| Log Aggregation | Elasticsearch + Kibana | Centralized logging, delivery logs |
3. System Architecture Overview
The GetResponse platform follows a service-oriented architecture (SOA) with clear domain boundaries. While not a pure microservices architecture — several domain boundaries are served by monolithic modules — the system exhibits clear separation between the web-facing API tier, the background processing layer, the delivery infrastructure, and the data persistence tier.
3.1 Architectural Principles
The architecture follows several key principles that are observable from the platform's external behavior and public engineering blog posts:
Domain-Driven Service Boundaries: The system is decomposed along business domain lines rather than technical layers. The Campaign Service owns the entire lifecycle of email campaigns — from creation through scheduling, personalization, and delivery triggering. The Contact Service owns contact CRUD, custom fields, and list membership. This decomposition minimizes cross-service transactions and allows each domain to scale independently.
Event-Driven Communication: Inter-service communication for non-blocking operations uses Kafka event streams. When a campaign is scheduled, the Campaign Service publishes a campaign.scheduled event. The Delivery Service subscribes to this event and manages the SMTP relay process. The Tracking Service subscribes to delivery events to initialize tracking pixels. This decoupling means the Campaign Service can acknowledge scheduling without waiting for delivery confirmation.
CQRS for Read/Write Separation: The Contact Service uses Command Query Responsibility Segregation. Writes go to PostgreSQL through the command side. Reads — particularly the complex segmentation queries — are served from Elasticsearch, which is populated via change data capture (CDC) from PostgreSQL. This separation is critical because segmentation queries involve complex boolean logic across dozens of custom fields against millions of contacts, which would be prohibitively slow on PostgreSQL alone.
Idempotent Delivery: Every email send is assigned a globally unique message_id at scheduling time. The SMTP relay layer is idempotent — if a send is retried due to a network failure, the same message_id is used, and receiving MTAs can deduplicate. This prevents duplicate delivery during failure scenarios.
3.2 Data Flow: Campaign Send Lifecycle
3.3 Capacity Planning
At 15 million emails per day, the platform must sustain an average throughput of approximately 174 emails per second, with peak throughput during campaign send windows reaching 5,000–10,000 emails per second. This peak occurs because customers overwhelmingly schedule campaigns for the top of the hour (9 AM, 10 AM, etc.), creating thundering herd patterns.
The SMTP relay cluster must maintain thousands of concurrent SMTP connections to hundreds of receiving ISPs, each with its own throughput limits and throttling policies. The relay layer implements per-domain connection pooling — for example, maintaining a separate connection pool for Gmail (gmail-smtp-in.l.google.com), another for Microsoft (smtp.office365.com), and so on — with per-pool rate limiting aligned to each ISP's published guidelines.
| Subsystem | Average Load | Peak Load | Scaling Strategy |
|---|---|---|---|
| API Gateway | 500 req/s | 5,000 req/s | Horizontal pod autoscaling |
| Campaign Service | 200 req/s | 2,000 req/s | Horizontal + read replicas |
| SMTP Relay | 174 emails/s | 10,000 emails/s | Per-ISP connection pools |
| Tracking Service | 1,000 req/s | 50,000 req/s | CDN edge caching + stateless pods |
| Kafka Cluster | 50K events/s | 500K events/s | Topic partitioning + consumer groups |
| PostgreSQL | 2,000 qps | 8,000 qps | Read replicas + connection pooling |
| Elasticsearch | 1,000 qps | 5,000 qps | Index sharding + hot-warm architecture |
| Redis | 10,000 qps | 80,000 qps | Cluster mode with hash slot distribution |
4. Email Ingestion and Campaign Creation Pipeline
The campaign creation pipeline is the primary write path for the email marketing platform. When a user creates and launches a campaign, the system must validate the campaign configuration, resolve the target audience, personalize content for each recipient, enqueue delivery tasks, and manage the entire lifecycle from scheduling through completion.
4.1 Campaign State Machine
Every campaign progresses through a well-defined state machine:
| State | Description | Transitions |
|---|---|---|
DRAFT | Campaign being edited | → SCHEDULED, → DELETED |
SCHEDULED | Campaign locked, waiting for send time | → PROCESSING, → CANCELLED |
PROCESSING | Resolving contacts, personalizing | SENDING, → FAILED |
SENDING | Actively dispatching to SMTP relay | → COMPLETED, → PAUSED |
PAUSED | Temporarily halted by user or system | → SENDING, → CANCELLED |
COMPLETED | All messages dispatched | Terminal |
FAILED | Irrecoverable error | Terminal (can retry as new) |
CANCELLED | User-initiated cancellation | Terminal |
4.2 Campaign Scheduling Service
The scheduling service must handle "time travel" sends — campaigns scheduled for a specific time in the future — as well as immediate sends. This requires a reliable job scheduling mechanism that can handle millions of scheduled jobs with second-level precision.
public class CampaignScheduler
{
private readonly IKafkaProducer _kafkaProducer;
private readonly ICampaignRepository _campaignRepo;
private readonly IRedisLock _distributedLock;
private readonly ILogger<CampaignScheduler> _logger;
public async Task<ScheduleResult> ScheduleCampaignAsync(
ScheduleCampaignRequest request)
{
var campaign = await _campaignRepo.GetByIdAsync(request.CampaignId);
if (campaign == null)
throw new CampaignNotFoundException(request.CampaignId);
// Prevent double-scheduling with distributed lock
var lockKey = $"campaign:schedule:{campaign.Id}";
var lockResult = await _distributedLock.AcquireAsync(
lockKey, TimeSpan.FromSeconds(30));
if (!lockResult.Acquired)
throw new ConcurrencyException(
$"Campaign {campaign.Id} is already being processed");
try
{
// Validate target list is not empty
var contactCount = await _campaignRepo
.GetMatchingContactCountAsync(campaign.ListId, campaign.SegmentFilter);
if (contactCount == 0)
throw new EmptyAudienceException(campaign.Id);
// Assign batch IDs for tracking
var batchId = Guid.NewGuid().ToString("N");
campaign.BatchId = batchId;
campaign.Status = CampaignStatus.Scheduled;
campaign.ScheduledAt = request.SendTime;
campaign.TotalContacts = contactCount;
await _campaignRepo.UpdateAsync(campaign);
// Publish scheduling event for downstream consumers
var scheduledEvent = new CampaignScheduledEvent
{
CampaignId = campaign.Id,
BatchId = batchId,
ListId = campaign.ListId,
SegmentFilter = campaign.SegmentFilter,
TemplateId = campaign.TemplateId,
ScheduledTime = request.SendTime,
TotalContacts = contactCount,
TenantId = campaign.TenantId,
CreatedAt = DateTime.UtcNow
};
await _kafkaProducer.ProduceAsync(
topic: "campaign.scheduled",
key: campaign.Id.ToString(),
value: scheduledEvent);
_logger.LogInformation(
"Campaign {CampaignId} scheduled for {SendTime}, " +
"{ContactCount} contacts, batch {BatchId}",
campaign.Id, request.SendTime, contactCount, batchId);
return new ScheduleResult
{
CampaignId = campaign.Id,
BatchId = batchId,
EstimatedSendTime = request.SendTime,
ContactCount = contactCount
};
}
finally
{
await _distributedLock.ReleaseAsync(lockKey);
}
}
}
4.3 Contact Resolution and Batching
When the scheduled time arrives, the Delivery Service consumes the campaign.scheduled event and begins contact resolution. This is the process of converting the campaign's target audience definition (a list ID plus optional segment filter) into an actual set of contact records with their personalization data.
The contact resolution process must handle several complexities. First, the target audience may include millions of contacts, but loading them all into memory simultaneously would cause out-of-memory conditions. Instead, the system streams contacts in paginated batches of 5,000–10,000 records. Second, contacts may have unsubscribed or bounced between campaign creation and send time. The system must re-validate each contact against current suppression lists. Third, personalization tokens in the email template must be resolved per-contact — merging first names, custom field values, and dynamic content blocks.
The batching strategy is critical for SMTP throughput. Sending emails one at a time incurs per-connection overhead for each SMTP transaction. Instead, the system aggregates contacts into batches of 5,000 and dispatches each batch as a single SMTP session with multiple RCPT TO commands. This reduces connection establishment overhead by orders of magnitude.
4.4 Template Personalization Engine
Template personalization is the process of resolving variable tokens in an email template with contact-specific values. GetResponse supports several personalization mechanisms: simple field merges ({{first_name}}), conditional content blocks ({{if custom_field == "value"}}...{{endif}}), and dynamic content sections that pull from external APIs.
public class TemplatePersonalizer
{
private readonly IContactRepository _contactRepo;
private readonly IDynamicContentResolver _dynamicResolver;
private readonly ITemplateCache _templateCache;
public async Task<PersonalizedEmail> PersonalizeAsync(
string templateId, ContactRecord contact, Guid campaignId)
{
var template = await _templateCache.GetTemplateAsync(templateId);
// Phase 1: Simple field merging (fast path)
var html = MergeFields(template.HtmlContent, contact);
var subject = MergeFields(template.Subject, contact);
// Phase 2: Conditional content blocks
html = ResolveConditionals(html, contact);
// Phase 3: Dynamic content (async, may call external APIs)
if (template.HasDynamicContent)
{
html = await _dynamicResolver.ResolveAsync(
html, contact, campaignId);
}
// Phase 4: Track pixel injection
var trackingPixel = GenerateTrackingPixel(campaignId, contact.Id);
html = InjectTrackingPixel(html, trackingPixel, campaignId);
// Phase 5: Click tracking rewrite
html = RewriteClickUrls(html, campaignId, contact.Id);
return new PersonalizedEmail
{
ContactId = contact.Id,
To = contact.Email,
Subject = subject,
HtmlBody = html,
TextBody = MergeFields(template.TextContent, contact),
CampaignId = campaignId,
MessageId = GenerateMessageId(campaignId, contact.Id)
};
}
private string MergeFields(string content, ContactRecord contact)
{
if (string.IsNullOrEmpty(content)) return content;
var result = content;
foreach (var field in contact.CustomFields)
{
var token = "{{" + field.Key + "}}";
result = result.Replace(token, field.Value ?? string.Empty);
}
// System fields
result = result.Replace("{{email}}", contact.Email ?? string.Empty);
result = result.Replace("{{first_name}}",
contact.Attributes.FirstName ?? string.Empty);
result = result.Replace("{{last_name}}",
contact.Attributes.LastName ?? string.Empty);
result = result.Replace("{{subscribe_date}}",
contact.CreatedAt.ToString("MMMM dd, yyyy"));
return result;
}
private string ResolveConditionals(string html, ContactRecord contact)
{
var pattern = @"\{\{if\s+(\w+)\s*==\s*""([^""]+)""\}\}(.*?)\{\{endif\}\}";
return Regex.Replace(html, pattern, match =>
{
var fieldName = match.Groups[1].Value;
var expectedValue = match.Groups[2].Value;
var content = match.Groups[3].Value;
var actualValue = contact.CustomFields
.GetValueOrDefault(fieldName, string.Empty);
return string.Equals(actualValue, expectedValue,
StringComparison.OrdinalIgnoreCase)
? content
: string.Empty;
});
}
}
4.5 Deduplication and Suppression
Before any email enters the delivery pipeline, the system must cross-reference the recipient against several suppression lists. This is non-negotiable for deliverability — sending to unsubscribed addresses triggers CAN-SPAM violations and ISP throttling. The suppression system maintains several real-time lists:
- Global Suppression List: Addresses that have hard-bounced across any GetResponse customer. This is shared across tenants and grows by approximately 50,000–100,000 entries per day.
- Tenant Suppression List: Addresses that have unsubscribed from a specific customer's lists. A contact may unsubscribe from Customer A but still be valid for Customer B.
- Complaint List: Addresses that have marked emails as spam via ISP feedback loops. These are suppressed immediately upon feedback loop receipt.
- Inactive Suppression: Optional feature where contacts who haven't opened in 12+ months are automatically suppressed to protect sender reputation.
The suppression check is implemented as a Redis Bloom filter for the fast path (positive match = definitely suppressed) with a PostgreSQL fallback for false positives. This provides sub-millisecond suppression lookups for the common case while maintaining 100% accuracy through the fallback path.
5. Contact Management and Segmentation Engine
The contact management system is arguably the most complex subsystem in GetResponse from a data modeling perspective. A single contact record may belong to multiple lists, have dozens of custom fields, carry behavioral data (opens, clicks, purchases), and be subject to real-time segmentation queries that combine demographic attributes with behavioral signals.
5.1 Contact Data Model
The contact data model must balance flexibility (customers define their own custom fields) with performance (segmentation queries must execute against billions of records in seconds). GetResponse uses a hybrid approach combining a normalized relational schema for core attributes with a JSONB column for custom fields, indexed via Elasticsearch for query performance.
-- PostgreSQL contact schema (simplified)
CREATE TABLE contacts (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tenant_id UUID NOT NULL REFERENCES tenants(id),
email VARCHAR(320) NOT NULL,
status VARCHAR(20) NOT NULL DEFAULT 'active',
-- active, unsubscribed, bounced, complained
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
last_activity_at TIMESTAMPTZ,
custom_fields JSONB NOT NULL DEFAULT '{}',
attributes JSONB NOT NULL DEFAULT '{}',
-- first_name, last_name, etc.
scoring INTEGER DEFAULT 0,
tags TEXT[] DEFAULT '{}',
UNIQUE(tenant_id, email)
);
CREATE INDEX idx_contacts_tenant_status
ON contacts(tenant_id, status);
CREATE INDEX idx_contacts_custom_fields
ON contacts USING GIN(custom_fields);
CREATE INDEX idx_contacts_tags
ON contacts USING GIN(tags);
CREATE INDEX idx_contacts_scoring
ON contacts(tenant_id, scoring DESC)
WHERE status = 'active';
-- List membership (many-to-many)
CREATE TABLE list_memberships (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
list_id UUID NOT NULL REFERENCES lists(id),
contact_id UUID NOT NULL REFERENCES contacts(id),
subscribed_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
unsubscribed_at TIMESTAMPTZ,
source VARCHAR(50) NOT NULL,
-- import, form, api, manual
UNIQUE(list_id, contact_id)
);
CREATE INDEX idx_memberships_list
ON list_memberships(list_id)
WHERE unsubscribed_at IS NULL;
-- Activity events (time-series)
CREATE TABLE contact_events (
id BIGSERIAL PRIMARY KEY,
contact_id UUID NOT NULL,
tenant_id UUID NOT NULL,
event_type VARCHAR(30) NOT NULL,
-- opened, clicked, bounced, converted
event_data JSONB NOT NULL DEFAULT '{}',
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
) PARTITION BY RANGE (created_at);
-- Monthly partitions for efficient query and archival
CREATE TABLE contact_events_2026_07
PARTITION OF contact_events
FOR VALUES FROM ('2026-07-01') TO ('2026-08-01');
5.2 Segmentation Engine Architecture
Segmentation is the process of defining a dynamic audience based on a set of conditions. A segment might be defined as: "Contacts who opened an email in the last 30 days AND clicked a link AND have custom_field 'plan' == 'enterprise' AND are NOT in list 'churned'." These conditions form a tree of boolean logic that must be evaluated against the full contact database.
GetResponse's segmentation engine translates these conditions into Elasticsearch queries. When a segment is created or edited, the system generates an Elasticsearch bool query and stores the query definition. When the segment is used for a campaign, the system executes the query and streams the resulting contact IDs into the campaign delivery pipeline.
5.3 Elasticsearch Index Strategy
Contact data is replicated from PostgreSQL to Elasticsearch via a CDC (Change Data Capture) pipeline using Debezium. The Elasticsearch index is structured with tenant-aware routing to ensure that queries are automatically scoped to a single tenant's data, preventing cross-tenant data leakage at the index level.
public class SegmentationQueryCompiler
{
private readonly IElasticClient _elasticClient;
public async Task<IAsyncEnumerable<ContactHit>> ExecuteSegmentAsync(
SegmentDefinition segment, Guid tenantId)
{
var query = CompileToElasticQuery(segment);
// Tenant-scoped search with scroll API for large result sets
var searchResponse = await _elasticClient
.SearchAsync<ContactDocument>(s => s
.Index($"contacts_tenant_{tenantId:N}")
.Size(1000)
.Scroll("2m")
.Query(q => q
.Bool(b => b
.Filter(query)
.Must(m => m
.Term(t => t
.Field(f => f.Status)
.Value("active")))
))
.Source(src => src
.Includes(i => i
.Fields(f => f.Id, f.Email, f.CustomFields))));
return ScrollResults(searchResponse.ScrollId, searchResponse.Hits);
}
private QueryContainer CompileToElasticQuery(SegmentDefinition segment)
{
var container = new QueryContainer();
foreach (var condition in segment.Conditions)
{
QueryContainer conditionQuery = condition switch
{
FieldEqualsCondition eq => new TermQuery(
FieldName(condition.Field))
{ Value = eq.Value },
FieldContainsCondition contains => new MatchQuery(
FieldName(condition.Field))
{ Query = contains.Value },
DateAfterCondition after => new DateRangeQuery(
FieldName(condition.Field))
{ GreaterThan = after.Date },
EventExistsCondition evt => new NestedQuery
{
Path = "events",
Query = new BoolQuery().Filter(
new TermQuery("events.type") { Value = evt.EventType },
new DateRangeQuery("events.timestamp")
{ GreaterThan = evt.Since })
},
TagContainsCondition tag => new TermsQuery(
FieldName("tags"))
{ Terms = new object[] { tag.TagName } },
ScoreAboveCondition score => new NumericRangeQuery(
FieldName("scoring"))
{ GreaterThan = score.MinScore },
_ => throw new NotSupportedException(
$"Condition type {condition.GetType().Name} not supported")
};
container = condition.LogicOperator switch
{
LogicOperator.And => container && conditionQuery,
LogicOperator.Or => container || conditionQuery,
_ => container && conditionQuery
};
}
return container;
}
private async IAsyncEnumerable<ContactHit> ScrollResults(
string scrollId, IReadOnlyCollection<IHit<ContactDocument>> initialHits)
{
foreach (var hit in initialHits)
{
yield return new ContactHit
{
ContactId = hit.Source.Id,
Email = hit.Source.Email
};
}
while (!string.IsNullOrEmpty(scrollId))
{
var scrollResponse = await _elasticClient
.ScrollAsync<ContactDocument>("2m", scrollId);
if (!scrollResponse.Hits.Any()) break;
foreach (var hit in scrollResponse.Hits)
{
yield return new ContactHit
{
ContactId = hit.Source.Id,
Email = hit.Source.Email
};
}
scrollId = scrollResponse.ScrollId;
}
if (!string.IsNullOrEmpty(scrollId))
await _elasticClient.ClearScrollAsync(scrollId);
}
}
5.4 Contact Scoring
Contact scoring assigns a numeric value to each contact based on their engagement history. High scores indicate engaged contacts who are likely to convert; low scores indicate inactive or disengaged contacts. The scoring model considers:
| Signal | Weight | Decay |
|---|---|---|
| Email opened | +5 | Halves every 30 days |
| Link clicked | +15 | Halves every 21 days |
| Website visited (via tracking) | +10 | Halves every 14 days |
| Form submitted | +25 | Halves every 60 days |
| Purchase completed | +50 | Halves every 90 days |
| Email bounced | -100 | Permanent |
| No activity 90+ days | -20 | Compounding |
| Marked as spam | -200 | Permanent |
Scoring is recalculated daily via a batch job that processes all active contacts. The job runs on a partitioned schedule — contacts are partitioned by tenant_id % 30, with one partition processed per day, ensuring full recalculation within a 30-day window while keeping daily batch size manageable.
6. Email Template Builder System Design
The drag-and-drop email template builder is one of GetResponse's most user-facing features and presents unique engineering challenges. It must provide a WYSIWYG editing experience in the browser, generate valid HTML email markup that renders correctly across hundreds of email clients (including Outlook, Gmail, Apple Mail, and various mobile clients), and store templates in a format that supports both editing and rendering efficiently.
6.1 Block-Based Editor Architecture
The template builder uses a block-based editor model where users assemble emails from pre-defined content blocks: text, image, button, divider, social links, video, product grid, and custom HTML. Each block has a configuration schema, a rendering template, and responsive behavior rules.
The block tree is stored as a JSON document. Each block contains a type, a config object with block-specific settings, and an optional children array for container blocks (columns, sections). This JSON representation serves as the source of truth — the HTML email is a derived artifact produced by the rendering engine.
6.2 Email HTML Rendering Challenges
Email HTML is not web HTML. Email clients support a severely limited subset of HTML and CSS. Outlook uses Word's HTML rendering engine. Gmail strips <style> tags from the <head> and inlines all CSS. Mobile clients add their own viewport handling. The template renderer must produce HTML that works across all these environments.
GetResponse's renderer applies several transformations to convert the block tree into valid email HTML:
- CSS Inlining: All CSS styles are inlined into element
styleattributes, since many clients strip<style>blocks. - Table-Based Layout: Multi-column layouts use nested
<table>elements rather than CSS Grid or Flexbox, which are unsupported in most email clients. - VML for Outlook: Outlook-specific features (background images, rounded corners) use Vector Markup Language (VML) fallbacks wrapped in conditional comments.
- Responsive Breakpoints: Media queries are included but supplemented with fluid tables that use
max-width: 100%and percentage-based widths. - Image Handling: All images are uploaded to the CDN and served via HTTPS. Alt text is required for accessibility and as fallback for image-blocking clients.
public class EmailHtmlRenderer
{
private readonly IBlockRendererRegistry _renderers;
private readonly ICssInliner _cssInliner;
private readonly IImageService _imageService;
public RenderedEmail Render(TemplateBlockTree blockTree,
RenderContext context)
{
var sb = new StringBuilder();
// Email document wrapper with MSO conditionals
sb.AppendLine("<!--[if mso]><xml>");
sb.AppendLine("<o:OfficeDocumentSettings>");
sb.AppendLine("<o:AllowPNG/>");
sb.AppendLine("<o:PixelsPerInch>96</o:PixelsPerInch>");
sb.AppendLine("</o:OfficeDocumentSettings></xml><![endif]-->");
sb.AppendLine("<table role=\"presentation\" cellspacing=\"0\" ");
sb.AppendLine("cellpadding=\"0\" border=\"0\" width=\"100%\" ");
sb.AppendLine("style=\"background-color:#f4f4f4;\">");
sb.AppendLine("<tr><td align=\"center\" style=\"padding:20px 0;\">");
// Main content container
sb.AppendLine("<table role=\"presentation\" cellspacing=\"0\" ");
sb.AppendLine("cellpadding=\"0\" border=\"0\" width=\"600\" ");
sb.AppendLine("style=\"background-color:#ffffff;\">");
foreach (var block in blockTree.Blocks)
{
var renderer = _renderers.GetRenderer(block.Type);
var blockHtml = renderer.Render(block, context);
sb.AppendLine(blockHtml);
}
sb.AppendLine("</table>"); // Close container
sb.AppendLine("</td></tr></table>"); // Close wrapper
var html = sb.ToString();
// Apply CSS inlining pass
html = _cssInliner.Process(html);
return new RenderedEmail
{
Html = html,
Preheader = blockTree.Metadata.Preheader,
Subject = blockTree.Metadata.Subject,
EstimatedSize = Encoding.UTF8.GetByteCount(html)
};
}
}
public interface IBlockRenderer
{
string BlockType { get; }
string Render(TemplateBlock block, RenderContext context);
}
public class ButtonBlockRenderer : IBlockRenderer
{
public string BlockType => "button";
public string Render(TemplateBlock block, RenderContext context)
{
var config = block.Config.Deserialize<ButtonConfig>();
var bgColor = config.BackgroundColor ?? "#0088ff";
var textColor = config.TextColor ?? "#ffffff";
var borderRadius = config.BorderRadius ?? 4;
// MSO button fallback (Outlook doesn't support border-radius)
var msoButton = $@"
<!--[if mso]>
<v:roundrect xmlns:v=""urn:schemas-microsoft-com:vml""
xmlns:w=""urn:schemas-microsoft-com:office:word""
href=""{context.TrackClickUrl(config.Url, block.Id)}""
style=""height:44px;v-text-anchor:middle;width:220px;""
arcsize=""{(borderRadius * 100) / 44}%""
stroke=""f"" fillcolor=""{bgColor}"">
<w:anchorlock/><center>
<![endif]-->";
return $@"
<tr><td align=""center"" style=""padding:10px 20px;"">
<table role=""presentation"" cellspacing=""0"" cellpadding=""0"">
<tr><td style=""border-radius:{borderRadius}px;
background-color:{bgColor};"">
{msoButton}
<a href=""{context.TrackClickUrl(config.Url, block.Id)}""
style=""background-color:{bgColor};
color:{textColor};
display:inline-block;
font-family:Verdana,sans-serif;
font-size:14px;
font-weight:bold;
line-height:44px;
text-align:center;
text-decoration:none;
border-radius:{borderRadius}px;
padding:0 30px;"">
{config.Text}
</a>
<!--[if mso]></center></v:roundrect><![endif]-->
</td></tr></table>
</td></tr>";
}
}
6.3 Template Versioning and Storage
Templates are stored as versioned JSON documents in PostgreSQL with the rendered HTML cached in Redis. When a template is saved, the system creates a new version, renders the HTML, and invalidates the cache. This ensures that in-flight campaigns using an older template version are not affected by edits to draft templates.
7. SMTP Relay and Email Delivery Infrastructure
The SMTP delivery infrastructure is the heart of GetResponse's operational reliability. This subsystem must deliver millions of emails per day with 99%+ inbox placement rates, handle bounces and complaints in real-time, manage sending reputation across hundreds of IP addresses and domains, and adapt to ISP throttling dynamically.
7.1 SMTP Relay Cluster Architecture
GetResponse operates a distributed SMTP relay cluster that abstracts the complexity of multi-IP, multi-domain email sending. The cluster comprises multiple relay nodes, each with its own set of outbound IP addresses and HELO domains. The relay layer implements a sophisticated routing algorithm that selects the optimal IP/domain combination for each email based on the recipient domain's current throttling state and historical reputation.
7.2 Smart Routing Algorithm
The smart router maintains real-time health scores for each IP/domain pair, segmented by recipient domain. For example, IP pool A may have an excellent reputation with Gmail but a degraded reputation with Yahoo due to recent bounce spikes. The routing algorithm uses a multi-armed bandit approach, balancing exploitation (using known-good IP/domain pairs) with exploration (testing new combinations) to maintain optimal delivery rates.
public class SmartRouter
{
private readonly IReputationStore _reputationStore;
private readonly IRedisCache _cache;
private readonly ILogger<SmartRouter> _logger;
public async Task<RoutingDecision> RouteAsync(
OutboundEmail email, CancellationToken ct)
{
var recipientDomain = ExtractDomain(email.To);
var recipientMx = await ResolveMxRecordAsync(recipientDomain);
// Get health scores for all available pools
var poolScores = await _reputationStore
.GetPoolScoresAsync(recipientDomain);
// Filter out pools that are throttled or below minimum score
var eligiblePools = poolScores
.Where(p => p.Score >= MinReputationScore
&& !p.IsThrottled
&& p.CurrentVolume < p.MaxVolume)
.OrderByDescending(p => p.Score)
.ThenBy(p => p.CurrentVolume) // Prefer less loaded
.ToList();
if (!eligiblePools.Any())
{
_logger.LogWarning(
"No eligible pools for domain {Domain}, " +
"falling back to default pool", recipientDomain);
eligiblePools = poolScores
.Where(p => !p.IsBlacklisted)
.Take(1)
.ToList();
}
var selectedPool = eligiblePools.First();
// Apply warming logic for new IPs
if (selectedPool.IsWarming)
{
var warmupLimit = CalculateWarmupLimit(selectedPool);
if (selectedPool.CurrentVolume >= warmupLimit)
{
selectedPool = eligiblePools
.FirstOrDefault(p => !p.IsWarming)
?? selectedPool;
}
}
// Track routing decision
await _reputationStore.IncrementVolumeAsync(
selectedPool.PoolId, recipientDomain);
return new RoutingDecision
{
PoolId = selectedPool.PoolId,
SmtpEndpoint = selectedPool.SmtpEndpoint,
MailFrom = selectedPool.MailFromAddress,
DkimDomain = selectedPool.DkimDomain,
Priority = selectedPool.Score
};
}
private string ExtractDomain(string email)
{
var atIndex = email.LastIndexOf('@');
return atIndex >= 0 ? email.Substring(atIndex + 1).ToLower() : email;
}
}
7.3 IP Warmup Management
New IP addresses must be gradually warmed up before they can handle full volume. ISPs track sending history per IP and will throttle or block IPs that suddenly send large volumes. GetResponse's warmup system follows a structured ramp-up schedule:
| Day | Max Emails/Day | Target ISPs | Requirements |
|---|---|---|---|
| 1–3 | 500 | Gmail, Outlook (engaged contacts only) | Bounce rate < 2%, complaint rate < 0.05% |
| 4–7 | 2,000 | Gmail, Outlook, Yahoo | Bounce rate < 2%, open rate > 15% |
| 8–14 | 10,000 | All major ISPs | Bounce rate < 1.5%, complaint rate < 0.03% |
| 15–21 | 25,000 | All ISPs | Consistent engagement metrics |
| 22–30 | 50,000 | All ISPs | No throttling events in 7 days |
| 31+ | Full volume | All ISPs | Stable reputation score > 85 |
7.4 Bounce and Complaint Processing
Bounce processing is critical for maintaining sender reputation. The system categorizes bounces into three types:
- Hard Bounces (5xx permanent failures): Invalid address, domain not found, mailbox full (some providers). Contacts are immediately marked as bounced and added to the global suppression list.
- Soft Bounces (4xx temporary failures): Server temporarily unavailable, message size exceeded, rate limiting. Messages are retried with exponential backoff — 5 minutes, 15 minutes, 1 hour, 4 hours — before being classified as undeliverable.
- Complaints (FBL responses): Recipient clicked "Mark as Spam." Contact is immediately suppressed and the event is used to adjust IP reputation scores. High complaint rates (> 0.1%) trigger automatic sending pauses.
public class BounceProcessor : BackgroundService
{
private readonly IKafkaConsumer _consumer;
private readonly IBounceRepository _bounceRepo;
private readonly ISuppressionList _suppressionList;
private readonly IReputationStore _reputationStore;
private readonly IEmailQueue _retryQueue;
protected override async Task ExecuteAsync(CancellationToken ct)
{
var consumerConfig = new ConsumerConfig
{
GroupId = "bounce-processor",
AutoOffsetReset = AutoOffsetReset.Latest,
EnableAutoCommit = false
};
using var consumer = _consumer.Create(consumerConfig);
consumer.Subscribe("email.bounced");
while (!ct.IsCancellationRequested)
{
try
{
var result = consumer.Consume(ct);
var bounceEvent = JsonSerializer
.Deserialize<BounceEvent>(result.Message.Value);
await ProcessBounceAsync(bounceEvent);
consumer.Commit(result);
}
catch (ConsumeException ex)
{
_logger.LogError(ex, "Error consuming bounce event");
}
}
}
private async Task ProcessBounceAsync(BounceEvent bounce)
{
var bounceType = ClassifyBounce(
bounce.SmtpCode, bounce.SmtpMessage);
switch (bounceType)
{
case BounceType.Hard:
// Immediate suppression
await _suppressionList.AddAsync(
bounce.RecipientEmail,
SuppressionReason.HardBounce,
bounce.TenantId);
// Update contact status
await _bounceRepo.MarkBouncedAsync(
bounce.ContactId, bounce.CampaignId);
// Update IP reputation
await _reputationStore.RecordBounceAsync(
bounce.PoolId, bounce.RecipientDomain, isHard: true);
_logger.LogInformation(
"Hard bounce: {Email} from campaign {CampaignId}",
bounce.RecipientEmail, bounce.CampaignId);
break;
case BounceType.Soft:
// Schedule retry with backoff
var retryCount = bounce.RetryCount + 1;
if (retryCount <= MaxSoftRetries)
{
var delay = CalculateBackoff(retryCount);
await _retryQueue.EnqueueAsync(
bounce.MessageId,
delay,
retryCount);
}
else
{
// Exceeded retries, treat as permanent
await _suppressionList.AddAsync(
bounce.RecipientEmail,
SuppressionReason.SoftBounceExhausted,
bounce.TenantId);
}
await _reputationStore.RecordBounceAsync(
bounce.PoolId, bounce.RecipientDomain, isHard: false);
break;
case BounceType.Complaint:
await _suppressionList.AddAsync(
bounce.RecipientEmail,
SuppressionReason.Complaint,
bounce.TenantId);
// Critical: Check complaint rate for the sending domain
var complaintRate = await _reputationStore
.GetComplaintRateAsync(bounce.PoolId,
TimeSpan.FromHours(24));
if (complaintRate > 0.001) // 0.1% threshold
{
_logger.LogCritical(
"ALERT: Pool {PoolId} complaint rate {Rate:P3} " +
"exceeds threshold. Pausing pool.",
bounce.PoolId, complaintRate);
await _reputationStore
.PausePoolAsync(bounce.PoolId,
PauseReason.HighComplaintRate);
}
break;
}
}
private BounceType ClassifyBounce(int smtpCode, string message)
{
return smtpCode switch
{
550 or 551 or 552 or 553 or 554 => BounceType.Hard,
421 or 450 or 451 or 452 => BounceType.Soft,
_ when message.Contains("spam", StringComparison.OrdinalIgnoreCase)
=> BounceType.Complaint,
_ when smtpCode >= 500 => BounceType.Hard,
_ => BounceType.Soft
};
}
}
8. Deliverability: SPF, DKIM, DMARC, BIMI
Email deliverability is governed by a constellation of authentication protocols that collectively verify that an email was authorized by the domain owner. GetResponse must configure and maintain these protocols across hundreds of sending domains (both shared and dedicated), making this a significant operational challenge.
8.1 Authentication Protocol Overview
| Protocol | Purpose | DNS Record Type | Failure Consequence |
|---|---|---|---|
| SPF | Authorizes sending IPs for a domain | TXT | Messages may be rejected or sent to spam |
| DKIM | Cryptographically signs email content | TXT | Signature verification fails, spam filtering |
| DMARC | Policy for SPF/DKIM alignment | TXT | Varies: none/quarantine/reject |
| BIMI | Brand logo display in inbox | TXT | No brand logo (requires DMARC p=quarantine or reject) |
8.2 DKIM Signing Architecture
DKIM (DomainKeys Identified Mail) signs each outgoing email with a private key. Receiving servers verify the signature using the public key published in DNS. GetResponse rotates DKIM keys periodically (every 90 days) and uses 2048-bit RSA keys. The signing process occurs in the SMTP relay layer, just before message handoff to the receiving MTA.
public class DkimSigner
{
private readonly IDkimKeyStore _keyStore;
private readonly ISystemClock _clock;
public async Task<SignedMessage> SignAsync(
MimeMessage message, string domain, string selector)
{
var keyPair = await _keyStore.GetKeyAsync(domain, selector);
// Canonicalization: use relaxed for both headers and body
// as it is more forgiving of minor modifications by intermediaries
var canonicalizer = new RelaxedCanonicalizer();
// Headers to sign (in order of importance)
var headersToSign = new[]
{
"From", "To", "Subject", "Date",
"Message-ID", "MIME-Version", "Content-Type"
};
// Compute body hash
var bodyHash = canonicalizer.CanonicalizeBody(message);
// Compute header hash
var headerHash = canonicalizer.CanonicalizeHeaders(
message, headersToSign);
// Sign
using var rsa = RSA.Create();
rsa.ImportPkcs8PrivateKey(keyPair.PrivateKeyBytes, out _);
var signature = rsa.SignData(
headerHash,
HashAlgorithmName.SHA256,
RSASignaturePadding.Pkcs1);
var signatureBase64 = Convert.ToBase64String(signature);
// Build DKIM-Signature header
var dkimHeader = new Header("DKIM-Signature",
$"v=1; a=rsa-sha256; d={domain}; s={selector};" +
$" c=relaxed/relaxed;" +
$" q=dns/txt; h={string.Join(":", headersToSign)};" +
$" bh={Convert.ToBase64String(bodyHash)};" +
$" b={signatureBase64}");
message.Headers.Insert(0, dkimHeader);
return new SignedMessage
{
MimeMessage = message,
Domain = domain,
Selector = selector,
SignedAt = _clock.UtcNow
};
}
}
8.3 IP and Domain Reputation Monitoring
GetResponse maintains sending reputation across multiple dimensions. Each sending IP address is tracked individually, as well as in aggregate per IP class C subnet. Domain reputation is tracked per sending domain. The reputation system ingests data from multiple sources: ISP feedback loops, bounce logs, Google Postmaster Tools API, Microsoft SNDS (Smart Network Data Services), and third-party reputation monitoring services like SenderScore and Talos Intelligence.
The reputation score is a composite metric weighted as follows: bounce rate (30%), complaint rate (30%), engagement rate (20%), volume consistency (10%), and authentication pass rate (10%). When any component drops below threshold, the system automatically triggers mitigation actions — reducing volume, shifting traffic to healthier IPs, or pausing the affected IPs entirely.
9. Open/Click Tracking and Event Processing
Email tracking is the system that records when recipients open emails or click links. This data drives engagement analytics, contact scoring, segmentation, and marketing automation triggers. The tracking infrastructure must handle extreme read-heavy workloads — a campaign sent to 10 million contacts generates 10 million potential open events and potentially tens of millions of click events, all concentrated within hours of send time.
9.1 Open Tracking Mechanism
Open tracking works by embedding a transparent 1x1 pixel image in each email. When the recipient's email client loads the image, it sends an HTTP GET request to the tracking server, which logs the open event. The tracking URL encodes the campaign ID, contact ID, and message ID as encrypted parameters.
Click tracking works by rewriting all links in the email to pass through the tracking server. When a recipient clicks a link, the tracking server logs the click event and issues an HTTP 302 redirect to the original URL. The redirect is processed in under 50 milliseconds to avoid perceptible latency for the recipient.
9.2 High-Performance Tracking Pipeline
The tracking pipeline must process millions of events per minute during peak campaign send windows. The design prioritizes write throughput over immediate consistency — it is acceptable for real-time counters to lag by a few seconds in exchange for handling the full event volume without dropping data.
[ApiController]
[Route("track")]
public class TrackingController : ControllerBase
{
private readonly ITrackingService _trackingService;
private readonly IKafkaProducer _kafkaProducer;
[HttpGet("open.gif")]
[ResponseCache(NoStore = true, Duration = 0)]
public async Task TrackOpen(
[FromQuery] string mid, // message ID (encrypted)
[FromQuery] string cid, // contact ID (encrypted)
[FromQuery] string cmp) // campaign ID
{
// Decode and validate parameters
var decoded = _trackingService.DecodeTrackingParams(mid, cid, cmp);
// Build tracking event
var trackingEvent = new TrackingEvent
{
EventType = TrackingEventType.Open,
MessageId = decoded.MessageId,
ContactId = decoded.ContactId,
CampaignId = decoded.CampaignId,
UserAgent = Request.Headers.UserAgent.ToString(),
IpAddress = GetClientIp(),
Timestamp = DateTime.UtcNow
};
// Fire-and-forget to Kafka for async processing
await _kafkaProducer.ProduceAsync(
topic: "tracking.events",
key: decoded.MessageId,
value: trackingEvent);
// Increment real-time counter in Redis (non-critical path)
_ = Task.Run(async () =>
{
await IncrementOpenCountAsync(decoded.CampaignId);
await UpdateContactLastActivityAsync(decoded.ContactId);
});
// Return 1x1 transparent GIF
var pixel = TransparentPixel();
return File(pixel, "image/gif");
}
[HttpGet("click/{encodedUrl}")]
[ResponseCache(NoStore = true, Duration = 0)]
public async Task TrackClick(string encodedUrl)
{
var decoded = _trackingService.DecodeClickUrl(encodedUrl);
var trackingEvent = new TrackingEvent
{
EventType = TrackingEventType.Click,
MessageId = decoded.MessageId,
ContactId = decoded.ContactId,
CampaignId = decoded.CampaignId,
OriginalUrl = decoded.OriginalUrl,
LinkId = decoded.LinkId,
UserAgent = Request.Headers.UserAgent.ToString(),
IpAddress = GetClientIp(),
Timestamp = DateTime.UtcNow
};
await _kafkaProducer.ProduceAsync(
topic: "tracking.events",
key: decoded.MessageId,
value: trackingEvent);
// Redirect to original URL with minimal latency
Response.StatusCode = 302;
Response.Headers.Location = decoded.OriginalUrl;
}
private byte[] TransparentPixel() => new byte[]
{
0x47, 0x49, 0x46, 0x38, 0x39, 0x61, 0x01, 0x00,
0x01, 0x00, 0x80, 0x00, 0x00, 0xff, 0xff, 0xff,
0x00, 0x00, 0x00, 0x21, 0xf9, 0x04, 0x01, 0x00,
0x00, 0x00, 0x00, 0x2c, 0x00, 0x00, 0x00, 0x00,
0x01, 0x00, 0x01, 0x00, 0x00, 0x02, 0x02, 0x44,
0x01, 0x00, 0x3b
};
}
9.3 Tracking Accuracy Challenges
Email open tracking has inherent accuracy limitations. Apple's Mail Privacy Protection (introduced in iOS 15, 2021) pre-fetches all email content including tracking pixels through proxy servers, inflating open rates and masking true engagement. Gmail's image proxy similarly complicates open tracking. GetResponse must account for these factors in analytics by:
- Flagging opens from known proxy IP ranges (Apple's 17.x.x.x range) as "proxy opens"
- Weighting click events more heavily than opens for engagement scoring
- Using link click as the primary engagement signal for automation triggers
- Providing customers with adjusted open rate metrics that account for proxy inflations
Click tracking remains more reliable than open tracking because clicks involve user interaction and are harder to proxy automatically. GetResponse has shifted its engagement model toward click-based metrics as the primary engagement signal, with opens treated as a secondary, less reliable indicator.
10. Marketing Automation Workflows and Journeys
Marketing automation is the capability that transforms GetResponse from a bulk email tool into a sophisticated marketing platform. Automation workflows (called "autoresponders" in legacy terminology and "marketing automation" for the newer visual builder) allow customers to create event-driven email sequences that respond to subscriber behavior in real-time.
10.1 Workflow Engine Architecture
The automation engine processes millions of active workflows concurrently. Each workflow consists of a trigger, a series of actions (send email, wait, condition check, score update, tag add/remove), and optional exit conditions. The engine must handle time-based triggers (wait 3 days), event-based triggers (contact clicked link), and conditional branching (if opened then path A else path B).
10.2 Workflow Execution Engine
The workflow engine uses a state machine approach where each contact's progress through a workflow is tracked independently. This allows millions of contacts to be at different stages of the same workflow simultaneously. The engine processes workflow steps as discrete tasks queued in Kafka, with time-based waits implemented via delayed message delivery.
public class WorkflowExecutor
{
private readonly IWorkflowRepository _workflowRepo;
private readonly IWorkflowStateStore _stateStore;
private readonly IKafkaProducer _kafkaProducer;
private readonly IActionExecutorRegistry _actionExecutors;
public async Task<WorkflowResult> ExecuteStepAsync(
WorkflowStepExecutionRequest request)
{
var workflow = await _workflowRepo
.GetWorkflowAsync(request.WorkflowId);
var contact = await _contactRepo
.GetContactAsync(request.ContactId);
var step = workflow.Steps.FirstOrDefault(
s => s.Id == request.StepId);
if (step == null)
throw new WorkflowStepNotFoundException(
request.WorkflowId, request.StepId);
// Execute the current step
var stepResult = await ExecuteStepAsync(
step, workflow, contact, request.Context);
// Determine next step based on result
var nextStep = DetermineNextStep(
step, stepResult, workflow);
if (nextStep != null)
{
if (nextStep.Type == StepType.Wait)
{
// Schedule future execution
var waitDuration = CalculateWaitDuration(
nextStep.WaitConfig, contact);
var executeAt = DateTime.UtcNow.Add(waitDuration);
await _kafkaProducer.ProduceAsync(
topic: "workflow.schedule",
key: $"{request.ContactId}:{nextStep.Id}",
value: new WorkflowScheduledEvent
{
WorkflowId = request.WorkflowId,
ContactId = request.ContactId,
StepId = nextStep.Id,
ExecuteAt = executeAt,
Context = request.Context
},
timestamp: new Timestamp(executeAt));
}
else
{
// Immediate execution - enqueue for next step
await _kafkaProducer.ProduceAsync(
topic: "workflow.execute",
key: $"{request.ContactId}:{nextStep.Id}",
value: new WorkflowStepExecutionRequest
{
WorkflowId = request.WorkflowId,
ContactId = request.ContactId,
StepId = nextStep.Id,
Context = request.Context
});
}
}
// Update contact's workflow position
await _stateStore.UpdatePositionAsync(
request.WorkflowId,
request.ContactId,
nextStep?.Id);
return new WorkflowResult
{
StepCompleted = step.Id,
NextStepId = nextStep?.Id,
ActionTaken = stepResult.ActionDescription,
Completed = nextStep == null
};
}
private async Task<StepExecutionResult> ExecuteStepAsync(
WorkflowStep step, AutomationWorkflow workflow,
ContactRecord contact, WorkflowContext context)
{
var executor = _actionExecutors.GetExecutor(step.Type);
return step.Type switch
{
StepType.SendEmail => await executor.ExecuteAsync(
new SendEmailAction
{
TemplateId = step.Config.TemplateId,
ContactId = contact.Id,
CampaignId = workflow.CampaignId,
FromName = step.Config.FromName,
FromEmail = step.Config.FromEmail,
Subject = step.Config.Subject
}),
StepType.Condition => await EvaluateConditionAsync(
step.Config.Condition, contact, context),
StepType.UpdateScore => await executor.ExecuteAsync(
new ScoreUpdateAction
{
ContactId = contact.Id,
Delta = step.Config.ScoreDelta
}),
StepType.AddTag => await executor.ExecuteAsync(
new TagAction
{
ContactId = contact.Id,
Tag = step.Config.TagName,
Operation = TagOperation.Add
}),
StepType.Webhook => await executor.ExecuteAsync(
new WebhookAction
{
Url = step.Config.WebhookUrl,
Method = step.Config.HttpMethod,
Headers = step.Config.Headers,
Body = BuildWebhookPayload(contact, context)
}),
_ => throw new NotSupportedException(
$"Step type {step.Type} not supported")
};
}
}
10.3 Latency Requirements
Marketing automation actions have different latency requirements depending on the trigger type:
| Trigger Type | Max Latency | Implementation |
|---|---|---|
| Immediate (tag added, form submit) | < 5 seconds | Kafka immediate consumption |
| Event-based (link click) | < 30 seconds | Kafka near-real-time processing |
| Time-delayed (wait 3 days) | ± 5 minutes of target | Delayed Kafka messages |
| Date-based (birthday, renewal) | Within configured send window | Daily batch scan + scheduling |
11. A/B Testing Engine
A/B testing allows marketers to test different versions of their emails to determine which performs better. GetResponse supports A/B testing on subject lines, sender names, content, and send times. The testing engine must handle statistical significance calculations, winner selection, and the operational complexity of running multiple test variants simultaneously.
11.1 A/B Test Architecture
An A/B test splits the target audience into variants (A, B, and optionally C, D, E), sends each variant to a subset of contacts, measures performance over a configurable evaluation window, and then automatically sends the winning variant to the remaining contacts. The system must ensure that each contact receives exactly one variant and that the evaluation is statistically rigorous.
public class AbTestEngine
{
private readonly ICampaignRepository _campaignRepo;
private readonly IStatisticalAnalyzer _statsAnalyzer;
private readonly IKafkaProducer _kafkaProducer;
public async Task<AbTestResult> EvaluateAndDeclareWinnerAsync(
Guid testCampaignId)
{
var testCampaign = await _campaignRepo
.GetAbTestCampaignAsync(testCampaignId);
var evaluationWindow = TimeSpan.FromHours(
testCampaign.Config.EvaluationHours);
// Collect metrics for each variant
var variantMetrics = new List<VariantMetrics>();
foreach (var variant in testCampaign.Variants)
{
var metrics = await CalculateVariantMetricsAsync(
variant.Id, evaluationWindow);
variantMetrics.Add(metrics);
}
// Determine the primary metric for comparison
var primaryMetric = testCampaign.Config.PrimaryMetric;
// open_rate, click_rate, conversion_rate, revenue_per_email
var winner = primaryMetric switch
{
"open_rate" => _statsAnalyzer.DetermineWinner(
variantMetrics.Select(v => new StatisticalSample
{
VariantId = v.VariantId,
Successes = v.UniqueOpens,
Trials = v.Delivered
}).ToList(), ConfidenceLevel: 0.95),
"click_rate" => _statsAnalyzer.DetermineWinner(
variantMetrics.Select(v => new StatisticalSample
{
VariantId = v.VariantId,
Successes = v.UniqueClicks,
Trials = v.Delivered
}).ToList(), ConfidenceLevel: 0.95),
"conversion_rate" => _statsAnalyzer.DetermineWinner(
variantMetrics.Select(v => new StatisticalSample
{
VariantId = v.VariantId,
Successes = v.Conversions,
Trials = v.Delivered
}).ToList(), ConfidenceLevel: 0.95),
_ => throw new NotSupportedException(
$"Metric {primaryMetric} not supported")
};
if (winner.HasWinner)
{
_logger.LogInformation(
"A/B test {TestId} declared winner: variant {VariantId} " +
"with {MetricValue:P2} {MetricName} " +
"(confidence: {Confidence:P1})",
testCampaignId, winner.WinningVariantId,
winner.WinningRate, primaryMetric,
winner.ConfidenceLevel);
// Send winning variant to remaining contacts
await SendWinnerToRemainingAsync(
testCampaign, winner.WinningVariantId);
// Record results for analytics
await RecordTestResultsAsync(testCampaign, variantMetrics, winner);
}
else
{
_logger.LogInformation(
"A/B test {TestId} has no clear winner at " +
"evaluation window. Extending or declaring draw.",
testCampaignId);
// Option 1: Extend evaluation window
// Option 2: Declare draw, send default variant
// Option 3: Send all variants with lowest unsubscribe rate
}
return new AbTestResult
{
TestCampaignId = testCampaignId,
Winner = winner,
VariantMetrics = variantMetrics
};
}
}
11.2 Statistical Methodology
GetResponse uses a two-proportion z-test for comparing variant performance. The test evaluates whether the difference in conversion rates between two variants is statistically significant at the 95% confidence level. For multi-variant tests (A/B/C), the system applies Bonferroni correction to maintain the overall family-wise error rate at 5%.
| Test Scenario | Min Sample Size | Min Duration | Statistical Test |
|---|---|---|---|
| Subject Line A/B | 1,000 per variant | 2 hours | Two-proportion z-test |
| Content A/B | 2,000 per variant | 4 hours | Two-proportion z-test |
| Send Time A/B | 5,000 per variant | 24 hours | Chi-squared test |
| A/B/C/D (4 variants) | 5,000 per variant | 6 hours | Chi-squared + post-hoc z-tests |
| Revenue per email | 2,000 per variant | 48 hours | Welch's t-test |
Minimum sample sizes ensure that even small differences in performance (e.g., 15.0% vs 15.5% open rate) can be detected with adequate statistical power. If the evaluation window ends before sufficient sample size is reached, the system extends the window automatically, up to a configured maximum.
12. Landing Page and Form Builder
GetResponse includes a landing page builder and form builder that integrate directly with the email marketing platform. Landing pages are hosted on GetResponse's infrastructure and serve as conversion points for marketing campaigns. Forms can be embedded on external websites or hosted on GetResponse domains.
12.1 Landing Page Architecture
Landing pages use a similar block-based editor as the email template builder but output standard web HTML rather than email-optimized HTML. The rendering pipeline is simpler since modern browser standards apply, but the hosting infrastructure must handle significant traffic spikes when campaigns drive traffic to landing pages.
Landing pages are pre-rendered to static HTML and served via CDN. When a user edits a landing page, the system generates the HTML, stores it in object storage (S3), and invalidates CDN cache. This ensures fast page loads without requiring server-side rendering at request time.
Form submissions trigger real-time events that flow through the same Kafka-based event pipeline as email tracking events. A form submission creates a new contact record (or updates an existing one), adds the contact to the specified list, and triggers any associated automation workflows.
12.2 Form Processing Pipeline
Forms must handle GDPR consent recording, double opt-in workflows, and anti-spam measures (CAPTCHA, honeypot fields). Each form submission is validated server-side, checked against existing contacts, and processed through the consent management pipeline before any automation triggers fire.
13. Webhook Integration System
Webhooks allow GetResponse customers to receive real-time HTTP callbacks when specific events occur — contact subscribed, email opened, link clicked, campaign completed, etc. The webhook system must handle delivery reliability, retry logic, payload security, and per-customer rate limiting.
13.1 Webhook Dispatcher Architecture
The webhook dispatcher consumes events from Kafka and delivers them as HTTP POST requests to customer-configured endpoints. The system guarantees at-least-once delivery and provides a webhook log for debugging failed deliveries.
public class WebhookDispatcher : BackgroundService
{
private readonly IKafkaConsumer _consumer;
private readonly IHttpClientFactory _httpClientFactory;
private readonly IWebhookRepository _webhookRepo;
private readonly IRetryPolicy _retryPolicy;
private readonly ILogger<WebhookDispatcher> _logger;
protected override async Task ExecuteAsync(CancellationToken ct)
{
while (!ct.IsCancellationRequested)
{
var result = _consumer.Consume(ct);
var webhookEvent = JsonSerializer
.Deserialize<WebhookPayload>(result.Message.Value);
var subscriptions = await _webhookRepo
.GetSubscriptionsAsync(
webhookEvent.TenantId,
webhookEvent.EventType);
foreach (var subscription in subscriptions)
{
await DeliverWebhookAsync(subscription, webhookEvent);
}
}
}
private async Task DeliverWebhookAsync(
WebhookSubscription subscription,
WebhookPayload payload)
{
var body = JsonSerializer.Serialize(new
{
event_type = payload.EventType,
timestamp = payload.Timestamp,
data = payload.Data,
webhook_id = Guid.NewGuid(),
delivery_attempt = payload.Attempt
});
// Compute HMAC signature for payload verification
var signature = ComputeHmac(
body, subscription.Secret);
var request = new HttpRequestMessage(
HttpMethod.Post, subscription.Url)
{
Content = new StringContent(body,
Encoding.UTF8, "application/json")
};
request.Headers.Add("X-Webhook-Signature", signature);
request.Headers.Add("X-Webhook-Event", payload.EventType);
request.Headers.Add("X-Delivery-Id",
payload.DeliveryId.ToString());
var stopwatch = Stopwatch.StartNew();
try
{
using var client = _httpClientFactory.CreateClient();
client.Timeout = TimeSpan.FromSeconds(10);
var response = await client.SendAsync(request);
stopwatch.Stop();
var success = response.IsSuccessStatusCode;
await LogDeliveryAsync(new WebhookDeliveryLog
{
SubscriptionId = subscription.Id,
Url = subscription.Url,
StatusCode = (int)response.StatusCode,
ResponseTime = stopwatch.ElapsedMilliseconds,
Success = success,
RequestBody = body,
Attempt = payload.Attempt
});
if (!success && IsRetryable(response.StatusCode))
{
await ScheduleRetryAsync(subscription, payload);
}
// Update endpoint health score
await UpdateEndpointHealthAsync(
subscription.Id, success, stopwatch.ElapsedMilliseconds);
}
catch (TaskCanceledException)
{
_logger.LogWarning(
"Webhook timeout for {Url}, attempt {Attempt}",
subscription.Url, payload.Attempt);
await ScheduleRetryAsync(subscription, payload);
}
catch (HttpRequestException ex)
{
_logger.LogError(ex,
"Webhook delivery failed for {Url}",
subscription.Url);
await ScheduleRetryAsync(subscription, payload);
}
}
private async Task ScheduleRetryAsync(
WebhookSubscription subscription,
WebhookPayload payload)
{
if (payload.Attempt >= MaxRetryAttempts)
{
_logger.LogWarning(
"Max retry attempts reached for webhook {DeliveryId}",
payload.DeliveryId);
await DisableEndpointIfFailingAsync(subscription.Id);
return;
}
var delay = CalculateRetryDelay(payload.Attempt);
// Exponential backoff: 30s, 2m, 10m, 30m, 2h
await _retryPolicy.ScheduleAsync(
() => DeliverWebhookAsync(subscription,
payload with { Attempt = payload.Attempt + 1 }),
delay);
}
}
13.2 Webhook Security
Webhook payloads are signed with HMAC-SHA256 using a per-subscription secret key. Customers should verify the signature to ensure the payload originated from GetResponse and was not tampered with in transit. The system also enforces TLS-only endpoints — HTTP URLs are rejected during webhook configuration.
| Security Feature | Implementation |
|---|---|
| Payload Signing | HMAC-SHA256 with per-subscription secret |
| Transport Security | TLS 1.2+ required, HTTP rejected |
| IP Whitelisting | Published GetResponse webhook IP ranges |
| Timeout | 10 second response timeout |
| Rate Limiting | 100 requests/minute per endpoint |
| Retry Policy | 5 attempts with exponential backoff |
| Failure Handling | Auto-disable after 50 consecutive failures |
14. Analytics and Reporting Dashboard
The analytics system must aggregate tracking events across millions of contacts and thousands of campaigns to provide real-time and historical reporting dashboards. The data volume is substantial — a single day generates hundreds of millions of tracking events that must be aggregated into readable metrics.
14.1 Analytics Data Pipeline
The analytics pipeline follows a Lambda architecture pattern with both a batch layer (for accurate historical reports) and a speed layer (for real-time dashboards). The batch layer processes events into pre-aggregated summaries stored in PostgreSQL. The speed layer provides real-time counters via Redis with near-real-time updates from Kafka.
14.2 Pre-Aggregation Strategy
To avoid querying raw event data for dashboard metrics, the system pre-aggregates events into summary tables at multiple granularities: hourly, daily, and monthly. Pre-aggregation reduces query latency from seconds (scanning millions of rows) to milliseconds (reading pre-computed summaries).
-- Pre-aggregation tables for campaign analytics
CREATE TABLE campaign_daily_stats (
campaign_id UUID NOT NULL,
stat_date DATE NOT NULL,
sent_count INTEGER NOT NULL DEFAULT 0,
delivered_count INTEGER NOT NULL DEFAULT 0,
bounce_count INTEGER NOT NULL DEFAULT 0,
hard_bounce_count INTEGER NOT NULL DEFAULT 0,
soft_bounce_count INTEGER NOT NULL DEFAULT 0,
open_count INTEGER NOT NULL DEFAULT 0,
unique_open_count INTEGER NOT NULL DEFAULT 0,
click_count INTEGER NOT NULL DEFAULT 0,
unique_click_count INTEGER NOT NULL DEFAULT 0,
unsubscribe_count INTEGER NOT NULL DEFAULT 0,
complaint_count INTEGER NOT NULL DEFAULT 0,
conversion_count INTEGER NOT NULL DEFAULT 0,
revenue_decimal NUMERIC(12,2) DEFAULT 0,
PRIMARY KEY (campaign_id, stat_date)
);
-- Materialized view for tenant-level dashboard
CREATE MATERIALIZED VIEW mv_tenant_daily_stats AS
SELECT
c.tenant_id,
cd.stat_date,
SUM(cd.sent_count) AS total_sent,
SUM(cd.delivered_count) AS total_delivered,
SUM(cd.open_count) AS total_opens,
SUM(cd.unique_open_count) AS total_unique_opens,
SUM(cd.click_count) AS total_clicks,
SUM(cd.unique_click_count) AS total_unique_clicks,
SUM(cd.unsubscribe_count) AS total_unsubscribes,
SUM(cd.complaint_count) AS total_complaints,
CASE
WHEN SUM(cd.delivered_count) > 0
THEN ROUND(SUM(cd.unique_open_count)::numeric /
SUM(cd.delivered_count) * 100, 2)
ELSE 0
END AS open_rate,
CASE
WHEN SUM(cd.delivered_count) > 0
THEN ROUND(SUM(cd.unique_click_count)::numeric /
SUM(cd.delivered_count) * 100, 2)
ELSE 0
END AS click_rate
FROM campaign_daily_stats cd
JOIN campaigns c ON c.id = cd.campaign_id
GROUP BY c.tenant_id, cd.stat_date
WITH DATA;
-- Refreshed every 5 minutes by background job
CREATE UNIQUE INDEX idx_mv_tenant_daily
ON mv_tenant_daily_stats(tenant_id, stat_date);
14.3 Dashboard Metrics
| Metric | Calculation | Update Frequency |
|---|---|---|
| Delivery Rate | delivered / sent × 100 | Real-time |
| Open Rate | unique opens / delivered × 100 | Real-time (with proxy adjustment) |
| Click-Through Rate (CTR) | unique clicks / delivered × 100 | Real-time |
| Click-to-Open Rate (CTOR) | unique clicks / unique opens × 100 | Real-time |
| Bounce Rate | bounces / sent × 100 | Real-time |
| Unsubscribe Rate | unsubscribes / delivered × 100 | Near real-time (5 min) |
| Complaint Rate | complaints / delivered × 100 | Near real-time (5 min) |
| Revenue per Email | total revenue / sent | Hourly |
| List Growth Rate | (new - removed) / total × 100 | Daily |
| Engagement Score | Weighted composite of opens, clicks, recency | Daily batch |
15. Multi-Tenancy and Data Isolation
As a SaaS platform, GetResponse must serve thousands of customers from shared infrastructure while ensuring strict data isolation. A customer must never see another customer's contacts, campaigns, or analytics data. The multi-tenancy architecture must also support per-tenant resource quotas, rate limiting, and billing metering.
15.1 Tenant Isolation Strategy
GetResponse uses a shared-database, shared-schema multi-tenancy model with tenant ID isolation. Every table includes a tenant_id column, and all queries are scoped to the current tenant. This approach maximizes infrastructure utilization while maintaining logical data isolation.
The isolation is enforced at multiple layers:
- API Layer: Every API request is authenticated and the tenant context is extracted from the JWT token. The tenant ID is propagated to all downstream services via HTTP headers and Kafka message headers.
- Data Layer: PostgreSQL Row-Level Security (RLS) policies enforce tenant scoping at the database level. Even if application code has a bug, RLS prevents cross-tenant data access.
- Elasticsearch: Indices are named with tenant ID prefix (
contacts_tenant_{id}), and all queries include a mandatory tenant filter. - Redis: All cache keys are prefixed with the tenant ID (
t:{tenantId}:key). - Kafka: Tenant ID is included in message headers and enforced by consumers.
-- PostgreSQL Row-Level Security policy
ALTER TABLE contacts ENABLE ROW LEVEL SECURITY;
ALTER TABLE campaigns ENABLE ROW LEVEL SECURITY;
ALTER TABLE lists ENABLE ROW LEVEL SECURITY;
-- Policy: tenants can only see their own contacts
CREATE POLICY tenant_contacts_isolation ON contacts
USING (tenant_id = current_setting('app.current_tenant')::uuid);
CREATE POLICY tenant_campaigns_isolation ON campaigns
USING (tenant_id = current_setting('app.current_tenant')::uuid);
CREATE POLICY tenant_lists_isolation ON lists
USING (tenant_id = current_setting('app.current_tenant')::uuid);
-- Application sets tenant context on each connection
-- SET app.current_tenant = '{tenant-id-here}';
-- Connection pool middleware sets this automatically
CREATE OR REPLACE FUNCTION set_tenant_context(p_tenant_id UUID)
RETURNS VOID AS $$
BEGIN
PERFORM set_config('app.current_tenant', p_tenant_id::text, true);
END;
$$ LANGUAGE plpgsql;
15.2 Tenant Tier Architecture
| Tier | Contacts | Emails/Month | Automation | Infrastructure |
|---|---|---|---|---|
| Free | 500 | 2,500 | Basic autoresponder | Shared queue, throttled |
| Email Marketing | 1,000–100,000 | Unlimited | Full automation | Shared queue, standard |
| Marketing Automation | 1,000–100,000 | Unlimited | Advanced + webinars | Priority queue |
| E-commerce Marketing | 1,000–100,000 | Unlimited | Full + e-commerce | Priority queue |
| GetResponse MAX | 100,000+ | Unlimited | All features | Dedicated IP option |
| Enterprise | Custom | Custom | Custom | Dedicated infrastructure |
Tier-based resource allocation is enforced at the API gateway level. When a request arrives, the system checks the tenant's tier and applies appropriate rate limits, feature flags, and queue priorities. Higher-tier customers get access to dedicated sending IPs, faster queue processing, and higher API rate limits.
15.3 Data Residency and GDPR Compliance
GetResponse operates data centers in the EU (Germany) and US, allowing customers to choose their data residency. GDPR compliance requires:
- Data Processing Agreements (DPA): Signed with all EU customers, specifying data processing purposes and safeguards.
- Right to Erasure: A contact deletion API that removes personal data from all systems — PostgreSQL, Elasticsearch, Redis, Kafka topic retention, and S3 object metadata — within 30 days.
- Data Portability: Contact export in CSV/JSON format including all custom fields and activity history.
- Consent Tracking: Every contact record tracks consent status, source, and timestamp. Marketing emails are only sent to contacts with valid consent.
- Access Controls: Audit logging for all data access operations, with 90-day retention.
16. Autoresponder and RSS-to-Email System
Autoresponders are time-based email sequences that are triggered when a contact subscribes to a list. Unlike marketing automation workflows (which are event-driven), autoresponders follow a fixed schedule — Day 1, Day 2, Day 7, Day 14, etc. — making them simpler but extremely common across GetResponse's customer base.
16.1 Autoresponder Scheduling
The autoresponder scheduler runs as a daily batch job that identifies contacts who should receive an autoresponder email on the current day. For each active autoresponder series, the job queries contacts whose subscription date plus the autoresponder day offset matches today. This requires efficient date-range queries across millions of contacts.
The RSS-to-email feature monitors RSS feeds and sends digest emails when new content is published. The system polls configured RSS feeds at configurable intervals (hourly, daily, weekly), detects new entries since the last poll, and generates email digests using the customer's configured template. New entries are tracked in a state table to prevent duplicate sends.
public class RssEmailService : BackgroundService
{
private readonly IRssFeedRepository _feedRepo;
private readonly IFeedStateStore _stateStore;
private readonly IEmailDispatcher _emailDispatcher;
private readonly IHttpClientFactory _httpClientFactory;
protected override async Task ExecuteAsync(CancellationToken ct)
{
while (!ct.IsCancellationRequested)
{
var activeFeeds = await _feedRepo
.GetActiveFeedsAsync();
foreach (var feed in activeFeeds)
{
if (ShouldPollNow(feed))
{
await ProcessFeedAsync(feed);
}
}
await Task.Delay(TimeSpan.FromMinutes(15), ct);
}
}
private async Task ProcessFeedAsync(RssFeedSubscription feed)
{
var lastState = await _stateStore
.GetLastStateAsync(feed.Id);
var feedContent = await FetchRssFeedAsync(feed.Url);
var newEntries = feedContent.Entries
.Where(e => e.PublishedDate > (lastState?.LastProcessedDate ?? DateTime.MinValue))
.OrderByDescending(e => e.PublishedDate)
.ToList();
if (!newEntries.Any()) return;
// Determine digest scope
var entriesToSend = feed.DigestMode switch
{
DigestMode.EveryEntry => newEntries,
DigestMode.DailyDigest => newEntries
.Where(e => e.PublishedDate.Date == DateTime.UtcNow.Date)
.ToList(),
DigestMode.WeeklyDigest => newEntries
.Where(e => e.PublishedDate >= DateTime.UtcNow.AddDays(-7))
.ToList(),
_ => newEntries.Take(1) // Latest only
};
if (entriesToSend.Any())
{
var email = await BuildDigestEmailAsync(
feed, entriesToSend);
await _emailDispatcher.SendAsync(
feed.ListId, email);
await _stateStore.UpdateStateAsync(feed.Id,
new FeedState
{
LastProcessedDate = DateTime.UtcNow,
LastEntryDate = entriesToSend.First().PublishedDate,
EntriesSentCount = entriesToSend.Count
});
}
}
}
17. Integration Marketplace Architecture
GetResponse offers 170+ integrations with third-party platforms including Shopify, WordPress, Salesforce, Zapier, and custom API integrations. The integration marketplace architecture must handle OAuth flows, API key management, bidirectional data synchronization, and per-integration rate limiting.
17.1 Integration Patterns
| Pattern | Description | Examples |
|---|---|---|
| One-Way Push | Data flows from third party to GetResponse | Shopify purchases → contacts |
| One-Way Pull | Data flows from GetResponse to third party | Campaign stats → Google Analytics |
| Bidirectional Sync | Continuous synchronization both ways | Salesforce contacts ↔ GetResponse contacts |
| Event-Driven | Triggered by specific events | Zapier webhooks on subscribe/unsubscribe |
| OAuth 2.0 | Secure delegated access | Google, Facebook, Shopify |
17.2 Integration Connector Framework
Each integration is implemented as a connector module following a standardized interface. Connectors handle authentication, data mapping, rate limiting, and error handling. The connector framework provides shared infrastructure for common concerns like retry logic, logging, and metrics collection.
public interface IIntegrationConnector
{
string ProviderId { get; }
IntegrationType Type { get; }
Task<ConnectionTestResult> TestConnectionAsync(
IntegrationConfig config);
Task<SyncResult> SyncContactsAsync(
IntegrationConfig config,
SyncOptions options,
CancellationToken ct);
Task<WebhookResult> HandleWebhookAsync(
HttpRequest request,
IntegrationConfig config);
}
public class ShopifyConnector : IIntegrationConnector
{
public string ProviderId => "shopify";
public IntegrationType Type => IntegrationType.BidirectionalSync;
private readonly IShopifyApiClient _shopifyApi;
private readonly IContactService _contactService;
private readonly ILogger<ShopifyConnector> _logger;
public async Task<SyncResult> SyncContactsAsync(
IntegrationConfig config,
SyncOptions options,
CancellationToken ct)
{
var shopifyConfig = config.Deserialize<ShopifyConfig>();
var shopUrl = shopifyConfig.ShopUrl;
var accessToken = await _credentialStore
.GetAccessTokenAsync(config.IntegrationId);
var syncResult = new SyncResult();
// Pull customers from Shopify
await foreach (var customer in _shopifyApi
.GetCustomersAsync(shopUrl, accessToken, ct))
{
var contact = MapShopifyCustomerToContact(
customer, config.DefaultListId);
// Check if contact already exists
var existing = await _contactService
.FindByEmailAsync(
config.TenantId, contact.Email);
if (existing != null)
{
// Update existing contact with Shopify data
await _contactService.UpdateFromIntegrationAsync(
existing, contact, ProviderId);
syncResult.Updated++;
}
else
{
// Create new contact
await _contactService.CreateFromIntegrationAsync(
contact, ProviderId);
syncResult.Created++;
}
// Tag with Shopify data
await _contactService.AddTagAsync(
contact.Id, "shopify-customer");
if (customer.TotalSpent > 1000)
await _contactService.AddTagAsync(
contact.Id, "shopify-high-value");
syncResult.TotalProcessed++;
}
// Push new GetResponse contacts back to Shopify
// (if bidirectional is enabled)
if (options.SyncDirection == SyncDirection.Bidirectional)
{
var newContacts = await _contactService
.GetContactsAddedSinceAsync(
config.TenantId,
config.DefaultListId,
options.LastSyncTime);
foreach (var contact in newContacts)
{
try
{
await _shopifyApi.CreateCustomerAsync(
shopUrl, accessToken,
MapContactToShopifyCustomer(contact));
syncResult.PushedToShopify++;
}
catch (ShopifyRateLimitException)
{
_logger.LogWarning(
"Shopify rate limit hit, pausing push");
break;
}
}
}
return syncResult;
}
}
18. Rate Limiting and Abuse Prevention
Rate limiting is essential for platform stability and fair resource allocation. GetResponse must protect against several abuse scenarios: accounts creating excessive API calls, accounts sending spam through the platform, and accounts attempting to harvest email addresses through form submissions.
18.1 Multi-Layer Rate Limiting
Rate limiting is applied at multiple layers to provide defense in depth:
| Layer | Mechanism | Limits |
|---|---|---|
| IP Level | Nginx limit_req | 100 requests/second per IP |
| API Key Level | Redis sliding window | Tier-based: 10–500 requests/minute |
| Tenant Level | Redis token bucket | Tier-based email volume limits |
| Contact Level | Application logic | Max 5 emails/24 hours per contact |
| SMTP Level | Connection pool limits | Per-ISP connection caps |
The Redis-based sliding window rate limiter uses a sorted set to track request timestamps. Each request is added to the set with its timestamp as the score. The rate limiter counts the number of entries within the sliding window and rejects requests that exceed the limit. This approach provides accurate rate limiting without the boundary issues of fixed-window counters.
18.2 Spam Detection and Prevention
GetResponse employs multiple spam detection mechanisms to prevent abuse of the platform. New accounts undergo enhanced scrutiny during their first 30 days, with lower sending limits and mandatory double opt-in. The system monitors for several spam indicators:
- High bounce rate: > 5% hard bounce rate on first campaign triggers manual review.
- Low engagement: Campaigns with < 1% open rate across multiple sends indicate purchased or scraped lists.
- Rapid list growth: Adding 10,000+ contacts in a single day from non-API sources is flagged for review.
- Content analysis: Spammy content patterns (excessive caps, known spam phrases, misleading subject lines) are detected and flagged.
- Complaint rate: Any account exceeding 0.1% complaint rate across any campaign is immediately suspended pending review.
19. Interview Q&A Section
Q1: How would you design the email delivery pipeline to handle 15 million emails per day with 99%+ deliverability?
Answer: The delivery pipeline should be event-driven with Kafka as the central message bus. Key design decisions include: (1) A smart routing layer that selects optimal IP/domain pairs based on real-time ISP reputation scores, implementing a multi-armed bandit algorithm. (2) Per-ISP connection pooling with dynamic rate limiting based on ISP feedback. (3) Idempotent delivery with globally unique message IDs to prevent duplicate sends during retries. (4) Real-time bounce and complaint processing via dedicated Kafka consumers that update suppression lists and reputation scores. (5) IP warmup management for new sending IPs following a structured 30-day ramp-up schedule. The pipeline should separate concerns: campaign scheduling, contact resolution, template personalization, SMTP dispatch, and event processing should all be independent services connected through Kafka.
Q2: How does the segmentation engine query millions of contacts with complex boolean conditions in seconds?
Answer: The segmentation engine uses Elasticsearch as the primary query engine, with data replicated from PostgreSQL via CDC (Debezium). The key architectural decisions are: (1) Tenant-aware index routing ensures queries are automatically scoped to a single tenant. (2) Segment definitions are compiled into Elasticsearch bool queries and stored as query templates. (3) Large result sets use the scroll API to stream contact IDs without loading all results into memory. (4) The Elasticsearch cluster uses a hot-warm architecture — active segment queries hit hot nodes with SSDs, while historical data sits on warm nodes with HDDs. (5) Commonly-used segments are cached with short TTLs since contact data changes frequently. The CQRS pattern is essential here — writes go to PostgreSQL, reads are served from Elasticsearch.
Q3: Explain the multi-tenancy data isolation strategy. What are the trade-offs between shared-database vs. database-per-tenant?
Answer: GetResponse uses a shared-database, shared-schema model with tenant_id columns on every table and PostgreSQL Row-Level Security for enforcement. The trade-offs are: Shared-database maximizes infrastructure utilization (no idle database resources), simplifies schema migrations (one schema to maintain), and reduces operational overhead. However, it requires careful query scoping, noisy neighbor mitigation, and per-tenant backup/restore is harder. Database-per-tenant provides stronger isolation (physically separate data), per-tenant performance tuning, and easier data deletion for GDPR. But it increases operational cost linearly with tenant count, complicates cross-tenant analytics, and makes schema migrations painful at scale. For a platform with 350K+ customers, database-per-tenant is impractical — you cannot run 350K PostgreSQL instances. Shared-database with RLS is the correct choice, with Elasticsearch providing the query performance needed for segmentation.
Q4: How would you handle the "thundering herd" problem when thousands of campaigns are scheduled for 9 AM simultaneously?
Answer: The thundering herd problem occurs because marketers overwhelmingly schedule campaigns for the top of the hour. At 9:00 AM, potentially thousands of campaigns begin processing simultaneously, creating spikes in database queries, Kafka message volume, and SMTP connections. Mitigation strategies include: (1) Micro-scheduling — stagger campaign starts within a 15-minute window around the scheduled time (±7 minutes jitter). (2) Pre-warming SMTP connection pools 5 minutes before scheduled send times. (3) Horizontal auto-scaling of delivery pods triggered by Kafka consumer lag metrics. (4) Priority queues in Kafka separating time-critical sends from batch imports. (5) Backpressure mechanisms that gracefully degrade — if the pipeline is saturated, campaigns are delayed by a few minutes rather than dropped. The key insight is that a 5-minute delay is imperceptible to email recipients but dramatically reduces infrastructure peak load.
Q5: Design the real-time analytics pipeline that can update campaign dashboards within seconds of an email being opened.
Answer: The pipeline uses a dual-speed architecture: A real-time path for the speed layer and a batch path for the accuracy layer. The real-time path: tracking events → Kafka → lightweight consumer that increments Redis counters (campaign:opens:count, campaign:opens:unique) → dashboard reads Redis. This provides sub-second latency for real-time counters. The batch path: Kafka → Parquet files in S3 → daily Spark/aggregation job → PostgreSQL summary tables. This provides accurate historical data for reports. The analytics API reads from Redis for real-time views and PostgreSQL for historical queries. For unique counts (which cannot be computed by simple counting), the real-time path uses Redis HyperLogLog data structures, which provide approximate unique counts with 0.81% standard error using only 12KB of memory per key — essential when tracking millions of unique opens per campaign.
Q6: How would you design the marketing automation workflow engine to handle millions of concurrent workflows?
Answer: The workflow engine must track independent state for millions of contacts progressing through thousands of different workflow definitions. The design uses Kafka as the execution backbone: each workflow step is an event in Kafka. When a step completes, it produces a "next step" event, which may be immediate or delayed. Time-based waits (e.g., "wait 3 days") use Kafka's timestamp-based scheduling with a daily sweep job for precision. The key design choice is that workflow state is stored per-contact-workflow pair in a dedicated state store (Redis for active workflows, PostgreSQL for historical). The executor is stateless — it reads the current state, executes the step, updates state, and enqueues the next step. This allows horizontal scaling by adding more executor pods. Workflow evaluation conditions (opened email? clicked link?) are resolved by querying the tracking data in Redis/Elasticsearch.
Q7: What are the key challenges in email template rendering across different email clients, and how does the system address them?
Answer: Email rendering is notoriously difficult because email clients use different rendering engines: Gmail strips <style> blocks, Outlook uses Microsoft Word's HTML engine, Apple Mail has its own WebKit-based renderer, and Yahoo applies its own transformations. The template renderer addresses these by: (1) Inlining all CSS into style attributes. (2) Using table-based layouts instead of CSS Grid/Flexbox. (3) Providing VML (Vector Markup Language) fallbacks for Outlook features like background images and rounded corners, wrapped in <!--[if mso]> conditional comments. (4) Using a "bulletproof" button technique that renders as both a rounded CSS button and a VML shape for Outlook. (5) Testing output against Litmus or Email on Acid before deployment. The block-based editor stores templates as structured JSON (the block tree) rather than raw HTML, which allows the renderer to generate client-specific HTML variants and simplifies the editing experience.
Q8: How does the IP warmup system work, and why is it necessary for maintaining email deliverability?
Answer: ISPs track sending history per IP address. A new IP with no history is treated with suspicion — ISPs will either reject or throttle its mail. The warmup process gradually increases sending volume over 30+ days, starting with small batches to highly engaged contacts (who are most likely to open and click), building a positive sending reputation. The system automatically selects recipients with high engagement scores for warmup sends, since positive signals (opens, clicks) from these recipients help build reputation faster. The warmup monitor tracks bounce rates, complaint rates, and throttling events daily. If any metric exceeds threshold, the warmup pace is reduced or paused. Once an IP reaches full volume capacity with consistent positive metrics, it graduates from warmup and joins the production pool. GetResponse manages this across hundreds of IP addresses, each at different stages of their lifecycle.
Q9: Explain the webhook delivery system's guarantee model. How do you ensure at-least-once delivery without duplicating events?
Answer: The webhook system provides at-least-once delivery semantics. Each delivery attempt is assigned a unique delivery_id (UUID) that the customer can use for deduplication. The system uses Kafka's consumer group mechanism for reliable message processing — messages are committed only after successful delivery or exhaustion of retries. For duplicate prevention: (1) Each webhook event has a unique event_id that is stable across retries. (2) Customers should implement idempotent webhook handlers that check the event_id before processing. (3) The webhook log stores delivery history per event for debugging. Retry policy uses exponential backoff (30s, 2m, 10m, 30m, 2h) with jitter. After 5 consecutive failures, the endpoint is auto-disabled and the customer is notified. Failed deliveries are retained in the webhook log for 30 days for manual replay.
Q10: How would you migrate GetResponse from its current architecture to a fully serverless/event-driven architecture? What are the trade-offs?
Answer: A serverless migration would involve: (1) Replacing API servers with Lambda/Azure Functions for request handling — works well for the tracking API (bursty, stateless) but problematic for the SMTP relay (requires persistent connections). (2) Using DynamoDB/CosmosDB for the contact store — good for individual contact lookups but poor for complex segmentation queries that need JOINs. (3) Step Functions/Azure Durable Functions for workflow orchestration — natural fit for sequential workflow steps with timers. The critical trade-offs: SMTP delivery cannot be truly serverless because it requires persistent TCP connections with ISPs. Template rendering with personalization may hit Lambda timeout limits for large batches. The segmentation engine's complex queries are better served by Elasticsearch than serverless databases. Cost at GetResponse's scale may be higher with serverless (pay-per-invocation vs. reserved capacity). The hybrid approach — serverless for API gateway, tracking, and webhooks; containerized for SMTP relay, segmentation, and workflow execution — is more practical than full serverless.
Conclusion
Building and operating an email marketing platform at GetResponse's scale requires solving a diverse set of distributed systems challenges. From the high-throughput SMTP delivery infrastructure with intelligent per-ISP routing, to the real-time segmentation engine powered by Elasticsearch, to the complex state machine of marketing automation workflows — each subsystem presents unique engineering trade-offs.
The key architectural insights from this analysis are: (1) Event-driven architecture with Kafka as the backbone enables loose coupling between subsystems while maintaining high throughput. (2) CQRS with Elasticsearch for reads and PostgreSQL for writes is essential for segmentation query performance. (3) IP reputation management and deliverability are ongoing operational challenges, not one-time engineering problems. (4) Multi-tenancy with shared-database isolation using RLS provides the right balance of isolation and efficiency at this scale. (5) Real-time analytics requires a dual-speed architecture balancing latency and accuracy.
The email marketing industry continues to evolve with privacy regulations (GDPR, CCPA), ISP policy changes (Apple MPP, Gmail throttling), and customer expectations for real-time personalization. The engineering team must continuously adapt the platform's architecture to these changing constraints while maintaining the reliability and deliverability that customers depend on.