Design a Tiered Messaging Platform

Design a billion-message platform where critical OTPs cannot be starved by bulk campaigns: tier isolation, providers, retries, scheduling, and failure handling.

Page content

An authentication service sends an OTP at the same moment Marketing launches a campaign to 50 million users.

Both messages enter the same platform, but they do not have the same value:

OTP                  useful for seconds; blocks a login
Marketing message    useful for hours; may wait

If bulk traffic fills the same queues, workers, or provider quota used by the OTP, calling one message “P0” does not help. Priority is meaningful only when the system reserves the resources needed to enforce it.

The central question is:

How do we deliver more than a billion SMS, email, and push messages per day while preventing low-priority traffic from starving critical messages?

We will start with requirements, estimate the traffic, and first examine the simple shared-queue design. Its failure will lead us to tier isolation.

1. Clarify the problem

The channels are:

SMS
Email
Push through FCM/APNs

Before designing, I would clarify latency, delivery guarantees, scheduling, tracking, providers, geography, and ordering. For this walkthrough, assume:

Tier SLAs       P0 <1s, P1 <10s, P2 <5m, P3 <1h
Traffic         1B messages/day, 10× peak
Delivery        durable acceptance, at-least-once processing
Scheduling      P2 and P3 only
Tracking        full lifecycle, query API, client webhooks
Tenants         trusted internal services with quotas and recipient controls
Providers       multiple providers selected dynamically
Regions         start single-region, then evolve globally
Ordering        optional key; P0 may overtake lower tiers

The latency SLA ends when a healthy provider accepts the request. The platform cannot promise that a carrier or powered-off phone will deliver an SMS within one second.

2. Requirements

Functional requirements

  1. Internal services submit SMS, email, and push messages.
  2. Every message carries a P0–P3 tier.
  3. P2/P3 may be scheduled.
  4. Messages use versioned templates and variables.
  5. Clients can query delivery status and receive signed webhooks.
  6. Retryable provider failures are retried.
  7. Multiple providers are supported per channel.
  8. Operators can inspect and replay operational DLQs.
  9. Tenant quotas, consent, quiet hours, and recipient limits are enforced.

Tier contracts

P0   Critical        OTP, payment/security alert     p99 < 1 second
P1   Important       booking/order confirmation      p99 < 10 seconds
P2   Normal          receipt/account update          p99 < 5 minutes
P3   Best effort     marketing campaign              within 1 hour

Non-functional requirements

  • Acknowledged messages must be durable.
  • P3 must not consume capacity reserved for P0.
  • Processing is at least once.
  • External duplicates remain possible after an ambiguous provider outcome.
  • Each tier scales and degrades independently.
  • Operational status is queryable for 30 days.
  • Sensitive recipient and message data is protected.

3. Scale estimation

Messages/day       1 billion
Average size       2 KB including metadata
Peak multiplier    10×

Throughput

Average
= 1B / 86,400
≈ 11,600 messages/second

Peak
≈ 116,000 messages/second

Assume:

TierShareAverage/secPeak/sec
P01%1161,160
P19%1,04010,400
P230%3,48034,800
P360%6,96069,600

P0 is small and strict. P3 dominates volume and often arrives as a bursty campaign. That mismatch drives the isolation strategy.

Channel capacity

Assume push 60%, email 30%, SMS 10%:

Push peak     ~69,600/sec
Email peak    ~34,800/sec
SMS peak      ~11,600/sec

Provider contracts need this capacity plus failover headroom. Provider B is not a real SMS backup if it cannot absorb Provider A’s traffic during an outage.

Storage and queue capacity

1B × 2 KB
≈ 2 TB/day of ingress payload

One day in Kafka with replication factor 3 is roughly 6 TB before overhead. Operational state at 1 KB/message is about 1 TB/day or 30 TB for 30 days before replication.

Historical events should move to object storage and an analytical database rather than remain in the serving store.

4. APIs

Submit a message

Request:

POST /v1/messages
Authorization: Bearer <service-token>
Idempotency-Key: order-981-confirmed

{
  "channel": "SMS",
  "tier": "P1",
  "recipient": "+919876543210",
  "template_id": "order-confirmed",
  "template_version": 7,
  "variables": {
    "order_id": "981",
    "eta": "30 minutes"
  },
  "expires_at": "2026-08-15T10:10:00Z",
  "ordering_key": "order:981",
  "status_webhook": "https://orders.internal/message-status"
}

Response:

HTTP 202 Accepted

{
  "message_id": "msg_7f31",
  "status": "QUEUED",
  "accepted_at": "2026-08-15T10:00:00Z"
}

Return 202 after durable acceptance, not after contacting a provider.

Other APIs

GET  /v1/messages/{message_id}
POST /v1/templates
GET  /v1/templates/{template_id}/versions/{version}
POST /v1/campaigns

Bulk campaigns upload a recipient manifest to object storage. A campaign expander emits individual P3 messages at a controlled rate instead of receiving millions of recipients in one request.

5. Access patterns and data model

The system must:

  1. Fetch current state by message_id.
  2. deduplicate (tenant_id, idempotency_key).
  3. List recent messages for a tenant.
  4. Fetch attempts for one message.
  5. correlate a provider callback by provider_message_id.
  6. find scheduled work due in a time range.
  7. retrieve an immutable template version.

Message

Message
-------
message_id
tenant_id
channel
tier
recipient
template_id
template_version
variables_reference
status
scheduled_at
expires_at
ordering_key
created_at
updated_at

Partition by hash(message_id). Maintain a tenant-history view partitioned by (tenant_id, date_bucket) and ordered by creation time.

Idempotency record

Key       (tenant_id, idempotency_key)
Value     request_hash, message_id, expires_at

The same key and payload return the original result. Reusing the key for different content returns 409.

Delivery attempt

Partition key   message_id
Sort key        attempt_number

provider_id
provider_message_id
outcome
normalized_error
started_at
completed_at

Index provider_message_id for webhook correlation.

Template

Partition key   (tenant_id, template_id)
Sort key        version

Template versions are immutable so a queued message renders exactly what was approved at acceptance.

Scheduled message

Partition key   (time_bucket, shard)
Sort key        (scheduled_at, message_id)

Schedulers query only current buckets, not every future message.

Data ownership

Operational truth    Message and current status
Attempt history      Provider calls and outcomes
Queue                Durable work and short-term replay
Redis state          Rate limits, provider health, configuration cache
Historical archive   Object storage / analytical database

6. Why one priority queue fails

The simplest design is:

Messaging API
One priority queue
One worker fleet
Providers

If P0 keeps arriving, workers always choose P0 and P3 starves.

If P3 adds 20 million records, it may consume broker disk, network, worker memory, and provider quota before P0 reaches the head.

Logical priority cannot isolate shared physical bottlenecks.

7. Tier isolation

Use separate queues and workers:

P0 → dedicated queue → dedicated workers
P1 → dedicated queue → dedicated workers
P2 → dedicated queue → dedicated workers
P3 → dedicated queue → dedicated workers

This gives independent lag, retention, scaling, and failure domains.

Reserve capacity:

P0   20%
P1   25%
P2   25%
P3   30%

Unused high-tier capacity may be borrowed downward. Lower tiers cannot consume the reservation needed by higher tiers.

Apply this rule to every bottleneck:

  • Kafka throughput and storage;
  • worker concurrency;
  • database writes;
  • network connections;
  • rate-limit tokens;
  • provider credentials and quotas.

Separate topics do not protect P0 if all tiers still share one exhausted provider account.

Start with logical isolation and Kafka quotas. Move P0 to dedicated brokers, worker nodes, network pools, and provider accounts when its SLA is contractual or shared infrastructure creates noisy-neighbor incidents.

8. Kafka design

Use topics per (channel, tier):

send.sms.p0       send.email.p0       send.push.p0
send.sms.p1       send.email.p1       send.push.p1
...
send.sms.p3       send.email.p3       send.push.p3

Twelve send topics allow different retention, partitions, consumers, and alerts.

Default partition key:

hash(message_id)

For optional ordering:

hash(tenant_id + ordering_key)

This preserves order within one channel and tier. There is no ordering across tiers because P0 is allowed to overtake P2.

Consumer groups are independently scalable:

sms-p0-senders
sms-p3-senders
email-p1-senders
...

Kafka fits because we need high throughput, durable buffering, replay, partition order, independent consumers, and measurable lag.

Managed queues such as SQS/Pub/Sub may be preferable when operational simplicity matters more than replay. RabbitMQ has rich routing but priority queues can starve and are harder at this retention/throughput. Redis Streams is useful at smaller scale but weaker as the primary billion-message durable log.

9. High-level architecture

 Internal services
┌─────────────────┐
│ Messaging API   │
│ auth, policy,   │
│ idempotency     │
└────────┬────────┘
┌─────────────────┐
│ Message Store   │
│ + Outbox / CDC  │
└────────┬────────┘
    Tier Router
    ┌────┼────┐
    ▼    ▼    ▼
 P0 topics ... P3 topics
    │             │
    ▼             ▼
Dedicated worker pools
┌─────────────────┐
│ Provider Router │
│ health, quota,  │
│ tier, cost      │
└───┬────┬────┬───┘
    ▼    ▼    ▼
   SMS Email Push providers
 Provider webhooks → Delivery Tracker → Client webhooks

Submission

  1. Authenticate the service and determine its tenant.
  2. Validate tier policy, channel, template, recipient, and TTL.
  3. Apply tenant admission limits.
  4. Atomically write message, idempotency record, and outbox event.
  5. Return 202.
  6. CDC/outbox routes the message to its channel/tier topic.

Sending

  1. A tier worker consumes the message.
  2. It checks expiry, consent, quiet hours, and recipient limits.
  3. It renders the immutable template.
  4. It reserves provider quota.
  5. Provider Router selects an adapter.
  6. The adapter sends with a stable idempotency key where supported.
  7. The worker persists the attempt before committing its Kafka offset.

10. Durable acceptance

This is unsafe:

write Message
Kafka publish fails
message never sends

Atomically write:

Message
Idempotency record
Outbox event

An outbox relay or CDC publishes later.

At 120K/sec, one relational outbox is a bottleneck. Use sharded relational stores, a distributed KV with conditional writes and streams, or another partitioned operational store with CDC.

11. Worker architecture

Separate fleets by channel and tier:

SMS P0       warm capacity, reliability-first
SMS P3       cost-first, aggressively throttled
Email P0     reserved provider accounts
Email P3     bulk batching
Push P0      reserved APNs/FCM capacity

Worker flow:

Consume
Acquire processing lease
Check expiry and policy
Render template
Reserve quota
Select provider
Send
Persist attempt
Commit offset

Scale by oldest message age, lag, ingress, provider latency, and available quota—not CPU alone. Adding workers while a provider quota is exhausted only creates more throttling.

Keep P0 workers warm. Waiting for autoscaling after backlog appears may already violate a one-second SLA.

12. Provider abstraction and routing

Use channel-specific adapters:

SmsProvider.send()
EmailProvider.send()
PushProvider.send()

Each adapter normalizes provider payloads, responses, errors, idempotency, and webhooks while preserving channel-specific capabilities.

Provider Router considers:

channel and tier
recipient country/carrier
regional/data-residency rules
health and latency
remaining quota
historical delivery success
cost
tenant/sender identity

Routing policy:

P0   best reliability, reserved quota, rapid failover
P1   reliability with moderate cost awareness
P2   balance cost and performance
P3   cheapest healthy provider, strict throttle

Circuit breaker flow:

Provider A failures: 5% → 10% → 30%
Open circuit
Route eligible traffic to B
Half-open probes
Close after sustained recovery

A timeout does not prove failure. Provider A may have sent the SMS and lost its response. Query A or retry with its idempotency key before failing over; otherwise accept the tier-specific duplicate risk.

13. Hierarchical rate limiting

Limits form a hierarchy:

Platform
└── Region
    └── Channel
        └── Provider account
            └── Tier reservation
                └── Tenant
                    └── Recipient

Examples:

Tenant              100K/sec
Provider A SMS       50K/sec
P0 reservation       10K/sec
Recipient OTP         5 / 10 minutes
Marketing             daily cap + quiet hours

Redis token buckets provide low-latency atomic coordination. At high scale, provider keys become hot, so grant workers short token leases that they spend locally.

If Redis fails, P0 may use conservative local reserved limits. P2/P3 pause. Recipient marketing limits fail closed.

14. Retry strategy

Retryable:

  • timeout or transport failure;
  • provider 5xx;
  • temporary provider failure;
  • 429 with Retry-After.

Non-retryable:

  • invalid address or device token;
  • missing template variable;
  • prohibited sender;
  • blocked or opted-out recipient.

Use exponential backoff with jitter and per-tier policies:

P0   alternate provider, then 1s, 3s, 10s within TTL
P1   2s, 10s, 30s, 2m
P2   30s, 2m, 10m, 30m
P3   minutes/hours within campaign window

Do not sleep inside a Kafka consumer. Persist RETRY_PENDING with next_attempt_at, then use delayed retry buckets/scheduler to re-enter the send topic.

Cap retry concurrency separately so retries cannot starve new P0 traffic.

15. Idempotency and external duplicates

Critical failure:

Worker sends SMS
Worker crashes before storing success
Kafka redelivers
SMS may send twice

At-least-once processing means a handler may run more than once. Kafka exactly-once cannot undo an external SMS.

Use:

provider_idempotency_key = message_id + logical_attempt

If supported, provider replay returns the original result. Otherwise a timeout is ambiguous. Persist UNKNOWN, reconcile by provider request id if possible, and apply a duplicate policy:

P0 OTP        duplicate may be safer than loss
P3 marketing  suppress duplicate when uncertain

Never claim exactly-once external delivery without provider support.

16. Delivery tracking

CREATED
SCHEDULED or QUEUED
PROCESSING
PROVIDER_ACCEPTED
SENT
DELIVERED

Branches:

RETRY_PENDING
FAILED
EXPIRED
SUPPRESSED

Provider callbacks may duplicate or arrive out of order. Deduplicate by (provider_id, provider_event_id) and prevent state regression:

DELIVERED cannot regress to SENT
FAILED cannot overwrite DELIVERED

Client webhook delivery uses a signed event id, retries, circuit breaker, and its own DLQ. The query API remains authoritative.

17. Scheduling

Do not put tomorrow’s message in Kafka and block a partition.

Store P2/P3 future work in time buckets:

Partition key   (minute/hour bucket, shard)
Sort key        (scheduled_at, message_id)

Schedulers:

  1. read current and near-future buckets;
  2. claim due rows with leases;
  3. verify cancellation and expiry;
  4. publish to P2/P3 send topics;
  5. mark dispatch complete.

Duplicate publication after a scheduler crash is safe because downstream handling is idempotent.

Large campaigns keep recipient manifests in object storage and expand near send time at a controlled rate.

18. Dead-letter queues

Expected business outcomes are not DLQs:

invalid recipient → FAILED
opted out         → SUPPRESSED
expired OTP       → EXPIRED

DLQs contain operationally unprocessable work:

  • unknown schema;
  • poison payload;
  • repeated worker crash;
  • renderer invariant violation;
  • exhausted retries caused by unexplained system failure.

Use channel/tier-aware DLQs and provide redacted inspection, attempt history, audited replay, and replay eligibility.

19. Backpressure

Normal P3 peak is ~70K/sec. A 100× spike is 7M/sec.

Ten minutes would create:

7M × 600
= 4.2B messages
≈ 8.4 TB payload
≈ 25 TB replicated

The platform cannot accept unlimited bulk work.

For P3:

  1. enforce campaign quotas;
  2. require manifests;
  3. throttle expansion;
  4. return 429 or delay the campaign;
  5. preserve P0/P1 reservations.

A 100× P0 spike reaches ~116K/sec. Isolation protects it from P3, but provider capacity is still finite. Use emergency quota and failover, then reject beyond the contracted capacity envelope. Priority cannot manufacture carrier capacity.

When lag grows, identify the true bottleneck. More workers do not fix provider quota, slow storage, poison retries, or insufficient Kafka partitions.

20. Storage by responsibility

ResponsibilitySuitable storageWhy
Operational message stateDistributed KV/wide-column or sharded SQLKey lookup, conditional transitions, TTL
Templates/configurationSQLLow volume, constraints, approval/audit
Scheduled workTime-bucketed KV/wide-columnDue-time range scans
Rate limits/provider healthRedisLow-latency ephemeral atomic state
Queue/replayKafkaDurable ordered streams
Historical eventsObject storage/ParquetCheap long retention
Operational analyticsClickHouse/DruidHigh-volume aggregations

Kafka transports events; Redis accelerates decisions. Neither is the operational source of message truth.

21. Consistency and ordering

Strong or conditional consistency is required for:

  • idempotency reservation;
  • message plus outbox creation;
  • message status transition;
  • immutable template version;
  • marketing opt-out checks;
  • provider quota when contractual limits must not be exceeded.

Eventual consistency is acceptable for:

  • provider health scores;
  • aggregate dashboards;
  • client webhooks;
  • provider delivery receipts;
  • cost analytics.

Optional ordering keys preserve order only within one channel and tier. Strict sequencing means a retrying message blocks later messages, so it is opt-in rather than default.

22. Failure scenarios

Kafka unavailable

Outbox rows accumulate. API acceptance continues only within a bounded safety window; stop accepting P0 before its SLA becomes impossible. Recovery replays outbox rows. Duplicate publication is safe.

Redis unavailable

P0 uses conservative local reserved quota if safe. P2/P3 pause. Rebuild ephemeral provider health and token state after recovery.

One provider down

Circuit opens. P0 uses reserved failover capacity. P3 may wait. Ambiguous timed-out sends may duplicate.

All providers for a channel down

Messages remain retryable until TTL, then become FAILED or EXPIRED. Do not retry forever.

Worker crashes before sending

Kafka redelivery safely retries.

Worker crashes after sending

Provider idempotency or reconciliation prevents duplication where possible. Otherwise external delivery is ambiguous.

Duplicate webhook

Webhook dedup and monotonic transitions make it harmless.

Provider throttles

Honor Retry-After, reduce route weight, preserve higher-tier reservations. Additional workers would worsen the incident.

P3 rises 100×

Admission control and campaign throttling protect P0/P1. Rejected bulk traffic is explicit, never silently dropped after acceptance.

P0 rises 100×

Use emergency providers and warm capacity. Reject beyond the provider capacity envelope rather than promise an impossible SLA.

Operational database unavailable

Stop accepting. Existing sends should pause if outcomes cannot be recorded safely.

Region fails

Promote one fenced region. Replayed ambiguous sends may duplicate, but durable accepted messages remain within the replication RPO.

Consumer lag grows

Alert on oldest age by tier. Determine whether workers, provider quota, partitions, retries, or storage is limiting before scaling.

23. Multi-region evolution

Start in one region across availability zones. Later assign each message an immutable home region:

Global routing
   ├── US region: US tenants/providers
   └── EU region: EU tenants/providers

Benefits:

  • recipient/provider locality;
  • data residency;
  • smaller failure domains.

Replicate durable state and Kafka asynchronously for disaster recovery. Do not actively process one message in two regions.

Use a fenced regional epoch. The DR region processes only after acquiring a newer epoch, preventing both regions from sending the same campaign during a partition.

P0 may justify synchronous metadata replication and warm provider capacity. P3 can tolerate a larger recovery delay.

24. Observability

Queue and tier

  • ingress by tenant/channel/tier;
  • consumer lag;
  • oldest message age;
  • retry age;
  • scheduler lateness;
  • DLQ rate.

Delivery and provider

  • acceptance-to-provider latency;
  • provider acceptance and delivery rates;
  • retry/suppression/expiry rate;
  • provider p99 latency, timeouts, 5xx, and 429;
  • quota remaining and circuit state.

Alert directly on tier SLOs:

P0 p99 provider acceptance > 1 second
P1 oldest age > 10 seconds
P2 oldest age > 5 minutes
P3 backlog threatens campaign window or retention

Oldest age is more meaningful than queue depth.

Trace by message, tenant, attempt, provider, tier, channel, and region. Redact recipient and content.

25. Security and abuse

  • Authenticate callers with workload identity, mTLS, or OAuth client credentials.
  • Authorize tenants by templates, channels, sender ids, and allowed tiers.
  • Prevent Marketing Service from labeling every campaign P0.
  • Isolate tenant status queries and encrypt sensitive payloads.
  • Keep provider credentials in a rotating secrets manager.
  • Enforce opt-outs, quiet hours, OTP frequency limits, and daily campaign caps.
  • Audit sensitive templates and sends.

If callers may self-declare all traffic critical, the tier model collapses.

26. Major trade-offs

Separate queues versus one queue

Separate queues provide isolation and independent scaling but add operational complexity and may leave capacity idle. Strict P0 requirements justify that cost.

Kafka versus managed queues

Kafka offers replay and stream control. Managed queues reduce operations. The essential decision is separate durable tier lanes, not Kafka by name.

Shared versus dedicated P0 infrastructure

Shared systems use resources efficiently. Dedicated systems reduce blast radius. Begin logically isolated and move to physical isolation when incidents or contracts justify it.

Fast failover versus duplicate risk

Rapid provider failover improves delivery probability after a timeout but can duplicate a message whose first send actually succeeded.

Ordering versus latency

Strict ordering creates head-of-line blocking during retry. Keep it optional and scoped.

27. Final architecture

                         INTERNAL SERVICES
                    ┌────────────────────────┐
                    │ Messaging API          │
                    │ auth, policy, quota,   │
                    │ idempotency, validate  │
                    └───────────┬────────────┘
                    ┌────────────────────────┐
                    │ Message Store + Outbox │
                    └───────────┬────────────┘
                                │ CDC
                         ┌────────────┐
                         │ Tier Router│
                         └─────┬──────┘
            ┌──────────────────┼──────────────────┐
            ▼                  ▼                  ▼
       P0 channel topics  P1/P2 topics       P3 topics
       reserved capacity  isolated workers   throttled bulk
            │                  │                  │
            └──────────────────┼──────────────────┘
                    ┌─────────────────────┐
                    │ Policy + Rate Limits│
                    │ Provider Router     │
                    └───┬────────┬────────┘
                        ▼        ▼
                 SMS / Email / FCM / APNs
                 Provider Webhooks
                 Delivery Tracker
                  ┌─────┴─────┐
                  ▼           ▼
           Client webhooks  Analytics

The mental model is:

P0 has a one-second SLA
P3 can arrive in campaigns
Shared resources create starvation
Reserve capacity at every bottleneck
Allow borrowing only downward
Use admission control when capacity is exhausted

28. Interview-ready summary

Top 10 decisions

  1. Define tier SLAs as provider-acceptance targets.
  2. Separate queues and workers by channel and tier.
  3. Reserve broker, worker, limiter, and provider capacity.
  4. Permit only downward borrowing.
  5. Atomically persist message, idempotency, and outbox.
  6. Use at-least-once processing with honest duplicate semantics.
  7. Route providers by reliability, quota, region, tier, and cost.
  8. Schedule with time buckets, not sleeping consumers.
  9. Separate business failures from operational DLQs.
  10. Give each message one fenced home region.

Top failure scenarios

  1. Kafka outage: durable outbox buffers within a safety window.
  2. Redis outage: P0 degrades conservatively; P2/P3 pause.
  3. Provider outage: circuit breaker and reserved failover.
  4. All providers down: retry until TTL, then fail.
  5. Crash after send: external result may duplicate.
  6. Duplicate webhook: dedupe and monotonic status.
  7. Provider throttling: reduce traffic; more workers do not help.
  8. P3 burst: campaign admission protects critical tiers.
  9. P0 burst: emergency capacity, then explicit rejection.
  10. Region loss: fenced failover prevents dual processing.

Senior-level differentiators

  • Reserve capacity at providers, not only Kafka.
  • Control which tenants may use P0.
  • Explain ambiguous provider timeouts.
  • Never claim exactly-once external delivery without provider support.
  • Treat retries as traffic with their own quota.
  • Alert on oldest message age, not only lag count.
  • Keep strict ordering opt-in.
  • Stop bulk expansion before queue storage becomes the incident.

Likely interviewer follow-ups

  • How do you calculate partition counts?
  • How does downward borrowing work?
  • What if P0 exceeds provider capacity?
  • How do you reconcile an ambiguous SMS timeout?
  • Why Kafka instead of SQS?
  • How do campaigns respect recipient time zones?
  • What happens if the outbox grows faster than Kafka recovery?
  • How do you migrate a tenant between regions?
  • How are distributed provider tokens implemented?
  • What prevents Marketing from using P0?

A 1–2 minute verbal answer

I would support SMS, email, and push across P0 through P3. At one billion messages per day, traffic averages about 11,600 per second and peaks around 116,000. P3 is most of the volume, while P0 is small but has a one-second provider-acceptance SLA.

The central decision is tier isolation. I would use separate channel-and-tier topics and independently scaled workers. P0 receives reserved broker, worker, network, limiter, and provider capacity. Lower tiers may borrow unused capacity but cannot consume its reservation. If shared brokers or provider accounts still create noisy neighbors, I would physically isolate P0.

The API enforces identity, tier policy, quotas, templates, and idempotency, then atomically writes the message and outbox before returning 202. Workers check TTL, consent, recipient limits, and provider quota before a reliability-, health-, region-, tier-, and cost-aware router selects a provider.

Processing is at least once. A crash after an external send can duplicate unless the provider supports idempotency or reconciliation. Provider callbacks are deduplicated into a monotonic status machine. Scheduling uses time-bucketed storage; retries re-enter through delayed buckets; poison operational failures go to inspectable DLQs. The key SLIs are oldest message age by tier, provider-acceptance latency, retry rate, and delivery success.

For the broader interview framework around this problem, see the System Design Interview Complete Guide.