Bird
Raised Fist0

Design a Notification System - Push, SMS, Email Fan-Out at 10M+ Scale

HLDhardInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
▶
Request Trace
Step 1/5
Notification API

Receive notification request

API accepts user notification request with message and target channels

Protocol:HTTP POST (REST)
BeforeUser has composed notification on client device
AfterNotification API received request and validated payload
Kafka Topic (Notif Events)

Publish notification event

Notification API publishes event to Kafka topic for fan-out

Protocol:Kafka Publish
BeforeNotification API holds validated notification event
AfterNotification event durably stored in Kafka topic
Delivery Workers

Consume notification event

Workers consume event from Kafka and prepare channel-specific messages

Protocol:Kafka Consume
BeforeNotification event is in Kafka topic, unprocessed
AfterDelivery Workers have parsed event and queued channel sends
Channel Gateways (Push GW, SMS GW, Email SMTP Relay)

Send notifications via channels

Delivery Workers send messages to respective gateways using channel protocols

Protocol:Push: HTTPS, SMS: SMPP, Email: SMTP
BeforeDelivery Workers have channel-specific messages ready
AfterChannel Gateways accepted messages for delivery to end users
Delivery Status Store (Cassandra)

Record delivery status

Delivery Workers update delivery status and metadata in Cassandra

Protocol:CQL (Cassandra Query Language)
BeforeDelivery Workers have sent messages to gateways
AfterDelivery status stored durably for tracking and retries
🏗️
Architecture Diagram
Architecture Diagram
┌───────────────────────┐
│   Notification API    │
└─────────────┬─────────┘
              │ HTTP POST (REST)
              ▼
┌───────────────────────────────┐
│ Kafka Topic (Notif Events)    │
└─────────────┬─────────────────┘
              │ Kafka Publish
              ▼
┌───────────────────────┐
│   Delivery Workers    │
│ (Push, SMS, Email)    │
└───────┬─────┬─────────┘
        │     │
        │     │
        │     │
        ▼     ▼
┌───────────────┐  ┌───────────────────────────────┐
│ Push GW       │  │ SMS GW                       │
└───────┬───────┘  └─────────────┬─────────────────┘
        │                  │
        ▼                  ▼
┌───────────────────────────────┐
│ Email SMTP Relay              │
└─────────────┬─────────────────┘
              │
              ▼
┌───────────────────────────────┐
│ Delivery Status Store (Cassandra) │
└───────────────────────────────┘
Design Rationale

This notification system uses a Kafka topic as a durable event queue to fan out notifications to multiple channels at scale. The Notification API receives requests and publishes events to Kafka, ensuring durability and decoupling. Delivery Workers consume these events and send channel-specific messages to external gateways (Push, SMS, Email). Delivery status is asynchronously recorded in Cassandra for tracking and retries. This architecture supports high throughput and fault tolerance by separating ingestion, processing, and delivery, enabling horizontal scaling of workers and gateways independently.

Primary Focus

✓

Practice

(1/5)
1. A global CDN operator needs to prevent a misconfiguration from propagating incorrect BGP routes that could cause widespread internet outages. Which component is primarily responsible for detecting and containing such route leaks?
easy
A. Using a centralized DNS firewall to block suspicious IP prefixes
B. Implementing BGP route filters and prefix limits on edge routers
C. Relying on client-side validation of IP addresses before routing
D. Deploying a distributed cache to store valid routing tables

Solution

  1. Step 1: Identify BGP route filtering role

    BGP route filters and prefix limits on edge routers prevent invalid or excessive route announcements, containing route leaks.

  2. Step 2: Eliminate other options

    DNS firewalls (A) do not control BGP routing; client-side validation (C) is ineffective for routing control; distributed caches (D) do not enforce routing policies.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    BGP route filters contain route leaks at the network edge [check]
Hint: Edge routers enforce BGP filters to block bad routes [check]
Common Mistakes:
2. In a composite index on columns (A, B), a query filters only on column B. At step 3 in the query execution plan, the database engine uses the composite index. What is the state of index usage at this step?
easy
A. The composite index cannot be used efficiently because column A is not filtered, causing a full index scan
B. The composite index is fully used to seek on column B alone efficiently
C. The database ignores the composite index and uses a table scan instead
D. The composite index is partially used by scanning leaf nodes but with poor performance

Solution

  1. Step 1: Understand composite index column order

    Composite indexes are ordered by the first column (A), then second (B). Queries filtering only on B cannot efficiently seek the index without filtering on A.

  2. Step 2: Analyze index usage at step 3

    Since column A is not filtered, the engine must scan the entire index or large portions, resulting in a full index scan rather than an efficient seek.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Composite index requires leading column filter for efficient seeks [check]
Hint: Composite index needs leading column filter for seeks [check]
Common Mistakes:
3. Without a distributed consensus system (like Paxos or Raft) coordinating leader election, what exactly breaks in a system designed for 99.999% availability?
medium
A. Clients receive stale data but system continues operating normally
B. Multiple nodes may believe they're leader simultaneously, causing conflicting writes and data corruption
C. The system automatically fails over to a backup leader without downtime
D. Leader election happens instantly with no impact on availability

Solution

  1. Step 1: Role of distributed consensus in leader election

    Consensus protocols ensure exactly one leader at a time, preventing split-brain scenarios.

  2. Step 2: Consequences of missing consensus

    Without consensus, multiple nodes may act as leader simultaneously, causing conflicting writes and data corruption, breaking availability guarantees.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    No consensus -> split-brain -> conflicting writes -> data corruption [check]
Hint: No consensus -> split-brain -> conflicting writes break availability [check]
Common Mistakes:
4. For a high-traffic API with strict memory limits and tolerance for bursty traffic, which rate limiting approach is best and why?
medium
A. Leaky bucket, because it enforces a steady output rate regardless of bursts
B. Sliding window counter, because it tracks exact request times for precise limits
C. Token bucket, because it smooths bursts and uses fixed memory for tokens
D. Fixed window counter, because it uses constant memory and handles bursts at window edges

Solution

  1. Step 1: Analyze memory constraints and burst tolerance

    Sliding window counters require storing timestamps, which grows with request volume, unsuitable for strict memory limits.

  2. Step 2: Fixed window counters use constant memory by counting requests per fixed interval, allowing bursts at window edges but fitting memory constraints

    Token and leaky buckets smooth bursts but require token state management and may not tolerate bursts as well.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Fixed window uses constant memory, tolerates bursts, fits strict memory limits [check]
Hint: Fixed window uses constant memory, sliding window stores timestamps [check]
Common Mistakes:
5. At 10x traffic scale, which component of a global server load balancing system is most likely to fail first and why?
hard
A. The data center ingress routers, since network bandwidth scales linearly and saturates quickly
B. The health monitoring system, due to increased frequency of health checks overwhelming it
C. The GSLB control plane, as it struggles to process and update routing decisions for all data centers in real-time
D. The DNS servers, because they cannot handle the surge in query volume and caching limits failover speed

Solution

  1. Step 1: Analyze scaling challenges

    Health monitoring (The health monitoring system, due to increased frequency of health checks overwhelming it) scales with checks but can be optimized; control plane (The GSLB control plane, as it struggles to process and update routing decisions for all data centers in real-time) is distributed and designed for scale; ingress routers (The data center ingress routers, since network bandwidth scales linearly and saturates quickly) scale with network upgrades.

  2. Step 2: Identify DNS bottleneck

    DNS servers face query volume spikes and caching TTLs limit failover responsiveness, making them the first to fail under 10x load.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    DNS query volume and caching limits cause first failure at scale [check]
Hint: DNS query volume overloads servers first at scale [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard