Bird
Raised Fist0

Alerting System Design - Threshold, Anomaly, Composite Alert Patterns

HLDmediumInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
▶
Request Trace
Step 1/6
User Dashboard

User configures alert rules

User submits threshold and anomaly alert patterns

Protocol:WebSocket
BeforeUser has no active alert rules configured
AfterAlert Evaluator Service receives new alert rules
Alert Evaluator Service

Publishes alert rules to event stream

Alert rules sent to Kafka for processing

Protocol:Kafka Produce
BeforeAlert rules only in Alert Evaluator Service memory
AfterAlert rules durably stored in Kafka topic
Flink Stream Processing

Consumes events and applies threshold/anomaly detection

Processes event stream to detect anomalies and threshold breaches

Protocol:Kafka Consume
BeforeRaw event stream available in Kafka
AfterDetected alerts emitted to Alert Evaluator Service
Alert Evaluator Service

Evaluates composite alert patterns

Combines threshold and anomaly alerts into composite alerts

Protocol:RPC Call
BeforeIndividual alerts detected by Flink but not combined
AfterComposite alerts generated and stored in Cassandra
Cassandra DB

Stores alert states

Persists composite alert status and history

Protocol:CQL Write
BeforeNo persistent record of composite alert
AfterComposite alert state durably stored
Notification Delivery Service

Fetches alert state and sends notifications

Delivers alert notifications to user dashboard

Protocol:REST API + WebSocket / Push
BeforeUser unaware of new alerts
AfterUser Dashboard receives alert notification
🏗️
Architecture Diagram
Architecture Diagram
┌───────────────────────┐
│    User Dashboard     │
└────────────┬──────────┘
             │ WebSocket
             ▼
┌───────────────────────┐
│ Alert Evaluator Srv   │
│ (receives user config)│
└────────────┬──────────┘
             │ Kafka Produce
             ▼
┌───────────────────────┐
│   Kafka Cluster       │
│ (event stream queue)  │
└────────────┬──────────┘
             │ Kafka Consume
             ▼
┌───────────────────────┐
│ Flink Stream Proc     │
│ (threshold & anomaly) │
└────────────┬──────────┘
             │ RPC Call
             ▼
┌───────────────────────┐
│ Alert Evaluator Srv   │
│ (composite alerts)    │
└────────────┬──────────┘
             │ CQL Write
             ▼
┌───────────────────────┐
│   Cassandra DB        │
│ (alert state storage) │
└────────────┬──────────┘
             │ REST API
             ▼
┌───────────────────────┐
│ Notification Delivery │
│       Service         │
└────────────┬──────────┘
             │ WebSocket / Push
             ▼
┌───────────────────────┐
│    User Dashboard     │
│ (receives alert)      │
└───────────────────────┘
Design Rationale

This alerting system ingests user-defined alert rules via the User Dashboard, which are sent to the Alert Evaluator Service. The service publishes these rules to a Kafka Cluster, enabling decoupled, durable event streaming. Flink Stream Processing consumes the event stream to detect threshold breaches and anomalies in real time. Detected alerts are sent back to the Alert Evaluator Service, which evaluates composite alert patterns combining multiple signals. Composite alerts are persisted in Cassandra DB for durability and historical queries. Finally, the Notification Delivery Service retrieves alert states and pushes notifications back to the User Dashboard via WebSocket or push protocols. This vertical pipeline ensures scalable, real-time alert detection and delivery with clear separation of concerns.

Primary Focus

✓

Practice

(1/5)
1. In the rate limiter request flow, at Step 4 the Distributed Counter service receives an increment request. What happens if the counter exceeds the configured limit at this step?
easy
A. The request is immediately rejected and an error is returned to the client
B. The counter increments anyway but a warning is logged for later analysis
C. The increment is delayed until the next time window resets the counter
D. The request is forwarded to a fallback service that applies a different limit

Solution

  1. Step 1: Trace Distributed Counter behavior at Step 4

    The counter checks if the increment would exceed the limit; if yes, it rejects the request immediately.

  2. Step 2: Eliminate incorrect states

    Allowing increment with warning (B) breaks enforcement; delaying increment (C) causes inconsistent state; forwarding (D) adds complexity and is not standard.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Counter rejects requests exceeding limit immediately [check]
Hint: Reject requests immediately when limit exceeded [check]
Common Mistakes:
2. In the request flow for generating a user's feed, at step 5 the Feed Aggregator receives multiple candidate posts from different microservices. What does the Feed Aggregator do at this step before sending the feed to the user?
easy
A. It merges, deduplicates, and ranks posts based on relevance scores before returning
B. It immediately returns the raw posts without filtering or ranking
C. It stores all posts in a persistent database before responding to the user
D. It sends posts to the notification service to alert the user before aggregation

Solution

  1. Step 1: Understand Feed Aggregator's role at Step 5

    The Feed Aggregator must combine posts from multiple sources, remove duplicates, and rank them to personalize the feed.

  2. Step 2: Eliminate incorrect options

    It immediately returns the raw posts without filtering or ranking skips ranking, leading to poor UX. It stores all posts in a persistent database before responding to the user delays response unnecessarily. It sends posts to the notification service to alert the user before aggregation misplaces notification logic before feed aggregation.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Feed Aggregator merges and ranks posts -> personalized feed [check]
Hint: Aggregate, dedupe, rank before sending feed [OK]
Common Mistakes:
3. A social network platform needs to efficiently query complex friend-of-friend relationships with low latency. Which database technology best suits this requirement?
easy
A. A wide-column store like Cassandra optimized for large-scale writes
B. A document store like MongoDB with embedded documents
C. A graph database designed to traverse relationships quickly
D. A relational database with normalized tables and JOINs

Solution

  1. Step 1: Identify query pattern

    Friend-of-friend queries require efficient traversal of relationships, which relational JOINs or document stores handle poorly at scale.

  2. Step 2: Match technology to pattern

    Graph databases optimize relationship traversal with index-free adjacency, enabling low-latency queries on connected data.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Graph DBs excel at relationship traversal, unlike wide-column or document stores [check]
Hint: Graph DBs excel at traversing relationships fast [check]
Common Mistakes:
4. In the Redlock algorithm request flow, at Step 4 the client attempts to acquire locks on all Redis masters. What happens if the client successfully acquires locks on only 3 out of 5 masters within the timeout?
easy
A. The client proceeds with the lock assuming majority success
B. The client releases any acquired locks and retries later
C. The client holds partial locks and waits for the rest to acquire
D. The client escalates to a centralized lock coordinator

Solution

  1. Step 1: Trace client lock acquisition at Step 4

    Redlock requires majority of Redis masters to be locked within timeout to consider lock acquired.

  2. Step 2: Handle partial lock acquisition

    If fewer than majority locks acquired, client must release any partial locks to avoid deadlocks and retry.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Partial lock acquisition -> release all and retry [check]
Hint: Partial locks -> release all, no partial ownership [check]
Common Mistakes:
5. For a system requiring both low latency and strong consistency in user profile updates, which architecture choice best balances these constraints?
medium
A. Use eventual consistency with asynchronous replication to maximize availability
B. Use a single primary database with synchronous replication for strong consistency
C. Use a distributed cache with write-behind to the database for low latency
D. Use client-side caching to reduce read latency without server coordination

Solution

  1. Step 1: Identify consistency and latency requirements

    Strong consistency requires synchronous replication; low latency demands minimal delay.

  2. Step 2: Evaluate architecture options

    Single primary DB with sync replication ensures consistency but may increase latency slightly; best trade-off.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Synchronous replication balances strong consistency and latency [check]
Hint: Strong consistency needs sync replication, not eventual or cache-only [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard