Bird
Raised Fist0

Metrics Collection System Design - Push vs Pull Model Trade-offs

HLDmediumInterview freq: common
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
▶
Request Trace
Step 1/6
Metrics Clients

Push metrics data

Client sends metrics via HTTP push to ingestion layer

Protocol:HTTP/GRPC
BeforeMetrics data generated on client, not yet sent
AfterMetrics data sent over network to ingestion layer
Ingestion Layer

Receive and validate metrics

Ingestion layer validates and transforms metrics, then produces to Kafka

Protocol:Kafka Produce
BeforeMetrics data received but not yet persisted
AfterMetrics data durably stored in Kafka topic partition
Message Broker (Apache Kafka)

Store and forward metrics events

Kafka stores metrics events partitioned by metric type for ordering

Protocol:Kafka Consume
BeforeMetrics events enqueued in Kafka topic
AfterMetrics events available for processing layer consumption
Processing Layer (Apache Flink/Beam)

Stream process metrics

Processes streams for aggregation, filtering, and enrichment

Protocol:Write Batch
BeforeRaw metrics events waiting in Kafka
AfterAggregated metrics batches written to Cassandra
Storage (Apache Cassandra)

Store aggregated metrics

Stores time-series metrics data optimized for fast writes and queries

Protocol:CQL (Cassandra Query Language)
BeforeAggregated metrics batch received from processing layer
AfterAggregated metrics durably stored and indexed in Cassandra
Query API

Serve metrics queries

Receives query requests and fetches data from Cassandra

Protocol:REST/gRPC
BeforeQuery request received from user or dashboard
AfterQuery results returned with latest aggregated metrics
🏗️
Architecture Diagram
Architecture Diagram
┌───────────────────┐
│  Metrics Clients  │
└─────────┬─────────┘
          │ HTTP/GRPC Push or Pull
          ▼
┌───────────────────┐
│  Ingestion Layer  │
│  (API Gateway)    │
└─────────┬─────────┘
          │ Kafka Produce
          ▼
┌───────────────────────────────┐
│ Message Broker (Apache Kafka) │
│  (Partitioned by metric type) │
└───────────────┬───────────────┘
                │ Kafka Consume
                ▼
┌───────────────────────────────────────┐
│ Processing Layer (Flink/Beam)          │
│  (Stream processing, aggregation, filt)│
└───────────────┬───────────────────────┘
                │ Write Batch
                ▼
┌───────────────────────────────┐
│ Storage (Apache Cassandra)     │
│  (Time-series optimized store)│
└───────────────┬───────────────┘
                │ Query
                ▼
┌───────────────┐
│   Query API   │
│ (REST/gRPC)   │
└───────────────┘
Design Rationale

This architecture illustrates a metrics collection system where clients push or pull metrics data to an ingestion layer. The ingestion layer validates and forwards data to a durable Kafka message broker, which partitions data by metric type to preserve ordering. The processing layer consumes Kafka streams to perform real-time aggregation and filtering, then writes batches to Cassandra, a time-series optimized database. Finally, the Query API serves user queries by fetching aggregated metrics from Cassandra. This design balances push and pull trade-offs by centralizing ingestion and decoupling processing via Kafka, enabling scalable, fault-tolerant metrics collection and querying.

Primary Focus

✓

Practice

(1/5)
1. In a gRPC streaming call, at step 4 the server-side stream handler checks the client connection status before sending a message. What happens if the client has disconnected at this step?
easy
A. The server immediately cancels the stream and releases resources
B. The server buffers the message indefinitely until the client reconnects
C. The server retries sending the message up to 3 times before dropping it
D. The server switches to unary RPC mode to deliver the message

Solution

  1. Step 1: Understand server stream handler behavior

    When client disconnects, server detects cancellation via context and stops streaming.

  2. Step 2: Trace server reaction at step 4

    Server cancels the stream and frees resources immediately; it does not buffer or retry indefinitely.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Client disconnect -> server cancels stream and frees resources [check]
Hint: Disconnected client -> server cancels stream immediately [check]
Common Mistakes:
2. A live video streaming service requires minimal latency and can tolerate occasional packet loss. Which transport protocol is most appropriate?
easy
A. UDP, because it provides low latency without retransmission delays
B. SCTP, because it supports multi-streaming and multi-homing
C. TCP, because it guarantees packet delivery and ordering
D. HTTP over TLS, because it ensures secure and reliable delivery

Solution

  1. Step 1: Identify latency and loss tolerance requirements

    Live video streaming prioritizes low latency and can tolerate some packet loss, so retransmission delays are undesirable.

  2. Step 2: Match protocol characteristics

    UDP provides minimal overhead and no retransmission, fitting low latency needs. TCP guarantees delivery but adds delay due to retransmissions and congestion control.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    UDP suits low-latency, lossy streaming [check]
Hint: UDP trades reliability for speed in latency-sensitive apps [check]
Common Mistakes:
3. At 10x scale, which component in an HTTP/3 deployment is most likely to become the bottleneck first and why?
hard
A. WebSocket servers because HTTP/3 does not support WebSockets natively
B. TLS handshake CPU load on servers due to QUIC's per-connection encryption overhead
C. UDP packet loss rate increases causing retransmissions and latency spikes
D. TCP connection limits because HTTP/3 uses TCP under the hood

Solution

  1. Step 1: Identify HTTP/3 transport and encryption

    HTTP/3 uses QUIC over UDP with built-in TLS encryption per connection, increasing CPU load for handshake and encryption at scale.

  2. Step 2: Eliminate other bottlenecks

    UDP packet loss is network-level, not server bottleneck; WebSockets are not natively supported by HTTP/3 but unrelated to scaling HTTP/3 itself; HTTP/3 does not use TCP.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    QUIC's per-connection TLS handshake CPU load limits HTTP/3 scale [check]
Hint: QUIC TLS handshake CPU load limits HTTP/3 scale first [check]
Common Mistakes:
4. At 10x traffic scale, which component in a Layer 7 load balancing architecture is most likely to become the bottleneck first, and why?
hard
A. Layer 7 load balancer CPU, due to expensive HTTP header parsing and SSL termination
B. Network switches, because increased packet forwarding saturates bandwidth
C. Backend servers, because they cannot handle increased HTTP request volume
D. DNS servers, because client-side resolution delays increase exponentially

Solution

  1. Step 1: Identify Layer 7 load balancer workload at scale

    Layer 7 load balancers perform CPU-intensive tasks like HTTP header parsing and SSL termination, which scale poorly with traffic.

  2. Step 2: Compare with other components

    Backend servers can be scaled horizontally; network switches handle packet forwarding efficiently; DNS servers are less impacted by traffic volume.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Layer 7 LB CPU bottleneck due to HTTP parsing and SSL termination [check]
Hint: Layer 7 LB CPU chokes first at high traffic scale [check]
Common Mistakes:
5. At 10x scale, which component in a multi-tier storage system is most likely to become the bottleneck first, and why?
hard
A. Archive storage because tape drives have limited concurrent access bandwidth
B. Metadata service because it maintains global access frequency and tiering decisions, causing high coordination overhead
C. Cold storage because object storage scales linearly with data size
D. Warm storage because HDDs have slower seek times but can be scaled horizontally easily

Solution

  1. Step 1: Identify component roles at scale

    Archive storage uses tape drives with limited concurrent access bandwidth and high latency.

  2. Step 2: Analyze scaling challenges

    At 10x scale, tape drives become bottleneck due to limited parallelism and slow seek times, unlike metadata which can be sharded.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Archive tape bandwidth limits -> first bottleneck [check]
Hint: Tape archive bandwidth limits scale poorly [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard