System Overview - Metrics collection
This system collects and stores metrics from various services to monitor performance and health. It must handle high volumes of data, provide near real-time insights, and support querying for analysis.
0
0
Metrics collection in HLD - Architecture Diagram
Architecture Diagram
User Services
|
v
Load Balancer
|
v
API Gateway
|
v
Metrics Collector Service
|
v
Message Queue
|
v
Metrics Processor Service
|
v
Time-Series Database
|
v
Cache
|
v
Dashboard/UIComponents
User Services
service
Generate and send metrics data
Load Balancer
load_balancer
Distribute incoming metric data requests evenly
API Gateway
api_gateway
Authenticate and route metric data requests
Metrics Collector Service
service
Receive and validate incoming metrics
Message Queue
queue
Buffer metrics for asynchronous processing
Metrics Processor Service
service
Aggregate and transform metrics before storage
Time-Series Database
database
Store time-stamped metrics efficiently
Cache
cache
Speed up frequent metric queries
Dashboard/UI
service
Display metrics and analytics to users
Request Flow - 8 Hops
User Services → Load Balancer
Load Balancer → API Gateway
API Gateway → Metrics Collector Service
Metrics Collector Service → Message Queue
Metrics Processor Service → Message Queue
Metrics Processor Service → Time-Series Database
Dashboard/UI → Cache
Cache → Dashboard/UI
Failure Scenario
Component Fails:Message Queue
Impact:Metrics processing delays; metrics may be lost if queue is full or down
Mitigation:Use replicated queue clusters and persistent storage to avoid data loss; alert on queue health
Architecture Quiz - 3 Questions
Test your understanding
Which component buffers metrics for asynchronous processing?
Design Principle
This design uses asynchronous processing with a message queue to handle high volumes of metrics efficiently. Caching improves query speed for dashboards, and a time-series database stores data optimized for time-based queries.