Health check endpoints in HLD - Scalability & System Analysis

Scalability Analysis - Health check endpoints

Growth Table: Health Check Endpoints Scaling

Users	Requests per Second (RPS)	Health Check Complexity	Infrastructure Impact
100 users	~10-50 RPS	Simple / Basic (e.g., HTTP 200)	Minimal load, single server handles checks
10,000 users	~1,000-5,000 RPS	Include dependency checks (DB, cache)	Load balancer health checks start impacting servers
1,000,000 users	~100,000 RPS	Lightweight checks, caching results	Dedicated health check service or endpoint rate limiting needed
100,000,000 users	~10,000,000+ RPS	Aggregated health status, asynchronous checks	Distributed health monitoring system, offload checks from main servers

First Bottleneck

The first bottleneck is the application servers' CPU and network resources. Health check endpoints, if not optimized, generate frequent requests that consume CPU cycles and network bandwidth. At medium scale (~10,000 users), the load balancer's frequent health checks can overwhelm servers if checks are heavy (e.g., checking multiple dependencies synchronously).

Scaling Solutions

• Lightweight Checks: Keep health checks simple and fast, returning cached or precomputed status.
• Rate Limiting: Limit frequency of health check requests to reduce load.
• Dedicated Health Check Service: Offload health checks to separate service or sidecar to avoid impacting main app.
• Asynchronous Checks: Perform dependency checks asynchronously and cache results for quick responses.
• Load Balancer Configuration: Adjust health check intervals and timeouts to balance freshness and load.
• Distributed Monitoring: Use external monitoring tools to reduce internal health check traffic.

Back-of-Envelope Cost Analysis

Assuming 10,000 users generating ~5,000 health check requests per second:

• Each health check response ~1 KB -> 5,000 KB/s (~5 MB/s) bandwidth per server.
• CPU usage: lightweight checks consume ~1-5% CPU per 1,000 RPS; at 5,000 RPS, ~5-25% CPU.
• Memory: minimal, mostly for caching health status (~few MBs).
• Scaling servers horizontally reduces per-server load.

Interview Tip

When discussing health check endpoint scalability, start by explaining the purpose of health checks. Then describe how load increases with users and how naive implementations can overload servers. Discuss bottlenecks clearly, then propose practical solutions like caching, rate limiting, and dedicated services. Use real numbers to show understanding of impact.

Self Check Question

Your database handles 1000 QPS. Traffic grows 10x. What do you do first?

Answer: Since the database is the bottleneck, first add read replicas and implement caching to reduce direct database queries. Also, optimize queries and consider rate limiting health check frequency to reduce load.

Key Result

Health check endpoints start simple but can overload servers as user count grows; optimizing checks, caching, and offloading are key to scaling.

Practice

1. What is the primary purpose of a health check endpoint in a system?

easy

A. To verify if the service is alive and functioning

B. To store user data securely

C. To handle user authentication

D. To serve static website content

Solution

1. Step 1: Understand the role of health check endpoints

   Health check endpoints are designed to tell if a service is alive and working properly.

2. Step 2: Identify the correct purpose among options

   Only To verify if the service is alive and functioning describes checking the service's health, while others describe unrelated functions.

3. Final Answer:

   To verify if the service is alive and functioning -> Option A

4. Quick Check:

   Health check = Verify service alive [OK]

Hint: Health checks confirm service status, not data or auth [OK]

Common Mistakes:

• Confusing health check with authentication
• Thinking health check stores data
• Assuming health check serves website content

2. Which URL path is commonly used for a readiness health check endpoint?

easy

A. /ready

B. /health

C. /status

D. /check

Solution

1. Step 1: Recall standard health check paths

   Readiness endpoints typically use the path /ready to indicate if the service is ready to accept traffic.

2. Step 2: Match the correct path

   Among the options, only /ready is the standard path for readiness checks.

3. Final Answer:

   /ready -> Option A

4. Quick Check:

   Readiness path = /ready [OK]

Hint: Readiness = /ready, Liveness = /health [OK]

Common Mistakes:

• Using /health for readiness instead of liveness
• Confusing /status or /check as standard paths
• Mixing readiness and liveness endpoints

3. Consider this pseudocode for a health check endpoint:

if database.isConnected() and cache.isAvailable():
    return 200
else:
    return 503

What HTTP status code will the endpoint return if the database is connected but the cache is down?

medium

A. 200

B. 503

C. 404

D. 500

Solution

1. Step 1: Analyze the condition logic

   The endpoint returns 200 only if both database.isConnected() and cache.isAvailable() are true.

2. Step 2: Evaluate the given scenario

   Database is connected (true), cache is down (false), so condition fails and returns 503.

3. Final Answer:

   503 -> Option B

4. Quick Check:

   Both must be true for 200; else 503 [OK]

Hint: Both services must be healthy for 200 OK [OK]

Common Mistakes:

• Assuming 200 if only one service is healthy
• Confusing 503 with 404 or 500
• Ignoring the AND condition in the code

4. A health check endpoint code snippet:

def health_check():
    if db.connected() or cache.available():
        return 200
    else:
        return 503

What is the main problem with this health check logic?

medium

A. It does not check any services

B. It returns 503 when both services are healthy

C. It returns 200 even if only one service is healthy

D. It uses incorrect HTTP status codes

Solution

1. Step 1: Understand the logical operator used

   The code uses OR, so it returns 200 if either db.connected() or cache.available() is true.

2. Step 2: Identify the issue with health check accuracy

   This means it reports healthy even if one service is down, which can mislead monitoring systems.

3. Final Answer:

   It returns 200 even if only one service is healthy -> Option C

4. Quick Check:

   OR allows partial health, which is incorrect [OK]

Hint: Use AND to ensure all services are healthy [OK]

Common Mistakes:

• Using OR instead of AND in health checks
• Ignoring partial failures in health status
• Misinterpreting HTTP status codes

5. You design a system with a /health endpoint for liveness and a /ready endpoint for readiness. Which of the following best describes when the /ready endpoint should return HTTP 503?

hard

A. When the service is completely down and unreachable

B. When the service is alive and ready to accept traffic

C. When the service is overloaded but still responding

D. When the service is alive but not ready to accept traffic due to missing dependencies

Solution

1. Step 1: Understand readiness vs liveness

   Liveness (/health) checks if service is alive; readiness (/ready) checks if it can handle requests safely.

2. Step 2: Identify readiness failure condition

   Readiness returns 503 if dependencies (like database) are missing, so service should not receive traffic yet.

3. Final Answer:

   When the service is alive but not ready to accept traffic due to missing dependencies -> Option D

4. Quick Check:

   Readiness 503 = alive but not ready [OK]

Hint: Ready means all dependencies OK; else 503 [OK]

Common Mistakes:

• Confusing liveness failure with readiness failure
• Returning 503 when service is fully down (should be liveness)
• Assuming overload means readiness failure