Liveness and readiness probes in Microservices - Scalability & System Analysis

Scalability Analysis - Liveness and readiness probes

Growth Table: Liveness and Readiness Probes at Different Scales

Users	What Changes?
100 users	Basic probes configured; simple health checks suffice; low frequency checks.
10,000 users	Increased probe frequency; readiness probes critical to avoid routing traffic to unhealthy pods; some probe failures start to impact service availability.
1,000,000 users	Probes must be lightweight and fast to avoid overhead; complex readiness logic to handle dependencies; automated restarts based on liveness probes prevent cascading failures.
100,000,000 users	Probes integrated with advanced monitoring and alerting; distributed health checks; probe endpoints optimized for minimal resource use; readiness probes coordinate with service mesh for traffic routing.

First Bottleneck

The first bottleneck is the application server CPU and memory due to probe overhead. As user traffic grows, frequent liveness and readiness probes add load. If probes are heavy or slow, they consume resources, reducing capacity to serve real requests.

Scaling Solutions

• Optimize probe logic: Make probes lightweight and fast to minimize resource use.
• Adjust probe frequency: Balance between timely detection and resource consumption.
• Horizontal scaling: Add more pod instances to distribute probe and user traffic load.
• Use caching: Cache probe results briefly if possible to reduce repeated expensive checks.
• Service mesh integration: Use mesh features to manage readiness and traffic routing efficiently.
• Separate probe endpoints: Design dedicated endpoints optimized for probes to avoid impacting main app performance.

Back-of-Envelope Cost Analysis

• Assuming 1 probe per pod every 10 seconds, 100 pods -> 10 probes/sec.
• At 1,000 pods, 100 probes/sec; at 10,000 pods, 1,000 probes/sec.
• Each probe request is small (~1 KB), so bandwidth is low (e.g., 1,000 probes/sec x 1 KB = ~1 MB/s).
• CPU overhead depends on probe complexity; simple HTTP GET probes cost minimal CPU.
• Storage impact negligible as probes do not store data but monitoring logs may grow.

Interview Tip

When discussing scalability of liveness and readiness probes, start by explaining their purpose. Then describe how probe frequency and complexity affect resource usage. Discuss how this overhead grows with scale and identify the bottleneck (CPU/memory). Finally, propose solutions like optimizing probes, adjusting frequency, horizontal scaling, and integration with service mesh.

Self Check

Your database handles 1000 QPS. Traffic grows 10x. What do you do first?

Answer: Since the database is the bottleneck, first add read replicas or caching to reduce load. For probes, ensure they remain lightweight to not add extra load on the database or app servers.

Key Result

Liveness and readiness probes must be lightweight and optimized as user scale grows to prevent CPU and memory bottlenecks; horizontal scaling and probe frequency tuning are key solutions.

Practice

1. What is the main purpose of a liveness probe in microservices?

easy

A. To check if the service is ready to accept traffic

B. To log user requests for debugging

C. To monitor the network latency between services

D. To check if the service is alive and restart it if it is not

Solution

1. Step 1: Understand the role of liveness probes

   Liveness probes detect if a service is stuck or dead and need restarting.

2. Step 2: Differentiate from readiness probes

   Readiness probes check if the service can handle requests, not if it is alive.

3. Final Answer:

   To check if the service is alive and restart it if it is not -> Option D

4. Quick Check:

   Liveness probe = check alive and restart [OK]

Hint: Liveness = alive and restart, Readiness = ready for traffic [OK]

Common Mistakes:

• Confusing liveness with readiness probes
• Thinking liveness probes check traffic readiness
• Assuming liveness probes monitor performance

2. Which of the following is the correct syntax to define a readiness probe in a Kubernetes pod spec?

easy

A. livenessProbe:
exec:
command: ["cat", "/tmp/healthy"]
timeoutSeconds: 1

B. livenessProbe:
tcpSocket:
port: 8080
initialDelaySeconds: 5
periodSeconds: 10

C. readinessProbe:
httpGet:
path: /healthz
port: 8080
initialDelaySeconds: 5
periodSeconds: 10

D. livenessProbe:
httpGet:
path: /ready
port: 8080
failureThreshold: 3

Solution

1. Step 1: Identify readiness probe syntax

   Readiness probes often use httpGet with path and port, plus delay and period settings.

2. Step 2: Confirm correct fields and indentation

   readinessProbe: httpGet: path: /healthz port: 8080 initialDelaySeconds: 5 periodSeconds: 10 correctly shows readinessProbe with httpGet, initialDelaySeconds, and periodSeconds.

3. Final Answer:

   readinessProbe: httpGet: path: /healthz port: 8080 initialDelaySeconds: 5 periodSeconds: 10 -> Option C

4. Quick Check:

   Readiness probe syntax = readinessProbe: httpGet: path: /healthz port: 8080 initialDelaySeconds: 5 periodSeconds: 10 [OK]

Hint: Readiness uses httpGet with path and port in YAML [OK]

Common Mistakes:

• Mixing livenessProbe and readinessProbe fields
• Incorrect indentation in YAML
• Using wrong probe type for readiness

3. Given this Kubernetes pod spec snippet, what will happen if the readiness probe fails continuously?

readinessProbe:
  httpGet:
    path: /ready
    port: 8080
  initialDelaySeconds: 5
  periodSeconds: 10
  failureThreshold: 3

medium

A. The pod will be restarted immediately

B. The pod will be marked as not ready and removed from service endpoints

C. The pod will ignore the failure and continue serving traffic

D. The pod will scale up automatically

Solution

1. Step 1: Understand readiness probe failure effect

   Readiness probe failure marks pod as not ready, so it stops receiving traffic.

2. Step 2: Differentiate from liveness probe effect

   Liveness probe failure triggers pod restart, readiness does not.

3. Final Answer:

   The pod will be marked as not ready and removed from service endpoints -> Option B

4. Quick Check:

   Readiness failure = pod not ready, no restart [OK]

Hint: Readiness failure removes pod from load balancer, no restart [OK]

Common Mistakes:

• Confusing readiness failure with pod restart
• Assuming pod scales automatically on probe failure
• Ignoring failureThreshold effect

4. A microservice has a liveness probe configured as an HTTP GET on /health. The service sometimes returns HTTP 500 during startup but is healthy afterward. What is the best fix to avoid unnecessary restarts?

medium

A. Increase initialDelaySeconds to allow startup time before probing

B. Change the probe to readiness probe instead of liveness probe

C. Remove the probe completely to avoid restarts

D. Set failureThreshold to 1 to detect failures faster

Solution

1. Step 1: Identify cause of restarts

   Liveness probe fails during startup because service returns HTTP 500 before ready.

2. Step 2: Adjust probe timing to avoid false failures

   Increasing initialDelaySeconds delays probe start, allowing service to become healthy first.

3. Final Answer:

   Increase initialDelaySeconds to allow startup time before probing -> Option A

4. Quick Check:

   Delay liveness probe start to avoid false failures [OK]

Hint: Delay liveness probe start to avoid false failure during startup [OK]

Common Mistakes:

• Removing probes which reduces reliability
• Confusing readiness and liveness probe roles
• Setting failureThreshold too low causing quick restarts

5. You have a microservice that takes time to initialize resources before it can serve requests. You want to ensure it is not restarted unnecessarily but also not receive traffic before ready. How should you configure liveness and readiness probes?

hard

A. Set liveness probe with a longer initialDelaySeconds and readiness probe to check resource initialization

B. Use only a liveness probe with a short periodSeconds to restart fast

C. Use only a readiness probe and no liveness probe

D. Set both probes to the same HTTP path and timing

Solution

1. Step 1: Prevent unnecessary restarts during initialization

   Set liveness probe initialDelaySeconds long enough to avoid restarting while initializing.

2. Step 2: Use readiness probe to block traffic until ready

   Readiness probe should check if resources are initialized before accepting traffic.

3. Final Answer:

   Set liveness probe with a longer initialDelaySeconds and readiness probe to check resource initialization -> Option A

4. Quick Check:

   Liveness delay + readiness check = safe startup [OK]

Hint: Delay liveness, readiness blocks traffic until ready [OK]

Common Mistakes:

• Using only one probe type causing traffic or restart issues
• Setting same path and timing for both probes
• Not delaying liveness probe causing premature restarts