Debugging service connectivity in Kubernetes - Mini Project: Build & Apply

Debugging service connectivity

📖 Scenario: You are managing a Kubernetes cluster where a simple web service is deployed. The service is supposed to be reachable from inside the cluster, but users report it is not responding. You need to check the service configuration and connectivity step-by-step to find the issue.

🎯 Goal: Learn how to check Kubernetes service and pod connectivity using basic kubectl commands and simple network tests inside the cluster.

📋 What You'll Learn

Use kubectl commands to inspect pods and services

Check pod status and service endpoints

Test connectivity using kubectl exec and curl

Understand basic troubleshooting steps for Kubernetes service connectivity

💡 Why This Matters

🌍 Real World

Kubernetes services are the main way to expose applications inside a cluster. Troubleshooting connectivity issues is a common real-world task to keep apps running smoothly.

💼 Career

DevOps engineers and site reliability engineers often debug service connectivity problems to ensure applications are reachable and healthy.

1

Check the pods in the default namespace

Run the command kubectl get pods -n default to list all pods in the default namespace and check their status.

Kubernetes

# Run kubectl get pods -n default to see pods
# Your code here

Hint

This command lists all pods in the default namespace with their status. Look for pods that are not Running.

2

Check the service details for web-service

Run the command kubectl get service web-service -n default to see the service details including its cluster IP and ports.

Kubernetes

kubectl get pods -n default
# Run kubectl get service web-service -n default to check service details
# Your code here

Hint

This command shows the IP and ports the service listens on inside the cluster.

3

Check the endpoints for web-service

Run the command kubectl get endpoints web-service -n default to verify which pod IPs the service routes to.

Kubernetes

kubectl get pods -n default
kubectl get service web-service -n default
# Run kubectl get endpoints web-service -n default to check endpoints
# Your code here

Hint

Endpoints show the actual pod IPs behind the service. If empty, the service has no pods to route to.

4

Test connectivity to web-service from a pod

Run the command kubectl exec -n default -it web-pod -- curl -sS http://web-service:80 to test if the service is reachable from inside the cluster. Replace web-pod with the name of a running pod in the default namespace.

Kubernetes

kubectl get pods -n default
kubectl get service web-service -n default
kubectl get endpoints web-service -n default
# Run kubectl exec -n default -it web-pod -- curl -sS http://web-service:80 to test connectivity
# Your code here

Hint

This command runs curl inside a pod to check if the service responds on port 80.

Practice

1. What is the primary command to check if a Kubernetes service has endpoints assigned?

easy

A. kubectl describe nodes

B. kubectl get pods

C. kubectl get endpoints

D. kubectl get configmaps

Solution

1. Step 1: Understand service connectivity basics

   Services route traffic to endpoints, so checking endpoints shows if pods are linked.

2. Step 2: Use the correct command to list endpoints

   kubectl get endpoints lists endpoints for services, showing if pods are ready.

3. Final Answer:

   kubectl get endpoints -> Option C

4. Quick Check:

   Check endpoints = kubectl get endpoints [OK]

Hint: Use 'kubectl get endpoints' to verify service pod connections [OK]

Common Mistakes:

• Using 'kubectl get pods' which shows pods but not service endpoints
• Checking nodes or configmaps which are unrelated to service endpoints
• Confusing 'kubectl describe svc' with listing endpoints

2. Which of the following commands correctly tests DNS resolution inside a Kubernetes pod named web-123?

easy

A. kubectl exec web-123 -- nslookup myservice

B. kubectl exec web-123 nslookup myservice

C. kubectl exec -it web-123 nslookup myservice

D. kubectl exec web-123 -- curl myservice

Solution

1. Step 1: Understand kubectl exec syntax

   The correct syntax to run a command inside a pod is kubectl exec [pod] -- [command].

2. Step 2: Identify the command to test DNS

   nslookup tests DNS resolution, so kubectl exec web-123 -- nslookup myservice is correct.

3. Final Answer:

   kubectl exec web-123 -- nslookup myservice -> Option A

4. Quick Check:

   Correct exec syntax + nslookup = kubectl exec web-123 -- nslookup myservice [OK]

Hint: Use '--' before command in kubectl exec to run inside pod [OK]

Common Mistakes:

• Omitting '--' which causes command to fail
• Using '-it' without need for interactive shell
• Using curl instead of nslookup for DNS test

3. You run kubectl describe svc myservice and see no endpoints listed. What will be the output of kubectl get endpoints myservice?

medium

A. Error from server (NotFound): endpoints "myservice" not found

B. NAME ENDPOINTS AGE
myservice 10.0.0.5:80,10.0.0.6:80 10m

C. NAME ENDPOINTS AGE
myservice 127.0.0.1:80 10m

D. NAME ENDPOINTS AGE
myservice <none> 10m

Solution

1. Step 1: Interpret service describe output

   No endpoints means no pods are linked to the service, so endpoints list is empty.

2. Step 2: Predict endpoints output

   kubectl get endpoints myservice will show the service name with <none> under ENDPOINTS.

3. Final Answer:

   NAME ENDPOINTS AGE myservice <none> 10m -> Option D

4. Quick Check:

   No endpoints = <none> shown [OK]

Hint: No endpoints in describe means endpoints show <none> [OK]

Common Mistakes:

• Assuming endpoints will list IPs even if none exist
• Expecting an error when endpoints exist but are empty
• Confusing endpoints with pod IPs

4. A pod cannot reach a service by its DNS name. You run kubectl exec pod1 -- nslookup myservice and get a timeout. What is the most likely cause?

medium

A. The pod is missing the DNS policy or DNS is misconfigured

B. The service has no endpoints, so DNS resolves but no response

C. The service selector labels do not match any pods

D. The pod is running in a different namespace without DNS search path

Solution

1. Step 1: Analyze DNS timeout symptom

   A DNS timeout means the pod cannot resolve the service name, indicating DNS issues.

2. Step 2: Identify DNS misconfiguration causes

   Missing DNS policy or broken DNS config in pod causes nslookup timeout, unlike no endpoints which still resolve DNS.

3. Final Answer:

   The pod is missing the DNS policy or DNS is misconfigured -> Option A

4. Quick Check:

   DNS timeout = DNS config issue [OK]

Hint: DNS timeout means DNS config or policy problem, not endpoints [OK]

Common Mistakes:

• Confusing DNS resolution failure with no endpoints
• Assuming label mismatch causes DNS timeout instead of no response
• Ignoring namespace DNS search path issues

5. You have a service myservice in namespace prod. A pod in namespace dev tries to connect using curl myservice but fails. Which is the best way to debug this connectivity issue?

hard

A. Run kubectl describe pod -n prod myservice to check pod details

B. Run kubectl exec -n dev pod -- curl myservice.prod.svc.cluster.local to test full DNS name

C. Run kubectl get svc -n dev myservice to check service in dev namespace

D. Run kubectl exec -n prod pod -- curl myservice to test from the service namespace

Solution

1. Step 1: Understand cross-namespace service access

   Pods in different namespaces must use the full DNS name including namespace to reach a service.

2. Step 2: Test connectivity using full DNS name from the pod in dev namespace

   Running kubectl exec -n dev pod -- curl myservice.prod.svc.cluster.local tests correct DNS and connectivity.

3. Final Answer:

   Run kubectl exec -n dev pod -- curl myservice.prod.svc.cluster.local to test full DNS name -> Option B

4. Quick Check:

   Cross-namespace access needs full DNS name [OK]

Hint: Use full DNS name with namespace for cross-namespace service access [OK]

Common Mistakes:

• Trying to curl service without namespace from another namespace
• Checking service in wrong namespace
• Describing pod instead of testing connectivity