Mutual TLS for service communication in Kubernetes - Mini Project: Build & Apply

Mutual TLS for service communication

📖 Scenario: You are setting up secure communication between two services in a Kubernetes cluster. To protect data and verify identities, you will configure mutual TLS (mTLS) so both services trust each other before exchanging information.

🎯 Goal: Build a simple Kubernetes setup where two services communicate securely using mutual TLS. You will create certificates, configure secrets, and update service deployment manifests to enable mTLS.

📋 What You'll Learn

Create Kubernetes secrets to hold TLS certificates

Configure service deployments to use TLS certificates

Enable mutual TLS between two services

Verify secure communication by checking pod logs

💡 Why This Matters

🌍 Real World

Mutual TLS is used in microservices to ensure encrypted and authenticated communication, preventing eavesdropping and impersonation.

💼 Career

Understanding mTLS setup in Kubernetes is essential for DevOps engineers and SREs to secure service-to-service communication in cloud-native environments.

1

Create TLS certificates as Kubernetes secrets

Create two Kubernetes secrets named service-a-tls and service-b-tls using TLS certificate files service-a.crt, service-a.key and service-b.crt, service-b.key respectively. Use the kubectl create secret tls command for each secret.

Kubernetes

# Create TLS secrets for service-a and service-b
# Your code here

Hint

Use kubectl create secret tls <secret-name> --cert=<cert-file> --key=<key-file> to create each secret.

2

Add volume mounts for TLS secrets in service deployments

Edit the deployment YAML files for service-a and service-b to add a volume named tls-certs that uses the corresponding TLS secret (service-a-tls for service-a, service-b-tls for service-b). Then add a volume mount at /etc/tls in the container spec to use this volume.

Kubernetes

kubectl create secret tls service-a-tls --cert=service-a.crt --key=service-a.key
kubectl create secret tls service-b-tls --cert=service-b.crt --key=service-b.key
# Add volumes and volumeMounts for TLS secrets in deployment YAMLs
# Your code here

Hint

Add a volumes section referencing the TLS secret, and a volumeMounts section in the container spec to mount at /etc/tls.

3

Configure services to use mutual TLS for communication

Update the service configurations or application environment variables to enable mutual TLS. Set variables TLS_CERT_PATH=/etc/tls/tls.crt and TLS_KEY_PATH=/etc/tls/tls.key for both services. Also configure each service to trust the other's certificate by mounting the peer's certificate as a trusted CA.

Kubernetes

kubectl create secret tls service-a-tls --cert=service-a.crt --key=service-a.key
kubectl create secret tls service-b-tls --cert=service-b.crt --key=service-b.key

# Set environment variables TLS_CERT_PATH and TLS_KEY_PATH in both deployments
# Mount peer service certificate as trusted CA
# Your code here

Hint

Set environment variables for TLS cert and key paths. Mount the peer service's TLS secret as a trusted CA at /etc/peer-ca.

4

Verify mutual TLS communication between services

Use kubectl logs to check the logs of service-a and service-b pods. Confirm that both services report successful TLS handshake and secure communication.

Kubernetes

kubectl create secret tls service-a-tls --cert=service-a.crt --key=service-a.key
kubectl create secret tls service-b-tls --cert=service-b.crt --key=service-b.key

# Use kubectl logs to check service-a and service-b pod logs for TLS handshake success
# Your code here

Hint

Use kubectl logs -l app=service-a and kubectl logs -l app=service-b to see logs from pods labeled app=service-a and app=service-b.

Practice

1. What is the main purpose of Mutual TLS (mTLS) in Kubernetes service communication?

easy

A. To disable encryption for faster debugging

B. To increase the speed of service communication

C. To allow services to communicate without authentication

D. To encrypt data and verify identities between services

Solution

1. Step 1: Understand mTLS purpose

   Mutual TLS encrypts data and verifies both client and server identities to secure communication.

2. Step 2: Compare options

   Only To encrypt data and verify identities between services correctly describes encryption and identity verification, others are incorrect or opposite.

3. Final Answer:

   To encrypt data and verify identities between services -> Option D

4. Quick Check:

   mTLS = encrypt + verify identities [OK]

Hint: mTLS means both encryption and identity check [OK]

Common Mistakes:

• Thinking mTLS only encrypts but doesn't verify identity
• Confusing mTLS with disabling security
• Assuming mTLS speeds up communication

2. Which PeerAuthentication mode in Istio allows both encrypted (mTLS) and plain traffic to a service?

easy

A. STRICT

B. PERMISSIVE

C. DISABLE

D. ENFORCED

Solution

1. Step 1: Recall PeerAuthentication modes

   STRICT enforces mTLS only, PERMISSIVE allows both mTLS and plain, DISABLE turns off mTLS.

2. Step 2: Match mode to description

   PERMISSIVE mode allows both encrypted and plain traffic, matching the question.

3. Final Answer:

   PERMISSIVE -> Option B

4. Quick Check:

   PERMISSIVE = both encrypted and plain allowed [OK]

Hint: PERMISSIVE means allow both secure and insecure traffic [OK]

Common Mistakes:

• Confusing STRICT with PERMISSIVE
• Thinking DISABLE allows encrypted traffic
• Assuming ENFORCED is a valid mode

3. Given this PeerAuthentication YAML snippet:

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: myapp
spec:
  mtls:
    mode: STRICT

What happens when a service in namespace myapp receives plain HTTP traffic?

medium

A. The traffic is rejected because mTLS is enforced

B. The traffic is accepted without encryption

C. The traffic is accepted but logged as insecure

D. The traffic is redirected to HTTPS automatically

Solution

1. Step 1: Analyze PeerAuthentication mode

   The mode is STRICT, which enforces mTLS for all incoming traffic in the namespace.

2. Step 2: Understand effect on plain HTTP

   Plain HTTP traffic without mTLS will be rejected because encryption and identity verification are mandatory.

3. Final Answer:

   The traffic is rejected because mTLS is enforced -> Option A

4. Quick Check:

   STRICT mode rejects plain HTTP [OK]

Hint: STRICT mode blocks non-mTLS traffic [OK]

Common Mistakes:

• Assuming plain HTTP is accepted in STRICT mode
• Thinking traffic is redirected automatically
• Confusing logging with rejection

4. You applied this PeerAuthentication config but your service still accepts plain HTTP traffic:

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: test
spec:
  mtls:
    mode: STRICT

What is the most likely reason?

medium

A. The service is in a different namespace than 'test'

B. PeerAuthentication resource is missing the selector field

C. STRICT mode allows plain HTTP by default

D. mTLS is disabled globally in Istio

Solution

1. Step 1: Check namespace scope

   PeerAuthentication applies only to the specified namespace 'test'. If the service is outside, it won't be affected.

2. Step 2: Understand effect on service

   If the service is in another namespace, it won't enforce STRICT mode and may accept plain HTTP.

3. Final Answer:

   The service is in a different namespace than 'test' -> Option A

4. Quick Check:

   Namespace mismatch causes no mTLS enforcement [OK]

Hint: PeerAuthentication applies per namespace only [OK]

Common Mistakes:

• Assuming STRICT mode allows plain HTTP
• Thinking selector is mandatory for namespace-wide policy
• Ignoring namespace differences

5. You want to enforce mTLS only for service payments in namespace finance, but allow other services to accept plain traffic. Which PeerAuthentication config achieves this?

hard

A. apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: permissive-all
namespace: finance
spec:
mtls:
mode: PERMISSIVE

B. apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: finance-wide
namespace: finance
spec:
mtls:
mode: STRICT

C. apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: payments-mtls
namespace: finance
spec:
selector:
matchLabels:
app: payments
mtls:
mode: STRICT

D. apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: payments-disable
namespace: finance
spec:
selector:
matchLabels:
app: payments
mtls:
mode: DISABLE

Solution

1. Step 1: Understand requirement

   Enforce mTLS STRICT only for 'payments' service, allow others to accept plain traffic.

2. Step 2: Analyze options

   apiVersion: security.istio.io/v1beta1
   kind: PeerAuthentication
   metadata:
     name: payments-mtls
     namespace: finance
   spec:
     selector:
       matchLabels:
         app: payments
     mtls:
       mode: STRICT

   applies STRICT mode with selector for 'payments' only.

   apiVersion: security.istio.io/v1beta1
   kind: PeerAuthentication
   metadata:
     name: finance-wide
     namespace: finance
   spec:
     mtls:
       mode: STRICT

   enforces STRICT for whole namespace, not desired.

   apiVersion: security.istio.io/v1beta1
   kind: PeerAuthentication
   metadata:
     name: permissive-all
     namespace: finance
   spec:
     mtls:
       mode: PERMISSIVE

   allows both for all services.

   apiVersion: security.istio.io/v1beta1
   kind: PeerAuthentication
   metadata:
     name: payments-disable
     namespace: finance
   spec:
     selector:
       matchLabels:
         app: payments
     mtls:
       mode: DISABLE

   disables mTLS for payments, opposite of requirement.

3. Final Answer:

   PeerAuthentication with selector for payments and STRICT mode -> Option C

4. Quick Check:

   Selector + STRICT = mTLS only for selected service [OK]

Hint: Use selector with STRICT mode for specific service enforcement [OK]

Common Mistakes:

• Applying STRICT mode to whole namespace accidentally
• Using DISABLE mode when enforcement is needed
• Not using selector to target specific service