Node affinity and anti-affinity in Kubernetes - Commands & Configuration

Introduction

Sometimes you want your app to run on specific computers in your cluster or avoid certain ones. Node affinity and anti-affinity help you tell Kubernetes where to place your app pods based on labels on the nodes.

When you want your app to run only on nodes with special hardware like GPUs.

When you want to keep your app pods away from nodes that run other critical apps to avoid resource conflicts.

When you want to spread your app pods across different nodes for better availability.

When you want to schedule pods on nodes in a specific data center or zone.

When you want to avoid placing multiple pods of the same app on the same node to reduce risk.

Config File - pod-node-affinity.yaml

pod-node-affinity.yaml

apiVersion: v1
kind: Pod
metadata:
  name: example-pod
spec:
  containers:
  - name: nginx
    image: nginx:1.23
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: disktype
            operator: In
            values:
            - ssd
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 1
        preference:
          matchExpressions:
          - key: zone
            operator: In
            values:
            - us-east-1a
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: app
            operator: In
            values:
            - example-app
        topologyKey: kubernetes.io/hostname

This file creates a pod named example-pod running nginx.

The nodeAffinity section has two parts:

• requiredDuringSchedulingIgnoredDuringExecution: The pod must run on nodes labeled disktype=ssd.
• preferredDuringSchedulingIgnoredDuringExecution: It prefers nodes in the us-east-1a zone but can run elsewhere if needed.

The podAntiAffinity section prevents this pod from running on the same node as other pods labeled app=example-app, spreading pods across nodes.

Commands

This command creates the pod with the node affinity and anti-affinity rules defined in the YAML file.

Terminal

kubectl apply -f pod-node-affinity.yaml

Expected Output

pod/example-pod created

This command shows the pods with details including which node they are running on, so you can verify the pod placement.

Terminal

kubectl get pods -o wide

Expected Output

NAME          READY   STATUS    RESTARTS   AGE   IP           NODE           NOMINATED NODE   READINESS GATES

example-pod   1/1     Running   0          10s   10.244.1.5   node-ssd-01    <none>           <none>

→

-o wide - Shows extra details including the node name where the pod is running

This command shows detailed information about the pod, including the node affinity and anti-affinity rules applied and scheduling decisions.

Terminal

kubectl describe pod example-pod

Expected Output

Name:           example-pod

Namespace:      default

Node:           node-ssd-01/192.168.1.10

Start Time:     Thu, 01 Jun 2023 12:00:00 +0000

Labels:         <none>

Annotations:    <none>

Status:         Running

IP:             10.244.1.5

Containers:

  nginx:

    Image:        nginx:1.23

    State:        Running

    Ready:        True

    Restart Count: 0

Conditions:

  Type           Status

  Initialized    True 

  Ready          True 

  ContainersReady True 

  PodScheduled   True 

Events:

  Type    Reason     Age   From               Message

  ----    ------     ----  ----               -------

  Normal  Scheduled  10s   default-scheduler  Successfully assigned default/example-pod to node-ssd-01

Key Concept

If you remember nothing else from this pattern, remember: node affinity lets you choose nodes for your pods, and anti-affinity helps you avoid placing pods together on the same node.

Common Mistakes

Using node labels that do not exist on any node

The pod will stay pending because Kubernetes cannot find a node matching the affinity rules.

Check node labels with 'kubectl get nodes --show-labels' and use existing labels in your affinity rules.

Mixing required and preferred affinity without understanding their difference

Required rules must be met or the pod won't schedule; preferred rules are just preferences and may be ignored.

Use requiredDuringSchedulingIgnoredDuringExecution for must-have conditions and preferredDuringSchedulingIgnoredDuringExecution for nice-to-have conditions.

Not specifying topologyKey correctly in pod anti-affinity

Without a proper topologyKey, Kubernetes cannot spread pods as intended, leading to pods possibly running on the same node.

Use a valid topologyKey like 'kubernetes.io/hostname' to spread pods across nodes.

Summary

Create a pod YAML with nodeAffinity to specify which nodes the pod should run on.

Add podAntiAffinity to prevent pods from running on the same node as certain other pods.

Apply the YAML with kubectl and verify pod placement with 'kubectl get pods -o wide' and 'kubectl describe pod'.

Practice

1. What does nodeAffinity in Kubernetes primarily control?

easy

A. Which nodes a pod can be scheduled on based on node labels

B. The order in which pods start in a deployment

C. The network policies applied to pods

D. The resource limits of a pod

Solution

1. Step 1: Understand nodeAffinity purpose

   Node affinity is used to specify rules about which nodes a pod can run on, based on labels assigned to nodes.

2. Step 2: Compare options to nodeAffinity function

   Options B, C, and D relate to pod startup order, network policies, and resource limits, which are unrelated to node affinity.

3. Final Answer:

   Which nodes a pod can be scheduled on based on node labels -> Option A

4. Quick Check:

   nodeAffinity controls pod placement on nodes [OK]

Hint: Node affinity = pod placement by node labels [OK]

Common Mistakes:

• Confusing node affinity with pod resource limits
• Thinking node affinity controls pod startup order
• Mixing node affinity with network policies

2. Which of the following is the correct syntax to specify a required node affinity in a pod spec?

easy

A. "nodeAffinity": {"preferredDuringSchedulingIgnoredDuringExecution": [{"weight": 1, "preference": {"matchExpressions": [{"key": "disktype", "operator": "In", "values": ["ssd"]}]}}]}

B. "podAffinity": {"requiredDuringSchedulingIgnoredDuringExecution": {"labelSelector": {"matchLabels": {"disktype": "ssd"}}}}

C. "nodeSelector": {"disktype": "ssd"}

D. "nodeAffinity": {"requiredDuringSchedulingIgnoredDuringExecution": {"nodeSelectorTerms": [{"matchExpressions": [{"key": "disktype", "operator": "In", "values": ["ssd"]}]}]}}

Solution

1. Step 1: Identify required node affinity syntax

   Required node affinity uses the key requiredDuringSchedulingIgnoredDuringExecution with nodeSelectorTerms and matchExpressions to specify node labels.

2. Step 2: Check options for correct structure

   "nodeAffinity": {"requiredDuringSchedulingIgnoredDuringExecution": {"nodeSelectorTerms": [{"matchExpressions": [{"key": "disktype", "operator": "In", "values": ["ssd"]}]}]}} matches the required node affinity syntax. "nodeAffinity": {"preferredDuringSchedulingIgnoredDuringExecution": [{"weight": 1, "preference": {"matchExpressions": [{"key": "disktype", "operator": "In", "values": ["ssd"]}]}}]} is preferred affinity, C is nodeSelector (different syntax), and D is pod affinity, not node affinity.

3. Final Answer:

   "nodeAffinity": {"requiredDuringSchedulingIgnoredDuringExecution": {"nodeSelectorTerms": [{"matchExpressions": [{"key": "disktype", "operator": "In", "values": ["ssd"]}]}]}} -> Option D

4. Quick Check:

   Required node affinity uses requiredDuringSchedulingIgnoredDuringExecution [OK]

Hint: Required node affinity uses requiredDuringSchedulingIgnoredDuringExecution [OK]

Common Mistakes:

• Confusing preferred and required affinity syntax
• Using podAffinity instead of nodeAffinity
• Mixing nodeSelector with nodeAffinity syntax

3. Given this pod spec snippet, what will happen if a node has label zone: us-east-1a but no label disk: ssd?

{
  "affinity": {
    "nodeAffinity": {
      "requiredDuringSchedulingIgnoredDuringExecution": {
        "nodeSelectorTerms": [
          {
            "matchExpressions": [
              {"key": "zone", "operator": "In", "values": ["us-east-1a"]},
              {"key": "disk", "operator": "In", "values": ["ssd"]}
            ]
          }
        ]
      }
    }
  }
}

medium

A. The pod will be scheduled only if preferredDuringScheduling is set

B. The pod will not be scheduled on the node because it lacks the required disk label

C. The pod will be scheduled but with a warning

D. The pod will be scheduled because the zone label matches

Solution

1. Step 1: Analyze required node affinity conditions

   Required node affinity means all match expressions must be true for scheduling. The node must have both zone: us-east-1a and disk: ssd labels.

2. Step 2: Check node labels against requirements

   The node has zone: us-east-1a but lacks disk: ssd. This fails the required condition, so the pod cannot be scheduled there.

3. Final Answer:

   The pod will not be scheduled on the node because it lacks the required disk label -> Option B

4. Quick Check:

   Required affinity needs all labels matched [OK]

Hint: Required affinity means all labels must match [OK]

Common Mistakes:

• Assuming partial label match is enough
• Confusing required with preferred affinity
• Thinking pod schedules with warnings on missing labels

4. You wrote this pod spec with pod anti-affinity but pods are still scheduled on the same node. What is the likely error?

affinity:
  podAntiAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
    - weight: 100
      podAffinityTerm:
        labelSelector:
          matchLabels:
            app: myapp
        topologyKey: "kubernetes.io/hostname"

medium

A. Incorrect labelSelector syntax causes failure

B. Missing nodeSelector in the pod spec

C. Using preferred instead of required anti-affinity allows scheduling on same node

D. topologyKey should be set to "zone" instead of hostname

Solution

1. Step 1: Understand pod anti-affinity types

   Preferred anti-affinity suggests avoiding co-location but does not enforce it strictly. Required anti-affinity enforces pods not to be scheduled together.

2. Step 2: Analyze the pod spec anti-affinity type

   The spec uses preferredDuringSchedulingIgnoredDuringExecution, which is a soft rule. This allows pods to be scheduled on the same node if needed.

3. Final Answer:

   Using preferred instead of required anti-affinity allows scheduling on same node -> Option C

4. Quick Check:

   Preferred anti-affinity is a soft rule [OK]

Hint: Preferred anti-affinity is soft; use required to enforce [OK]

Common Mistakes:

• Confusing preferred and required anti-affinity
• Assuming labelSelector syntax is wrong without checking
• Thinking topologyKey must be zone for anti-affinity

5. You want to ensure pods with label app=frontend never run on the same node as pods with label app=backend. Which affinity configuration achieves this?

hard

A. Use podAntiAffinity with requiredDuringSchedulingIgnoredDuringExecution and labelSelector matching app=backend

B. Use nodeAffinity with requiredDuringSchedulingIgnoredDuringExecution and labelSelector matching app=backend

C. Use podAffinity with preferredDuringSchedulingIgnoredDuringExecution and labelSelector matching app=backend

D. Use nodeSelector with label app=backend

Solution

1. Step 1: Identify requirement to prevent co-location of pods

   Pod anti-affinity prevents pods from running on the same node as other pods matching certain labels.

2. Step 2: Choose required anti-affinity for strict enforcement

   Using requiredDuringSchedulingIgnoredDuringExecution enforces pods never co-locate on the same node.

3. Step 3: Match labelSelector to pods to avoid

   The labelSelector should match pods with app=backend to prevent frontend pods from scheduling on same node.

4. Final Answer:

   Use podAntiAffinity with requiredDuringSchedulingIgnoredDuringExecution and labelSelector matching app=backend -> Option A

5. Quick Check:

   Pod anti-affinity + required = strict pod separation [OK]

Hint: Use required podAntiAffinity to strictly separate pods [OK]

Common Mistakes:

• Using nodeAffinity instead of podAntiAffinity for pod separation
• Using preferred instead of required anti-affinity for strict rules
• Confusing podAffinity with podAntiAffinity