Taints and tolerations in Kubernetes - Commands & Configuration

Introduction

Sometimes you want to keep certain computers in your Kubernetes cluster reserved for special tasks. Taints and tolerations help you do this by marking those computers and allowing only certain tasks to run on them.

When you want to reserve some nodes for high-priority workloads only.

When you want to prevent regular tasks from running on nodes with limited resources.

When you want to isolate workloads that need special hardware like GPUs.

When you want to avoid running test workloads on production nodes.

When you want to control where certain pods can be scheduled in your cluster.

Config File - taint-node.yaml

taint-node.yaml

apiVersion: v1
kind: Node
metadata:
  name: example-node
spec:
  taints:
  - key: special
    value: reserved
    effect: NoSchedule

This file adds a taint to the node named example-node. The taint has a key special with value reserved and effect NoSchedule. This means no pod can run on this node unless it has a matching toleration.

Commands

This command adds a taint to the node named example-node. It marks the node so that pods without a matching toleration will not be scheduled on it.

Terminal

kubectl taint nodes example-node special=reserved:NoSchedule

Expected Output

node/example-node tainted

This command lists all nodes with their taints to verify that the taint was applied correctly.

Terminal

kubectl get nodes -o json | jq '.items[] | {name: .metadata.name, taints: .spec.taints}'

Expected Output

{

  "name": "example-node",

  "taints": [

    {

      "key": "special",

      "value": "reserved",

      "effect": "NoSchedule"

    }

  ]

}

{

  "name": "other-node",

  "taints": null

}

→

-o json - Outputs node details in JSON format

This command creates a pod that has a toleration matching the taint on example-node, allowing it to be scheduled there.

Terminal

kubectl apply -f pod-with-toleration.yaml

Expected Output

pod/my-pod created

This command shows the pods and the nodes they are running on to confirm the pod was scheduled on the tainted node.

Terminal

kubectl get pods -o wide

Expected Output

NAME     READY   STATUS    RESTARTS   AGE   NODE          

my-pod   1/1     Running   0          1m    example-node

→

-o wide - Shows extra details including node name

Key Concept

If you remember nothing else from this pattern, remember: taints mark nodes to repel pods, and tolerations let pods ignore those marks to run there.

Common Mistakes

Adding a taint to a node but not adding a matching toleration to the pod.

The pod will not be scheduled on the tainted node and may stay pending.

Always add a toleration in the pod spec that matches the node's taint key, value, and effect.

Using the wrong effect type in the taint or toleration (e.g., NoExecute instead of NoSchedule).

The pod scheduling behavior will not be as expected; pods might be evicted or blocked incorrectly.

Use the correct effect type that matches your scheduling needs: NoSchedule to block scheduling, NoExecute to evict running pods.

Forgetting to remove a taint when it is no longer needed.

Nodes remain reserved unnecessarily, reducing cluster resource availability.

Remove taints with kubectl taint nodes example-node special- when reservation is no longer required.

Summary

Use 'kubectl taint nodes' to mark nodes with taints that repel pods without matching tolerations.

Add matching tolerations in pod specs to allow pods to run on tainted nodes.

Verify taints with 'kubectl get nodes' and pod scheduling with 'kubectl get pods -o wide'.

Practice

1. What is the main purpose of a taint in Kubernetes?

easy

A. To repel pods from nodes unless they have matching tolerations

B. To allow pods to run on any node without restrictions

C. To label pods for easier identification

D. To increase node resource limits automatically

Solution

1. Step 1: Understand what a taint does

   A taint is applied to a node to repel pods that do not tolerate it.

2. Step 2: Compare with other options

   Options B, C, and D describe unrelated features: pod scheduling without restrictions, pod labeling, and resource limits, which are not taint functions.

3. Final Answer:

   To repel pods from nodes unless they have matching tolerations -> Option A

4. Quick Check:

   Taints repel pods without tolerations = A [OK]

Hint: Taints push pods away; tolerations allow pods to stay [OK]

Common Mistakes:

• Confusing taints with labels
• Thinking taints increase resources
• Assuming taints allow all pods

2. Which of the following is the correct syntax to add a taint to a node named node1 that repels pods without the matching toleration?

easy

A. kubectl taint nodes node1 key=value:NoSchedule

B. kubectl label nodes node1 key=value:NoSchedule

C. kubectl taint pods node1 key=value:NoSchedule

D. kubectl tolerate nodes node1 key=value:NoSchedule

Solution

1. Step 1: Identify the correct command for tainting nodes

   The command to add a taint is kubectl taint nodes followed by node name and taint details.

2. Step 2: Check the options

   kubectl taint nodes node1 key=value:NoSchedule uses the correct syntax. kubectl label nodes node1 key=value:NoSchedule uses label instead of taint. kubectl taint pods node1 key=value:NoSchedule tries to taint pods, which is invalid. kubectl tolerate nodes node1 key=value:NoSchedule uses a non-existent command.

3. Final Answer:

   kubectl taint nodes node1 key=value:NoSchedule -> Option A

4. Quick Check:

   Use kubectl taint nodes for taints = A [OK]

Hint: Use 'kubectl taint nodes' to add taints, not labels or pods [OK]

Common Mistakes:

• Using 'label' instead of 'taint'
• Trying to taint pods instead of nodes
• Using wrong command like 'tolerate'

3. Given the following pod spec snippet, what will happen if the node has the taint key=value:NoSchedule?

tolerations:
- key: "key"
  operator: "Equal"
  value: "value"
  effect: "NoSchedule"

medium

A. The pod will be scheduled but evicted immediately

B. The pod will be scheduled on the tainted node

C. The pod will be rejected and not scheduled

D. The pod will ignore the taint and schedule anywhere

Solution

1. Step 1: Understand the pod toleration

   The pod tolerates the taint with matching key, value, and effect, so it can be scheduled on the node.

2. Step 2: Analyze the taint effect

   The taint effect is NoSchedule, which prevents pods without tolerations from scheduling. Since this pod has the matching toleration, it is allowed.

3. Final Answer:

   The pod will be scheduled on the tainted node -> Option B

4. Quick Check:

   Matching toleration allows scheduling despite taint = C [OK]

Hint: Matching toleration lets pod schedule on tainted node [OK]

Common Mistakes:

• Ignoring the value match in toleration
• Confusing NoSchedule with NoExecute
• Assuming tolerations always allow scheduling

4. You applied the taint key=value:NoSchedule to a node, but pods without tolerations are still scheduling on it. What is the most likely cause?

medium

A. Pods have tolerations matching the taint

B. Taints do not affect scheduling

C. The node is in a NotReady state

D. The taint was applied with a typo in the key or value

Solution

1. Step 1: Check taint correctness

   If pods without tolerations schedule on a tainted node, the taint might have a typo or was not applied correctly.

2. Step 2: Evaluate other options

   Pods have tolerations matching the taint means pods tolerate the taint, which contradicts the question. The node is in a NotReady state is unrelated to taints. Taints do not affect scheduling is false because taints do affect scheduling.

3. Final Answer:

   The taint was applied with a typo in the key or value -> Option D

4. Quick Check:

   Incorrect taint syntax causes scheduling errors = B [OK]

Hint: Check taint spelling carefully if pods ignore it [OK]

Common Mistakes:

• Assuming taints don't affect scheduling
• Ignoring typos in taint keys or values
• Confusing node status with taint issues

5. You want to ensure that only pods with a specific toleration can run on a node, and all other pods are evicted if already running. Which taint effect should you use?

hard

A. NoSchedule

B. PreferNoSchedule

C. NoExecute

D. NoEvict

Solution

1. Step 1: Understand taint effects

   NoSchedule prevents new pods without tolerations from scheduling. PreferNoSchedule tries to avoid scheduling but doesn't guarantee. NoExecute evicts existing pods without tolerations and prevents new ones. NoEvict is not a valid effect.

2. Step 2: Match requirement to effect

   Since you want to evict existing pods without tolerations, NoExecute is the correct effect.

3. Final Answer:

   NoExecute -> Option C

4. Quick Check:

   Evict pods without tolerations = NoExecute = D [OK]

Hint: Use NoExecute to evict pods lacking tolerations [OK]

Common Mistakes:

• Choosing NoSchedule which doesn't evict existing pods
• Confusing PreferNoSchedule with eviction
• Using invalid effect NoEvict