Resource quotas per namespace in Kubernetes - Deep Dive

Overview - Resource quotas per namespace

What is it?

Resource quotas per namespace in Kubernetes are limits set on the amount of resources like CPU, memory, and storage that can be used by all the containers and pods within a specific namespace. A namespace is like a separate space or room inside a Kubernetes cluster where resources and objects are grouped. Setting quotas helps control how much resource each namespace can consume to avoid one group using too much and affecting others.

Why it matters

Without resource quotas, one team or application could use all the cluster's resources, causing other teams' applications to slow down or fail. Quotas ensure fair sharing and prevent accidental or malicious overuse. This keeps the cluster stable and predictable, which is important for running many applications together smoothly.

Where it fits

Before learning resource quotas, you should understand Kubernetes namespaces and basic resource concepts like CPU and memory requests and limits. After mastering quotas, you can explore advanced resource management like LimitRanges, Quality of Service classes, and cluster autoscaling.

Mental Model

Core Idea

Resource quotas act like a budget that limits how much computing power and storage each namespace can spend inside a Kubernetes cluster.

Think of it like...

Imagine a shared apartment where each roommate has a monthly budget for utilities and groceries. Resource quotas are like setting those budgets so no one spends more than their share, keeping the household running smoothly.

┌───────────────────────────────┐
│ Kubernetes Cluster             │
│ ┌───────────────┐             │
│ │ Namespace A   │             │
│ │ ┌───────────┐ │             │
│ │ │ Quota:    │ │             │
│ │ │ CPU: 2    │ │             │
│ │ │ Memory: 4G│ │             │
│ │ └───────────┘ │             │
│ └───────────────┘             │
│ ┌───────────────┐             │
│ │ Namespace B   │             │
│ │ ┌───────────┐ │             │
│ │ │ Quota:    │ │             │
│ │ │ CPU: 1    │ │             │
│ │ │ Memory: 2G│ │             │
│ │ └───────────┘ │             │
│ └───────────────┘             │
└───────────────────────────────┘

Build-Up - 7 Steps

1

Foundation

Understanding Kubernetes namespaces

Concept: Namespaces divide a Kubernetes cluster into virtual sections to organize and isolate resources.

A Kubernetes cluster can run many applications for different teams or projects. Namespaces help keep these separate by grouping resources like pods, services, and deployments into named spaces. This prevents name conflicts and allows different teams to work independently.

Result

You can create and manage resources in separate namespaces, keeping them organized and isolated.

Knowing namespaces is essential because resource quotas apply at this level, controlling resource use per namespace.

2

Foundation

Basics of Kubernetes resource requests and limits

Concept: Pods and containers declare how much CPU and memory they need and the maximum they can use.

When you create a pod, you specify resource requests (minimum needed) and limits (maximum allowed) for CPU and memory. The scheduler uses requests to place pods on nodes with enough resources. Limits prevent pods from using too much and affecting others.

Result

Pods run with guaranteed minimum resources and capped maximum usage.

Understanding requests and limits is key because resource quotas count these values to enforce limits per namespace.

3

Intermediate

What resource quotas control in namespaces

🤔Before reading on: do you think resource quotas limit actual usage or just resource requests? Commit to your answer.

Concept: Resource quotas limit the total resource requests and object counts within a namespace, not just actual usage.

A resource quota sets maximum totals for resource requests (like CPU and memory), number of pods, services, and other objects in a namespace. It does not limit actual usage directly but restricts how much can be requested or created. For example, if a quota limits CPU requests to 4 cores, all pods combined cannot request more than 4 cores.

Result

Namespaces cannot create or run pods that exceed the quota limits on requests or object counts.

Knowing quotas limit requests and object counts, not actual usage, helps avoid confusion about how Kubernetes enforces resource fairness.

4

Intermediate

How to define a resource quota YAML

🤔Before reading on: do you think resource quotas are defined per cluster or per namespace? Commit to your answer.

Concept: Resource quotas are defined in YAML files and applied to specific namespaces.

A resource quota YAML includes apiVersion, kind: ResourceQuota, metadata with namespace, and spec with hard limits like cpu, memory, pods, and services. For example: apiVersion: v1 kind: ResourceQuota metadata: name: example-quota namespace: team-a spec: hard: requests.cpu: "4" requests.memory: 8Gi pods: "10" This limits the team-a namespace to 4 CPU cores requested, 8 GB memory requested, and 10 pods total.

Result

Applying this YAML enforces the specified limits on the namespace.

Understanding the YAML structure lets you create and customize quotas to fit your team's needs.

5

Intermediate

Enforcing quotas and observing effects

🤔Before reading on: do you think creating a pod that exceeds quota fails immediately or runs then gets killed? Commit to your answer.

Concept: Kubernetes prevents creation of pods or resources that would exceed the quota limits.

When you try to create a pod or resource that would push the namespace over its quota, Kubernetes rejects the request with an error. For example, if the quota limits pods to 10 and you already have 10, creating an 11th pod fails. This enforcement happens at creation time, not after running.

Result

Users get clear errors preventing resource overuse, keeping the cluster stable.

Knowing enforcement happens at creation avoids confusion about pods being killed later for quota reasons.

6

Advanced

Combining resource quotas with LimitRanges

🤔Before reading on: do you think LimitRanges and ResourceQuotas do the same thing or complement each other? Commit to your answer.

Concept: LimitRanges set default and maximum resource requests and limits per pod/container, complementing quotas that limit totals per namespace.

LimitRanges define minimum, maximum, and default resource requests and limits for pods and containers in a namespace. For example, they can require every pod to request at least 100m CPU and limit max memory to 2Gi. Resource quotas limit the total sum of requests and counts. Together, they control both individual pod sizes and total namespace usage.

Result

Namespaces have controlled pod sizes and total resource consumption, improving fairness and predictability.

Understanding how quotas and LimitRanges work together helps design robust resource policies.

7

Expert

Surprising quota behavior with burstable pods

🤔Before reading on: do you think resource quotas count actual pod usage or only requests? Commit to your answer.

Concept: Resource quotas count only resource requests, so pods with burstable QoS can use more than their request up to their limit without breaking quotas.

Pods with requests less than their limits are burstable. Quotas count only the requests, not limits or actual usage. This means a namespace can have pods that collectively request less than the quota but use more CPU or memory at runtime, potentially causing node pressure. Quotas do not prevent this burst usage, which can surprise admins expecting strict caps.

Result

Burstable pods can cause resource spikes beyond quota totals, requiring careful cluster monitoring and node capacity planning.

Knowing quotas count requests only prevents false security assumptions about resource caps and guides better resource planning.

Under the Hood

Resource quotas work by Kubernetes API server tracking the sum of resource requests and object counts in each namespace. When a new resource creation or update request arrives, the API server checks if adding it would exceed any quota limit. If yes, it rejects the request with a quota exceeded error. This check happens before the resource is persisted. The quota controller continuously monitors usage and updates status to reflect current consumption.

Why designed this way?

This design centralizes quota enforcement at the API server to ensure consistency and prevent race conditions. It avoids complex runtime enforcement inside nodes, which would be harder to coordinate. Counting requests rather than actual usage simplifies enforcement and scheduling decisions, though it allows burstable usage beyond quotas.

┌───────────────────────────────┐
│ Kubernetes API Server          │
│ ┌───────────────────────────┐ │
│ │ Receives resource requests │ │
│ └─────────────┬─────────────┘ │
│               │ Checks quota limits
│               ▼               │
│ ┌───────────────────────────┐ │
│ │ Quota Controller tracks   │ │
│ │ resource usage per ns     │ │
│ └─────────────┬─────────────┘ │
│               │ Accept or reject
│               ▼               │
│ ┌───────────────────────────┐ │
│ │ Resource stored if allowed │ │
│ └───────────────────────────┘ │
└───────────────────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Do resource quotas limit actual CPU and memory usage or only resource requests? Commit to your answer.

Common Belief: Resource quotas limit the actual CPU and memory usage of pods in a namespace.

Reality: Resource quotas limit the total resource requests and object counts, not the actual runtime usage, which can be higher for burstable pods.

Why it matters: Believing quotas limit actual usage can cause admins to underestimate resource spikes and node pressure, leading to instability.

Quick: Can resource quotas be applied cluster-wide or only per namespace? Commit to your answer.

Common Belief: Resource quotas can be set globally for the entire Kubernetes cluster.

Reality: Resource quotas apply only per namespace, not cluster-wide. Cluster-wide limits require other tools like cluster resource quotas or admission controllers.

Why it matters: Expecting cluster-wide enforcement from namespace quotas can cause uncontrolled resource use across namespaces.

Quick: Does creating a pod that exceeds quota limits run then get killed, or is it blocked upfront? Commit to your answer.

Common Belief: Pods that exceed resource quotas start running but get killed later if they use too much.

Reality: Kubernetes blocks creation of pods that would exceed quotas upfront with an error; they never start.

Why it matters: Misunderstanding enforcement timing can lead to confusion and wasted troubleshooting.

Quick: Do resource quotas count limits or requests for resource usage? Commit to your answer.

Common Belief: Resource quotas count the resource limits set on pods and containers.

Reality: Resource quotas count only the resource requests, not limits.

Why it matters: This affects how much burstable pods can consume beyond quotas, impacting cluster stability.

Expert Zone

1

Resource quotas count resource requests and object counts but do not track actual usage, so burstable pods can exceed quotas in practice.

2

Quota enforcement happens at the API server during resource creation, preventing race conditions but not controlling runtime spikes.

3

Combining resource quotas with LimitRanges allows fine-grained control over both total namespace usage and individual pod resource sizes.

When NOT to use

Resource quotas are not suitable for enforcing strict runtime resource caps or cluster-wide limits. For runtime enforcement, use cgroups or node-level controls. For cluster-wide limits, consider ClusterResourceQuota (in OpenShift) or admission controllers. Also, quotas do not prevent resource starvation caused by burstable pods exceeding requests.

Production Patterns

In production, teams use resource quotas per namespace to allocate fair resource budgets to different projects or teams. They combine quotas with LimitRanges to enforce minimum and maximum pod sizes. Monitoring tools track actual usage to detect burstable pod spikes. Quotas are integrated into CI/CD pipelines to prevent resource overuse before deployment.

Connections

Linux cgroups

Resource quotas limit resource requests at the Kubernetes API level, while cgroups enforce actual resource usage limits at the OS level.

Understanding cgroups helps grasp why Kubernetes quotas count requests but cannot strictly cap runtime usage.

Project budgeting in finance

Resource quotas are like financial budgets that limit spending per project (namespace) to avoid overspending cluster resources.

Knowing budgeting principles clarifies why quotas prevent resource overuse and ensure fair sharing.

Traffic shaping in networking

Both resource quotas and traffic shaping control how much resource (CPU/memory or bandwidth) a group can consume to prevent congestion.

Recognizing this pattern across domains highlights the importance of fair resource allocation for system stability.

Common Pitfalls

#1 Trying to limit actual pod CPU usage with resource quotas.

Wrong approach:

apiVersion: v1
kind: ResourceQuota
metadata:
name: cpu-limit-quota
namespace: team-a
spec:
hard:
limits.cpu: "4"

Correct approach:

apiVersion: v1
kind: ResourceQuota
metadata:
name: cpu-request-quota
namespace: team-a
spec:
hard:
requests.cpu: "4"

Root cause:

Resource quotas count resource requests, not limits; using limits.cpu in quotas has no effect.

#2 Applying resource quotas without namespaces or to the wrong namespace.

Wrong approach:

kubectl apply -f quota.yaml
# quota.yaml missing or wrong namespace metadata

Correct approach:

apiVersion: v1
kind: ResourceQuota
metadata:
name: example-quota
namespace: correct-namespace
spec:
hard:
pods: "10"

Root cause:

Resource quotas must specify the target namespace; otherwise, they do not apply or apply to default namespace.

#3 Expecting pods to be killed if they exceed quota after creation.

Wrong approach:

Create pod exceeding quota and expect it to run then be terminated by quota enforcement.

Correct approach:

Pods exceeding quota are blocked at creation time with an error; they never run.

Root cause:

Misunderstanding that quota enforcement happens at resource creation, not runtime.

Key Takeaways

Resource quotas limit the total resource requests and object counts per namespace to ensure fair resource sharing in Kubernetes clusters.

Quotas enforce limits at resource creation time by the API server, preventing overuse before pods run.

They count resource requests, not actual usage or limits, so burstable pods can exceed quotas in runtime usage.

Combining resource quotas with LimitRanges provides control over both total namespace usage and individual pod resource sizes.

Understanding quotas helps maintain cluster stability and fairness among multiple teams or projects sharing the same cluster.