Custom Resource Definitions (CRDs) in Kubernetes - Deep Dive

Overview - Custom Resource Definitions (CRDs)

What is it?

Custom Resource Definitions (CRDs) let you add your own types of objects to Kubernetes. They extend Kubernetes by defining new resource kinds beyond the built-in ones like Pods or Services. This means you can manage your own custom data and behavior using Kubernetes tools. CRDs make Kubernetes flexible to handle many different use cases.

Why it matters

Without CRDs, Kubernetes would only manage its built-in resources, limiting what you can automate or control. CRDs solve the problem of extending Kubernetes without changing its core code. This lets teams build custom automation, operators, and workflows that fit their unique needs while still using Kubernetes as the platform. It makes Kubernetes a universal control plane.

Where it fits

Before learning CRDs, you should understand basic Kubernetes concepts like Pods, Services, and how Kubernetes API works. After CRDs, you can learn about Operators, Controllers, and how to build automation that reacts to custom resources. CRDs are a foundation for advanced Kubernetes extensibility.

Mental Model

Core Idea

CRDs let you teach Kubernetes new types of objects so it can manage anything you want, just like it manages built-in resources.

Think of it like...

Imagine Kubernetes as a smartphone that comes with default apps like Phone and Messages. CRDs are like installing new apps that add new features, letting the phone do things it couldn't before.

┌─────────────────────────────┐
│        Kubernetes API        │
├─────────────┬───────────────┤
│ Built-in    │ Custom Types  │
│ Resources   │ (CRDs)        │
│ (Pods, etc) │               │
└─────────────┴───────────────┘
         ▲                 ▲
         │                 │
   Controllers        Custom Controllers
   manage built-in    manage custom types
   resources          defined by CRDs

Build-Up - 7 Steps

1

Foundation

What is a Custom Resource Definition

Concept: Introduces the idea that CRDs define new resource types in Kubernetes.

A Custom Resource Definition is a Kubernetes object that tells the system about a new kind of resource you want to manage. For example, you can create a CRD called 'Database' to represent database instances. Once created, Kubernetes treats 'Database' as a first-class resource, letting you create, update, and delete these objects like Pods or Services.

Result

You can create new resource types in Kubernetes that behave like built-in ones.

Understanding that CRDs are the way to add new resource types unlocks Kubernetes extensibility.

2

Foundation

How to Define a CRD YAML

Concept: Shows the structure of a CRD manifest and its key fields.

A CRD YAML includes apiVersion: apiextensions.k8s.io/v1, kind: CustomResourceDefinition, metadata with a name, and a spec that defines the group, versions, scope, and names of the new resource. For example, spec.names.kind is the resource's kind name, and spec.scope defines if it's cluster-wide or namespaced.

Result

You can write a valid CRD manifest to register a new resource type.

Knowing the CRD YAML structure helps you customize and create your own resource types.

3

Intermediate

Creating and Using Custom Resources

🤔Before reading on: do you think creating a custom resource requires special Kubernetes code or just a YAML manifest? Commit to your answer.

Concept: Explains how to create instances of the new resource type after CRD registration.

After applying a CRD, you can create custom resource objects using kubectl with the new kind. For example, if you defined a 'Database' CRD, you create a Database object with a YAML manifest specifying its spec fields. Kubernetes stores these objects in etcd and makes them accessible via kubectl and API.

Result

You can manage your custom resource objects like any other Kubernetes resource.

Understanding that CRDs enable new resource objects without changing Kubernetes code shows the power of declarative APIs.

4

Intermediate

CRD Versions and Schema Validation

🤔Before reading on: do you think CRDs support multiple versions of the same resource kind? Commit to yes or no.

Concept: Introduces versioning and validation schemas in CRDs.

CRDs can define multiple versions (like v1alpha1, v1beta1, v1) to evolve resource definitions safely. Each version can have an OpenAPI v3 schema to validate the resource's fields. This ensures users create valid custom resources and allows smooth upgrades between versions.

Result

You can safely evolve your custom resource definitions and enforce data correctness.

Knowing CRD versioning and validation prevents breaking changes and improves reliability in production.

5

Intermediate

Scope and Namespaces in CRDs

Concept: Explains how CRDs can be cluster-scoped or namespaced.

When defining a CRD, you set spec.scope to 'Namespaced' or 'Cluster'. Namespaced means each resource exists within a namespace, like Pods. Cluster means the resource is global across the cluster, like Nodes. This choice affects how you organize and access your custom resources.

Result

You can control the visibility and isolation of your custom resources.

Understanding scope helps design resource access and security boundaries.

6

Advanced

Custom Controllers for CRD Automation

🤔Before reading on: do you think CRDs alone automate behavior or do you need extra components? Commit to your answer.

Concept: Shows that CRDs define data but need controllers to automate actions.

CRDs only define the shape of data. To make Kubernetes act on custom resources, you write controllers that watch these resources and perform tasks. For example, a Database controller might create actual database instances when a Database resource is created. This separation keeps Kubernetes flexible and modular.

Result

You can build full automation by combining CRDs with controllers.

Knowing CRDs separate data definition from behavior clarifies Kubernetes extensibility design.

7

Expert

CRD Internals and API Server Integration

🤔Before reading on: do you think CRDs require changes to Kubernetes API server code? Commit to yes or no.

Concept: Explains how CRDs integrate with the Kubernetes API server without core code changes.

The Kubernetes API server dynamically loads CRD definitions and serves their REST endpoints. It stores custom resources in etcd like built-in ones. This dynamic registration means CRDs add new resource types without recompiling or restarting the API server. The API server uses the CRD schema for validation and version conversion.

Result

CRDs extend Kubernetes API dynamically and safely.

Understanding this dynamic integration reveals why CRDs are powerful yet safe to use in production.

Under the Hood

When you apply a CRD manifest, the Kubernetes API server records the new resource definition in its internal storage. It then dynamically adds REST endpoints for the new resource kind. When users create custom resource objects, the API server validates them against the CRD schema and stores them in etcd. Controllers watch these objects via the API server to react and automate tasks.

Why designed this way?

Kubernetes was designed to be extensible without changing its core code. CRDs allow users to add new resource types dynamically, avoiding the need to fork or patch Kubernetes. This design balances flexibility with stability, letting the community innovate while keeping the core system stable and secure.

┌───────────────────────────────┐
│       Kubernetes API Server    │
│ ┌───────────────┐             │
│ │ CRD Registry  │<────────────┤
│ └───────────────┘             │
│        ▲                      │
│        │                      │
│ ┌───────────────┐             │
│ │ REST Endpoints│             │
│ └───────────────┘             │
│        ▲                      │
│        │                      │
│ ┌───────────────┐             │
│ │ Validation &  │             │
│ │ Versioning    │             │
│ └───────────────┘             │
│        ▲                      │
│        │                      │
│ ┌───────────────┐             │
│ │ etcd Storage  │             │
│ └───────────────┘             │
└───────────────────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Do CRDs automatically add new behaviors to Kubernetes without extra code? Commit yes or no.

Common Belief: CRDs alone add new automation and behavior to Kubernetes.

Reality: CRDs only define new resource types and data schemas; automation requires separate controllers.

Why it matters: Assuming CRDs automate behavior leads to confusion and broken systems when expected actions don't happen.

Quick: Can you change a CRD's schema without any impact on existing custom resources? Commit yes or no.

Common Belief: You can freely change CRD schemas anytime without issues.

Reality: Changing CRD schemas can break existing resources or clients; versioning and careful migration are needed.

Why it matters: Ignoring schema evolution risks data loss or API errors in production.

Quick: Are CRDs cluster-wide by default? Commit yes or no.

Common Belief: CRDs always create cluster-scoped resources.

Reality: CRDs can be either cluster-scoped or namespaced, depending on the spec.scope setting.

Why it matters: Misunderstanding scope can cause access control and resource management mistakes.

Quick: Do CRDs require modifying Kubernetes API server code? Commit yes or no.

Common Belief: Adding CRDs means changing Kubernetes core code.

Reality: CRDs are dynamically registered by the API server without code changes or restarts.

Why it matters: Thinking core changes are needed discourages users from extending Kubernetes.

Expert Zone

1

CRD schema validation uses OpenAPI v3, but some complex validations require additional admission webhooks.

2

CRD version conversion can be customized with conversion webhooks to handle complex upgrades.

3

CRDs can cause performance issues if too many or very large custom resources exist, requiring careful design.

When NOT to use

Avoid CRDs when you need very high-performance or low-latency APIs; native Kubernetes resources or external systems may be better. Also, for very complex logic, consider building full operators or external controllers rather than relying solely on CRDs.

Production Patterns

In production, CRDs are often combined with Operators that automate lifecycle management. Teams use versioned CRDs with conversion webhooks for smooth upgrades. Admission webhooks complement CRDs for advanced validation and security policies.

Connections

API Design

CRDs build on RESTful API principles by dynamically adding new resource endpoints.

Understanding REST APIs helps grasp how CRDs extend Kubernetes API without core changes.

Database Schema Migration

CRD versioning and schema evolution parallels database schema migrations and version control.

Knowing database migrations clarifies why CRD versioning and conversion are critical for stability.

Plugin Systems in Software Engineering

CRDs act like plugins that extend Kubernetes functionality without modifying core code.

Recognizing CRDs as plugins helps understand modular software design and extensibility.

Common Pitfalls

#1 Creating a CRD without specifying a proper schema leads to unvalidated and potentially invalid custom resources.

Wrong approach:

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: databases.example.com
spec:
group: example.com
versions:
- name: v1
served: true
storage: true
scope: Namespaced
names:
plural: databases
singular: database
kind: Database
shortNames:
- db

Correct approach:

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: databases.example.com
spec:
group: example.com
versions:
- name: v1
served: true
storage: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
size:
type: integer
version:
type: string
scope: Namespaced
names:
plural: databases
singular: database
kind: Database
shortNames:
- db

Root cause:

Beginners often omit schema because they focus on resource creation, not validation, leading to poor data quality.

#2 Assuming CRDs automatically trigger actions without writing controllers.

Wrong approach:

Apply a CRD and create custom resources expecting Kubernetes to perform tasks automatically.

Correct approach:

Write and deploy a controller that watches custom resources and implements desired automation.

Root cause:

Misunderstanding that CRDs define data only, not behavior.

#3 Changing CRD schema fields in incompatible ways without versioning causes API errors.

Wrong approach:

Modify existing CRD schema by removing or changing field types directly in the active version.

Correct approach:

Add a new version to the CRD with updated schema and use conversion webhooks to migrate data.

Root cause:

Ignoring best practices for API evolution and backward compatibility.

Key Takeaways

Custom Resource Definitions let you add new resource types to Kubernetes without changing its core code.

CRDs define the shape and validation of custom resources, but automation requires separate controllers.

Versioning and schema validation in CRDs ensure safe evolution and data correctness.

CRDs can be cluster-scoped or namespaced, affecting resource visibility and management.

The Kubernetes API server dynamically integrates CRDs, making Kubernetes highly extensible and flexible.

Practice

1. What is the main purpose of a Custom Resource Definition (CRD) in Kubernetes?

easy

A. To add new resource types to Kubernetes that are not built-in

B. To update the Kubernetes version automatically

C. To manage user permissions in Kubernetes

D. To monitor cluster health and performance

Solution

1. Step 1: Understand what CRDs do

   CRDs allow users to create their own resource types beyond the default Kubernetes resources.

2. Step 2: Compare options

   Automatic version updates, user permissions (RBAC), and cluster monitoring are separate Kubernetes features unrelated to CRDs.

3. Final Answer:

   To add new resource types to Kubernetes that are not built-in -> Option A

4. Quick Check:

   CRDs = add custom resource types [OK]

Hint: CRDs = add your own resource types [OK]

Common Mistakes:

• Confusing CRDs with RBAC for permissions
• Thinking CRDs update Kubernetes versions
• Assuming CRDs monitor cluster health

2. Which of the following is the correct YAML key to define the API version for a CRD?

easy

A. version

B. api_version

C. apiVersion

D. apiVer

Solution

1. Step 1: Recall Kubernetes YAML syntax

   Kubernetes uses camelCase keys like apiVersion to specify API versions.

2. Step 2: Check other options

   version, api_version, and apiVer use incorrect formats not recognized by Kubernetes.

3. Final Answer:

   apiVersion -> Option C

4. Quick Check:

   Correct YAML key = apiVersion [OK]

Hint: Use camelCase keys like apiVersion in Kubernetes YAML [OK]

Common Mistakes:

• Using underscores instead of camelCase
• Using incomplete or shortened keys
• Confusing version with apiVersion

3. Given this CRD snippet, what is the scope of the custom resource?

spec:
  scope: Namespaced
  group: example.com
  names:
    kind: Widget
    plural: widgets

medium

A. Cluster-wide resource available in all namespaces

B. Resource limited to a specific namespace

C. Resource only available in the default namespace

D. Resource scoped to nodes

Solution

1. Step 1: Read the scope field

   The scope is set to Namespaced, meaning the resource exists within namespaces.

2. Step 2: Understand scope meanings

   Namespaced means the resource is limited to a specific namespace, not cluster-wide or node-scoped.

3. Final Answer:

   Resource limited to a specific namespace -> Option B

4. Quick Check:

   scope Namespaced = namespace-limited resource [OK]

Hint: scope: Namespaced means resource lives inside namespaces [OK]

Common Mistakes:

• Confusing Namespaced with Cluster scope
• Assuming default namespace only
• Thinking scope relates to nodes

4. You applied a CRD YAML but get an error: "unknown field 'kindd'". What is the most likely cause?

medium

A. Typo in the YAML key 'kindd' instead of 'kind'

B. Missing apiVersion field

C. CRD name is not unique

D. Cluster role permissions missing

Solution

1. Step 1: Analyze the error message

   The error says "unknown field 'kindd'", indicating a typo in the YAML key.

2. Step 2: Identify the correct key

   The correct key is kind, so kindd is a misspelling causing the error.

3. Final Answer:

   Typo in the YAML key 'kindd' instead of 'kind' -> Option A

4. Quick Check:

   YAML key typos cause unknown field errors [OK]

Hint: Check YAML keys carefully for typos [OK]

Common Mistakes:

• Ignoring spelling errors in YAML keys
• Assuming missing fields cause unknown field errors
• Blaming permissions for syntax errors

5. You want to create a CRD for a resource named Gadget that is cluster-scoped and has a plural name gadgets. Which YAML snippet correctly defines this?

hard

A. spec:
group: example.com
scope: Cluster
names:
kind: Gadget
plural: gadget

B. spec:
group: example.com
scope: Namespaced
names:
kind: Gadget
plural: gadgets

C. spec:
group: example.com
scope: Cluster
names:
kind: gadget
plural: gadget

D. spec:
group: example.com
scope: Cluster
names:
kind: Gadget
plural: gadgets

Solution

1. Step 1: Check scope value

   The resource must be cluster-scoped, so scope: Cluster is correct.

2. Step 2: Verify kind and plural names

   kind should be capitalized as Gadget and plural should be gadgets (plural lowercase).

3. Step 3: Compare options

   The snippet with scope: Cluster, kind: Gadget, plural: gadgets matches all requirements. Snippets with scope: Namespaced, lowercase kind: gadget, or singular plural: gadget are incorrect.

4. Final Answer:

   spec: group: example.com scope: Cluster names: kind: Gadget plural: gadgets -> Option D

5. Quick Check:

   Cluster scope + correct kind/plural = spec: group: example.com scope: Cluster names: kind: Gadget plural: gadgets [OK]

Hint: Cluster scope means scope: Cluster; kind capitalized, plural lowercase [OK]

Common Mistakes:

• Using Namespaced instead of Cluster scope
• Incorrect casing for kind or plural
• Using singular for plural name