Secrets encryption at rest in Kubernetes - Step-by-Step Execution

Process Flow - Secrets encryption at rest

Create Secret in Kubernetes

↓

Secret stored in etcd (unencrypted)

↓

Enable Encryption Configuration

↓

Kubernetes API Server encrypts Secret data

↓

Encrypted Secret stored in etcd

↓

When Secret requested

↓

API Server decrypts Secret before returning

↓

User gets decrypted Secret

This flow shows how Kubernetes encrypts Secrets before saving them in etcd and decrypts them when accessed.

Execution Sample

Kubernetes

apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources:
    - secrets
    providers:
    - aescbc:
        keys:
        - name: key1
          secret: <base64-encoded-key>
    - identity: {}

This config enables AES-CBC encryption for Secrets stored in etcd.

Process Table

Step	Action	Secret State in etcd	API Server Behavior	Result
1	Create Secret 'mysecret'	Plaintext stored	No encryption configured	Secret saved unencrypted
2	Apply EncryptionConfiguration	New secrets encrypted; existing unencrypted	API Server encrypts Secrets before saving	New secrets stored encrypted
3	Request Secret 'mysecret'	Encrypted data in etcd	API Server decrypts Secret before returning	User receives plaintext Secret
4	Update Secret 'mysecret'	Encrypted data updated	API Server encrypts updated Secret	Updated Secret stored encrypted
5	Delete EncryptionConfiguration	Secrets remain encrypted	API Server stops encrypting new Secrets	New Secrets stored unencrypted
6	Request Secret 'mysecret'	Encrypted data in etcd	API Server decrypts Secret	User receives decrypted Secret
7	Exit	N/A	N/A	Process ends

💡 Process ends after Secret retrieval and encryption configuration changes

Status Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4	After Step 5	After Step 6	Final
Secret Data in etcd	None	Plaintext	Encrypted	Encrypted	Encrypted	Encrypted	Encrypted	Encrypted
API Server Encryption Enabled	False	False	True	True	True	False	False	False
User Receives	None	Plaintext Secret	Plaintext Secret	Plaintext Secret	Plaintext Secret	Plaintext Secret	Plaintext Secret	Plaintext Secret

Key Moments - 3 Insights

Why is the Secret stored unencrypted at first (Step 1) but new Secrets encrypted after applying EncryptionConfiguration (Step 2)?

Initially, Kubernetes stores Secrets as plaintext in etcd because encryption is not enabled. After applying EncryptionConfiguration, the API Server encrypts new Secrets before saving them, so etcd stores encrypted data (see execution_table rows 1 and 2).

If the EncryptionConfiguration is deleted (Step 5), are existing Secrets decrypted in etcd?

No, existing Secrets remain encrypted in etcd. Deleting the config stops encrypting new Secrets, but stored Secrets stay encrypted until manually decrypted or rotated (see execution_table row 5).

When a Secret is requested, why does the user always get the decrypted Secret even though etcd stores encrypted data?

The API Server decrypts Secrets on retrieval before returning them to the user, so users always see plaintext Secrets regardless of encryption at rest (see execution_table rows 3 and 6).

Visual Quiz - 3 Questions

Look at the execution_table at Step 2. What changes in the API Server behavior?

A It stops encrypting Secrets

B It deletes Secrets from etcd

C It starts encrypting Secrets before saving to etcd

D It returns Secrets unencrypted to users

💡 Hint

Check the 'API Server Behavior' column at Step 2 in the execution_table

At which step does the user receive the Secret for the first time?

A Step 3

B Step 1

C Step 5

D Step 7

💡 Hint

Look at the 'User Receives' column in variable_tracker after Step 1 and Step 3

If EncryptionConfiguration is removed, what happens to new Secrets stored after Step 5?

A They are stored unencrypted

B They are deleted automatically

C They are encrypted with the old key

D They cause an error on save

💡 Hint

See 'API Server Encryption Enabled' variable_tracker value after Step 5 and execution_table Step 5

Concept Snapshot

Kubernetes Secrets encryption at rest:
- Secrets stored in etcd by default are unencrypted.
- Enable encryption via EncryptionConfiguration in API Server.
- API Server encrypts Secrets before saving to etcd.
- Secrets are decrypted by API Server when accessed.
- Removing encryption config stops encrypting new Secrets but existing remain encrypted.
- Protects sensitive data stored in cluster backend.

Full Transcript

This visual execution shows how Kubernetes handles Secrets encryption at rest. Initially, Secrets are stored in plaintext in etcd. When an EncryptionConfiguration is applied, the API Server encrypts Secrets before saving them, so etcd stores encrypted data. When a Secret is requested, the API Server decrypts it before returning it to the user, ensuring users always see plaintext Secrets. If the encryption config is removed, new Secrets are stored unencrypted, but existing encrypted Secrets remain encrypted in etcd. This process protects sensitive data stored in the cluster backend.