TTL-based expiry in Redis - Deep Dive

Overview - TTL-based expiry

What is it?

TTL-based expiry is a way to automatically remove data from a database after a set time. TTL stands for Time To Live, which means each piece of data has a countdown timer. When the timer reaches zero, the data is deleted. This helps keep the database clean and saves space.

Why it matters

Without TTL-based expiry, old or unused data would pile up, making the database slower and harder to manage. It also helps in scenarios like caching, where data should only be kept temporarily. TTL makes sure data is fresh and storage is efficient without manual cleanup.

Where it fits

Before learning TTL expiry, you should understand basic database storage and key-value pairs. After TTL, you can explore advanced caching strategies, data eviction policies, and persistence mechanisms in Redis.

Mental Model

Core Idea

TTL-based expiry is like setting a countdown timer on data so it disappears automatically when time runs out.

Think of it like...

Imagine putting food in the fridge with a label that says 'Eat by this date.' Once the date passes, the food is thrown away to avoid spoilage.

┌─────────────┐
│   Data Key  │
├─────────────┤
│ Value       │
├─────────────┤
│ TTL Timer   │───▶ Counts down every second
└─────────────┘
       │
       ▼
  TTL reaches 0
       │
       ▼
  Data is deleted automatically

Build-Up - 6 Steps

1

Foundation

What is TTL in Redis

Concept: Introducing the basic idea of TTL as a countdown timer for keys.

In Redis, TTL means Time To Live. You can set a TTL on any key, which tells Redis to delete that key after the time expires. TTL is measured in seconds or milliseconds. For example, setting TTL to 10 means the key will live for 10 seconds.

Result

Keys with TTL will automatically disappear after the set time.

Understanding TTL as a timer helps you see how Redis manages temporary data without manual deletion.

2

Foundation

Setting and Checking TTL

Concept: How to assign TTL to keys and check remaining time.

You use the EXPIRE command to set TTL in seconds, e.g., EXPIRE mykey 30 sets 30 seconds TTL. To check TTL, use TTL mykey, which returns remaining seconds. If TTL returns -1, the key has no expiry; if -2, the key does not exist.

Result

You can control and monitor how long keys live in Redis.

Knowing how to set and check TTL is essential to managing data lifecycle in Redis.

3

Intermediate

TTL Expiry Behavior and Access

🤔Before reading on: Do you think accessing a key resets its TTL or leaves it unchanged? Commit to your answer.

Concept: Understanding how TTL behaves when keys are accessed or modified.

Accessing a key (like reading its value) does NOT reset or change its TTL. However, modifying the key (like setting a new value) removes the old TTL unless you set a new one. TTL counts down independently of key access.

Result

Keys expire exactly when TTL ends, regardless of how often they are read.

Knowing that TTL is independent of reads prevents mistakes in expecting data to live longer just because it is accessed.

4

Intermediate

Different TTL Commands and Precision

🤔Before reading on: Do you think TTL can be set with millisecond precision or only seconds? Commit to your answer.

Concept: Exploring commands like PEXPIRE and their precision differences.

Besides EXPIRE (seconds), Redis offers PEXPIRE to set TTL in milliseconds for finer control. Similarly, PTTL returns TTL in milliseconds. This is useful for short-lived keys needing precise expiry.

Result

You can manage key expiry with both second and millisecond precision.

Understanding TTL precision helps design systems that require exact timing for data expiry.

5

Advanced

How Redis Handles Expired Keys Internally

🤔Before reading on: Do you think Redis deletes expired keys immediately at TTL expiry or lazily when accessed? Commit to your answer.

Concept: Learning Redis's lazy and active expiration strategies.

Redis does not delete keys exactly at TTL expiry time. Instead, it uses lazy expiration: keys are checked for expiry only when accessed. Additionally, Redis runs periodic active expiration cycles to find and delete expired keys proactively. This balances performance and memory cleanup.

Result

Expired keys may linger briefly but are removed efficiently without slowing Redis.

Knowing Redis's expiration strategy explains why expired keys might still appear briefly and how Redis balances speed with cleanup.

6

Expert

TTL Expiry Impact on Redis Performance and Memory

🤔Before reading on: Do you think setting many keys with TTL always improves Redis performance? Commit to your answer.

Concept: Understanding how TTL affects Redis memory and CPU usage in large-scale systems.

While TTL helps free memory, having millions of expiring keys can increase CPU load due to active expiration cycles. Also, keys with very short TTLs can cause spikes in expiration processing. Redis uses sampling and limits expiration work per cycle to avoid performance hits.

Result

TTL improves memory usage but requires careful tuning to avoid CPU spikes.

Knowing TTL's performance tradeoffs helps design scalable Redis systems that balance memory and CPU.

Under the Hood

Redis stores TTL as metadata alongside each key. When a key is set with TTL, Redis records the expiry timestamp internally. On key access, Redis compares current time with expiry time to decide if the key is expired. Expired keys are deleted lazily during access or actively during periodic scans. This avoids constant overhead of checking all keys.

Why designed this way?

Redis uses lazy and active expiration to optimize speed and memory. Immediate deletion of expired keys would slow down every operation. Periodic scanning balances cleanup without blocking commands. This design fits Redis's goal of being a fast in-memory store.

┌───────────────┐       ┌───────────────┐
│   Key Stored  │──────▶│ TTL Metadata  │
│  (Value)     │       │ (Expiry Time)  │
└───────────────┘       └───────────────┘
        │                        │
        ▼                        ▼
  On Access:             Periodic Scan:
  Check current time     Sample keys
  vs expiry time         Delete expired
        │                        │
        ▼                        ▼
  If expired:            Remove keys
  Delete key             to free memory

Myth Busters - 4 Common Misconceptions

Quick: Does reading a key reset its TTL? Commit to yes or no.

Common Belief: Reading or accessing a key resets its TTL timer, extending its life.

Reality: TTL is not reset by reading; it only changes if TTL is explicitly updated or key is modified.

Why it matters: Expecting TTL to reset on read can cause unexpected data expiry and bugs in caching logic.

Quick: Are expired keys immediately removed from Redis at TTL expiry? Commit to yes or no.

Common Belief: Keys are deleted exactly when TTL reaches zero, instantly freeing memory.

Reality: Redis deletes expired keys lazily on access or during periodic scans, so expired keys may exist briefly after expiry.

Why it matters: Assuming immediate deletion can lead to confusion when expired keys appear accessible for a short time.

Quick: Can you set TTL on a key and then update the key without affecting TTL? Commit to yes or no.

Common Belief: Updating a key's value does not affect its TTL; the timer continues as before.

Reality: Updating a key removes its TTL unless TTL is explicitly reset after the update.

Why it matters: Not resetting TTL after update can cause keys to live indefinitely, breaking expiry expectations.

Quick: Does setting TTL on keys always improve Redis performance? Commit to yes or no.

Common Belief: Using TTL on many keys always reduces memory and improves performance.

Reality: While TTL frees memory, too many expiring keys can increase CPU usage due to expiration processing.

Why it matters: Ignoring TTL performance tradeoffs can cause unexpected CPU spikes and slowdowns in production.

Expert Zone

1

Expired keys might still be returned briefly if accessed before Redis deletes them lazily.

2

Setting TTL on volatile keys can cause uneven memory usage spikes due to clustered expirations.

3

Redis does not guarantee exact expiry timing; TTL is a best-effort mechanism optimized for speed.

When NOT to use

TTL expiry is not suitable for data that must persist reliably or for precise timing guarantees. Use persistent storage with explicit deletion or scheduled jobs instead. For complex expiration logic, consider application-level management or Redis modules.

Production Patterns

In production, TTL is widely used for caching, session management, and rate limiting. Patterns include setting TTL on user sessions to auto-expire inactive users, caching API responses with short TTLs, and using Redis keyspace notifications to trigger events on expiry.

Connections

Cache Invalidation

TTL expiry is a form of automatic cache invalidation.

Understanding TTL helps grasp how caches keep data fresh by removing stale entries automatically.

Garbage Collection in Programming

Both TTL expiry and garbage collection remove unused data to free resources.

Knowing TTL expiry clarifies how systems manage memory by cleaning up data no longer needed.

Perishable Goods Management

TTL expiry mirrors managing expiration dates in inventory systems.

Recognizing TTL as similar to product expiry helps understand why automatic removal is crucial for freshness and safety.

Common Pitfalls

#1 Expecting TTL to reset on key read.

Wrong approach:

GET mykey
// Expect TTL to restart here

Correct approach:

GET mykey
EXPIRE mykey 30
// Explicitly reset TTL after read if needed

Root cause:

Misunderstanding that TTL is independent of key access and only changes on explicit commands.

#2 Updating a key without resetting TTL causes key to live forever.

Wrong approach:

SET mykey newvalue
// TTL lost, key never expires

Correct approach:

SET mykey newvalue
EXPIRE mykey 30
// Reset TTL after update

Root cause:

Not realizing that SET removes TTL and it must be reapplied.

#3 Assuming expired keys are instantly deleted and never accessible.

Wrong approach:

Expecting TTL key to be gone immediately after expiry time.

Correct approach:

Understand Redis lazy expiration and check TTL or key existence explicitly.

Root cause:

Believing Redis deletes keys exactly at expiry time without delay.

Key Takeaways

TTL-based expiry automatically removes keys after a set time, keeping Redis data fresh and memory efficient.

TTL timers count down independently of key access and must be explicitly set or reset when updating keys.

Redis uses lazy and active expiration strategies to balance performance and memory cleanup, so expired keys may linger briefly.

While TTL helps manage memory, too many expiring keys can increase CPU load, requiring careful system design.

Understanding TTL expiry is essential for building reliable caching, session management, and temporary data storage in Redis.