KEYS pattern matching (avoid in production) in Redis - Deep Dive

Overview - KEYS pattern matching (avoid in production)

What is it?

In Redis, the KEYS command lets you search for keys that match a specific pattern using wildcards like * and ?. It returns all keys that fit the pattern you give. This is useful for quick lookups but can be slow if you have many keys. Because it scans all keys, it can block Redis and slow down your app.

Why it matters

Without KEYS, you might struggle to find keys matching a pattern quickly. But using KEYS in a busy system can freeze your database, causing delays or crashes. This command exists to help with debugging or small datasets, but in big systems, it can cause serious performance problems. Understanding this helps you avoid slowdowns and keep your app fast.

Where it fits

Before learning KEYS, you should know basic Redis commands and how keys and values work. After this, you should learn about SCAN, a safer alternative for pattern matching in production. Later, you can explore Redis data structures and how to design keys for efficient access.

Mental Model

Core Idea

The KEYS command scans every key in Redis to find matches, which can block the database if the dataset is large.

Think of it like...

Imagine searching for a book in a huge library by checking every single book one by one instead of using the catalog. It works but takes a long time and blocks others from using the library.

┌───────────────┐
│ Redis Database│
│ ┌───────────┐ │
│ │ All Keys  │ │
│ └───────────┘ │
│       │       │
│       ▼       │
│  KEYS command │
│  scans all    │
│  keys for     │
│  pattern      │
└───────────────┘

Build-Up - 7 Steps

1

Foundation

What is Redis KEYS command

Concept: Introducing the KEYS command and its basic usage.

The KEYS command in Redis lets you find keys that match a pattern. For example, KEYS user:* finds all keys starting with 'user:'. It uses wildcards like * (any characters) and ? (single character).

Result

Returns a list of keys matching the pattern.

Understanding KEYS is the first step to searching keys by pattern in Redis.

2

Foundation

How pattern matching works in KEYS

Concept: Explaining the pattern syntax and matching rules.

Patterns use * to match any number of characters and ? to match exactly one character. For example, KEYS order:202? matches keys like order:2021 or order:2022. KEYS * matches all keys.

Result

Returns keys that fit the pattern rules.

Knowing pattern syntax helps you write queries that find exactly what you want.

3

Intermediate

Performance impact of KEYS command

🤔Before reading on: do you think KEYS runs fast even with millions of keys? Commit to your answer.

Concept: Understanding that KEYS scans all keys and can be slow on large datasets.

KEYS checks every key in the database to see if it matches the pattern. If you have many keys, this takes time and blocks Redis from doing other work. This can cause delays or timeouts in your app.

Result

Using KEYS on large datasets causes slow response and blocks Redis.

Knowing KEYS scans all keys explains why it can freeze your database under load.

4

Intermediate

Why KEYS blocks Redis server

🤔Before reading on: do you think KEYS runs in the background or blocks other commands? Commit to your answer.

Concept: Explaining Redis's single-threaded nature and how KEYS blocks it.

Redis runs commands one at a time on a single thread. KEYS scans all keys in one go, so no other commands run until it finishes. This blocking causes delays for all clients connected to Redis.

Result

Redis becomes unresponsive during KEYS execution.

Understanding Redis's single-threaded design clarifies why KEYS blocks everything else.

5

Intermediate

Alternatives to KEYS: Using SCAN command

🤔Before reading on: do you think SCAN blocks Redis like KEYS? Commit to your answer.

Concept: Introducing SCAN as a non-blocking alternative for pattern matching.

SCAN lets you find keys matching a pattern without blocking Redis. It returns a small batch of keys each time you call it, letting Redis handle other commands between calls. This makes it safe for production use.

Result

You can search keys gradually without freezing Redis.

Knowing SCAN helps you avoid the performance problems of KEYS in real systems.

6

Advanced

When KEYS might still be useful

🤔Before reading on: do you think KEYS is always bad and never useful? Commit to your answer.

Concept: Exploring scenarios where KEYS is acceptable or helpful.

KEYS can be useful in development, debugging, or small datasets where performance impact is minimal. It gives quick results without writing complex code. But it should never be used in production with large data.

Result

You can use KEYS safely in limited, controlled situations.

Understanding the right context for KEYS prevents misuse and performance issues.

7

Expert

Internal cost of KEYS on Redis architecture

🤔Before reading on: do you think KEYS uses indexes internally to speed up search? Commit to your answer.

Concept: Explaining that KEYS does a full scan because Redis has no key index for patterns.

Redis stores keys in a hash table without indexes for pattern matching. KEYS must check each key one by one. This design keeps Redis fast for direct key access but makes pattern searches costly.

Result

KEYS is inherently slow due to Redis's data structure choices.

Knowing Redis's internal design explains why KEYS can't be optimized further.

Under the Hood

The KEYS command iterates over the entire keyspace stored in Redis's main dictionary. It compares each key against the given pattern using simple wildcard matching. Because Redis is single-threaded, this iteration blocks all other commands until complete. No indexes or shortcuts exist for pattern matching, so the scan is linear in the number of keys.

Why designed this way?

Redis prioritizes speed for direct key access and simplicity. Adding indexes for pattern matching would increase memory use and slow down writes. KEYS was designed as a simple tool for debugging and small datasets, not for production-scale pattern searches. The SCAN command was later introduced to provide a safer alternative.

┌───────────────┐
│ Redis Server  │
│ ┌───────────┐ │
│ │ Keyspace  │ │
│ └───────────┘ │
│       │       │
│       ▼       │
│  KEYS command │
│  iterates all │
│  keys one by  │
│  one, matches │
│  pattern     │
│       │       │
│       ▼       │
│ Returns list  │
└───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does KEYS run fast even with millions of keys? Commit yes or no.

Common Belief: KEYS is fast and safe to use anytime because Redis is very fast.

Reality: KEYS scans every key and blocks Redis, so it is slow and unsafe on large datasets.

Why it matters: Using KEYS in production can freeze your database, causing downtime and lost user requests.

Quick: Does KEYS run in the background allowing other commands? Commit yes or no.

Common Belief: KEYS runs asynchronously and does not block other Redis commands.

Reality: Redis is single-threaded; KEYS blocks all other commands until it finishes.

Why it matters: Misunderstanding this leads to unexpected delays and poor app performance.

Quick: Can you use KEYS safely in production if you limit the pattern? Commit yes or no.

Common Belief: Using KEYS with specific patterns is safe in production because it searches fewer keys.

Reality: Even specific patterns require scanning all keys; performance impact remains high.

Why it matters: Relying on pattern specificity does not prevent blocking and slowdowns.

Quick: Does SCAN command have the same blocking problem as KEYS? Commit yes or no.

Common Belief: SCAN blocks Redis just like KEYS because it also searches keys.

Reality: SCAN returns keys incrementally and does not block Redis, making it safe for production.

Why it matters: Confusing SCAN with KEYS causes missed opportunities for safe pattern matching.

Expert Zone

1

The KEYS command's blocking time depends not only on the number of keys but also on the size of each key and the complexity of the pattern.

2

Redis's internal dictionary resizing during KEYS execution can cause additional latency spikes, making performance unpredictable.

3

Using namespaces (prefixes) in keys can reduce the impact of KEYS by limiting the keyspace, but it does not eliminate blocking.

When NOT to use

Avoid KEYS in any production environment with large or growing datasets. Instead, use SCAN for incremental, non-blocking key searches. For structured data, design your keys and data models to avoid pattern searches altogether.

Production Patterns

In production, developers use SCAN with cursors to paginate through keys safely. They also design keys with predictable prefixes and maintain secondary indexes in Redis sets or sorted sets to find keys efficiently without pattern matching.

Connections

Database Indexing

KEYS lacks indexing, causing full scans similar to unindexed database queries.

Understanding why KEYS is slow helps appreciate the importance of indexes in databases for fast lookups.

Event Loop in Node.js

Both Redis and Node.js use single-threaded event loops where blocking operations freeze all other tasks.

Knowing Redis's single-threaded model is like understanding Node.js event loop blocking, which helps avoid performance pitfalls.

Library Book Cataloging

Searching keys with KEYS is like searching books without a catalog, scanning every book manually.

This cross-domain view highlights why indexes or catalogs are essential for efficient searching.

Common Pitfalls

#1 Using KEYS in production on large datasets causing Redis to freeze.

Wrong approach:

KEYS *

Correct approach:

Use SCAN 0 MATCH * COUNT 1000 and iterate until cursor returns 0

Root cause:

Misunderstanding that KEYS scans all keys at once and blocks Redis.

#2 Assuming KEYS with a specific pattern is safe in production.

Wrong approach:

KEYS user:123*

Correct approach:

Use SCAN with MATCH user:123* to incrementally fetch keys

Root cause:

Believing pattern specificity reduces scanning cost, which it does not.

#3 Confusing SCAN with KEYS and avoiding SCAN due to fear of blocking.

Wrong approach:

Avoid SCAN and use KEYS for simplicity

Correct approach:

Use SCAN for non-blocking incremental key search in production

Root cause:

Lack of understanding of SCAN's incremental, non-blocking design.

Key Takeaways

The KEYS command scans all keys in Redis to find matches, which blocks the server and slows down your app on large datasets.

Because Redis is single-threaded, KEYS blocks all other commands until it finishes, causing delays or downtime.

Use KEYS only for debugging or small datasets; in production, use the SCAN command for safe, incremental key searching.

Redis's design favors fast direct key access over pattern matching, so KEYS cannot be optimized with indexes.

Understanding KEYS's limitations helps you design better Redis usage patterns and avoid costly performance problems.

Practice

1. What does the Redis command KEYS * do?

easy

A. Returns all keys in the current database

B. Deletes all keys in the current database

C. Returns the number of keys in the current database

D. Sets a new key with a wildcard name

Solution

1. Step 1: Understand the KEYS command

   The KEYS command searches for keys matching a given pattern.

2. Step 2: Analyze the pattern *

   The pattern * matches all keys in the current database.

3. Final Answer:

   Returns all keys in the current database -> Option A

4. Quick Check:

   KEYS * lists all keys [OK]

Hint: Remember * matches everything in KEYS command [OK]

Common Mistakes:

• Thinking KEYS deletes keys
• Confusing KEYS with COUNT commands
• Assuming KEYS sets keys

2. Which of the following is the correct unquoted syntax to find keys starting with 'user:' using KEYS?

easy

A. KEYS user:*

B. KEYS user*

C. KEYS 'user:*'

D. KEYS 'user*'

Solution

1. Step 1: Understand pattern syntax in KEYS

   Patterns do not require quotes; wildcards like * match any characters.

2. Step 2: Match keys starting with 'user:'

   The pattern should be user:* without quotes to match keys starting with 'user:'.

3. Final Answer:

   KEYS user:* -> Option A

4. Quick Check:

   Use unquoted patterns with * wildcard [OK]

Hint: Use unquoted patterns with * for KEYS [OK]

Common Mistakes:

• Adding quotes around pattern
• Using colon without wildcard
• Using wrong wildcard placement

3. Given keys: user:1, user:2, admin:1, what is the output of KEYS user:*?

medium

A. ["admin:1"]

B. ["user:1", "user:2", "admin:1"]

C. ["user:1", "user:2"]

D. []

Solution

1. Step 1: Identify keys matching pattern user:*

   Keys starting with 'user:' are user:1 and user:2.

2. Step 2: Exclude keys not matching pattern

   admin:1 does not start with 'user:', so it is excluded.

3. Final Answer:

   ["user:1", "user:2"] -> Option C

4. Quick Check:

   Pattern matches keys starting with 'user:' [OK]

Hint: Match only keys starting with pattern prefix [OK]

Common Mistakes:

• Including keys with different prefixes
• Returning empty list incorrectly
• Confusing pattern syntax

4. What is wrong with this command: KEYS user?*?

medium

A. The question mark is not a valid wildcard in KEYS

B. The pattern should be quoted

C. The command will delete keys instead of listing

D. The pattern matches only keys with exactly one character after 'user' followed by any characters

Solution

1. Step 1: Understand the question mark wildcard in KEYS

   The question mark ? matches exactly one character in Redis patterns.

2. Step 2: Analyze the pattern user?*

   This pattern matches keys starting with 'user', followed by exactly one character, then any characters.

3. Final Answer:

   The pattern matches only keys with exactly one character after 'user' -> Option D

4. Quick Check:

   ? matches exactly one character [OK]

Hint: Remember ? matches exactly one character in KEYS [OK]

Common Mistakes:

• Thinking ? is invalid wildcard
• Assuming pattern deletes keys
• Forgetting ? matches exactly one character

5. You want to find all keys containing the word 'session' anywhere in their name without blocking Redis in production. Which approach is best?

hard

A. Use KEYS *session* command directly

B. Use SCAN 0 MATCH *session* iteratively

C. Use GET *session* command

D. Use DEL *session* to remove keys first

Solution

1. Step 1: Understand why KEYS is discouraged in production

   KEYS blocks Redis and can cause performance issues on large datasets.

2. Step 2: Use SCAN with MATCH for safe iteration

   SCAN iterates keys incrementally without blocking, using MATCH *session* to filter keys containing 'session'.

3. Final Answer:

   Use SCAN 0 MATCH *session* iteratively -> Option B

4. Quick Check:

   SCAN is safe for production key matching [OK]

Hint: Use SCAN with MATCH to avoid blocking Redis [OK]

Common Mistakes:

• Using KEYS in production causing blocking
• Using GET with wildcards (invalid)
• Deleting keys without checking