EXEC to execute in Redis - Deep Dive

Overview - EXEC to execute

What is it?

EXEC is a Redis command used to execute all commands queued in a transaction. When you start a transaction with MULTI, commands are queued but not run immediately. EXEC runs all those queued commands atomically, meaning all succeed or none do.

Why it matters

Without EXEC, you cannot run multiple commands as a single unit in Redis. This means you can't guarantee that a group of commands will all happen together, which is important for data consistency. EXEC solves this by making sure either all commands run or none do, preventing partial updates.

Where it fits

Before learning EXEC, you should understand basic Redis commands and the concept of transactions using MULTI. After EXEC, you can explore WATCH for optimistic locking and how Redis handles atomicity and concurrency.

Mental Model

Core Idea

EXEC runs all queued commands in a Redis transaction atomically, ensuring they all succeed or fail together.

Think of it like...

Imagine writing a shopping list (commands) and handing it to a cashier (EXEC) who processes all items at once. Either you buy everything on the list or nothing, no partial checkout.

┌───────────┐   MULTI   ┌───────────────┐   EXEC   ┌───────────────┐
│ Client    │──────────▶│ Redis Queue   │────────▶│ Execute All   │
│ queues    │          │ commands      │         │ commands atomically│
│ commands  │          └───────────────┘         └───────────────┘

Build-Up - 6 Steps

1

Foundation

What is a Redis Transaction?

Concept: Introduce the idea of grouping commands in Redis using MULTI.

In Redis, a transaction starts with the MULTI command. This tells Redis to start queuing commands instead of running them immediately. You can then send multiple commands, and Redis will remember them but not execute yet.

Result

Commands are stored in a queue inside Redis, waiting to be executed.

Understanding that MULTI queues commands is key to knowing why EXEC is needed to run them.

2

Foundation

How EXEC Runs Queued Commands

Concept: EXEC executes all commands queued after MULTI in one atomic operation.

After queuing commands with MULTI, you send EXEC. Redis then runs all queued commands in order, as one single operation. If any command fails, the whole transaction still runs but you get errors for those commands.

Result

All queued commands run together, and their results are returned as a list.

EXEC is the trigger that turns queued commands into actual changes in Redis.

3

Intermediate

Atomicity Guarantees of EXEC

🤔Before reading on: Do you think EXEC partially applies commands if one fails, or all-or-nothing? Commit to your answer.

Concept: EXEC ensures atomic execution of all commands in the transaction.

EXEC runs all commands atomically, meaning no other client can see partial results. However, if a command inside the transaction is invalid, Redis still runs all commands but returns errors for the invalid ones. So atomicity means commands run together, but not rollback on errors.

Result

Commands run as a batch, but errors in some commands do not cancel others.

Knowing that EXEC does not rollback on command errors prevents false assumptions about full rollback.

4

Intermediate

Using WATCH with EXEC for Optimistic Locking

🤔Before reading on: Does EXEC alone prevent data changes by others during a transaction? Yes or no? Commit your answer.

Concept: WATCH monitors keys for changes to abort EXEC if data was modified by others.

WATCH lets you watch keys before MULTI. If any watched key changes before EXEC, the transaction aborts and EXEC returns null. This helps avoid race conditions by retrying transactions only if data is unchanged.

Result

EXEC runs only if watched keys are unchanged, otherwise transaction is discarded.

Understanding WATCH with EXEC shows how Redis handles concurrency safely.

5

Advanced

Limitations of EXEC and Error Handling

🤔Before reading on: Will EXEC rollback all commands if one command fails? Commit your answer.

Concept: EXEC does not rollback commands on errors; errors are reported but other commands still run.

If a command inside EXEC fails (like wrong argument), Redis still executes all commands. The client receives error replies for failed commands but successful commands remain applied. This means no automatic rollback.

Result

Partial success with error details returned; no rollback happens.

Knowing EXEC's error behavior helps design safer transactions and error handling.

6

Expert

Internals: How Redis Executes EXEC Atomically

🤔Before reading on: Do you think Redis locks the entire database during EXEC? Yes or no? Commit your answer.

Concept: EXEC uses a single-threaded event loop to run commands atomically without explicit locks.

Redis is single-threaded and processes commands one at a time. When EXEC runs, Redis executes all queued commands sequentially without interruption. This guarantees atomicity without complex locking. Other clients wait until EXEC finishes.

Result

Atomic execution achieved by Redis's single-threaded design, no partial interleaving.

Understanding Redis's single-threaded model explains how EXEC achieves atomicity efficiently.

Under the Hood

Redis queues commands after MULTI. When EXEC is called, Redis processes all queued commands sequentially in its single-threaded event loop. This means no other commands run in between, ensuring atomicity. Errors in commands do not rollback previous commands; they are reported individually.

Why designed this way?

Redis uses a single-threaded model for simplicity and speed, avoiding complex locks. EXEC leverages this to run transactions atomically without overhead. The design favors performance and simplicity over full rollback support, which would require more complexity.

┌─────────────┐
│ Client sends│
│ MULTI      │
└─────┬───────┘
      │
┌─────▼───────┐
│ Redis queues │
│ commands    │
└─────┬───────┘
      │
┌─────▼───────┐
│ Client sends│
│ EXEC       │
└─────┬───────┘
      │
┌─────▼───────┐
│ Redis runs  │
│ all queued  │
│ commands    │
│ atomically  │
└─────────────┘

Myth Busters - 3 Common Misconceptions

Quick: Does EXEC rollback all commands if one command fails? Commit yes or no.

Common Belief: EXEC rolls back all commands if any command inside the transaction fails.

Reality: EXEC runs all commands regardless of errors; it does not rollback previous commands on failure.

Why it matters: Assuming rollback leads to data inconsistency and bugs because partial changes remain applied.

Quick: Does EXEC lock the entire Redis database during execution? Commit yes or no.

Common Belief: EXEC locks the whole database to ensure atomicity.

Reality: Redis is single-threaded, so commands run one at a time; no explicit locks are needed for EXEC atomicity.

Why it matters: Misunderstanding locking can lead to wrong assumptions about performance and concurrency.

Quick: Can EXEC prevent other clients from modifying data during a transaction without WATCH? Commit yes or no.

Common Belief: EXEC alone prevents other clients from changing data during the transaction.

Reality: EXEC does not prevent other clients from modifying data before EXEC runs; WATCH is needed for that.

Why it matters: Ignoring WATCH can cause race conditions and unexpected data overwrites.

Expert Zone

1

EXEC does not provide rollback, so applications must handle partial failures manually.

2

WATCH combined with EXEC implements optimistic locking, but it requires retry logic in clients.

3

Redis transactions are not isolated like in traditional databases; commands run atomically but intermediate states are visible before EXEC.

When NOT to use

Do not use EXEC for complex multi-key transactions requiring rollback or isolation. Instead, use Lua scripting (EVAL) for atomic scripts or external transaction managers.

Production Patterns

In production, EXEC is often combined with WATCH for concurrency control. Lua scripts replace EXEC when atomicity with rollback or complex logic is needed. Monitoring EXEC failures helps detect transaction conflicts.

Connections

Database Transactions

EXEC implements a form of transaction atomicity similar to database commit operations.

Understanding EXEC helps grasp how Redis provides atomic operations without full ACID compliance.

Optimistic Locking

EXEC combined with WATCH implements optimistic locking to handle concurrent data changes.

Knowing EXEC's role clarifies how Redis manages concurrency without locks.

Single-threaded Event Loop

EXEC relies on Redis's single-threaded event loop to achieve atomic command execution.

Recognizing this connection explains why Redis can guarantee atomicity without complex locking.

Common Pitfalls

#1 Expecting EXEC to rollback all commands on error.

Wrong approach:

MULTI
SET key1 value1
INCR key2
SET key3
EXEC

Correct approach:

MULTI
SET key1 value1
INCR key2
EXEC
// Check for errors and handle manually

Root cause:

Misunderstanding that Redis transactions do not rollback on command errors.

#2 Using EXEC without WATCH and expecting safe concurrency.

Wrong approach:

MULTI
GET key
SET key newvalue
EXEC

Correct approach:

WATCH key
MULTI
GET key
SET key newvalue
EXEC

Root cause:

Ignoring the need for WATCH to detect concurrent modifications.

#3 Assuming EXEC locks the database causing performance issues.

Wrong approach:

Using EXEC in high concurrency without understanding Redis single-threading.

Correct approach:

Design transactions knowing Redis processes commands sequentially without explicit locks.

Root cause:

Confusing Redis's single-threaded model with traditional locking.

Key Takeaways

EXEC runs all queued commands in a Redis transaction atomically, ensuring they execute together.

Redis transactions do not rollback on errors; all commands run and errors are reported individually.

WATCH combined with EXEC enables optimistic locking to prevent race conditions.

Redis achieves atomicity through its single-threaded event loop, not by locking the database.

For complex atomic operations with rollback, Lua scripting is a better alternative than EXEC.