Maxmemory setting in Redis - Deep Dive

Overview - Maxmemory setting

What is it?

The maxmemory setting in Redis controls the maximum amount of memory the Redis server can use. When Redis reaches this limit, it uses a policy to decide which data to remove to free up space. This setting helps manage memory usage and keeps Redis from using too much memory on a system.

Why it matters

Without maxmemory, Redis could use all available memory on a server, causing the system to slow down or crash. This setting protects the server by limiting Redis's memory use and ensures Redis continues to work smoothly by removing old or less important data when needed.

Where it fits

Before learning maxmemory, you should understand basic Redis concepts like keys, values, and how Redis stores data in memory. After maxmemory, you can learn about eviction policies and persistence to manage data durability and memory efficiently.

Mental Model

Core Idea

Maxmemory sets a memory limit for Redis, and when reached, Redis removes data based on a chosen policy to stay within that limit.

Think of it like...

Imagine a backpack with a fixed size. You can only carry so much inside it. When it gets full, you have to decide what to take out before adding new items. Maxmemory is like the backpack size, and eviction policies decide what to remove.

┌───────────────┐
│   Redis RAM   │
│  Maxmemory    │
│  Limit Set   │
├───────────────┤
│ Data Stored   │
│ (Keys/Values) │
├───────────────┤
│ When Full →   │
│ Eviction      │
│ Policy Runs   │
└───────────────┘

Build-Up - 7 Steps

1

Foundation

What is Maxmemory in Redis

Concept: Introducing the maxmemory setting as a limit on Redis memory usage.

Redis stores all data in memory for fast access. Maxmemory is a configuration that sets the maximum memory Redis can use. If Redis tries to use more than this limit, it triggers actions to free memory.

Result

Redis will not exceed the set memory limit, protecting the server from running out of memory.

Understanding maxmemory is key to controlling Redis's memory footprint and preventing system crashes.

2

Foundation

How Redis Uses Memory

Concept: Explaining that Redis keeps all data in RAM and why memory management is critical.

Redis keeps data in RAM for speed. Unlike databases that store data on disk, Redis's speed depends on fitting data in memory. Without limits, Redis can consume all RAM, affecting other programs.

Result

Learners see why memory limits like maxmemory are necessary for stable Redis operation.

Knowing Redis is an in-memory database clarifies why maxmemory is a fundamental setting.

3

Intermediate

Configuring Maxmemory Setting

🤔Before reading on: Do you think maxmemory is set in bytes, megabytes, or as a percentage? Commit to your answer.

Concept: How to set maxmemory in Redis configuration using different units.

You can set maxmemory in the redis.conf file or at runtime with commands. It accepts bytes or human-readable units like 'mb' for megabytes. For example, maxmemory 100mb limits Redis to 100 megabytes of RAM.

Result

Redis enforces the memory limit you set, measured in bytes or convenient units.

Knowing how to configure maxmemory precisely lets you tailor Redis memory use to your server's capacity.

4

Intermediate

Eviction Policies When Maxmemory Reached

🤔Before reading on: Do you think Redis stops accepting writes when maxmemory is full, or does it remove some data? Commit to your answer.

Concept: Introducing eviction policies that decide what data Redis removes when maxmemory is reached.

When Redis hits maxmemory, it uses an eviction policy to free space. Policies include removing least recently used keys (LRU), least frequently used (LFU), or no eviction (reject writes). You choose the policy based on your needs.

Result

Redis continues working by removing data according to the chosen policy, preventing crashes.

Understanding eviction policies is crucial to controlling data loss and performance under memory pressure.

5

Intermediate

Noeviction Policy and Its Effects

Concept: Explaining the noeviction policy where Redis refuses writes when maxmemory is reached.

If you set maxmemory-policy to noeviction, Redis will reject commands that add data once the limit is reached. This protects existing data but can cause errors in your application if not handled.

Result

Redis stops accepting new data when memory is full, preserving current data but risking write failures.

Knowing noeviction helps you decide if data safety or availability is more important for your use case.

6

Advanced

Memory Fragmentation and Maxmemory

🤔Before reading on: Do you think maxmemory counts only used data memory or also includes fragmentation? Commit to your answer.

Concept: Memory fragmentation can cause Redis to use more memory than expected, affecting maxmemory enforcement.

Memory fragmentation happens when free memory is split into small pieces, making it hard to allocate large blocks. Redis's maxmemory counts total allocated memory including fragmentation, so actual usable memory can be less than maxmemory.

Result

Redis may hit maxmemory sooner than expected due to fragmentation, triggering eviction or errors.

Understanding fragmentation helps in tuning maxmemory and interpreting memory usage metrics correctly.

7

Expert

Maxmemory and Persistence Trade-offs

🤔Before reading on: Does maxmemory affect Redis persistence files on disk? Commit to your answer.

Concept: How maxmemory interacts with Redis persistence mechanisms like RDB and AOF.

Maxmemory limits RAM usage but does not limit disk space used for persistence. However, eviction can cause data loss before persistence saves. Also, persistence files can grow large independently, so monitoring both memory and disk is important.

Result

You learn that maxmemory controls RAM but persistence requires separate management to avoid data loss or disk issues.

Knowing this prevents surprises where data is evicted from memory but still saved on disk, or disk fills up despite memory limits.

Under the Hood

Redis tracks memory usage internally using allocator statistics and counts all memory allocated for keys, values, and overhead. When maxmemory is reached, Redis triggers the eviction process based on the configured policy. It scans keys according to the policy (e.g., LRU) and removes candidates until enough memory is freed. This process is efficient to avoid blocking Redis operations.

Why designed this way?

Redis was designed as an in-memory database prioritizing speed. Memory limits prevent Redis from destabilizing the host system. Eviction policies provide flexible trade-offs between data retention and availability. Alternatives like blocking writes were less flexible, so eviction policies became the standard.

┌───────────────┐
│ Redis Memory  │
│ Usage Track   │
├───────────────┤
│ Maxmemory Set │
├───────────────┤
│ If Usage >    │
│ Maxmemory →   │
│ Eviction Run  │
├───────────────┤
│ Eviction      │
│ Policy Check  │
├───────────────┤
│ Remove Keys   │
│ Until Enough  │
│ Memory Freed  │
└───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does setting maxmemory guarantee Redis will never use more memory? Commit yes or no.

Common Belief: Setting maxmemory means Redis will never exceed that memory limit.

Reality: Redis can temporarily exceed maxmemory due to memory fragmentation or internal overhead before eviction frees space.

Why it matters: Assuming strict limits can cause unexpected crashes or performance issues if memory usage spikes beyond maxmemory.

Quick: When maxmemory is full, does Redis always delete the oldest data? Commit yes or no.

Common Belief: Redis always removes the oldest data when maxmemory is reached.

Reality: Redis removes data based on the configured eviction policy, which may be least recently used, least frequently used, or other strategies, not necessarily oldest.

Why it matters: Misunderstanding eviction can lead to wrong assumptions about which data will be lost under memory pressure.

Quick: Does maxmemory affect how much disk space Redis uses for persistence? Commit yes or no.

Common Belief: Maxmemory also limits the disk space Redis uses for saving data.

Reality: Maxmemory only limits RAM usage; disk persistence files like RDB or AOF are not limited by maxmemory.

Why it matters: Ignoring disk usage can cause disk space exhaustion even if memory is controlled, leading to system failures.

Quick: If maxmemory-policy is set to noeviction, will Redis accept all writes? Commit yes or no.

Common Belief: With noeviction policy, Redis will keep accepting writes even if memory is full.

Reality: Redis rejects write commands when memory is full under noeviction policy, causing errors.

Why it matters: Not handling write rejections can cause application failures or data loss.

Expert Zone

1

Eviction policies can be combined with volatile or allkeys modes, affecting whether only keys with expiration or all keys are candidates for eviction.

2

Memory fragmentation can vary greatly depending on workload and allocator, so maxmemory tuning requires monitoring fragmentation metrics.

3

Redis's internal memory accounting includes overhead for data structures, so actual usable memory for data is less than maxmemory.

When NOT to use

Maxmemory is not suitable if you need guaranteed data persistence without loss; in such cases, consider Redis in append-only mode with no eviction or use disk-based databases. Also, for workloads with unpredictable memory spikes, external memory management or sharding may be better.

Production Patterns

In production, maxmemory is set based on available system RAM minus overhead for OS and other processes. Eviction policies like allkeys-lru are common for caching use cases. Monitoring tools track memory usage and eviction events to adjust settings dynamically.

Connections

Cache Eviction Algorithms

Maxmemory eviction policies implement cache eviction algorithms like LRU and LFU.

Understanding cache eviction helps grasp how Redis decides which data to remove under memory pressure.

Operating System Memory Management

Maxmemory setting interacts with OS memory limits and fragmentation behavior.

Knowing OS memory management clarifies why Redis memory usage can exceed limits temporarily and how fragmentation affects performance.

Project Management Resource Limits

Both set limits to prevent overuse of resources and require policies to handle excess demand.

Recognizing resource limits and policies in different fields helps appreciate why maxmemory and eviction policies are essential for system stability.

Common Pitfalls

#1 Setting maxmemory too high, leaving no RAM for OS and other processes.

Wrong approach:

maxmemory 100% # Trying to use all system RAM

Correct approach:

maxmemory 75% # Reserving RAM for OS and other apps

Root cause:

Misunderstanding that Redis shares system memory and needs headroom for OS and other applications.

#2 Using noeviction policy without handling write errors in the application.

Wrong approach:

maxmemory-policy noeviction
# Application ignores write failures

Correct approach:

maxmemory-policy noeviction
# Application checks and handles write errors gracefully

Root cause:

Not anticipating that Redis will reject writes when memory is full under noeviction.

#3 Assuming maxmemory limits disk usage and ignoring persistence file growth.

Wrong approach:

# Set maxmemory but no disk monitoring
maxmemory 100mb

Correct approach:

# Set maxmemory and monitor disk space for persistence
maxmemory 100mb
# Use monitoring tools for disk usage

Root cause:

Confusing memory limits with disk storage limits.

Key Takeaways

Maxmemory sets a hard limit on how much RAM Redis can use to store data.

When Redis reaches maxmemory, it removes data based on eviction policies to free space.

Choosing the right eviction policy balances data retention and availability under memory pressure.

Memory fragmentation can cause Redis to exceed maxmemory temporarily, affecting performance.

Maxmemory controls RAM usage but does not limit disk space used for persistence.

Practice

1. What does the maxmemory setting in Redis control?

easy

A. The maximum amount of memory Redis can use

B. The maximum number of clients connected

C. The maximum size of a single key

D. The maximum number of commands per second

Solution

1. Step 1: Understand the purpose of maxmemory

   The maxmemory setting limits how much memory Redis can use to avoid overloading the server.

2. Step 2: Compare options with this purpose

   Only the maximum amount of memory Redis can use matches this description, others relate to clients, keys, or commands, which are unrelated.

3. Final Answer:

   The maximum amount of memory Redis can use -> Option A

4. Quick Check:

   maxmemory limits Redis memory use [OK]

Hint: maxmemory always limits Redis memory size [OK]

Common Mistakes:

• Confusing maxmemory with client limits
• Thinking maxmemory limits key size
• Assuming maxmemory limits command rate

2. Which of the following is the correct syntax to set maxmemory to 512 megabytes using CONFIG SET in Redis?

easy

A. CONFIG SET maxmemory 512mb

B. CONFIG SET maxmemory 512MB

C. CONFIG SET maxmemory 512m

D. CONFIG SET maxmemory 512

Solution

1. Step 1: Recall Redis size suffix format

   Redis accepts size suffixes like KB, MB, GB in uppercase for memory settings.

2. Step 2: Check each option's suffix

   CONFIG SET maxmemory 512MB uses 'MB' uppercase which is correct; CONFIG SET maxmemory 512mb uses lowercase 'mb' which is invalid; CONFIG SET maxmemory 512m uses 'm' which is not recognized; CONFIG SET maxmemory 512 has no suffix, so it's bytes, not megabytes.

3. Final Answer:

   CONFIG SET maxmemory 512MB -> Option B

4. Quick Check:

   Use uppercase suffixes like MB for megabytes [OK]

Hint: Use uppercase suffixes like MB, GB for sizes [OK]

Common Mistakes:

• Using lowercase suffixes like mb
• Using incomplete suffix like m
• Omitting suffix and expecting megabytes

3. Given the command CONFIG SET maxmemory 1GB executed on a Redis server, what will be the effect?

medium

A. Redis will limit its memory usage to 1 gigabyte

B. Redis will allow unlimited memory usage

C. Redis will limit memory to 1 byte

D. Redis will throw a syntax error

Solution

1. Step 1: Interpret the command meaning

   The command sets maxmemory to 1GB, meaning Redis should limit memory use to 1 gigabyte.

2. Step 2: Validate the command syntax and effect

   The syntax is correct and Redis supports 'GB' suffix, so it will enforce the memory limit.

3. Final Answer:

   Redis will limit its memory usage to 1 gigabyte -> Option A

4. Quick Check:

   CONFIG SET maxmemory 1GB limits memory to 1GB [OK]

Hint: CONFIG SET maxmemory with GB limits memory to gigabytes [OK]

Common Mistakes:

• Thinking it sets unlimited memory
• Confusing bytes with gigabytes
• Expecting syntax error from correct command

4. You run CONFIG SET maxmemory 100mb on Redis but it returns an error. What is the likely cause?

medium

A. Redis does not support changing maxmemory at runtime

B. maxmemory must be set in bytes only, no suffix allowed

C. The value 100mb is too large for Redis

D. The suffix 'mb' is lowercase and invalid

Solution

1. Step 1: Check Redis suffix case sensitivity

   Redis requires uppercase suffixes like MB, GB for memory sizes.

2. Step 2: Analyze the error cause

   Using lowercase 'mb' causes syntax error; changing to 'MB' fixes it.

3. Final Answer:

   The suffix 'mb' is lowercase and invalid -> Option D

4. Quick Check:

   Use uppercase suffixes for memory sizes [OK]

Hint: Always use uppercase suffixes like MB, GB [OK]

Common Mistakes:

• Assuming maxmemory can't be changed at runtime
• Using lowercase suffix causing errors
• Thinking maxmemory only accepts bytes

5. You want Redis to use at most 2 gigabytes of memory and automatically evict keys when the limit is reached. Which two commands should you run?

hard

A. CONFIG SET maxmemory 2gb and CONFIG SET maxmemory-policy allkeys-random

B. CONFIG SET maxmemory 2GB and CONFIG SET maxmemory-policy noeviction

C. CONFIG SET maxmemory 2GB and CONFIG SET maxmemory-policy allkeys-lru

D. CONFIG SET maxmemory 2048 and CONFIG SET maxmemory-policy volatile-lru

Solution

1. Step 1: Set maxmemory to 2 gigabytes correctly

   Use CONFIG SET maxmemory 2GB with uppercase suffix for 2 gigabytes.

2. Step 2: Choose eviction policy that evicts keys automatically

   allkeys-lru evicts least recently used keys from all keys, enabling automatic eviction.

3. Step 3: Verify other options

   Lowercase '2gb' with allkeys-random uses invalid suffix and random eviction; 2GB with noeviction disables eviction; 2048 (bytes, not GB) with volatile-lru limits to tiny memory and only evicts expiring keys.

4. Final Answer:

   CONFIG SET maxmemory 2GB and CONFIG SET maxmemory-policy allkeys-lru -> Option C

5. Quick Check:

   Set maxmemory + allkeys-lru for eviction [OK]

Hint: Use uppercase GB and allkeys-lru for eviction [OK]

Common Mistakes:

• Using lowercase suffix like '2gb'
• Choosing noeviction policy
• Setting maxmemory without suffix
• Using volatile-lru when all keys need eviction