Embedding vs referencing in Redis - Trade-offs & Expert Analysis

Overview - Embedding vs referencing

What is it?

Embedding and referencing are two ways to organize related data in Redis, a fast key-value database. Embedding means storing related data together inside one record, while referencing means storing data separately and linking them by keys. This helps manage how data is saved and accessed efficiently.

Why it matters

Without choosing between embedding or referencing, data can become slow to access or hard to update. Embedding makes reading related data fast but can cause duplication, while referencing keeps data clean but needs extra steps to gather information. Picking the right method improves app speed and simplicity.

Where it fits

Before learning this, you should understand basic Redis data types like strings, hashes, and sets. After this, you can learn about Redis data modeling and performance tuning to build efficient applications.

Mental Model

Core Idea

Embedding stores related data together inside one Redis key, while referencing stores data separately and links them by keys.

Think of it like...

Imagine a photo album: embedding is like putting all photos of a trip in one album page, while referencing is like having separate photo pages linked by a table of contents.

┌───────────────┐       ┌───────────────┐
│ Embedded Key  │──────▶│ All data inside│
│ (one record)  │       │ one place     │
└───────────────┘       └───────────────┘

┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│ Reference Key │──────▶│ Separate Data │       │ Separate Data │
│ (links keys)  │       │ (key 1)      │       │ (key 2)      │
└───────────────┘       └───────────────┘       └───────────────┘

Build-Up - 6 Steps

1

Foundation

Understanding Redis Basic Storage

Concept: Learn how Redis stores simple data types like strings and hashes.

Redis stores data as keys with values. Values can be simple strings or more complex types like hashes, which hold multiple fields and values inside one key.

Result

You can store and retrieve simple and grouped data using Redis commands like SET and HSET.

Knowing Redis data types is essential before deciding how to organize related data using embedding or referencing.

2

Foundation

What Is Embedding in Redis

Concept: Embedding means putting all related data inside one Redis key, often using a hash.

For example, a user profile with name, email, and age can be stored in one hash key: HSET user:1 name 'Alice' email '[e-mail]' age 30. All data is together and easy to get in one command.

Result

Fetching user:1 returns all fields at once quickly.

Embedding simplifies data access by grouping related fields, reducing the number of commands needed.

3

Intermediate

What Is Referencing in Redis

Concept: Referencing means storing related data in separate keys and linking them by storing keys or IDs.

For example, user:1 stores basic info, and user:1:posts stores a list of post IDs. Each post is stored separately as post:123. To get all posts, you first get the list of IDs, then fetch each post.

Result

Data is split but linked, requiring multiple commands to gather full info.

Referencing keeps data modular and avoids duplication but needs extra steps to combine data.

4

Intermediate

Comparing Embedding and Referencing

🤔Before reading on: do you think embedding or referencing is always faster? Commit to your answer.

Concept: Understand the trade-offs between embedding and referencing in speed, storage, and complexity.

Embedding is faster for reading all related data because it's in one key, but updating parts can be slower or cause duplication. Referencing uses less space by avoiding duplication and allows independent updates but needs more commands to read full data.

Result

Choosing embedding or referencing depends on your app's read/write patterns and data size.

Knowing trade-offs helps design data models that balance speed and maintainability.

5

Advanced

Handling Data Consistency with Referencing

🤔Before reading on: do you think Redis automatically keeps referenced data consistent? Commit to yes or no.

Concept: Learn how to keep data consistent when using references in Redis.

Redis does not enforce consistency between referenced keys. For example, deleting a user key does not delete their posts automatically. You must write application logic to update or delete related keys to avoid stale data.

Result

You must manage consistency manually when using referencing.

Understanding Redis's lack of built-in joins or foreign keys prevents bugs from inconsistent data.

6

Expert

Optimizing Redis Models with Hybrid Approaches

🤔Before reading on: do you think embedding and referencing can be combined? Commit to yes or no.

Concept: Explore mixing embedding and referencing to optimize performance and flexibility.

For example, embed small, frequently accessed fields inside a hash, but reference large or rarely changed data separately. This reduces duplication and speeds up common reads. Use Redis pipelines or Lua scripts to fetch related data efficiently.

Result

Hybrid models balance speed and storage, tailored to app needs.

Knowing hybrid approaches unlocks advanced Redis data modeling for real-world apps.

Under the Hood

Redis stores data as key-value pairs in memory for fast access. Embedding stores multiple fields inside one key using hashes, which are efficient for grouped data. Referencing stores separate keys and requires multiple lookups to gather related data. Redis does not support joins or foreign keys, so references are managed by the application.

Why designed this way?

Redis was designed for speed and simplicity, focusing on key-value access. Embedding fits this by grouping related data in one key, minimizing lookups. Referencing allows flexible, normalized data but requires extra commands. This design avoids complex joins to keep Redis fast and scalable.

┌───────────────┐
│ Redis Memory  │
│  (Key-Value)  │
└──────┬────────┘
       │
┌──────▼───────┐          ┌───────────────┐
│ Embedded Key │─────────▶│ Hash with     │
│ (one record) │          │ multiple fields│
└──────────────┘          └───────────────┘

┌───────────────┐          ┌───────────────┐          ┌───────────────┐
│ Reference Key │─────────▶│ Separate Key 1│          │ Separate Key 2│
│ (links keys)  │          │ (data part 1) │          │ (data part 2) │
└───────────────┘          └───────────────┘          └───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does embedding always use more memory than referencing? Commit yes or no.

Common Belief: Embedding always uses more memory because it duplicates data.

Reality: Embedding can use less memory if it avoids storing repeated keys or references separately.

Why it matters: Assuming embedding wastes memory may lead to overusing referencing, causing slower reads.

Quick: Does referencing automatically keep data consistent in Redis? Commit yes or no.

Common Belief: Referencing keys in Redis automatically keeps data consistent like foreign keys in SQL.

Reality: Redis does not enforce consistency; the application must handle updates and deletes to keep references valid.

Why it matters: Ignoring this causes stale or broken references, leading to bugs and incorrect data.

Quick: Is embedding always faster than referencing? Commit yes or no.

Common Belief: Embedding is always faster because all data is in one key.

Reality: Embedding is faster for reads but can slow writes or cause duplication; referencing can be faster for updates or large data.

Why it matters: Choosing embedding blindly can hurt performance in write-heavy or large data scenarios.

Quick: Can you use Redis commands to join referenced data automatically? Commit yes or no.

Common Belief: Redis supports automatic joins between referenced keys like SQL databases.

Reality: Redis has no join commands; combining referenced data requires multiple commands or scripting.

Why it matters: Expecting automatic joins leads to inefficient or incorrect data access patterns.

Expert Zone

1

Embedding small, frequently accessed fields reduces network round-trips and improves latency.

2

Referencing large or rarely changed data avoids duplication and reduces memory usage.

3

Using Redis Lua scripts or pipelines can efficiently fetch referenced data in fewer commands.

When NOT to use

Avoid embedding when data grows large or changes independently, as it causes duplication and slow writes. Avoid referencing when you need very fast reads of all related data. Instead, use hybrid models or specialized databases with joins if complex relations are needed.

Production Patterns

In production, developers embed user profile info but reference user-generated content like posts or comments. They use Redis pipelines to batch fetch referenced keys and Lua scripts to maintain consistency during updates.

Connections

Normalization in Relational Databases

Referencing in Redis is similar to normalization, separating data to avoid duplication.

Understanding normalization helps grasp why referencing reduces data duplication but adds complexity.

Caching Strategies

Embedding resembles caching full objects, while referencing is like caching pointers or IDs.

Knowing caching trade-offs clarifies when embedding or referencing improves Redis performance.

Object Composition in Software Design

Embedding is like composing objects with all parts inside, referencing is like linking separate objects.

Recognizing this helps software developers design data models that match application logic.

Common Pitfalls

#1 Storing all related data in separate keys without linking them.

Wrong approach:

HSET user:1 name 'Alice'
HSET user:1:posts post1 'Hello'
# No key storing list of posts

Correct approach:

HSET user:1 name 'Alice'
LPUSH user:1:posts post1
HSET post:post1 content 'Hello'

Root cause:

Not linking referenced data causes inability to find related records.

#2 Embedding large, frequently changing data causing slow writes.

Wrong approach:

HSET user:1 name 'Alice' posts 'very large JSON string with all posts'

Correct approach:

HSET user:1 name 'Alice'
LPUSH user:1:posts post1
HSET post:post1 content 'Hello'

Root cause:

Embedding large data causes duplication and slow updates.

#3 Assuming Redis enforces reference integrity automatically.

Wrong approach:

DEL user:1
# expecting posts to be deleted automatically

Correct approach:

DEL user:1
DEL user:1:posts
DEL post:post1
# application deletes all related keys

Root cause:

Redis lacks foreign key constraints; app must manage consistency.

Key Takeaways

Embedding stores related data together inside one Redis key for fast, simple reads.

Referencing stores data separately and links them by keys, reducing duplication but needing extra commands.

Choosing embedding or referencing depends on data size, update frequency, and access patterns.

Redis does not enforce data consistency between referenced keys; applications must handle it.

Hybrid models combining embedding and referencing often provide the best balance in real-world apps.

Practice

1. Which statement best describes embedding in Redis?

easy

A. Storing related data together in one key.

B. Storing data in separate keys linked by IDs.

C. Using multiple databases for related data.

D. Storing only references without actual data.

Solution

1. Step 1: Understand embedding concept

   Embedding means putting related data inside one Redis key, so all data is together.

2. Step 2: Compare with referencing

   Referencing stores data separately and links them by IDs, which is different from embedding.

3. Final Answer:

   Storing related data together in one key. -> Option A

4. Quick Check:

   Embedding = data together [OK]

Hint: Embedding means one key holds all related data [OK]

Common Mistakes:

• Confusing embedding with referencing
• Thinking embedding uses multiple keys
• Assuming embedding stores only IDs

2. Which Redis data structure is best suited for embedding small related data together?

easy

A. Sorted Set

B. String

C. List

D. Hash

Solution

1. Step 1: Identify data structure for embedding

   Hashes store multiple fields and values inside one key, ideal for embedding related small data.

2. Step 2: Compare other structures

   Strings hold single values, Lists and Sorted Sets are for ordered collections, less suited for embedding small related fields.

3. Final Answer:

   Hash -> Option D

4. Quick Check:

   Embedding small data = Hash [OK]

Hint: Use Hash for embedding small related fields [OK]

Common Mistakes:

• Choosing String for multiple fields
• Confusing List with Hash
• Using Sorted Set without score relevance

3. Given these Redis commands:

HSET user:100 name "Alice" age 30
HSET user:100:address city "NY" zip 10001

Which describes the data design?

medium

A. Referencing address data in a separate key

B. Embedding all user data in one key

C. Using a List to store user info

D. Storing user data as a String

Solution

1. Step 1: Analyze keys and data

   user:100 holds name and age, user:100:address holds address fields separately.

2. Step 2: Identify design pattern

   Address is stored in a separate key linked by user ID, which is referencing.

3. Final Answer:

   Referencing address data in a separate key -> Option A

4. Quick Check:

   Separate keys linked by ID = referencing [OK]

Hint: Separate keys with related IDs mean referencing [OK]

Common Mistakes:

• Thinking all data is embedded in one key
• Confusing List with Hash usage
• Assuming address is embedded inside user key

4. You want to store user profiles with embedded address data in Redis using hashes. Which command has an error?

medium

A. HSET user:200 name "Bob" age 25

B. HSET user:200:address city "LA" zip 90001

C. HSET user:200 name "Bob" city "LA" zip 90001

D. HSET user:200 name "Bob" age 25 city "LA" zip 90001

Solution

1. Step 1: Check embedding address in user hash

   Embedding means all fields in one key, so address fields city and zip should be in user:200 hash.

2. Step 2: Identify error in command C

   Command C uses a separate key user:200:address, storing address separately linked by ID. This is referencing, not embedding.

3. Final Answer:

   HSET user:200:address city "LA" zip 90001 -> Option B

4. Quick Check:

   Separate keys = referencing, not embedding [OK]

Hint: Embedding: all data in ONE key, no sub-keys [OK]

Common Mistakes:

• Missing value for a field
• Using separate keys when embedding
• Mixing field-value pairs incorrectly

5. You have a Redis database storing product info. You want to optimize for fast reads of product details including reviews. Which design is best?

hard

A. Embed product details and all reviews in one hash key per product.

B. Store all reviews in a single list key for all products.

C. Store product details in one hash and each review in separate keys linked by product ID.

D. Store product details as strings and reviews as sorted sets.

Solution

1. Step 1: Consider data size and access pattern

   Reviews can be many and large, embedding all in one key can cause large data and slow updates.

2. Step 2: Choose referencing for scalability

   Storing product details separately and reviews in linked keys allows efficient updates and reads without large keys.

3. Step 3: Evaluate other options

   Embedding all reviews in one hash risks large keys; storing all reviews in a single list key for all products mixes products; storing product details as strings and reviews as sorted sets don't fit review structure well.

4. Final Answer:

   Store product details in one hash and each review in separate keys linked by product ID. -> Option C

5. Quick Check:

   Large related data = referencing for performance [OK]

Hint: Large or many related items? Use referencing [OK]

Common Mistakes:

• Embedding large lists causing slow reads
• Mixing all reviews in one key
• Ignoring update frequency of reviews