Sparse index pattern in DynamoDB - Step-by-Step Execution

Concept Flow - Sparse index pattern

Create main table with items

↓

Add GSI with attribute only on some items

↓

Insert items with and without GSI attribute

↓

Query GSI returns only items with attribute

↓

Result: Sparse index filters items

The sparse index pattern uses a secondary index that only includes items having a specific attribute, so queries on that index return only those items.

Execution Sample

DynamoDB

CREATE TABLE Products (
  ProductID STRING HASH KEY,
  Category STRING,
  Price NUMBER,
  OnSale BOOLEAN
);

CREATE GLOBAL SECONDARY INDEX OnSaleIndex ON Products (OnSale);

Creates a table and a sparse GSI that indexes only items where OnSale is true.

Execution Table

Step	Action	Item Inserted	GSI Attribute Present?	GSI Index Updated	Query OnSaleIndex Result
1	Insert item	{"ProductID": "P1", "Category": "Books", "Price": 10, "OnSale": true}	Yes	Yes	P1 included
2	Insert item	{"ProductID": "P2", "Category": "Books", "Price": 15}	No	No	P1 only
3	Insert item	{"ProductID": "P3", "Category": "Electronics", "Price": 100, "OnSale": true}	Yes	Yes	P1, P3 included
4	Query OnSaleIndex	N/A	N/A	N/A	Returns items P1 and P3 only
5	Insert item	{"ProductID": "P4", "Category": "Electronics", "Price": 200}	No	No	P1, P3 only
6	Query OnSaleIndex	N/A	N/A	N/A	Still returns P1 and P3 only

💡 Query on sparse index returns only items with OnSale attribute true; items without attribute are excluded.

Variable Tracker

Variable	Start	After 1	After 2	After 3	After 4	After 5	Final
Items in Table	[]	[P1]	[P1, P2]	[P1, P2, P3]	[P1, P2, P3]	[P1, P2, P3, P4]	[P1, P2, P3, P4]
Items in GSI	[]	[P1]	[P1]	[P1, P3]	[P1, P3]	[P1, P3]	[P1, P3]
Query Result	[]	[P1]	[P1]	[P1, P3]	[P1, P3]	[P1, P3]	[P1, P3]

Key Moments - 2 Insights

Why doesn't item P2 appear in the GSI or query results?

Because P2 does not have the OnSale attribute set, it is excluded from the sparse index as shown in execution_table rows 2 and 4.

What happens if an item does not have the attribute used for the sparse index?

The item is not included in the GSI, so queries on that index will not return it, as seen with items P2 and P4 in the execution_table.

Visual Quiz - 3 Questions

Look at the execution_table, which items are returned by the OnSaleIndex query after step 3?

A Only P3

B P1 and P3

C P1, P2, and P3

D P2 and P4

💡 Hint

Check the 'Query OnSaleIndex Result' column at step 4 in the execution_table.

At which step is item P4 inserted and does it appear in the GSI?

A Step 5, Yes

B Step 6, Yes

C Step 5, No

D Step 6, No

💡 Hint

Look at the 'Item Inserted' and 'GSI Index Updated' columns at step 5 in the execution_table.

If the OnSale attribute was added to P2 with value true after insertion, what would change in the variable_tracker?

A P2 would appear in Items in GSI and Query Result after update

B No change, P2 stays excluded

C P2 would be removed from Items in Table

D P2 would appear only in Query Result but not in GSI

💡 Hint

Sparse index includes items with the attribute; see how Items in GSI and Query Result track items.

Concept Snapshot

Sparse index pattern in DynamoDB:
- Create a GSI on an attribute present only on some items.
- Only items with that attribute appear in the GSI.
- Queries on the GSI return only those items.
- Useful to filter items efficiently without scanning the whole table.
- Items missing the attribute are excluded from the index.

Full Transcript

The sparse index pattern in DynamoDB uses a global secondary index (GSI) that only includes items having a specific attribute. When you create a GSI on an attribute that only some items have, only those items appear in the index. For example, if you create a GSI on an 'OnSale' attribute, only items where OnSale is true will be indexed. When you query this GSI, you get only those items, effectively filtering the data. Items without the attribute do not appear in the GSI or query results. This pattern helps efficiently query subsets of data without scanning the entire table.

Practice

1. What is the main benefit of using a sparse index pattern in DynamoDB?

easy

A. It indexes all items regardless of attributes, increasing storage.

B. It indexes only items with a specific attribute, making queries faster and cheaper.

C. It duplicates all data to another table for backup.

D. It automatically deletes items without a key attribute.

Solution

1. Step 1: Understand sparse index purpose

   Sparse index only includes items that have the indexed attribute, skipping others.

2. Step 2: Identify benefit of selective indexing

   This selective indexing reduces query cost and speeds up queries for that subset.

3. Final Answer:

   It indexes only items with a specific attribute, making queries faster and cheaper. -> Option B

4. Quick Check:

   Sparse index = selective fast queries [OK]

Hint: Sparse index targets only items with the attribute [OK]

Common Mistakes:

• Thinking sparse index includes all items
• Confusing sparse index with full table scan
• Assuming sparse index duplicates data

2. Which of the following is the correct way to define a sparse index in DynamoDB?

easy

A. Create a Global Secondary Index (GSI) with a partition key only on items having the attribute.

B. Create a Local Secondary Index (LSI) on all items regardless of attribute presence.

C. Create a GSI with a partition key that is always null.

D. Create a GSI with a partition key that duplicates the primary key.

Solution

1. Step 1: Identify sparse index creation method

   Sparse indexes use GSIs with keys only on items having the attribute.

2. Step 2: Check options for correctness

   Create a Global Secondary Index (GSI) with a partition key only on items having the attribute. matches this method; others either index all items or use invalid keys.

3. Final Answer:

   Create a Global Secondary Index (GSI) with a partition key only on items having the attribute. -> Option A

4. Quick Check:

   Sparse index = GSI with selective keys [OK]

Hint: Sparse index uses GSI with keys only on some items [OK]

Common Mistakes:

• Confusing LSI with sparse index
• Using null keys in GSI partition key
• Duplicating primary key in GSI

3. Given a DynamoDB table with items where only some have attribute status, and a sparse GSI on status, what will this query return?

Query on GSI where status = 'active'

medium

A. Only items without the status attribute.

B. All items in the table regardless of status.

C. All items in the table with status 'active'.

D. An error because status is not a primary key.

Solution

1. Step 1: Understand sparse GSI query behavior

   The sparse GSI includes only items with the status attribute, so querying status='active' returns those items.

2. Step 2: Eliminate incorrect options

   All items in the table regardless of status. is wrong because items without status are not in GSI; C is wrong because query targets status='active'; D is wrong because GSI keys can be queried.

3. Final Answer:

   All items in the table with status 'active'. -> Option C

4. Quick Check:

   Query sparse GSI = items with attribute value [OK]

Hint: Query sparse GSI returns only items with that attribute [OK]

Common Mistakes:

• Expecting all table items in result
• Thinking query fails without primary key
• Confusing sparse index with full scan

4. You created a sparse GSI on attribute category, but your query returns no results even though items have category. What is a likely cause?

medium

A. Sparse indexes do not support queries.

B. The table has no items at all.

C. You queried the base table instead of the GSI.

D. The GSI partition key is not set to category.

Solution

1. Step 1: Check GSI key configuration

   If the GSI partition key is not category, the index won't include those items properly.

2. Step 2: Evaluate other options

   Table having no items is unlikely if items exist; querying base table returns items but not via GSI; sparse indexes do support queries.

3. Final Answer:

   The GSI partition key is not set to category. -> Option D

4. Quick Check:

   Wrong GSI key = no results [OK]

Hint: Check GSI partition key matches attribute queried [OK]

Common Mistakes:

• Querying base table instead of GSI
• Assuming sparse index can't be queried
• Ignoring GSI key setup

5. You want to quickly find all orders with a deliveryDate attribute set, but most orders don't have this attribute. How can you use a sparse index pattern to optimize queries?

hard

A. Create a GSI with deliveryDate as the partition key; only orders with this attribute appear in the index.

B. Add deliveryDate as a sort key in the main table to speed up queries.

C. Create a GSI with a constant partition key for all items and deliveryDate as sort key.

D. Scan the entire table filtering on deliveryDate attribute.

Solution

1. Step 1: Use sparse index with deliveryDate attribute

   Creating a GSI with deliveryDate as partition key includes only items having that attribute, making queries efficient.

2. Step 2: Evaluate other options

   A constant partition key approach requires adding an extra attribute to all items, incurring write costs; main table sort key doesn't filter missing attributes; full table scan is inefficient.

3. Final Answer:

   Create a GSI with deliveryDate as the partition key; only orders with this attribute appear in the index. -> Option A

4. Quick Check:

   Sparse GSI on attribute = fast filtered queries [OK]

Hint: Sparse GSI on attribute filters only items having it [OK]

Common Mistakes:

• Using scan instead of sparse index
• Indexing all items losing sparsity
• Misusing sort key for filtering