Sparse index pattern in DynamoDB

Introduction

The sparse index pattern helps you find only certain items in a big table by using an index that has fewer entries. This makes searching faster and easier.

You want to quickly find items that have a specific attribute without scanning the whole table.

You have a large table but only need to query a small subset of items often.

You want to save costs by avoiding full table scans in DynamoDB.

You want to organize data so that only items with a certain property appear in an index.

Syntax

DynamoDB

Create a secondary index with a partition key that only some items have.

Example:
- Table items have attribute "status".
- Only items with status = "active" have a special attribute "activeId".
- Create a Global Secondary Index (GSI) with partition key = "activeId".

Only items with "activeId" appear in this index.

A sparse index only includes items where the index key attribute exists.

This pattern reduces the size of the index and speeds up queries for that subset.

Examples

Only the first item appears in the GSI because it has the "activeId" attribute.

DynamoDB

Table items:
{
  "id": "1",
  "status": "active",
  "activeId": "A1"
}

{
  "id": "2",
  "status": "inactive"
}

GSI partition key: activeId

This query returns just the items marked active, ignoring others.

DynamoDB

Query the GSI with activeId = "A1" to get only active items.

Sample Program

This example uses PartiQL to insert items into a DynamoDB table with a sparse GSI on ActiveId. Only users with ActiveId appear in the index. The query returns only active users.

DynamoDB

-- Create table and GSI using AWS Console, CLI, or SDK:
-- Table: Users, Partition Key: UserId (String)
-- GSI: ActiveUsersIndex, Partition Key: ActiveId (String), Projection: ALL

INSERT INTO "Users" VALUE {
  "UserId": "1",
  "Status": "active",
  "ActiveId": "A1"
};

INSERT INTO "Users" VALUE {
  "UserId": "2",
  "Status": "inactive"
};

-- Query the sparse index
SELECT * FROM "Users"."ActiveUsersIndex" WHERE "ActiveId" = 'A1';

Important Notes

Make sure the attribute used as the index key exists only on the items you want in the index.

Sparse indexes help reduce costs by limiting the size of the index and speeding queries.

Summary

Sparse index pattern uses an index with keys only on some items.

This makes queries faster and cheaper for that subset of data.

It works well when you want to find items with a specific attribute quickly.

Practice

1. What is the main benefit of using a sparse index pattern in DynamoDB?

easy

A. It indexes all items regardless of attributes, increasing storage.

B. It indexes only items with a specific attribute, making queries faster and cheaper.

C. It duplicates all data to another table for backup.

D. It automatically deletes items without a key attribute.

Solution

1. Step 1: Understand sparse index purpose

   Sparse index only includes items that have the indexed attribute, skipping others.

2. Step 2: Identify benefit of selective indexing

   This selective indexing reduces query cost and speeds up queries for that subset.

3. Final Answer:

   It indexes only items with a specific attribute, making queries faster and cheaper. -> Option B

4. Quick Check:

   Sparse index = selective fast queries [OK]

Hint: Sparse index targets only items with the attribute [OK]

Common Mistakes:

• Thinking sparse index includes all items
• Confusing sparse index with full table scan
• Assuming sparse index duplicates data

2. Which of the following is the correct way to define a sparse index in DynamoDB?

easy

A. Create a Global Secondary Index (GSI) with a partition key only on items having the attribute.

B. Create a Local Secondary Index (LSI) on all items regardless of attribute presence.

C. Create a GSI with a partition key that is always null.

D. Create a GSI with a partition key that duplicates the primary key.

Solution

1. Step 1: Identify sparse index creation method

   Sparse indexes use GSIs with keys only on items having the attribute.

2. Step 2: Check options for correctness

   Create a Global Secondary Index (GSI) with a partition key only on items having the attribute. matches this method; others either index all items or use invalid keys.

3. Final Answer:

   Create a Global Secondary Index (GSI) with a partition key only on items having the attribute. -> Option A

4. Quick Check:

   Sparse index = GSI with selective keys [OK]

Hint: Sparse index uses GSI with keys only on some items [OK]

Common Mistakes:

• Confusing LSI with sparse index
• Using null keys in GSI partition key
• Duplicating primary key in GSI

3. Given a DynamoDB table with items where only some have attribute status, and a sparse GSI on status, what will this query return?

Query on GSI where status = 'active'

medium

A. Only items without the status attribute.

B. All items in the table regardless of status.

C. All items in the table with status 'active'.

D. An error because status is not a primary key.

Solution

1. Step 1: Understand sparse GSI query behavior

   The sparse GSI includes only items with the status attribute, so querying status='active' returns those items.

2. Step 2: Eliminate incorrect options

   All items in the table regardless of status. is wrong because items without status are not in GSI; C is wrong because query targets status='active'; D is wrong because GSI keys can be queried.

3. Final Answer:

   All items in the table with status 'active'. -> Option C

4. Quick Check:

   Query sparse GSI = items with attribute value [OK]

Hint: Query sparse GSI returns only items with that attribute [OK]

Common Mistakes:

• Expecting all table items in result
• Thinking query fails without primary key
• Confusing sparse index with full scan

4. You created a sparse GSI on attribute category, but your query returns no results even though items have category. What is a likely cause?

medium

A. Sparse indexes do not support queries.

B. The table has no items at all.

C. You queried the base table instead of the GSI.

D. The GSI partition key is not set to category.

Solution

1. Step 1: Check GSI key configuration

   If the GSI partition key is not category, the index won't include those items properly.

2. Step 2: Evaluate other options

   Table having no items is unlikely if items exist; querying base table returns items but not via GSI; sparse indexes do support queries.

3. Final Answer:

   The GSI partition key is not set to category. -> Option D

4. Quick Check:

   Wrong GSI key = no results [OK]

Hint: Check GSI partition key matches attribute queried [OK]

Common Mistakes:

• Querying base table instead of GSI
• Assuming sparse index can't be queried
• Ignoring GSI key setup

5. You want to quickly find all orders with a deliveryDate attribute set, but most orders don't have this attribute. How can you use a sparse index pattern to optimize queries?

hard

A. Create a GSI with deliveryDate as the partition key; only orders with this attribute appear in the index.

B. Add deliveryDate as a sort key in the main table to speed up queries.

C. Create a GSI with a constant partition key for all items and deliveryDate as sort key.

D. Scan the entire table filtering on deliveryDate attribute.

Solution

1. Step 1: Use sparse index with deliveryDate attribute

   Creating a GSI with deliveryDate as partition key includes only items having that attribute, making queries efficient.

2. Step 2: Evaluate other options

   A constant partition key approach requires adding an extra attribute to all items, incurring write costs; main table sort key doesn't filter missing attributes; full table scan is inefficient.

3. Final Answer:

   Create a GSI with deliveryDate as the partition key; only orders with this attribute appear in the index. -> Option A

4. Quick Check:

   Sparse GSI on attribute = fast filtered queries [OK]

Hint: Sparse GSI on attribute filters only items having it [OK]

Common Mistakes:

• Using scan instead of sparse index
• Indexing all items losing sparsity
• Misusing sort key for filtering