Why Many-to-many with GSI overloading in DynamoDB? - Purpose & Use Cases

The Big Idea

What if you could instantly find all connections between two groups without juggling multiple lists?

The Scenario

Imagine you have two groups of friends, and you want to keep track of which friends belong to which groups using a simple list on paper.

Every time a friend joins or leaves a group, you have to rewrite the entire list, and if a friend belongs to many groups, you must write their name multiple times.

The Problem

This manual method is slow and confusing because you have to constantly update multiple lists.

It's easy to make mistakes, like forgetting to remove a friend from one group or mixing up which friend belongs where.

Finding all groups a friend belongs to or all friends in a group becomes a big headache.

The Solution

Using many-to-many with GSI overloading in DynamoDB lets you store all relationships in one table and use a special index to quickly find all friends in a group or all groups a friend belongs to.

This way, you don't have to rewrite multiple lists or duplicate data manually.

Before vs After

✗ Before

List<Group> groups = getAllGroups();
for (Group g : groups) {
  List<Friend> friends = getFriendsInGroup(g);
  // manual search and update
}

✓ After

QueryRequest query = new QueryRequest()
  .withTableName("FriendGroups")
  .withIndexName("GSI1")
  .withKeyConditionExpression("GSI1PK = :groupId")
  .withExpressionAttributeValues(Map.of(":groupId", AttributeValue.builder().s(groupId).build()));
List<Map<String, AttributeValue>> results = dynamoDB.query(query).items();

What It Enables

This approach makes it easy and fast to manage complex many-to-many relationships without duplicating data or writing complicated code.

Real Life Example

Think of a music app where songs belong to many playlists, and playlists contain many songs.

Using many-to-many with GSI overloading, the app can quickly find all playlists a song is in or all songs in a playlist without slow searches.

Key Takeaways

Manual tracking of many-to-many relationships is slow and error-prone.

GSI overloading in DynamoDB stores all relationships in one place for easy querying.

This method simplifies data management and speeds up lookups.

Practice

1. What is the main advantage of using a Global Secondary Index (GSI) for many-to-many relationships in DynamoDB?

easy

A. It duplicates data across multiple tables for backup.

B. It allows querying related items from both sides without extra tables.

C. It automatically creates foreign key constraints.

D. It enforces strict schema validation on items.

Solution

1. Step 1: Understand GSI role in many-to-many

   A GSI lets you query items using alternate keys, enabling access from either side of a relationship.

2. Step 2: Compare with other methods

   Unlike extra tables or joins, GSI overloading stores relationships in one table, simplifying queries.

3. Final Answer:

   It allows querying related items from both sides without extra tables. -> Option B

4. Quick Check:

   GSI enables two-way queries without extra tables [OK]

Hint: GSI lets you query both sides in one table [OK]

Common Mistakes:

• Thinking GSI enforces foreign keys
• Assuming GSI duplicates data in multiple tables
• Believing GSI validates schema strictly

2. Which of the following is the correct way to define a GSI for overloading many-to-many relationships in DynamoDB?

easy

A. { "IndexName": "GSI1", "KeySchema": [{"AttributeName": "UserId", "KeyType": "HASH"}], "Projection": {"ProjectionType": "KEYS_ONLY"}}

B. { "IndexName": "GSI1", "KeySchema": [{"AttributeName": "PK", "KeyType": "HASH"}, {"AttributeName": "SK", "KeyType": "RANGE"}], "Projection": {"ProjectionType": "ALL"}}

C. { "IndexName": "GSI1", "KeySchema": [{"AttributeName": "SK", "KeyType": "HASH"}, {"AttributeName": "PK", "KeyType": "RANGE"}], "Projection": {"ProjectionType": "INCLUDE", "NonKeyAttributes": ["Name"]}}

D. { "IndexName": "GSI1", "KeySchema": [{"AttributeName": "ID", "KeyType": "HASH"}], "Projection": {"ProjectionType": "ALL"}}

Solution

1. Step 1: Identify correct GSI key schema for many-to-many

   The GSI should use SK as HASH and PK as RANGE keys to support querying from both sides.

2. Step 2: Check projection type

   Projection INCLUDE ensures required attributes like Name are available in the index for queries.

3. Final Answer:

   Definition with SK as HASH, PK as RANGE, and Projection INCLUDE -> Option C

4. Quick Check:

   GSI keys SK HASH + PK RANGE + Projection INCLUDE [OK]

Hint: GSI keys: SK as HASH, PK as RANGE, Projection INCLUDE [OK]

Common Mistakes:

• Using only HASH key without RANGE
• Swapping PK and SK roles incorrectly
• Choosing Projection KEYS_ONLY when full data needed

3. Given a DynamoDB table with items storing user-group relationships using GSI overloading, what will this query return?

Query({
  IndexName: 'GSI1',
  KeyConditionExpression: 'SK = :groupId',
  ExpressionAttributeValues: { ':groupId': 'GROUP#123' }
})

medium

A. All users belonging to group with ID 'GROUP#123'.

B. All groups that user 'GROUP#123' belongs to.

C. All items with primary key 'GROUP#123'.

D. An error because PK cannot be used in GSI queries.

Solution

1. Step 1: Understand GSI key usage in query

   The query uses GSI1 with SK = 'GROUP#123', so it fetches items where GSI partition key (SK attribute) equals this group ID.

2. Step 2: Interpret the meaning of PK in many-to-many

   SK in GSI represents the group, PK (range) the users, so we get users in that group.

3. Final Answer:

   All users belonging to group with ID 'GROUP#123'. -> Option A

4. Quick Check:

   Query by GSI SK = group ID returns users in group [OK]

Hint: GSI SK query with group ID returns users in that group [OK]

Common Mistakes:

• Confusing group ID with user ID in query
• Expecting groups for a user instead of users for a group
• Thinking PK cannot be queried in GSI

4. You wrote this query to get all groups for a user but it returns no results:

Query({
  IndexName: 'GSI1',
  KeyConditionExpression: 'SK = :userId',
  ExpressionAttributeValues: { ':userId': 'USER#456' }
})

What is the likely mistake?

medium

A. The GSI's HASH key is actually SK, not PK, for user queries.

B. The query syntax is invalid; KeyConditionExpression must use SK.

C. ExpressionAttributeValues should use ':user' instead of ':userId'.

D. You must use the main table's primary key, not GSI keys.

Solution

1. Step 1: Recall GSI key overloading pattern

   In many-to-many GSI overloading, use main table (PK=userId as HASH) for user-to-groups queries, GSI (SK=groupId as HASH) for group-to-users.

2. Step 2: Check query KeyConditionExpression

   The query uses GSI with SK = userId, but SK holds group IDs, so no results; use main table instead.

3. Final Answer:

   You must use the main table's primary key, not GSI keys. -> Option D

4. Quick Check:

   Main table PK=user for user queries [OK]

Hint: Main table for user->groups, GSI for group->users [OK]

Common Mistakes:

• Using wrong GSI key attribute in query
• Confusing main table keys with GSI keys
• Incorrect ExpressionAttributeValues naming

5. You want to store many-to-many relationships between authors and books in one DynamoDB table using GSI overloading. Which design correctly supports querying all books by an author and all authors of a book efficiently?

hard

A. Use PK = 'AUTHOR#<authorId>', SK = 'BOOK#<bookId>' for author-to-book items, and GSI with PK = SK, SK = PK to query from book side.

B. Create separate tables for authors and books, then join them in application code.

C. Use PK = 'BOOK#<bookId>', SK = 'AUTHOR#<authorId>' only, and no GSI.

D. Store all relationships as JSON arrays inside author items without GSI.

Solution

1. Step 1: Understand many-to-many with GSI overloading

   Store relationships with PK as author and SK as book to query books by author.

2. Step 2: Use GSI to invert keys for reverse query

   Define GSI with PK = SK and SK = PK to query authors by book efficiently.

3. Step 3: Evaluate other options

   Separate tables or JSON arrays do not provide efficient two-way queries without extra work.

4. Final Answer:

   Use PK = 'AUTHOR#<authorId>', SK = 'BOOK#<bookId>' and GSI with PK = SK, SK = PK. -> Option A

5. Quick Check:

   Overload GSI keys to invert PK and SK for two-way queries [OK]

Hint: Invert PK and SK in GSI for reverse many-to-many queries [OK]

Common Mistakes:

• Not using GSI for reverse lookup
• Storing relationships only one way
• Using separate tables instead of one with GSI