Parallel scan in DynamoDB - Step-by-Step Execution

Concept Flow - Parallel scan

Start Scan Request

↓

Divide Table into Segments

↓

Each Segment Scanned in Parallel

↓

Collect Results from All Segments

↓

Combine Results

↓

Return Full Scan Output

Parallel scan splits the table into segments and scans each segment at the same time, then combines all results.

Execution Sample

DynamoDB

Scan with Segment=0, TotalSegments=2
Scan with Segment=1, TotalSegments=2

This example scans a table in 2 parallel segments, each scanning half the table.

Execution Table

Step	Segment	Scan Range	Items Scanned	Items Returned	Action
1	0	First half of table	50	45	Scan segment 0 starts
2	1	Second half of table	50	48	Scan segment 1 starts
3	0	First half of table	50	45	Segment 0 scan completes
4	1	Second half of table	50	48	Segment 1 scan completes
5	-	-	100	93	Combine results from both segments
6	-	-	-	-	Return combined scan results

💡 All segments scanned and results combined, scan completes.

Variable Tracker

Variable	Start	After Segment 0	After Segment 1	Final
ItemsScanned	0	50	100	100
ItemsReturned	0	45	93	93
SegmentsCompleted	0	1	2	2

Key Moments - 2 Insights

Why do we need to specify Segment and TotalSegments in a parallel scan?

Each scan worker must know which part of the table to scan (Segment) and how many total parts exist (TotalSegments) to avoid overlapping scans, as shown in execution_table rows 1 and 2.

Does parallel scan return results in order?

No, each segment scans independently and results are combined after all segments finish, so order is not guaranteed. See execution_table rows 3-6.

Visual Quiz - 3 Questions

Look at the execution_table, how many items were scanned in total after both segments finished?

A 93

B 100

C 50

D 45

💡 Hint

Check the ItemsScanned column in rows 3 and 4 and the combined total in row 5.

At which step do both segments complete their scans?

A Step 2

B Step 3

C Step 4

D Step 5

💡 Hint

Look at the Action column for segment scan completions in rows 3 and 4.

If TotalSegments was increased to 4, how would the execution_table change?

A There would be 4 segment scan rows instead of 2

B ItemsScanned would double

C SegmentsCompleted would stay at 2

D Scan results would be combined earlier

💡 Hint

Parallel scan divides the table into TotalSegments parts, so more segments mean more scan rows.

Concept Snapshot

Parallel scan splits a DynamoDB table into multiple segments.
Each segment is scanned independently and simultaneously.
Specify Segment and TotalSegments to control scanning.
Results from all segments are combined after all finish.
Improves scan speed by using parallel workers.

Full Transcript

Parallel scan in DynamoDB divides the table into segments. Each segment is scanned in parallel by separate workers. You specify which segment to scan and how many total segments exist. Each segment scans only its part of the table. After all segments finish scanning, their results are combined and returned as one complete scan output. This method speeds up scanning large tables by using multiple workers at once. The execution table shows two segments scanning halves of the table, then combining results. Variables track items scanned and returned per segment and overall. Key points include specifying segments correctly and understanding that results are combined after all segments finish. The visual quiz tests understanding of total items scanned, completion steps, and effect of changing total segments.

Practice

1. What is the main purpose of using Parallel Scan in DynamoDB?

easy

A. To update multiple items in a table simultaneously

B. To backup data in parallel to another region

C. To create multiple tables for faster access

D. To split a table scan into multiple parts that run at the same time

Solution

1. Step 1: Understand the concept of Parallel Scan

   Parallel Scan divides the scan operation into segments that run concurrently to speed up reading the entire table.

2. Step 2: Identify the main purpose

   The main goal is to speed up scanning by running parts in parallel, not updating or backing up data.

3. Final Answer:

   To split a table scan into multiple parts that run at the same time -> Option D

4. Quick Check:

   Parallel Scan = split scan parts [OK]

Hint: Parallel scan means splitting scan into parts running together [OK]

Common Mistakes:

• Confusing scan with update operations
• Thinking parallel scan creates multiple tables
• Assuming parallel scan is for backup

2. Which two parameters are required to perform a parallel scan in DynamoDB?

easy

A. Segment and TotalSegments

B. PartitionKey and SortKey

C. Limit and FilterExpression

D. IndexName and ProjectionExpression

Solution

1. Step 1: Recall parameters for parallel scan

   Parallel scan requires specifying which segment to scan and how many total segments exist.

2. Step 2: Match parameters to options

   Segment and TotalSegments control the parts of the scan; other options relate to different operations.

3. Final Answer:

   Segment and TotalSegments -> Option A

4. Quick Check:

   Parallel scan params = Segment + TotalSegments [OK]

Hint: Remember: Segment and TotalSegments split the scan [OK]

Common Mistakes:

• Using PartitionKey and SortKey which are for queries
• Confusing Limit with segment control
• Mixing index parameters with scan parameters

3. Given a table with 1000 items and a parallel scan with TotalSegments=5, what does setting Segment=2 do?

medium

A. Scans the first part of the table items

B. Scans all items in the table

C. Scans the third part of the table items

D. Scans only 2 items from the table

Solution

1. Step 1: Understand segment numbering

   Segments are zero-based, so Segment=2 means the third segment out of 5.

2. Step 2: Identify what scanning Segment=2 means

   It scans only the third part of the table, not the whole table or just two items.

3. Final Answer:

   Scans the third part of the table items -> Option C

4. Quick Check:

   Segment=2 means third part scanned [OK]

Hint: Segments start at 0; Segment=2 is third part [OK]

Common Mistakes:

• Thinking Segment=2 scans whole table
• Assuming segments start at 1
• Confusing segment number with item count

4. You wrote this code for parallel scan but it returns incomplete data:

for segment in range(3):
    response = table.scan(Segment=segment, TotalSegments=3)
    print(response['Items'])

What is the likely problem?

medium

A. TotalSegments should be 1 for parallel scan

B. You must combine results from all segments to get full data

C. Segment numbers should start from 1, not 0

D. You cannot use scan with Segment parameter

Solution

1. Step 1: Analyze the code behavior

   The code scans each segment separately but prints results immediately without combining.

2. Step 2: Understand why data is incomplete

   Each segment returns part of data; to get full data, results must be combined from all segments.

3. Final Answer:

   You must combine results from all segments to get full data -> Option B

4. Quick Check:

   Combine all segment results for full scan [OK]

Hint: Combine all segment results to get full table data [OK]

Common Mistakes:

• Starting segments at 1 instead of 0
• Setting TotalSegments to 1 disables parallelism
• Believing scan can't use Segment parameter

5. You want to speed up scanning a large DynamoDB table with 10 million items. You set TotalSegments=10 and run scans in parallel. Which approach ensures you get all items without missing or duplicating data?

hard

A. Run scans for all segments (0 to 9) and combine all results

B. Run scan only on Segment=0 with TotalSegments=10

C. Run scans on segments 1 to 10 (1-based) and combine results

D. Run a single scan without segments to avoid duplicates

Solution

1. Step 1: Understand segment indexing and coverage

   Segments are zero-based, so with TotalSegments=10, segments are 0 through 9.

2. Step 2: Ensure full coverage without overlap

   Running all segments from 0 to 9 and combining results covers entire table exactly once.

3. Step 3: Identify incorrect options

   Running only Segment=0 misses data; segments 1 to 10 are off by one; single scan is slower and not parallel.

4. Final Answer:

   Run scans for all segments (0 to 9) and combine all results -> Option A

5. Quick Check:

   All segments 0-9 combined = full scan [OK]

Hint: Run all zero-based segments and combine results [OK]

Common Mistakes:

• Using 1-based segment numbers instead of 0-based
• Running only one segment expecting full data
• Avoiding parallel scan due to fear of duplicates