Scan pagination in DynamoDB - Mini Project: Build & Apply

Scan Pagination in DynamoDB

📖 Scenario: You are working with a DynamoDB table that stores customer orders. The table has many items, and you want to retrieve all orders but only a few at a time to avoid overloading your application.

🎯 Goal: Build a DynamoDB scan operation with pagination to fetch all items in small batches.

📋 What You'll Learn

Create a dictionary called orders_table to simulate the DynamoDB table with 10 items.

Create a variable called page_size and set it to 3 to limit items per scan.

Write a function called scan_with_pagination that scans the orders_table in pages of page_size.

Add a variable called last_evaluated_key to keep track of the scan position and update it after each page.

💡 Why This Matters

🌍 Real World

In real applications, DynamoDB tables can have thousands or millions of items. Scanning all at once can be slow and costly. Pagination helps fetch data in smaller, manageable chunks.

💼 Career

Understanding scan pagination is essential for backend developers and database administrators working with DynamoDB to optimize data retrieval and application performance.

1

DATA SETUP: Create the orders table data

Create a dictionary called orders_table with 10 items. Each item should have keys OrderId (from 1 to 10) and CustomerName with values 'Customer1' to 'Customer10'.

DynamoDB

# Create the orders_table dictionary with 10 items
# Your code here

Need a hint?

Use a dictionary with keys 1 to 10. Each value is another dictionary with keys 'OrderId' and 'CustomerName'.

2

CONFIGURATION: Set the page size for scan

Create a variable called page_size and set it to 3 to limit the number of items returned per scan page.

DynamoDB

orders_table = {
    1: {'OrderId': 1, 'CustomerName': 'Customer1'},
    2: {'OrderId': 2, 'CustomerName': 'Customer2'},
    3: {'OrderId': 3, 'CustomerName': 'Customer3'},
    4: {'OrderId': 4, 'CustomerName': 'Customer4'},
    5: {'OrderId': 5, 'CustomerName': 'Customer5'},
    6: {'OrderId': 6, 'CustomerName': 'Customer6'},
    7: {'OrderId': 7, 'CustomerName': 'Customer7'},
    8: {'OrderId': 8, 'CustomerName': 'Customer8'},
    9: {'OrderId': 9, 'CustomerName': 'Customer9'},
    10: {'OrderId': 10, 'CustomerName': 'Customer10'}
}

# Set the page_size variable to 3
# Your code here

Need a hint?

Just create a variable named page_size and assign it the value 3.

3

CORE LOGIC: Write the scan_with_pagination function

Write a function called scan_with_pagination that takes orders_table and page_size as parameters. Inside, create a variable last_evaluated_key set to None. Use a while loop to scan the table in pages of page_size. In each loop, select the next page of items starting after last_evaluated_key. Update last_evaluated_key to the last item's key in the current page. Stop when all items are scanned. Return a list of all scanned items.

DynamoDB

def scan_with_pagination(orders_table, page_size):
    last_evaluated_key = None
    scanned_items = []
    keys = sorted(orders_table.keys())
    while True:
        if last_evaluated_key is None:
            start_index = 0
        else:
            start_index = keys.index(last_evaluated_key) + 1
        page_keys = keys[start_index:start_index + page_size]
        if not page_keys:
            break
        for key in page_keys:
            scanned_items.append(orders_table[key])
        last_evaluated_key = page_keys[-1]
    return scanned_items
# Create last_evaluated_key variable outside the function

Need a hint?

Use a while True loop and update last_evaluated_key after each page. Use keys.index() to find the start position.

4

COMPLETION: Add the last_evaluated_key variable outside the function

Create a variable called last_evaluated_key and set it to None outside the function to simulate the initial scan state.

DynamoDB

last_evaluated_key = None

Need a hint?

Just create the variable last_evaluated_key and set it to None outside the function.