Scan pagination in DynamoDB

Introduction

Scan pagination helps you get all items from a big table in small parts. This avoids slow or heavy requests.

When you want to read all data but the table is too large to get at once.

When you want to show data page by page in an app or website.

When you want to avoid timeouts or limits on reading data.

When you want to process data in chunks to save memory.

When you want to control how much data you read each time.

Syntax

DynamoDB

Scan operation with ExclusiveStartKey and Limit parameters.

Example:

Scan(
  TableName='YourTable',
  Limit=10,
  ExclusiveStartKey={ 'PrimaryKey': { 'S': 'LastEvaluatedKeyValue' } }
)

Limit controls how many items you get per scan call.

ExclusiveStartKey tells DynamoDB where to continue scanning from.

Examples

Get first 5 items from the Products table.

DynamoDB

Scan(
  TableName='Products',
  Limit=5
)

Get next 5 items starting after ProductID 123.

DynamoDB

Scan(
  TableName='Products',
  Limit=5,
  ExclusiveStartKey={ 'ProductID': { 'S': '123' } }
)

Sample Program

This code reads all items from 'ExampleTable' in chunks of 3 using scan pagination. It keeps scanning until no more items remain.

DynamoDB

import boto3

client = boto3.client('dynamodb')

def scan_all_items(table_name):
    last_evaluated_key = None
    all_items = []

    while True:
        if last_evaluated_key:
            response = client.scan(
                TableName=table_name,
                Limit=3,
                ExclusiveStartKey=last_evaluated_key
            )
        else:
            response = client.scan(
                TableName=table_name,
                Limit=3
            )

        items = response.get('Items', [])
        all_items.extend(items)

        last_evaluated_key = response.get('LastEvaluatedKey')
        if not last_evaluated_key:
            break

    return all_items

# Example usage
items = scan_all_items('ExampleTable')
print(f"Total items fetched: {len(items)}")

Important Notes

Scan reads the whole table, so it can be slow for big tables.

Use pagination to avoid timeouts and large data loads.

Always check for LastEvaluatedKey to know if more data exists.

Summary

Scan pagination breaks big scans into small parts.

Use Limit to set chunk size and ExclusiveStartKey to continue.

Keep scanning until LastEvaluatedKey is empty.