Tables, items, and attributes in AWS - Deep Dive

Overview - Tables, items, and attributes

What is it?

In AWS DynamoDB, data is stored in tables. Each table holds multiple items, which are like rows in a spreadsheet. Each item is made up of attributes, which are the individual pieces of data or columns. This structure helps organize and access data efficiently in a flexible way.

Why it matters

Without tables, items, and attributes, storing and retrieving data in a fast, scalable way would be very hard. These concepts let you organize data so you can quickly find what you need, even when your data grows very large. Without them, apps would be slow or unreliable when handling lots of users or data.

Where it fits

You should first understand basic database ideas like tables and records. After this, you can learn about querying DynamoDB, indexing, and data modeling to use these building blocks effectively.

Mental Model

Core Idea

A DynamoDB table holds many items, and each item is a collection of attributes that describe it.

Think of it like...

Think of a table as a filing cabinet, each item as a folder inside it, and attributes as the papers inside the folder with details about that item.

┌─────────────┐
│   Table     │
│ ┌─────────┐ │
│ │  Item   │ │
│ │ ┌─────┐ │ │
│ │ │Attr │ │ │
│ │ │ibute│ │ │
│ │ └─────┘ │ │
│ └─────────┘ │
└─────────────┘

Build-Up - 7 Steps

1

Foundation

Understanding DynamoDB Tables

Concept: Tables are containers for data in DynamoDB, similar to tables in other databases.

A DynamoDB table is where you store your data. It has a name and a primary key that uniquely identifies each item. You create a table before adding any data. Think of it as an empty box ready to hold many items.

Result

You have a named container ready to hold data items, each uniquely identified.

Knowing that tables are the top-level container helps you organize your data storage and plan how to access it.

2

Foundation

What Are Items in DynamoDB?

Concept: Items are individual records stored inside a table, each identified by a unique key.

An item is like a row in a spreadsheet. It represents one entity or record. Each item must have a primary key attribute that makes it unique in the table. Items can have different sets of attributes, meaning they don't all need the same columns.

Result

You can store multiple unique records inside a table, each with its own data.

Understanding items as flexible records lets you design your data to fit your app's needs without rigid schemas.

3

Intermediate

Attributes: The Data Pieces

Concept: Attributes are the individual data fields that make up an item.

Attributes are like columns in a spreadsheet but can vary between items. They can be simple types like strings or numbers, or complex types like lists and maps. Attributes hold the actual data values for each item.

Result

Each item contains meaningful data organized into attributes, which can be simple or complex.

Knowing attributes can be flexible and varied allows you to store rich, nested data without strict schemas.

4

Intermediate

Primary Keys and Uniqueness

🤔Before reading on: do you think a table can have two items with the same primary key? Commit to yes or no.

Concept: Primary keys uniquely identify each item in a table and prevent duplicates.

Every item must have a primary key. It can be a simple key (one attribute) or a composite key (two attributes: partition key and sort key). This key ensures each item is unique and helps DynamoDB find items quickly.

Result

You can uniquely identify and retrieve any item in the table using its primary key.

Understanding primary keys is crucial because they control data uniqueness and access speed.

5

Intermediate

Flexible Schema with Attributes

🤔Before reading on: do you think all items in a DynamoDB table must have the same attributes? Commit to yes or no.

Concept: DynamoDB tables do not require all items to have the same attributes, allowing flexible data models.

Unlike traditional databases, DynamoDB lets each item have different attributes. This means you can add new data fields to some items without changing the whole table. This flexibility supports evolving applications.

Result

You can store diverse data types and structures in the same table without schema changes.

Knowing DynamoDB's flexible schema helps you design adaptable data models that evolve with your app.

6

Advanced

Handling Large Items and Attribute Limits

🤔Before reading on: do you think DynamoDB allows items of unlimited size? Commit to yes or no.

Concept: DynamoDB has size limits on items and attributes that affect how you design your data.

Each item can be up to 400 KB in size. Attributes can be large but must fit within this limit. If your data is bigger, you need strategies like splitting data across items or storing large files elsewhere (e.g., S3).

Result

You design data storage that respects size limits to avoid errors and performance issues.

Understanding size limits prevents common mistakes that cause failed writes or slow queries.

7

Expert

Sparse Indexes and Attribute Absence

🤔Before reading on: do you think missing attributes in items cause errors in DynamoDB queries? Commit to yes or no.

Concept: DynamoDB allows items to omit attributes, enabling sparse indexes and efficient queries.

Because items can have different attributes, indexes can be created on attributes that only some items have. These are called sparse indexes and help optimize queries by indexing only relevant items. Missing attributes do not cause errors but affect query results.

Result

You can build efficient queries and indexes that target subsets of data without extra storage cost.

Knowing how sparse indexes work unlocks powerful optimization techniques for large datasets.

Under the Hood

DynamoDB stores tables as distributed partitions across many servers. Each item is stored as a key-value pair where the key is the primary key, and the value is a JSON-like document of attributes. The system uses the primary key to quickly locate the partition holding the item. Attributes are stored flexibly, allowing different items to have different sets of attributes without a fixed schema.

Why designed this way?

DynamoDB was designed for high scalability and flexibility. Fixed schemas slow down scaling and make evolving data models hard. Using flexible attributes and primary keys allows DynamoDB to distribute data efficiently and adapt to changing application needs. Alternatives like rigid relational schemas were rejected to achieve speed and scale.

┌───────────────┐
│   DynamoDB    │
│   Table      │
│ ┌───────────┐ │
│ │ Partition │ │
│ │  Server   │ │
│ └───────────┘ │
│      │        │
│ ┌───────────┐ │
│ │   Item    │ │
│ │(Primary   │ │
│ │  Key +    │ │
│ │ Attributes)│ │
│ └───────────┘ │
└───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Can two items in the same DynamoDB table share the same primary key? Commit to yes or no.

Common Belief: Yes, items can share the same primary key as long as other attributes differ.

Reality: No, the primary key must be unique for each item in the table to avoid overwriting data.

Why it matters: If you assume duplicates are allowed, you risk data loss because new items overwrite existing ones with the same key.

Quick: Do all items in a DynamoDB table need to have the same attributes? Commit to yes or no.

Common Belief: Yes, all items must have the same attributes like columns in a traditional database.

Reality: No, DynamoDB allows items to have different attributes, enabling flexible and evolving data models.

Why it matters: Assuming fixed schemas limits your ability to adapt data models and can cause unnecessary complexity.

Quick: Does DynamoDB automatically index all attributes in a table? Commit to yes or no.

Common Belief: Yes, all attributes are indexed by default for fast queries.

Reality: No, only the primary key and explicitly created secondary indexes are indexed. Other attributes are not indexed.

Why it matters: Expecting automatic indexing can lead to slow queries and poor performance if indexes are not planned.

Quick: Can DynamoDB items be larger than 400 KB? Commit to yes or no.

Common Belief: Yes, DynamoDB can store items of any size.

Reality: No, each item has a maximum size of 400 KB. Larger data must be split or stored elsewhere.

Why it matters: Ignoring size limits causes write failures and application errors.

Expert Zone

1

Sparse indexes only include items that have the indexed attribute, saving storage and improving query speed.

2

Composite primary keys enable sorting and range queries within partitions, which is key for complex access patterns.

3

Attributes can be nested documents (maps) or lists, allowing rich data structures without multiple tables.

When NOT to use

Avoid using DynamoDB tables with highly relational data requiring complex joins or transactions; use relational databases like Amazon RDS instead. Also, if your data items exceed 400 KB regularly, consider storing large objects in Amazon S3 and referencing them.

Production Patterns

In production, tables are designed with careful primary key selection to distribute load evenly. Items use sparse attributes to optimize indexes. Data modeling often involves denormalization and composite keys to support fast queries without joins.

Connections

Relational Database Tables and Rows

Similar structure but DynamoDB tables are schema-less and more flexible.

Understanding relational tables helps grasp DynamoDB tables, but knowing DynamoDB's flexibility shows how NoSQL differs.

JSON Documents

Attributes in DynamoDB items can be nested like JSON objects.

Knowing JSON structure helps understand how DynamoDB stores complex nested data within attributes.

Filing Systems

DynamoDB tables and items function like filing cabinets and folders organizing information.

This connection helps appreciate how data organization impacts retrieval speed and flexibility.

Common Pitfalls

#1 Using a primary key that causes all items to be stored in one partition.

Wrong approach:

PrimaryKey: { "userId": "sameValue" }

Correct approach:

PrimaryKey: { "userId": "uniqueValue" }

Root cause:

Misunderstanding that partition keys must distribute data evenly to avoid hot partitions.

#2 Assuming all items must have the same attributes and forcing a fixed schema.

Wrong approach:

Item1: { "name": "Alice", "age": 30 }
Item2: { "name": "Bob", "age": 25, "address": "123 St" } // Error: schema mismatch

Correct approach:

Item1: { "name": "Alice", "age": 30 }
Item2: { "name": "Bob", "age": 25, "address": "123 St" } // Allowed in DynamoDB

Root cause:

Applying relational database schema rules to DynamoDB leads to unnecessary constraints.

#3 Not creating secondary indexes for attributes used in queries.

Wrong approach:

Querying on attribute 'email' without an index, causing full table scan.

Correct approach:

Create a Global Secondary Index on 'email' attribute before querying.

Root cause:

Assuming DynamoDB automatically indexes all attributes leads to inefficient queries.

Key Takeaways

DynamoDB organizes data into tables, items, and attributes, where tables hold items and items hold attributes.

Each item must have a unique primary key to identify it and enable fast access.

Attributes are flexible and can vary between items, allowing schema-less data models.

Understanding primary keys and attribute flexibility is essential for designing scalable and efficient DynamoDB tables.

Knowing DynamoDB's limits and indexing behavior prevents common mistakes and improves application performance.