Sharding and partitioning in DBMS Theory - Mini Project: Build & Apply

Understanding Sharding and Partitioning in Databases

📖 Scenario: You are working with a large online store database that holds millions of customer orders. To improve performance and manage data efficiently, the database team wants to organize the data using sharding and partitioning techniques.

🎯 Goal: Build a simple conceptual model that shows how data can be divided using partitioning and sharding. You will create data groups, set rules for dividing data, and apply the main logic to separate data into shards and partitions.

📋 What You'll Learn

Create a data structure representing customer orders with order IDs and customer regions

Add a configuration variable to define the partitioning key (e.g., region)

Write logic to assign each order to a partition based on the region

Add a final step to assign each partition to a shard based on a shard ID

💡 Why This Matters

🌍 Real World

Sharding and partitioning help large databases handle huge amounts of data by splitting it into manageable pieces. This improves speed and reliability for online stores, social networks, and other big data systems.

💼 Career

Database administrators and backend engineers use sharding and partitioning to design scalable systems that can grow with user demand and keep data organized.

1

Create the initial data structure for orders

Create a dictionary called orders with these exact entries: 101: 'North', 102: 'South', 103: 'East', 104: 'West', and 105: 'North'. Each key is an order ID and each value is the customer region.

DBMS Theory

# Create the orders dictionary with order IDs and regions
# Your code here

Need a hint?

Use curly braces to create a dictionary with order IDs as keys and regions as values.

2

Define the partitioning key

Create a variable called partition_key and set it to the string 'region'. This will represent the attribute used to divide data into partitions.

DBMS Theory

orders = {101: 'North', 102: 'South', 103: 'East', 104: 'West', 105: 'North'}
# Define the partitioning key
# Your code here

Need a hint?

Assign the string 'region' to the variable partition_key.

3

Assign orders to partitions based on region

Create a dictionary called partitions where keys are region names and values are lists of order IDs from orders that belong to that region. Use a for loop with variables order_id and region to iterate over orders.items().

DBMS Theory

orders = {101: 'North', 102: 'South', 103: 'East', 104: 'West', 105: 'North'}
partition_key = 'region'
# Create partitions dictionary and assign orders to regions
# Your code here

Need a hint?

Use a loop to check each order's region and add the order ID to the correct list in partitions.

4

Assign each partition to a shard

Create a dictionary called shards that assigns each region partition to a shard ID. Use these exact mappings: 'North': 1, 'South': 2, 'East': 1, 'West': 2. Then create a dictionary called sharded_data where keys are shard IDs and values are lists of order IDs from all partitions assigned to that shard.

DBMS Theory

orders = {101: 'North', 102: 'South', 103: 'East', 104: 'West', 105: 'North'}
partition_key = 'region'
partitions = {}
for order_id, region in orders.items():
    if region not in partitions:
        partitions[region] = []
    partitions[region].append(order_id)
# Assign partitions to shards and group orders by shard
# Your code here

Need a hint?

Map each region to a shard ID and combine orders from partitions into the correct shard lists.