Shard sizing strategy in Elasticsearch - Mini Project: Build & Apply

Shard sizing strategy

📖 Scenario: You are managing an Elasticsearch cluster for a growing online store. You want to organize your data into shards efficiently to keep search fast and storage balanced.

🎯 Goal: Build a simple shard sizing strategy by creating a dictionary of index names with their document counts, setting a maximum shard size, calculating the number of shards needed for each index, and printing the results.

📋 What You'll Learn

Create a dictionary called index_docs with exact entries for three indexes and their document counts

Create a variable called max_shard_size and set it to 1000000

Create a new dictionary called shard_counts using dictionary comprehension that calculates the number of shards needed for each index by dividing document count by max shard size and rounding up

Print the shard_counts dictionary

💡 Why This Matters

🌍 Real World

Shard sizing helps keep Elasticsearch fast and balanced by splitting data into manageable pieces.

💼 Career

Understanding shard sizing is important for roles managing search infrastructure and large data systems.

1

Create the index document counts

Create a dictionary called index_docs with these exact entries: 'products': 2500000, 'customers': 1200000, 'orders': 800000

Elasticsearch

# Create the index_docs dictionary with document counts
# Your code here

Need a hint?

Use curly braces to create a dictionary with keys and values separated by colons.

2

Set the maximum shard size

Create a variable called max_shard_size and set it to 1000000

Elasticsearch

index_docs = {'products': 2500000, 'customers': 1200000, 'orders': 800000}
# Set max_shard_size to 1000000
# Your code here

Need a hint?

Just assign the number 1000000 to the variable max_shard_size.

3

Calculate shard counts using dictionary comprehension

Create a dictionary called shard_counts using dictionary comprehension that calculates the number of shards needed for each index. Use index_docs.items() to get index and document count, divide document count by max_shard_size, and round up using int((doc_count + max_shard_size - 1) / max_shard_size).

Elasticsearch

index_docs = {'products': 2500000, 'customers': 1200000, 'orders': 800000}
max_shard_size = 1000000
# Create shard_counts dictionary with shard numbers
# Your code here

Need a hint?

Use dictionary comprehension with a for loop over index_docs.items() and calculate shards by rounding up the division.

4

Print the shard counts

Write print(shard_counts) to display the dictionary with the number of shards for each index.

Elasticsearch

index_docs = {'products': 2500000, 'customers': 1200000, 'orders': 800000}
max_shard_size = 1000000
shard_counts = {index: int((doc_count + max_shard_size - 1) / max_shard_size) for index, doc_count in index_docs.items()}
# Print the shard_counts dictionary
# Your code here

Need a hint?

Use the print function to show the shard_counts dictionary.