Crawl budget optimization in SEO Fundamentals - Mini Project: Build & Apply

Crawl Budget Optimization

📖 Scenario: You manage a website and want to make sure search engines crawl your important pages efficiently. You will organize your website's URLs and set rules to help optimize the crawl budget.

🎯 Goal: Build a simple plan to organize URLs by priority and set rules to guide search engine crawlers on which pages to crawl or avoid.

📋 What You'll Learn

Create a list of website URLs with their priority levels

Add a threshold variable to filter high priority URLs

Use a loop to select URLs above the priority threshold

Add a final rule to block crawling of low priority URLs

💡 Why This Matters

🌍 Real World

Website owners and SEO specialists use crawl budget optimization to help search engines focus on important pages, improving site indexing and search rankings.

💼 Career

Understanding crawl budget helps digital marketers and SEO professionals manage large websites efficiently and improve organic traffic.

1

Create a list of URLs with priority

Create a list called urls with these exact entries as tuples: ("/home", 10), ("/about", 5), ("/blog/post1", 8), ("/blog/post2", 3), ("/contact", 6).

SEO Fundamentals

# Create the list called urls with the given tuples
# Your code here

Need a hint?

Use a list of tuples where each tuple has the URL string and its priority number.

2

Set a priority threshold

Create a variable called priority_threshold and set it to 6 to filter URLs with priority above this value.

SEO Fundamentals

urls = [
    ("/home", 10),
    ("/about", 5),
    ("/blog/post1", 8),
    ("/blog/post2", 3),
    ("/contact", 6)
]
# Create priority_threshold variable and set it to 6
# Your code here

Need a hint?

This variable helps decide which URLs are important enough to crawl.

3

Select URLs above the priority threshold

Create a list called high_priority_urls using a for loop with variables url and priority to iterate over urls. Add only URLs with priority greater than priority_threshold.

SEO Fundamentals

urls = [
    ("/home", 10),
    ("/about", 5),
    ("/blog/post1", 8),
    ("/blog/post2", 3),
    ("/contact", 6)
]
priority_threshold = 6
# Create high_priority_urls list with URLs having priority > priority_threshold
# Your code here

Need a hint?

Use a loop to check each URL's priority and add only the important ones to the new list.

4

Add rule to block low priority URLs

Create a list called blocked_urls with URLs from urls where priority is less than or equal to priority_threshold using a for loop with variables url and priority.

SEO Fundamentals

urls = [
    ("/home", 10),
    ("/about", 5),
    ("/blog/post1", 8),
    ("/blog/post2", 3),
    ("/contact", 6)
]
priority_threshold = 6

high_priority_urls = []
for url, priority in urls:
    if priority > priority_threshold:
        high_priority_urls.append(url)

# Create blocked_urls list with URLs having priority <= priority_threshold
# Your code here

Need a hint?

This list will help you block less important pages from being crawled.