[Solved] Given the following robots.txt content, which URL will be blocked from crawling? User-agent: Googlebot Disallow: /private/ User-agent: * Disallow: /temp/ | SEO Fundamentals

SEO Fundamentals - Technical SEO Basics

Given the following robots.txt content, which URL will be blocked from crawling?

User-agent: Googlebot
Disallow: /private/

User-agent: *
Disallow: /temp/

Ahttps://example.com/private/info.html by Bingbot

Bhttps://example.com/temp/info.html by Googlebot

Chttps://example.com/public/page.html by Googlebot

Dhttps://example.com/private/data.html by Googlebot

Step-by-Step Solution

Solution:

Step 1: Analyze rules for Googlebot
Googlebot is blocked from /private/ but not from /temp/ because the specific rule for Googlebot disallows /private/ only.
Step 2: Analyze rules for other bots
All other bots (like Bingbot) are blocked from /temp/ but not /private/.
Final Answer:
https://example.com/private/data.html by Googlebot -> Option D
Quick Check:
Googlebot blocked /private/ = B [OK]

Quick Trick: Specific user-agent rules override general ones [OK]

Common Mistakes:

Master "Technical SEO Basics" in SEO Fundamentals

9 interactive learning modes - each teaches the same concept differently

More SEO Fundamentals Quizzes

Given the following robots.txt content, which URL will be blocked from crawling?