You want to allow Googlebot to crawl everything except the /private/ folder, but block all other bots from the entire site. Which robots.txt configuration achieves this?

hard📝 Application Q15 of 15

SEO Fundamentals - Technical SEO Basics

You want to allow Googlebot to crawl everything except the /private/ folder, but block all other bots from the entire site. Which robots.txt configuration achieves this?

AUser-agent: Googlebot Allow: / User-agent: * Disallow: /private/

BUser-agent: * Disallow: / User-agent: Googlebot Allow: /private/

CUser-agent: Googlebot Disallow: /private/ User-agent: * Disallow: /

DUser-agent: * Disallow: /private/ User-agent: Googlebot Disallow: /

Step-by-Step Solution

Solution:

Step 1: Understand Googlebot's rule
Googlebot should be allowed everywhere except /private/, so Disallow: /private/ applies to Googlebot.
Step 2: Understand other bots' rule
All other bots (*) should be blocked from the entire site, so Disallow: / applies to them.
Step 3: Check options
User-agent: Googlebot Disallow: /private/ User-agent: * Disallow: / matches these rules exactly. Other options either allow or block incorrectly.
Final Answer:
User-agent: Googlebot Disallow: /private/ User-agent: * Disallow: / -> Option C
Quick Check:
Googlebot partial block, others full block = C [OK]

Quick Trick: Use specific user-agent rules before general ones [OK]

Common Mistakes:

Reversing Allow and Disallow for Googlebot
Blocking Googlebot fully by mistake
Using Allow incorrectly for blocking

Master "Technical SEO Basics" in SEO Fundamentals

9 interactive learning modes - each teaches the same concept differently

Learn Why Deep Visual Try Challenge Project Recall Time

Want More Practice?

15+ quiz questions · All difficulty levels · Free

Free Signup - Practice All Questions

More SEO Fundamentals Quizzes

You want to allow Googlebot to crawl everything except the /private/ folder, but block all other bots from the entire site. Which robots.txt configuration achieves this?

Step 1: Understand Googlebot's rule

Step 2: Understand other bots' rule

Step 3: Check options

Final Answer:

Quick Check:

Want More Practice?