Bird
0
0

Which directive in robots.txt is used to prevent crawlers from accessing specific pages?

easy📝 Conceptual Q2 of 15
SEO Fundamentals - Technical SEO Basics
Which directive in robots.txt is used to prevent crawlers from accessing specific pages?
AAllow
BCrawl-delay
CDisallow
DSitemap
Step-by-Step Solution
Solution:

  1. Step 1: Identify the directive that blocks URLs

    The Disallow directive tells crawlers which pages or folders they cannot access.

  2. Step 2: Differentiate from other directives

    Allow permits access, Crawl-delay sets wait time, and Sitemap points to sitemap files.

  3. Final Answer:

    Disallow -> Option C

  4. Quick Check:

    Disallow = Blocks pages [OK]
Quick Trick: Disallow blocks URLs from crawlers [OK]
Common Mistakes:
  • Using Allow to block pages
  • Confusing Crawl-delay with blocking

Want More Practice?

15+ quiz questions · All difficulty levels · Free

Free Signup - Practice All Questions
More SEO Fundamentals Quizzes
How Search Engines Work - Search intent types (informational, navigational, transactional) - Quiz 4medium How Search Engines Work - Search intent types (informational, navigational, transactional) - Quiz 7medium How Search Engines Work - Search intent types (informational, navigational, transactional) - Quiz 9hard Keyword Research - Search volume and keyword difficulty - Quiz 2easy On-Page SEO - Internal linking strategy - Quiz 3easy On-Page SEO - Header tag hierarchy (H1, H2, H3) - Quiz 9hard Technical SEO Basics - Page speed fundamentals - Quiz 9hard Technical SEO Basics - Page speed fundamentals - Quiz 1easy Technical SEO Basics - Core Web Vitals overview - Quiz 15hard Technical SEO Basics - HTTPS and security - Quiz 5medium