HtmlHow-ToBeginner · 3 min read

How to Add Robots Meta Tag in HTML for SEO Control

Add the <meta name="robots" content="value"> tag inside the <head> section of your HTML. Replace value with instructions like index, noindex, follow, or nofollow to control how search engines handle your page.

📐

Syntax

The robots meta tag uses the <meta> element with name="robots" and a content attribute specifying instructions for search engines.

name="robots": Identifies the tag as a robots directive.
content: Contains one or more comma-separated values like index, noindex, follow, nofollow.

html

<meta name="robots" content="index, follow">

💻

Example

This example shows how to add a robots meta tag that tells search engines to not index the page but still follow links on it.

html

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <meta name="robots" content="noindex, follow">
  <title>Robots Meta Tag Example</title>
</head>
<body>
  <h1>Page with Robots Meta Tag</h1>
  <p>This page tells search engines not to index it but to follow links.</p>
</body>
</html>

Output

A webpage with a heading 'Page with Robots Meta Tag' and a paragraph explaining the robots tag effect.

⚠️

Common Pitfalls

Common mistakes when adding the robots meta tag include:

Placing the tag outside the <head> section, which may cause it to be ignored.
Using incorrect or misspelled values in the content attribute.
Conflicting directives like index, noindex together, which confuse search engines.
Forgetting to add nofollow if you want to prevent link crawling.

html

<!-- Wrong: placed in body -->
<body>
  <meta name="robots" content="noindex">
</body>

<!-- Right: placed in head -->
<head>
  <meta name="robots" content="noindex">
</head>

📊

Quick Reference

Here are common content values for the robots meta tag:

Value	Meaning
index	Allow search engines to index the page
noindex	Prevent the page from being indexed
follow	Allow search engines to follow links on the page
nofollow	Prevent search engines from following links
noarchive	Prevent search engines from saving a cached copy
nosnippet	Prevent search engines from showing a snippet in results

✅

Key Takeaways

Place the robots meta tag inside the section of your HTML.

Use the content attribute to control indexing and link following with values like index, noindex, follow, and nofollow.

Avoid conflicting or misspelled directives to ensure search engines understand your instructions.

The robots meta tag helps manage how search engines crawl and display your pages.

Test your pages with browser developer tools to confirm the meta tag is correctly placed.