What is meta robots tag in html

HtmlConceptBeginner · 3 min read

Meta Robots Tag in HTML: What It Is and How It Works

The meta robots tag in HTML tells search engines how to handle a web page, such as whether to index it or follow its links. It is placed inside the <head> section and helps control a page's visibility in search results.

⚙️

How It Works

The meta robots tag acts like a set of instructions for search engines, telling them what they should do with a web page. Imagine you have a book and you want to tell a librarian whether they can list it in the library catalog or share its contents with others. The meta robots tag is like that note for search engines.

Placed inside the <head> part of your HTML, this tag uses special words called directives, such as index or noindex, to control if the page should appear in search results. It can also tell search engines whether to follow the links on the page or ignore them.

This helps website owners manage privacy, avoid duplicate content, or keep unfinished pages out of search results.

💻

Example

This example shows a meta robots tag that tells search engines not to index the page but to follow the links on it.

html

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <meta name="robots" content="noindex, follow">
  <title>Example Page</title>
</head>
<body>
  <h1>Welcome to the Example Page</h1>
  <p>This page will not appear in search results, but links here will be followed.</p>
  <a href="https://example.com">Visit Example</a>
</body>
</html>

Output

A simple webpage with a heading, paragraph, and a link. Search engines will not list this page in results but will follow the link.

🎯

When to Use

Use the meta robots tag when you want to control how search engines treat your pages. For example:

To keep private or unfinished pages out of search results by using noindex.
To prevent duplicate content issues by telling search engines not to index certain pages.
To allow search engines to follow links on a page but not show the page itself.
To manage SEO by controlling which pages appear in search engines and which do not.

It is especially useful for pages like login screens, thank-you pages, or staging sites.

✅

Key Points

The meta robots tag is placed in the HTML <head> section.
It uses directives like index, noindex, follow, and nofollow.
It helps control search engine indexing and link following behavior.
It is important for SEO and managing page visibility.

✅

Key Takeaways

The meta robots tag controls if search engines index a page and follow its links.

Place the meta robots tag inside the HTML head section with appropriate directives.

Use it to hide private or duplicate pages from search results.

It helps improve SEO by managing what search engines see and list.

Common directives include noindex, nofollow, index, and follow.