Last updated
What Is the Robots Meta Tag?
The robots meta tag is an HTML element that gives instructions to web crawlers about
how to index a specific page. Unlike robots.txt which controls access to
entire URL paths, the robots meta tag applies only to the page it appears on and
controls indexing and link-following behavior.
Robots Meta Tag Directives
| Directive | Effect |
|---|---|
index | Allow the page to be indexed (default) |
noindex | Prevent the page from appearing in search results |
follow | Follow links on the page (default) |
nofollow | Do not follow links on the page |
noarchive | Do not show a cached version in search results |
nosnippet | Do not show a text snippet in search results |
noimageindex | Do not index images on the page |
none | Equivalent to noindex, nofollow |
Usage Examples
HTML
<!-- Allow indexing (default, usually omitted) -->
<meta name="robots" content="index, follow">
<!-- Block indexing entirely -->
<meta name="robots" content="noindex, nofollow">
<!-- Index but don't follow links -->
<meta name="robots" content="index, nofollow">
<!-- Googlebot-specific directive -->
<meta name="googlebot" content="noindex">
<!-- No snippet, no cache -->
<meta name="robots" content="nosnippet, noarchive">
ℹ️
The robots meta tag only works if the page is accessible to crawlers. If a page is blocked by robots.txt, the meta tag is never read. To reliably prevent indexing, use noindex in the meta tag AND allow crawlers in robots.txt.