What does <meta name="robots"> do?
The <meta name="robots"> tag controls how search engines index the page and how they handle its links.
It gives search bots (Googlebot, Bingbot, and others) instructions about whether they may:
- add the page to search results,
- follow links inside it,
- save a cached copy of the page, and so on.
1. Syntax
html
<meta name="robots" content="index, follow">Here:
name="robots": indicates that these are instructions for search bots;content: sets what they are allowed or forbidden to do.
2. Main values of content
| Value | What it does | Example |
|---|---|---|
index | Allows indexing the page (include in search) | <meta name="robots" content="index"> |
noindex | Forbids indexing (do not add to search) | <meta name="robots" content="noindex"> |
follow | Allows following links on the page | <meta name="robots" content="follow"> |
nofollow | Forbids following links (does not pass weight) | <meta name="robots" content="nofollow"> |
noarchive | Forbids caching the page in the search engine | <meta name="robots" content="noarchive"> |
nosnippet | Forbids showing a description (snippet) in results | <meta name="robots" content="nosnippet"> |
noimageindex | Forbids indexing images from this page | <meta name="robots" content="noimageindex"> |
3. Commonly used combinations
Allow everything (default):
html
<meta name="robots" content="index, follow">Do not index, but allow following links:
html
<meta name="robots" content="noindex, follow">(often used for cart pages, account pages, forms, and so on)
Completely forbid indexing and links:
html
<meta name="robots" content="noindex, nofollow">Forbid cache and description:
html
<meta name="robots" content="noarchive, nosnippet">4. When to use it
- For auxiliary pages that should not appear in search (cart, profile, filters).
- For duplicate pages, to avoid SEO cannibalization.
- For admin or private sections, if there is no access to the
robots.txtfile.
5. The difference between <meta name="robots"> and robots.txt
| Feature | <meta name="robots"> | robots.txt |
|---|---|---|
| Applies to | A specific page | The whole site or section |
| Control level | HTML | Server level |
| Can control indexing | Yes | No (only blocks access) |
| Location | Inside <head> | Separate file at the site root |
Summary:
| Parameter | Purpose |
|---|---|
| Tag | <meta name="robots"> |
| Location | Inside <head> |
| Purpose | Controls indexing and link handling |
| Key values | index, noindex, follow, nofollow, noarchive, nosnippet |
| SEO importance | Lets you control which pages search engines can see |
In simple terms:
<meta name="robots"> is an instruction for search engines:
"May this page be shown in search, and may its links be followed?"
It helps manage indexing precisely, without removing the page from the site.
Short Answer
Interview readyPremium
A concise answer to help you respond confidently on this topic during an interview.