What is robots metadata?

Next.jsMiddle~1 min read

Robots metadata is metadata that tells search bots what they can and cannot do with a specific page: index it, follow its links, show snippets, and so on.

In other words:

robots metadata is a set of behavior rules for search bots.

Why robots metadata is needed

It lets you control how a page participates in search:

whether the page ends up in the index
whether the links on the page get counted
whether previews, descriptions, and cache can be shown
whether service or temporary pages need to be hidden

What robots metadata looks like in HTML

The classic form:

html

<meta name="robots" content="noindex, nofollow" />

Search bots read this tag while crawling the page.

Robots metadata in Next.js

In Next.js, robots is set through the Metadata API:

export const metadata = {
  robots: {
    index: false,
    follow: false
  }
}

Next.js itself turns this into a correct <meta name="robots">.

Main directives

The most commonly used:

index / noindex - whether the page can be added to search
follow / nofollow - whether its links can be followed
noimageindex - don't index images
nosnippet - don't show a snippet
noarchive - don't store a cached copy of the page

You can combine several rules.

Usage examples

Block a page from indexing

export const metadata = {
  robots: {
    index: false
  }
}

Allow indexing but block links

export const metadata = {
  robots: {
    index: true,
    follow: false
  }
}

Service pages

Fits for:

/login
/admin
/profile
error pages
temporary pages

Robots metadata ≠ robots.txt

Important not to confuse the two:

robots metadata
- works at the level of a specific page
- is read after the HTML loads
robots.txt
- works at the level of the whole site
- controls crawling, not indexing

They solve different tasks and are often used together.

Inheritance

Like other metadata:

robots can be set in a layout
pages can override the rules
the closest rule in the hierarchy applies

Summary

Robots metadata is:

instructions for search bots
control over page indexing
part of an SEO strategy
a convenient setting through the Metadata API

Short Answer

Interview ready

Premium

A concise answer to help you respond confidently on this topic during an interview.