What is a noindex tag?
A noindex tag is the instruction that keeps a page out of search. It takes two forms: a robots meta tag in the page's head, <meta name="robots" content="noindex">, or an X-Robots-Tag header sent by the server, which can also be applied to non-HTML files such as PDFs. When a crawler fetches a page carrying the directive, it drops the page from its index or never adds it. The page can still be crawled, its links can still be followed (unless nofollow is added too) and it still works for anyone who visits it directly.
It is distinct from robots.txt, which stops crawling rather than indexing. A page blocked in robots.txt can still appear in results if other pages link to it; a page with noindex cannot, as long as the crawler is allowed to fetch it and read the tag.
Why it matters
Why a noindex tag matters
Because not every page should rank. Search results pages, filtered and sorted views, tag archives, thank-you pages, internal tools, staging environments, thin pages that exist for a form, duplicate print versions: each is a page a visitor might need and no searcher should land on. Leaving them indexed wastes crawl budget, dilutes the site's quality in the engine's eyes and sometimes sends a searcher to a page that makes no sense out of context.
It also matters because it is the most common way a site accidentally disappears. A noindex set on a staging site and copied to production, or a plugin setting applied globally, removes a site from search in a single deploy. Knowing where the tag lives is part of launching anything.
In practice
How a noindex tag works in practice
- 01
Decide which page types should be excluded and apply the directive at the template level so new pages inherit the rule. On WordPress and similar systems, SEO plugins expose index and noindex settings per post type, taxonomy and individual page. For files and anything served outside the CMS, set the X-Robots-Tag header on the server. Keep noindexed pages out of the XML sitemap, since listing them sends a mixed message.
- 02
Then make sure crawlers can read it. A noindexed page must not be blocked in robots.txt, or the crawler will never see the tag. Check Search Console's indexing report, which lists pages excluded by noindex, and confirm the list matches your intent. Before any launch, crawl the site and search the output for noindex to catch the one that was left behind.
Watch out
What people get wrong about noindex tags
Blocking the page in robots.txt as well, which stops the crawler reading the tag and leaves the page indexed from its links. The other mistake is using noindex on pages that have earned links; the page vanishes and the links pointing at it stop helping anyone. Consolidate with a redirect or a canonical instead.
The third is forgetting the tag is permanent until removed. A noindex applied "for now" on a launch day is still there a year later, and nobody remembers why.
