Creative agency glossarySEO and Analytics

Indexability

Indexability is whether a search engine is able and permitted to add a page to its index, which is the precondition for that page appearing in any search result.

What is indexability?

Indexability is a page's eligibility for the index. A page is indexable when a search engine can reach it, can read its content, and has not been told to leave it out. That makes it a chain of conditions rather than a single switch: the URL has to return a success code, it cannot be blocked from crawling in robots.txt, it cannot carry a noindex directive in its meta tags or HTTP headers, its canonical tag has to point at itself rather than at another page, and the content has to be visible to the crawler once the page is rendered.

It is often paired with crawlability, and the two are different. Crawlability is whether the crawler can get to a page; indexability is whether, having got there, it is allowed to keep it. A page can be crawlable and not indexable, and that combination is the most common way good content disappears.

Why it matters

Why indexability matters

Because an unindexable page has no search presence at all. It does not rank low; it does not rank. Every hour spent on its content, its design and its links is wasted until the directive that keeps it out is found. And those directives are easy to set by accident: a noindex left on from the staging site, a plugin setting applied to a whole post type, a canonical tag pointing every page at the homepage, a redirect that loops. Launches are lost this way with some regularity.

It matters in the other direction too. Deliberately making a page unindexable is how a site keeps checkout steps, thank-you pages, internal search results and duplicate variants out of the results, so indexability is a control to be used, not only a fault to be avoided.

In practice

How indexability works in practice

  1. 01

    The check is a short list, and it is run on every important page before launch and after any template change. Does the URL return 200? Is it allowed in robots.txt? Does the page's meta robots tag or X-Robots-Tag header say index, or say nothing? Does the canonical point to this URL? Does the content appear in the rendered HTML rather than only after a script runs? Is the page linked from somewhere the crawler already knows, and is it in the sitemap? The URL Inspection tool in Search Console answers all of these for a single page in one screen.

  2. 02

    At scale the same questions are asked by a crawler such as Screaming Frog or Sitebulb across the whole site, which flags every page that is non-indexable and why. The report is read against intent: pages that should be out and are out are fine, pages that should be in and are out are the bugs.

Watch out

What people get wrong about indexability

Assuming the sitemap decides it. Listing a page in the sitemap is a request; a noindex tag on the page is an instruction, and the instruction wins. The other mistake is blocking a page in robots.txt to keep it out of the index, which prevents Google reading the noindex tag and can leave the page indexed as a bare URL. The third is forgetting rendered content: a page whose text loads only after a client-side script may be indexable in theory and empty in practice, because what the crawler stores is what it managed to render.

Questions

People also ask

What is the difference between crawlability and indexability?

Crawlability is whether a search engine can reach and read a page. Indexability is whether it is allowed to store that page and show it in results. A page needs both; it can be crawlable but blocked from the index by a noindex tag or a canonical pointing elsewhere.

How do I check if a page is indexable?

Use the URL Inspection tool in Google Search Console, which reports crawl status, indexing status, the canonical Google chose and any blocking directives. For a whole site, run a crawler that lists every non-indexable page with the reason.

Why would I make a page non-indexable on purpose?

To keep pages that exist for function rather than discovery out of search: checkout and account pages, internal search results, thank-you pages, filtered duplicates and staging copies. Keeping those out makes the pages you do want found look stronger.

Next to it in the glossary

  1. Inbound LinksAn inbound link is a link from another website to yours, and the number and quality of them is one of the strongest signals search engines use to judge a site's authority.
  2. IndexThe index is the database in which a search engine stores the pages it has crawled and understood, and only pages in the index can appear in search results.
  3. Index BloatIndex bloat is what happens when a search engine indexes far more pages from a site than have any value, so the pages that matter are diluted by pages that do not.
  4. Informational QueryAn informational query is a search made to learn something rather than to buy something or reach a particular site, and it is the largest category of searches there is.
  5. Interaction to Next Paint (INP)Interaction to Next Paint (INP) is the Core Web Vitals metric that measures how quickly a page responds visually after a user taps, clicks or types.
  6. Internal LinksAn internal link is a link from one page on a website to another page on the same site, and together they form the structure that readers and search engines use to move through it.

Every SEO and Analytics term →

In the work

Where it shows up

All our creations →

From the journal

Read around it

The journal →

Talking the talk is one thing

We make the work.

RIOT is a creative agency and production studio in New York. Strategy, brand, film, photography, design and the campaigns that carry them, made by one team. A few lines is enough to start: we read every brief the same day.

    Footage