What is a sitemap?
A sitemap is an index of a website's pages. It comes in two forms that share a name and little else. An XML sitemap is a machine-readable file, usually at /sitemap.xml, that lists every URL a site wants search engines to know about, with optional notes on when each was last changed. An HTML sitemap is an ordinary page for people, a long list of links organized by section, the kind that used to live in every footer and still earns its place on large sites.
Both answer the same question: what is here? A search engine crawler can find most pages by following links, but it can miss pages that are new, deep or poorly linked. A sitemap hands it the full list so nothing depends on luck.
Why it matters
Why a sitemap matters
Search engines cannot rank what they have not found. On a small site with good internal links a sitemap changes little, because the crawler finds everything anyway. On a large site, a new site or a site with pages that are hard to reach, the sitemap is often the only reason certain pages get crawled at all. It also carries timing: the last-modified date tells a crawler which pages have changed since its last visit, so it spends its limited attention on them rather than re-reading pages that are the same as last month.
A sitemap matters for diagnosis too. Google Search Console reports how many of the URLs in a sitemap are indexed, and the gap between submitted and indexed is one of the first numbers to check when a site is underperforming.
In practice
How a sitemap works in practice
- 01
Most content management systems generate the XML sitemap automatically and keep it current as pages are published and deleted. The file is then referenced in robots.txt and submitted in Google Search Console and Bing Webmaster Tools, after which the engines fetch it on their own schedule. Large sites split the list into several files under a sitemap index, because a single file is capped at fifty thousand URLs. Specialized sitemaps exist for images, video and news, each carrying extra fields the general file does not.
- 02
The maintenance is in the hygiene. A sitemap should only contain pages that are meant to be indexed: no redirects, no 404s, no pages set to noindex, no duplicate URL variants. Every entry that fails those rules teaches the crawler that the file is unreliable, and it starts trusting the sitemap less. The HTML version needs the same care, kept in sync with the site and organized so a person can actually use it.
At RIOT
Sitemaps at RIOT
This glossary is the reason riot.nyc needs one. More than a thousand terms, each on its own page, linked from category listings and from one another, is more than a crawler will reliably discover on its own. The sitemap lists every term, so a new entry is on the list the day it goes live rather than waiting to be stumbled upon.
Watch out
What people get wrong about sitemaps
Thinking a sitemap makes pages rank. It makes them discoverable; ranking is decided by everything else. The reverse mistake is neglecting it: a sitemap full of dead URLs, redirected pages and noindexed junk is worse than none, because it wastes crawl budget and undermines trust. And many sites still confuse the two kinds, submitting an HTML page to Search Console or burying the XML file where no robots.txt points to it.
