What is an XML sitemap?
An XML sitemap is a file, usually at /sitemap.xml, written in a standard format that search engines read. Each entry is a URL, optionally with the date it was last modified, and nothing else of importance; the change frequency and priority fields the format allows are ignored by Google. The file is a list, not a map in any visual sense, and it exists for one reason: to tell crawlers which pages exist, so that discovery does not depend on following links.
Large sites use a sitemap index, a file that lists other sitemap files, because a single sitemap is capped at fifty thousand URLs and fifty megabytes. Specialized sitemaps extend the format for images, video and news with the extra fields those formats need, and a separate sitemap can be kept for each content type so that reports in Search Console are easier to read.
Why it matters
Why an XML sitemap matters
Crawlers find pages by following links, and links miss things: new pages that nothing points to yet, deep pages with few internal links, pages on large sites that are simply far down the queue. The sitemap makes discovery complete. The last-modified date adds efficiency: a crawler can see which pages changed since its last visit and spend its budget there rather than refetching the unchanged. On a site with tens of thousands of pages that difference decides whether updates are seen in days or months.
Search Console reports indexing per sitemap, so the file is also the measuring stick: how many URLs were submitted and how many were indexed, and for those that were not, why.
In practice
How an XML sitemap works in practice
- 01
The sitemap is generated automatically by the content management system or an SEO plugin and updated as pages are published, edited and removed. It is referenced in robots.txt with a Sitemap: line and submitted in Google Search Console and Bing Webmaster Tools. For large sites it is split by type or section, each under a sitemap index, with video and image sitemaps where those assets matter.
- 02
The rules keep it honest. It lists only canonical, indexable URLs that return a 200 status: no redirects, no errors, no noindexed pages, no parameter variants, no pages behind a login. Last-modified dates reflect real changes, not the time the file was generated. The sitemap is checked after every migration or template change, and the submitted-versus-indexed gap in Search Console is read monthly, because a widening gap is an early warning of a quality or technical problem.
At RIOT
XML sitemaps at RIOT
AltSounds, our music video channel, has far more URLs than a single sitemap file is allowed to hold, so it runs a sitemap index that splits the catalog across many files. For a catalog that size the sitemap is not a courtesy; it is how search engines keep up with what has been added.
Watch out
What people get wrong about XML sitemaps
Including everything. A sitemap stuffed with redirects, noindexed pages and duplicates trains the crawler to distrust it, and the useful entries lose their weight. The other mistake is faking the dates: setting last-modified to today on every URL every time the file is built, which makes the field meaningless. And many sites never submit it or reference it in robots.txt, so the file exists and nobody has told the engines.
