An XML sitemap is a machine-readable list of the pages on your site. It is the table of contents your website hands to search engines and AI crawlers. They read it to find your pages fast. Without one, they have to hunt for your pages by following links. Deep pages can get missed entirely.

This guide explains what an XML sitemap is in plain terms. We'll cover the format rules, the hygiene rules, and the errors that show up in audits. You'll also learn why honest "last modified" dates matter more than the tags everyone obsesses over.

What is an XML sitemap?

An XML sitemap is a special file that lists every page on your website. Search engines and AI crawlers read it to discover your pages. XML just means a strict, machine-readable text format. The file usually lives at https://yoursite.com/sitemap.xml.

Think of it as a library catalog. A crawler can wander the shelves and stumble on books. Or it can read the catalog and know exactly what's there. The sitemap is the catalog.

Here's why it matters. Crawlers find pages two ways: by following links, and by reading your sitemap. Without a sitemap, they rely on links alone. Pages buried deep in your site can be slow to find or missed for good.

This matters for AI search too. AI crawlers use sitemaps to discover pages, just like search engines do. Without that pointer, a deep page may never be fetched. And a page that is never fetched can never be cited in an AI answer.

The good news: you rarely build a sitemap by hand. Most platforms generate one for you. WordPress, Webflow, and Shopify all create a sitemap automatically. Your job is to make sure it follows the rules below.

The format rules: valid XML or it does not count

Your sitemap must be valid XML, or crawlers ignore it. The file has to parse against the standard sitemap protocol. That means the right structure, the right namespace line, and UTF-8 text encoding. A broken sitemap file is ignored entirely.

The structure is simple. A <urlset> wraps the file. Each page sits in its own <url> block. Inside that, the <loc> tag holds the page address. Here is a small example:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://example.com/</loc>
    <lastmod>2026-06-01</lastmod>
  </url>
  <url>
    <loc>https://example.com/pricing</loc>
    <lastmod>2026-05-18</lastmod>
  </url>
</urlset>

There is a hard limit too. One sitemap file can hold at most 50,000 URLs. Go over it and the file breaks the rules. In our audit, exceeding the limit is rated Critical.

Big sites get around this by splitting the list. They use many smaller sitemap files. A sitemap index ties them together. Think of the index as a master list of lists. The crawler reads the index, then reads each file it points to.

Splitting also helps in a smarter way. Group your sitemaps by content type. You might have sitemap-pages.xml, sitemap-posts.xml, and one for images. Then a problem in one content area is easy to spot.

Where do you tell crawlers your sitemap lives?

You point crawlers to your sitemap from your robots.txt file. Your robots.txt file is a small text file at the root of your site. It tells crawlers where they may go. It should also name your sitemap with one line:

Sitemap: https://example.com/sitemap.xml

This line is how crawlers find your sitemap on their own. That includes AI bots. No one has to submit anything by hand. Our audit checks for this line from two sides. It is worth real points in the AI-readiness score.

For more on this file, see AI crawlers and robots.txt: the complete guide.

Why honest lastmod dates matter

Each page in your sitemap can carry a <lastmod> date. It says when the page last changed. Crawlers use it to decide what to re-read first. Accurate dates help them prioritize your updated pages. That gets your fresh content noticed faster.

There is a catch, and many sites trip on it by accident. The date has to be real. Our audit checks whether every page claims the same date. If they all match, the dates were almost certainly auto-generated. That makes them meaningless.

Fake dates train crawlers to ignore your sitemap. If every "last modified" date is wrong, the signal is worthless. So crawlers learn to skip it. That hurts the pages you actually did update.

The fix is a settings change, not a rewrite. Configure your sitemap generator to use each page's true update date. Pull it from your content system. Do not use the moment the file was generated.

Do priority and changefreq tags do anything?

No. The <priority> and <changefreq> tags do nothing for Google. Google has confirmed it ignores both. They were meant to hint at which pages matter and how often they change. Google does not use them.

Many people were told to tune these values. It was never worth the effort. In our audit, these tags are rated Info, the lowest severity. They are harmless clutter. But they also suggest an outdated setup.

You can read Google's own sitemaps documentation for confirmation. The cleaner move is to drop them. Regenerate your sitemap with just <loc> and an accurate <lastmod>.

The inclusion rules: only good pages belong

A sitemap should list only live, official, listable, secure pages. This is the hygiene part, and it matters most. A sitemap full of junk wastes a crawler's attention. It also signals a poorly maintained site.

Back to the library catalog. A good catalog lists books that are actually on the shelf. It lists them once, by their real title. A bad catalog lists missing books, duplicates, and books no one should read. Your sitemap should be the good catalog.

Here are the five rules every listed page must pass:

Rule What it means If broken
Status 200 The page loads (not a 404 dead page or 500 error) High severity — wastes crawler attention
Canonical only The address matches the page's official version Sends mixed signals about which page to index
No noindex The page is not hidden from search High severity — the list contradicts itself
No redirects The address is final, not one that forwards Medium severity — list the destination instead
HTTPS only The address starts with https://, not http:// Insecure pages should not be listed

A quick note on jargon. A status code is how a page reports its health. 200 means "works". A 3xx means "forwards somewhere else". A 404 means "dead". A canonical is the one official address for a page when several point to the same content. Noindex is a tag that asks search engines to hide a page.

The noindex rule trips people up most. Putting a hidden page in your sitemap is a contradiction. The sitemap exists to invite crawlers in. A noindex tag tells them to stay out. Pick one. For more, see noindex explained: when to use it and common mistakes and canonical tags: a complete guide.

Is everything you care about actually listed?

A sitemap should list every page you want found. Missing pages are a real problem. Our audit cross-checks the pages it finds by crawling against your sitemap. It flags real pages that were left off the list.

Why does this hurt? Pages missing from the sitemap are found later, or not at all. So they are slower to appear in search results. They are also slower to show up in AI answers. The fix is to correct your generator's rules so every indexable page is listed.

Why curation beats completeness

A clean sitemap beats a big one. The goal is a list of real, worthwhile, working pages. The opposite is a list padded with junk, duplicates, and near-identical filler. That dilutes the value of the whole file.

A common example is mass location pages. A business spins up hundreds of pages that differ only by the city name. The content is otherwise identical. Stuffing these into your sitemap looks like padding.

Here is the payoff for getting it right. A clean, honest sitemap focuses crawler attention on pages worth ranking and citing. A bloated one spreads that attention thin. It can even trigger quality penalties. To go deeper, see crawl budget and index bloat.

How our audit reads your sitemap

Our visibility audit parses your sitemap, fetches its URLs, and runs 14 distinct checks. It starts by fetching your sitemap and sitemap index. That tells it which pages you have. It uses that list to pick pages to test.

Then it checks the details. It fetches sample URLs to confirm they return 200. It compares your URL count against the pages it expects to find. It tests the 50,000-entry limit. And it checks whether your "last modified" dates are real or auto-stamped fakes.

This sits inside the Technical SEO category of the audit. The GEO side also re-checks the basics: that a sitemap exists, that the dates are accurate, and that robots.txt points to it. Sitemap health is one of the most concentrated parts of the whole score. To see how it all fits together, read how an AI-visibility audit works.

Your sitemap checklist

Run through this list to keep your sitemap clean:

  • It exists at /sitemap.xml
  • It is valid XML, in UTF-8, with the right namespace
  • Each file holds under 50,000 URLs (use a sitemap index beyond that)
  • Your robots.txt names the sitemap
  • Each page has a real, per-page <lastmod> date
  • You dropped the <priority> and <changefreq> tags
  • Every URL returns 200, is canonical, is indexable, is final, and uses HTTPS
  • Every indexable page you care about is included
  • No thin, duplicate, or filler pages padding the list

This pillar is the deep guide. If you want the short version, start with what is an XML sitemap and why your site needs one. The sitemap also fits a wider picture in the technical SEO checklist for the AI search era.

Frequently asked questions

Do I need an XML sitemap? Most sites benefit from one. It is most useful for large sites, new sites with few links, and sites with deep pages. Even small sites gain a little. A sitemap removes guesswork for both search engines and AI crawlers.

Where should my sitemap be located? The standard spot is https://yoursite.com/sitemap.xml. You can name it differently. But you must then point to it from your robots.txt file so crawlers can find it.

How often should I update my sitemap? It should update on its own. Your platform should add new pages and refresh the <lastmod> dates as you publish and edit. You should not be editing the file by hand.

Will an XML sitemap improve my rankings? Not directly. A sitemap helps crawlers find and index your pages. It does not make a page rank higher. But pages that are never found cannot rank or get cited at all.

Should I include priority and changefreq tags? No. Google ignores both. They add clutter and hint at an outdated setup. Use only <loc> and an accurate <lastmod>.

The bottom line

An XML sitemap is the table of contents your site hands to search engines and AI crawlers. Get the format right, then keep the list clean. List only live, official, listable, secure pages. Give each one an honest update date. And drop the tags Google ignores.

The simplest first step is to check what you already have. Run a free visibility audit to see how your sitemap scores and the fixes that matter most.