Two Cloudflare Pages defaults to fix before you launch
Without a 404.html, Cloudflare Pages answers every missing URL with your homepage, and your pages.dev address can be indexed. Both fixes are small.
This site runs on Cloudflare Pages. While setting it up today, I checked two of Pages' defaults on my own deploys. Both are in Cloudflare's documentation, both are easy to miss, and both affect how search engines see your site.
Missing pages return your homepage
If your build has no 404.html file at the top level, Cloudflare Pages assumes your site is a single-page app. Any URL that doesn't match a file gets your homepage, with a 200 "OK" status.
Here is what my two deploys answered today for a URL that doesn't exist, /this-page-does-not-exist/:
| Deploy | Has a 404.html | What came back |
|---|---|---|
| A placeholder page on my production address | No | 200 OK, with the homepage |
| The full site on a branch preview | Yes | 404 Not Found, with my 404 page |
Google asks sites to return a 404 response code for pages that don't exist. With the fallback, a deleted page or a typo in a link still answers "OK" and shows your homepage, so neither a search engine nor a link checker can tell the page is gone.
The fix is a 404.html at the top of your build output. In Astro, that's a src/pages/404.astro page. Cloudflare serves it with a real 404 status. The only sites that want the fallback are single-page apps that handle their own routes in the browser.
Your pages.dev address can be indexed
Every Pages project gets a free address on pages.dev, and it keeps working after you connect your own domain. Cloudflare adds an X-Robots-Tag: noindex header to preview deployments by default, which tells search engines not to index them. Your production pages.dev address doesn't get that header, so once your domain is live, the same pages can be indexed at two addresses.
Cloudflare's documentation gives the fix. Put a file called _headers in the folder your build copies to the output unchanged (public/ in Astro):
https://:project.pages.dev/*
X-Robots-Tag: noindex
https://:version.:project.pages.dev/*
X-Robots-Tag: noindex
The first rule matches your production pages.dev address. The second matches every preview and branch address. Your own domain matches neither, so it stays indexable. If you'd rather send visitors to your domain, Cloudflare's account-level Bulk Redirects can redirect the pages.dev address instead.
To check it, request your pages.dev address and look for the header:
curl -sI https://your-project.pages.dev/ | grep -i x-robots-tag
That should print x-robots-tag: noindex. The same command on your own domain should print nothing.
Check the deployed site, not your machine
Run both checks against the deployed site. A local server can answer a missing page with a 404 while Cloudflare falls back to your homepage, so a test on your own machine passes without proving anything about the host.