Real fixes from seven live sites. No pitch.Log in / My reports
Idea to Traffic Pressure Test

Your missing pages return 200 OK, and it is breaking things you have not noticed

If a nonsense URL on your site returns 200 instead of 404, your host is serving your homepage for every missing file. That silently breaks tools that fetch data, makes ads.txt look missing and fills Search Console with soft 404s. Add a real 404.html and check content, not status codes.

Three sites, two weeks, every check green

This cost two weeks on three separate sites before anyone spotted it, because every check being run came back green. The sites were up, pages returned 200, and monitoring was happy. What was broken was a tool on the page that quietly did nothing when a visitor used it, with no error in the console and nobody to report it, since most people who use a tool that does nothing assume they used it wrong.

The cause was a catch-all rule handing back index.html with status 200 for any path that did not match a real file. A data file had not deployed, so the fetch received homepage HTML, res.ok was true, the guard never fired, and res.json() threw a parse error inside a promise nobody awaited.

This one cost two weeks on three separate sites before anyone spotted it, because every check that was being run came back green.

The symptom, if you can call it that, is nothing. The site loads. Pages return 200. Monitoring is happy. But a tool on the page quietly does nothing when you use it, and no error appears in the console.

What is actually happening

Many static hosts and single page app configurations include a catch-all rule: any path that does not match a real file gets your index.html, with status 200. That is correct and necessary for client side routing. It is a disaster for everything else.

Consider a tool that loads a dictionary at startup:

const res = await fetch('/data/words.json');
if (!res.ok) throw new Error('missing data');
const words = await res.json();

If words.json did not deploy, the fetch returns your homepage HTML with status 200. res.ok is true, so the guard never fires. Then res.json() throws a parse error inside a promise nobody awaited, and the page carries on looking perfectly normal with a tool that silently does nothing.

How to detect it in one command

Ask for something that definitely does not exist:

curl -s -o /dev/null -w "%{http_code}\n" https://yourdomain.com/zzz-does-not-exist-9912

You want 404. If you get 200, you have this problem.

Confirm by comparing checksums. If a nonsense URL and your homepage hash identically, every missing file on your site is being served as your homepage:

curl -sL https://yourdomain.com/           | md5sum
curl -sL https://yourdomain.com/zzz-9912  | md5sum

What it breaks, in order of how much it costs you

  • Your tools, silently. Any res.ok check is defeated. This is the expensive one because there is no error to find.
  • ads.txt. Ad networks read HTML where a text file should be and report the file as missing.

Make missing pages return a real 404

Add a real 404.html at the site root. Most static hosts serve it automatically for unmatched paths once it exists. If you genuinely need client side routing, scope the catch-all to your app routes only, and never to asset paths like /data/*, /*.txt, or /*.json.

Change how you verify. A 200 proves a request was answered. It does not prove the right thing answered. When checking that a page exists, compare its content against your homepage rather than reading its status code. That single habit would have caught this in minutes instead of weeks.

Write fetch code that cannot be fooled by this

Fixing the host is the right fix. Writing client code that survives the situation anyway is the cheap insurance, because you will eventually deploy to a host you do not control.

The guard everyone writes checks res.ok, which is exactly the check this defeats. Check the content type instead, or in addition:

const res = await fetch('/data/words.json');
    const type = res.headers.get('content-type') || '';
    if (!res.ok || !type.includes('json')) {
      throw new Error(`expected JSON, got ${res.status} ${type}`);
    }

For a plain text asset the same idea applies, and there is a second sanity check worth adding: a dictionary file that comes back as one enormous line, or whose first line starts with <, is not a dictionary. Validating the shape of the data rather than the status of the response turns a silent failure into a visible one.

The important part is not the specific check. It is that the error surfaces. A thrown error inside an unawaited promise disappears, so make sure the failure path actually renders something to the user rather than only logging to a console nobody has open.

The same trap makes deleted files look alive

This behaviour has a mirror image that is worth knowing, because it wastes time in a completely different situation.

When you delete a file or a serverless function and redeploy, the route keeps returning 200, because the catch-all is now serving your homepage from that path. Every check you run says the route is still there. It looks like the deploy failed, or like something is cached, and people go around the loop several times before working it out.

A route that returns your homepage HTML is a route that does not exist, whether you were expecting a file to be there or expecting it to be gone:

curl -s https://yourdomain.com/api/thing | head -c 80

If that starts with <!DOCTYPE, the endpoint is gone. If you wanted it gone, you are finished.

What it does to your search presence

The indexing damage is slower than the tool breakage and harder to undo.

Every mistyped URL anyone has ever linked to becomes a real, indexable page serving your homepage content. So does every stale URL from a previous version of the site, and every path a crawler invented while probing. Each one is a distinct URL with identical content, which is the textbook definition of the duplicate content problem.

In Search Console this shows up as a growing pile of Soft 404 entries, and alongside them Duplicate, Google chose a different canonical. The soft 404 label is Google telling you directly that a page returned a success status while looking like an error page, which is a fairly precise description of what your host is doing.

Crawl budget is the second cost and it matters more for a small site than people assume. A crawler that keeps discovering new URLs will keep fetching them, and on a site with no authority that budget is small. Spending it on infinite variations of your homepage means less of it spent on the pages you actually wrote.

The audit that should run after every deploy

Two checks, both fast, both catching things a status code cannot.

First, confirm that a nonsense path returns 404 rather than 200. If that single assertion holds, the whole class of problem is gone.

Second, confirm that the files nobody links to are actually present. These are the ones no visitor will ever report as broken, and they are the ones most likely to go missing:

HOME=$(curl -s https://yourdomain.com/ | md5sum | cut -d' ' -f1)
    for f in ads.txt robots.txt sitemap.xml favicon.ico og-image.png; do
      H=$(curl -s https://yourdomain.com/$f | md5sum | cut -d' ' -f1)
      [ "$H" = "$HOME" ] && echo "$f MISSING (served as homepage)" || echo "$f ok"
    done

Why this survives so long in the wild

Every instinct you have for verifying a site is status-based. Uptime monitors check status codes. Link checkers check status codes. A quick manual check is opening the page and seeing that it loads. All of those pass. The site genuinely is up, genuinely is fast, and genuinely answers every request.

Soft 404s Google has already recorded

Adding a real 404.html stops the masking from that point onwards. It does not undo the indexing damage, which is slower and harder to reverse: soft 404 entries already logged, the duplicate canonical findings alongside them, and crawl budget already spent on endless variations of your homepage. It does not put back the file that never deployed, so the missing asset still has to be found and shipped. It does not help on a host you do not control, which is why the fetch code needs to check content type and the shape of the data rather than res.ok. And it does not change the habit that let this run for two weeks, which is verifying by status code instead of comparing the body against your homepage.

Questions about 200 responses for missing pages

How do I check whether my host returns 404 for missing files?

Ask for something that definitely does not exist and read the status: curl -s -o /dev/null -w "%{http_code}\n" https://yourdomain.com/zzz-does-not-exist-9912. You want 404. A 200 means you have this problem, and hashing a nonsense URL against your homepage confirms it.

Why did my res.ok check not catch a missing JSON file?

Because the file was not reported as missing. The host returned your homepage with status 200, so res.ok was true and the guard was bypassed, and the parse error that followed vanished inside an unawaited promise.

Why is Search Console filling up with soft 404s and duplicate content?

Every mistyped URL, dead link, stale path and crawler-invented path becomes a real indexable page serving your homepage. Each is a distinct URL with identical content, and the soft 404 label is Google saying a page returned a success status while looking like an error page.

I deleted a serverless function but the route still returns 200. Did the deploy fail?

Probably not. The catch-all is now serving your homepage from that path. Check the body with curl -s https://yourdomain.com/api/thing | head -c 80, and if it starts with <!DOCTYPE the endpoint is gone.

Is checking that /about returns 200 a useful test?

No. On a host with a catch-all it was always going to return 200. Compare the response against your homepage instead, and treat an identical hash as proof the page does not exist.

How do I write fetch code that cannot be fooled by this?

Check the content type as well as the status, so a response that is not JSON throws even when the status is 200, and validate the shape of the data rather than only the response. Make sure the failure path renders something to the user instead of only logging to a console nobody has open.