Skip to content

Find orphan and zombie pages

Two lists on one report answer two opposite questions. An orphan is a page people reach and the crawler does not. A zombie is a page the crawler keeps fetching and nobody reads. Both are candidates, not verdicts.

Time: about 20 minutes · You need: at least 30 days of data, and a sitemap inventory for step 4

Log Hero → Bots vs Humans starts on Googlebot. A toggle in the page toolbar switches the bot side to bingbot, Search bots, AI bots or Any bot.

Three tiles run across the top. With the default bot they read:

  • Crawled by Googlebot and visited by humans
  • Crawled by Googlebot, never visited
  • Visited by humans, never crawled by Googlebot

Four rules for reading the page:

  • The tiles count paths, not hits. The hit counts are in the small hint line under each tile.
  • The two lists show 20 rows. The panel sub-lines say Top 20 by {bot} hits. and Top 20 by human hits. A path outside the top 20 is not listed anywhere.
  • The tiles and lists include assets and every status class. The chart Crawls and visits, day by day counts only status 200 and leaves assets out, so the two sets of numbers do not add up.
  • There is no column sorting and no export on this page, or on any other Log Hero report.

The panel By section is worth a look before the lists. It gives one row per first path segment with a Crawl : visit ratio, so you see which part of the site is crawl-heavy or visit-heavy before you read single paths.

2. Read “Visited, never crawled” as orphan candidates

Section titled “2. Read “Visited, never crawled” as orphan candidates”

The panel Visited, never crawled by Googlebot lists paths with human hits and no Googlebot hit in the window. Its sub-line names the caveat itself:

Orphan candidates — or pages {bot} is deliberately kept out of. Check before acting. Top 20 by human hits.

Three causes account for most rows:

  • The page is not linked internally. People arrive from a newsletter, a social post or a bookmark. The crawler has no path to it.
  • The page is blocked. A robots.txt rule or a WAF rule keeps the bot out.
  • The page is new. The crawler has not got to it yet.

Widen the date range to 90 days before you conclude that a page is an orphan. A seasonal page or a long-tail page looks orphaned in a short window.

For a real orphan, add internal links from pages the crawler already visits often. Use Crawl Budget & Waste ▸ Crawl by section to find those pages.

3. Read “Crawled, never visited” as zombie candidates

Section titled “3. Read “Crawled, never visited” as zombie candidates”

The panel Crawled by Googlebot, never visited is the reverse. Googlebot keeps fetching the path and no human hit arrived in the window.

Three causes again:

  • Old links. Internal links, an old sitemap entry or a backlink keeps pointing there.
  • Parameter URLs. One page multiplied into many near-duplicate URLs. Crawl Budget & Waste ▸ Parameter keys names the parameters.
  • Thin pages. Tag pages, paginated archives and empty filter results.

Decide per page. There are three outcomes:

Decision When What to do
Improve The page has a real job and no traffic Add content, add internal links, fix the title
Redirect Another page covers the topic Redirect once, to the exact replacement
Remove The page has no job Return 410 and remove the internal links

A clean 410 is cheap for the crawler. The cost is whatever keeps linking to the URL.

Coverage holds the strongest signal, in the tile Advertised, never requested by anyone. Its hint reads no bot, no human, {n} days. Those are pages your sitemap advertises that nothing at all requested in the window.

An orphan candidate that is also in the sitemap is the strongest case for action. You advertise the page, people find it, and the crawler does not.

Log Hero has no built-in join between the two reports, so do this by hand. Take the paths from Visited, never crawled by Googlebot and look each one up in Coverage ▸ Advertised, not crawled by Googlebot.

Bots vs Humans does show a grey paragraph under the section table when an inventory exists. It gives counts of visited-only paths that are advertised and not advertised. It gives counts only, never the paths, and it says itself that the two sets of numbers do not reconcile.

If no inventory is loaded, the whole Coverage page is an empty state. See Add a sitemap.

Open URL Drilldown for one path you changed. The panel Status served, by day shows the day the code flipped, in the form flipped {prev} → {next} on {YYYY-MM-DD}.

For a new internal link, wait a full window and open Bots vs Humans again. The path leaves the orphan list once Googlebot fetches it once.

  • The lists are empty. No bot hits at all is a setup problem. See No data arriving.
  • Every page looks like an orphan. An origin integration behind a CDN misses cached requests. See Choose your integration.
  • The bot side is empty in a group mode. Search bots and AI bots resolve only from the bots seen in the window, and the page says No hits from {Search bots|AI bots} in the selected period.

Human is a residual. It is every request that Log Hero did not match as a bot, so unrecognised crawlers, prefetches and login probes land in it. The report says so:

"Human". Rows Log Hero did not classify as a bot. Login and XML-RPC probes show up here as human hits — filter them before drawing conclusions.

Read it as not identified as a bot, not as people. These are requests, not sessions.

  • No side-by-side error rate. The panel Humans hitting errors gives a human 404 rate and a human status mix. The bot error rate lives on Bot Activity. Nothing puts the two next to each other.
  • No response times. No panel on this page reads response time at all.
  • No section filter. A section is the first path segment and the depth is fixed. You cannot narrow the page to one section.