Find orphan and zombie pages
Two lists on one report answer two opposite questions. An orphan is a page people reach and the crawler does not. A zombie is a page the crawler keeps fetching and nobody reads. Both are candidates, not verdicts.
Time: about 20 minutes · You need: at least 30 days of data, and a sitemap inventory for step 4
1. Open Bots vs Humans
Section titled “1. Open Bots vs Humans”Log Hero → Bots vs Humans starts on Googlebot. A toggle in the page toolbar
switches the bot side to bingbot, Search bots, AI bots or Any bot.
Three tiles run across the top. With the default bot they read:
- Crawled by Googlebot and visited by humans
- Crawled by Googlebot, never visited
- Visited by humans, never crawled by Googlebot
Four rules for reading the page:
- The tiles count paths, not hits. The hit counts are in the small hint line under each tile.
- The two lists show 20 rows. The panel sub-lines say
Top 20 by {bot} hits.andTop 20 by human hits.A path outside the top 20 is not listed anywhere. - The tiles and lists include assets and every status class. The chart Crawls and
visits, day by day counts only status
200and leaves assets out, so the two sets of numbers do not add up. - There is no column sorting and no export on this page, or on any other Log Hero report.
The panel By section is worth a look before the lists. It gives one row per first
path segment with a Crawl : visit ratio, so you see which part of the site is
crawl-heavy or visit-heavy before you read single paths.
2. Read “Visited, never crawled” as orphan candidates
Section titled “2. Read “Visited, never crawled” as orphan candidates”The panel Visited, never crawled by Googlebot lists paths with human hits and no Googlebot hit in the window. Its sub-line names the caveat itself:
Orphan candidates — or pages {bot} is deliberately kept out of. Check before acting. Top 20 by human hits.
Three causes account for most rows:
- The page is not linked internally. People arrive from a newsletter, a social post or a bookmark. The crawler has no path to it.
- The page is blocked. A
robots.txtrule or a WAF rule keeps the bot out. - The page is new. The crawler has not got to it yet.
Widen the date range to 90 days before you conclude that a page is an orphan. A seasonal page or a long-tail page looks orphaned in a short window.
For a real orphan, add internal links from pages the crawler already visits often. Use Crawl Budget & Waste ▸ Crawl by section to find those pages.
3. Read “Crawled, never visited” as zombie candidates
Section titled “3. Read “Crawled, never visited” as zombie candidates”The panel Crawled by Googlebot, never visited is the reverse. Googlebot keeps fetching the path and no human hit arrived in the window.
Three causes again:
- Old links. Internal links, an old sitemap entry or a backlink keeps pointing there.
- Parameter URLs. One page multiplied into many near-duplicate URLs. Crawl Budget & Waste ▸ Parameter keys names the parameters.
- Thin pages. Tag pages, paginated archives and empty filter results.
Decide per page. There are three outcomes:
| Decision | When | What to do |
|---|---|---|
| Improve | The page has a real job and no traffic | Add content, add internal links, fix the title |
| Redirect | Another page covers the topic | Redirect once, to the exact replacement |
| Remove | The page has no job | Return 410 and remove the internal links |
A clean 410 is cheap for the crawler. The cost is whatever keeps linking to the URL.
4. Cross-check with the sitemap
Section titled “4. Cross-check with the sitemap”Coverage holds the strongest signal, in the tile Advertised, never requested by
anyone. Its hint reads no bot, no human, {n} days. Those are pages your sitemap
advertises that nothing at all requested in the window.
An orphan candidate that is also in the sitemap is the strongest case for action. You advertise the page, people find it, and the crawler does not.
Log Hero has no built-in join between the two reports, so do this by hand. Take the paths from Visited, never crawled by Googlebot and look each one up in Coverage ▸ Advertised, not crawled by Googlebot.
Bots vs Humans does show a grey paragraph under the section table when an inventory exists. It gives counts of visited-only paths that are advertised and not advertised. It gives counts only, never the paths, and it says itself that the two sets of numbers do not reconcile.
If no inventory is loaded, the whole Coverage page is an empty state. See Add a sitemap.
Make sure that the change worked
Section titled “Make sure that the change worked”Open URL Drilldown for one path you changed. The panel Status served, by day
shows the day the code flipped, in the form flipped {prev} → {next} on {YYYY-MM-DD}.
For a new internal link, wait a full window and open Bots vs Humans again. The path leaves the orphan list once Googlebot fetches it once.
If the lists look wrong
Section titled “If the lists look wrong”- The lists are empty. No bot hits at all is a setup problem. See No data arriving.
- Every page looks like an orphan. An origin integration behind a CDN misses cached requests. See Choose your integration.
- The bot side is empty in a group mode.
Search botsandAI botsresolve only from the bots seen in the window, and the page saysNo hits from {Search bots|AI bots} in the selected period.
What “human” means here
Section titled “What “human” means here”Human is a residual. It is every request that Log Hero did not match as a bot, so unrecognised crawlers, prefetches and login probes land in it. The report says so:
"Human". Rows Log Hero did not classify as a bot. Login and XML-RPC probes show up here as human hits — filter them before drawing conclusions.
Read it as not identified as a bot, not as people. These are requests, not sessions.
What this report does not show
Section titled “What this report does not show”- No side-by-side error rate. The panel Humans hitting errors gives a human
404rate and a human status mix. The bot error rate lives on Bot Activity. Nothing puts the two next to each other. - No response times. No panel on this page reads response time at all.
- No section filter. A section is the first path segment and the depth is fixed. You cannot narrow the page to one section.
