Missing requests
Data is arriving, but a request you know happened is not in the report. Work through these checks in order. They are ordered by how often each one is the cause.
Start from one fact about the pipeline:
- There is no sampling. Log Hero records every request the integration hands over.
- There is no deduplication. A retried delivery counts twice rather than going missing.
- There are no quotas. No plan limits hits, domains or requests per day.
- There is no asset filter. Images, CSS, JavaScript and health probes are hits like any other request.
So a gap is always a problem at the source, never a decision Log Hero made about your volume.
1. The request never reached the integration
Section titled “1. The request never reached the integration”This is the most common cause by a wide margin. Where you log decides what you see.
- An origin integration behind a CDN. The WordPress plugin, the PHP SDK and the Apache module run on your server. Every request your CDN answers from cache stops at the edge, and your server never learns about it.
- An edge integration with traffic going around it. Cloudflare, CloudFront, Fastly and Azure Front Door see only what passes through them. A bot that reaches your origin address directly is invisible to them.
- A static file on an origin integration. The web server serves images, CSS, JavaScript and PDFs without PHP. The WordPress plugin and the PHP SDK never see them. The Apache module and the edge integrations record them.
- A request that killed PHP. The WordPress plugin and the PHP SDK record a
request from inside your application. A request that failed before that code
ran is never logged. This is why origin integrations under-report
5xx. - A request stopped before logging. A WAF rule, a rate limit or a bot mitigation rule ends the request before your integration records it.
To close this gap, move to the integration that sits closer to the visitor. Read Choose your integration.
2. The record was dropped at ingestion
Section titled “2. The record was dropped at ingestion”Log Hero drops a record in a small number of defined cases. None of them produce a warning in the UI. Anatomy of a log line covers the fields these cases turn on.
| Cause | What happens |
|---|---|
| No client IP in the log line | The record is dropped. Every identity field derives from the address. |
| The log line does not parse | The record is skipped. Log Hero writes one warning for the whole delivery, not one per line. |
| The token is wrong but present | The endpoint answers 200, and the record is dropped one step later because no account matches. |
| An Azure Front Door record of another category | Only records of the FrontDoorAccessLog category are kept. |
| A user-agent filter on the token | If your token carries a user-agent filter, Log Hero keeps only the requests that match it. This is off unless it was set for you. |
The wrong-token case deserves attention, because it looks like success everywhere. Read A wrong token looks like success.
One rare loss exists on Log Hero’s side. If the storage layer fails, the hits from that short window are lost and are not sent again.
3. The buffer dropped records
Section titled “3. The buffer dropped records”This case is specific to the WordPress plugin and the PHP SDK. When delivery to Log Hero fails, the buffer on your server grows. When it passes 5 MB, the integration stops adding to it. Records collected after that point are lost.
The sign is a clean gap in time, not one kind of request missing everywhere. WordPress shows a warning in the backend while this happens.
Work through Asynchronous flush failed and Permission denied.
4. The request is there under another hostname
Section titled “4. The request is there under another hostname”Log Hero stores the hostname exactly as the integration sent it.
www.example.comandexample.comare two hostnames.- A port stays attached.
example.com:8443is a third. - Case is kept.
Example.comis a fourth. - Azure Front Door is the exception. It lowercases the hostname and drops the port.
Open the hostname selector in the top bar and choose All hosts. If the request appears, it arrived under a hostname you did not expect. The selector appears only once two or more hostnames had bot hits in the window.
5. The request is there under another day
Section titled “5. The request is there under another day”Log Hero dates every hit in UTC. Your date picker builds the range in your browser’s local time. A request near midnight therefore lands on the day before or after the one you expect.
A second cause is rarer and quieter. If the timestamp in the log line is missing or unreadable, Log Hero dates the hit at the moment it processed the record. A delayed batch then lands on the processing day, not the request day.
Widen the date range by one day on each side and look again.
6. The request is counted as human
Section titled “6. The request is counted as human”The report you are reading almost certainly filters on bots.
- An unrecognised user agent counts as human. Log Hero names a bot from a
user-agent table and verifies it against published IP ranges. A request that
matches neither is stored with
is_bot = 0. It is not missing. It is in the human half. - Log Hero’s own fetcher is not classified.
KeywordHeroBotfetches yourrobots.txtand your sitemap. The reports filter that name out of every query, but the ingestion side does not assign it, so those hits arrive unnamed or as human and stay in the data.
Open Bots vs Humans to see the human side, or read How bot verification works.
7. The request is a parameter URL
Section titled “7. The request is a parameter URL”Log Hero keeps the query string on the path, so /search?q=shoes and /search
are different rows. URL Drilldown cannot open a path that contains a ? and
says so:
Parameter URLs can't be drilled down yet — hits are counted against the base path only.
The hits exist. Only the drilldown is unavailable. Use the Parameter keys panel on Crawl Budget & Waste to see which parameters bots request.
One integration adds a wrinkle. Amazon CloudFront percent-decodes the path and the query string once before Log Hero stores them. The other integrations store them exactly as they arrived.
8. Today’s data is partial
Section titled “8. Today’s data is partial”Today is included in every date range you pick, and today is still being
collected. A figure for today is therefore lower than the finished day will be.
The reports carry the line Today's data is still being collected — charts end yesterday. That is a warning about partial data, not a filter.
What Changed is the exception. Its comparison always uses the last seven complete days against the seven before them.
If none of these match
Section titled “If none of these match”Send an email to integrations@keyword-hero.com with the subject “Requests are missing from my Log Hero reports”. Include:
- The domain as registered in Log Hero.
- The integration you use.
- One example request: the full URL, the hostname, and the time it happened.
- The date range you looked in.
