LinkedIn
Your page ranks for “rooftop unit maintenance checklist.” A facilities manager asks an AI tool what a quarterly maintenance contract should include, and the answer cites someone else. Your content-gap dashboard records one missing citation. That is an outcome, not a diagnosis.
I used to tell clients the fix was more content. I was wrong. A page can fail at three separate stages: it is never fetched, it is fetched but not retrieved for that prompt, or it is retrieved but not chosen for the answer. Each stage calls for a different fix. Most gap tools show you the citation you missed, not the stage where you lost it.
Start with the three things a cited page has to survive
For a live-retrieval answer to cite a passage, the system needs access to the page, a reason to retrieve it for the question, and a passage it can use in the answer. Access, prompt match, and a usable passage are separate requirements. That is the model for the rest of this piece.
Say you run commercial HVAC maintenance in Dallas and spend around $20k a month on search. A facilities manager asks, “What should a quarterly rooftop unit maintenance contract include?” Your page about rooftop units may be excellent. It still has to clear three filters:
- Fetch: Can the relevant bot reach the page?
- Retrieval: Does the page surface as a candidate for that wording?
- Selection: Does the answer use and cite a passage from it?
The order matters. There is no point polishing a paragraph a bot cannot reach. There is little point adding a quotable statistic to a page that does not match the buyer’s question. Start with the first filter you can show is failing.
The answer may not use live retrieval at all
There is no single AI index behind every answer. Perplexity uses live-web retrieval; ChatGPT can answer from training data or use search and live fetches; Gemini combines training data with live Google retrieval. One analysis of 5.5 million responses found that 35 to 40% of queries had zero cited-domain overlap across the three engines. They do not necessarily start with the same set of pages.
If ChatGPT names an HVAC company from training data without browsing, your freshly edited contract page may never enter the picture. If a system retrieves live pages, that same page has a chance to compete for the facilities manager’s question. Training-data recognition and live-page retrieval reward different kinds of visibility. Before diagnosing a page, check whether the answer actually relied on live retrieval.
A Google ranking is not an AI citation
An 11,500-query study found that Google Search and AI Overviews shared only about 18% of sources, with even lower overlap between Search and Gemini and between Overviews and Gemini. Traditional results and generative answers drew on different source mixes.
That is why I do not treat a page-one ranking for “rooftop unit maintenance checklist” as proof that an AI system considered the page for “what should a quarterly rooftop unit maintenance contract include.” One is a useful signal about classic search. It does not tell you which page the AI retrieved, much less which passage it chose. Organic rank cannot diagnose the retrieval gap.
A bot visit is not always a crawl
Server logs can show several routes to your site, and the bot names matter. OpenAI distinguishes GPTBot for training, OAI-SearchBot for ChatGPT Search, and ChatGPT-User for live fetches prompted by a user. Anthropic uses ClaudeBot, Claude-SearchBot, and Claude-User for corresponding jobs. A rule intended to block training can also catch a search bot if it is written too broadly. That can affect search visibility for a reason unrelated to your copy.
Live-fetch bots complicate the picture. ChatGPT-User and Perplexity-User fetch on behalf of users, and their operators argue that those requests are exempt from robots.txt restrictions. OpenAI removed ChatGPT-User from its robots.txt tag list in December 2025. For those fetches, a robots.txt check alone is not an access test; server and CDN controls matter too. Keep the three jobs separate when you read logs or change a bot rule.
Gap 1: The page is not being fetched
The first symptom is quieter than a bad ranking: you find no relevant bot requests for the page in the logs you checked. I start here because access is cheaper to rule out than content quality. Pull 30 days of edge logs and filter for GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, and PerplexityBot. For the HVAC contract page, look particularly at the search and user-fetch bots for the platform you are testing.
Zero hits in that window do not prove the page has never been fetched. Check that the logs cover the right page and bot traffic. Then inspect the practical obstacles: a training block that also catches a search crawler, JavaScript that leaves the answer unavailable to the bot, a CDN challenge, or a login or geo wall in front of the content. These are access problems. A better opening paragraph will not fix them.
The remedy is to let the search indexers and user-fetch bots reach pages you want cited, while keeping any training restriction you intend to keep. Confirm the result at the server or CDN, not just in robots.txt. If the HVAC contract page is inaccessible, stop commissioning contract-content rewrites until the access problem is solved.
Gap 2: The page is fetched but misses the prompt
Now suppose the HVAC page gets fetched and ranks for “rooftop unit maintenance checklist,” yet it rarely appears among sources for “what should a quarterly rooftop unit maintenance contract include.” Being available is not the same as matching the question. The page talks about maintenance tasks. The buyer asks what belongs in a contract. Those subjects overlap, but they are not interchangeable.
Wording matters, and one test is not enough. The same 11,500-query study found AI Overviews less consistent across repeated runs and less robust to small query-syntax changes than traditional search. A missing source in one answer does not prove the system never considered the page. Test several phrasings of the buyer’s question and look for a pattern.
If the pattern points to a prompt-match problem, put a direct answer near the top using the buyer’s language. Then give that answer enough structure to stand on its own: short paragraphs, a list, or a table where those forms actually help. Use valid Article, FAQ, HowTo, or Product schema where it fits the page, but do not mistake markup for a guarantee of retrieval. Research on generative surfacing found that cited sources, quotations, and statistics can increase the odds of a page appearing. The practical test comes first: try five phrasings, not one, before deciding what to rewrite.
Gap 3: The answer chooses another passage
At the third stage, the page is available and there is evidence it surfaces for the topic, but the answer uses a competitor’s passage. Your dashboard can see the final citation. It usually cannot show every candidate the system considered, so do not label every missing citation a proven selection loss. Compare the passages you can observe before diagnosing quotability.
This is where a page that reads fine to a human can lose. One citability framework recommends self-contained, fact-rich passages of roughly 100 to 200 words, with the subject named early and concrete details close to the claim. The useful principle is not a magic word count. It is that a passage should make sense when separated from the rest of the page.
An HVAC category page that says “explore our plans below,” then hides the details in a filter widget, gives an answer little to work with. A standalone paragraph that names a quarterly rooftop unit contract and explains its inspections, checks, and written reporting is easier to use. A quotable block names its subject instead of relying on “it” or “this,” includes a specific fact, and does not need the preceding paragraph to explain it. Copy one paragraph into an empty document. If it no longer answers the buyer’s question, rewrite that paragraph before you buy another page.
What our own pages showed us
At groas, one Google Ads updates post fed 463 live ChatGPT answers in 30 days while category pages on the same domain fed none. That contrast does not reveal every page the system fetched or considered. It does show why I look closely at the passage an answer can use. The post answered dated questions with names, numbers, and change logs in standalone sections. The category pages pointed readers elsewhere. They did not offer the same kind of liftable answer.
The post put a direct answer near the top. Each dated block named the change and its effect in plain sentences, so it still made sense on its own. The category pages had a headline, a grid of links, a thin introduction, and pricing behind a filter. If a system reached one of those pages, there was little standalone text to select. Fix the unit the answer can cite: one self-contained block, not an entire page in the abstract.
Know when this model is the wrong one
The three gaps describe a page’s path through a live-retrieval answer. They do not explain every mention of a brand. If ChatGPT names your company without browsing or a link, you may be looking at training-data recognition rather than a missed retrieval. That is not a problem you solve by moving one paragraph up the page.
Local prompts can also change the emphasis. For “best rooftop contractor near me,” maps and review signals may matter more than a neatly written contract passage. And if the useful content sits behind a login, paywall, or aggressive bot challenge, passage edits wait until access is addressed. I keep the diagnostic order short: check access, then prompt match, then the passage. Skip the order and you can pay for the wrong fix.
What a content-gap platform should show you
Most dashboards start with who appeared in the answer. A tracking-tool roundup warns that basic tools report mention counts without the source content behind them, and that tracking alone produces reports rather than a content strategy. A citation report is useful. It is not, by itself, evidence that your page was fetched, retrieved, and rejected at the final stage.
A platform worth paying for should help you work through the stages in order:
- Fetch evidence: bot requests by page and bot type, with access problems visible.
- Prompt coverage: results across five to ten phrasings of the same buyer question, rather than a verdict from one run.
- Passage comparison: the text used in the answer beside the text your page offers.
That sequence determines where effort goes. At groas, we use it to avoid rewriting a page before checking whether bots can reach it, or polishing a passage before testing whether the page matches the prompt. A scoreboard that cannot help distinguish those problems will usually send you back to “write more content.”
Run the model on a different buyer question
Say you sell project management software for construction firms, spend $30k a month, and Perplexity keeps citing a competitor for “how to track subcontractor change orders.” The HVAC example gives you a way to investigate without assuming this is a writing problem.
First, check edge logs for PerplexityBot and Perplexity-User requests to your how-to page. In this example, the hits are present. You can move past an obvious fetch failure, though those hits alone do not prove what happened inside any particular answer.
Second, run eight phrasings of the question. Your page appears as a source in two; the competitor appears in seven. That pattern points you toward prompt match. Your page says “change order workflows,” while the buyer asks about “tracking subcontractor change orders,” and the direct answer sits three screens down beneath product copy. Move an answer in the buyer’s language near the top, then test again.
Third, compare what the answers actually use on the prompts where your page surfaces. The competitor offers a standalone description of a change order log that names its fields: date, cost impact, approval status, and revised completion date. Your passage begins, “With our flexible solution, you can…” It asks the reader to keep reading before it says what belongs in the log. Rewrite one roughly 150-word block so it names the subject and answers the question without that runway.
The analogy to three filters is useful because it forces an order of operations. It breaks if you treat the filter results as fully visible: logs, repeated prompts, and cited passages are evidence, not a complete record of the model’s internal choices. Still, they tell you far more than a missing-citation count. In the construction-software case, the first move is not a content program. It is a prompt-matched answer and one passage that can stand alone.
If your vendor cannot help you tell those failures apart, you do not have a diagnosis. You have a scoreboard. Start with access, test the words buyers use, and make the answer worth citing before you buy more pages.