LinkedIn
A vendor shows you a competitor cited in a Google AI Overview, rewrites a few of your pages, and later shows you a citation with your name on it. Did the rewrite work, or did Google rotate its sources? Before you buy the software, run a six-week, 20-page holdout test: rewrite 10 pages, leave 10 matched pages alone, and count citations for both.
I would not claim a bidding change improved a paid search account after changing every ad group at once during a seasonal rush. Yet that is roughly the standard of proof behind many AI search optimization pitches. Profound reported 59.3% month-over-month domain citation drift in Google AI Overviews. When cited domains change that often, a before-and-after screenshot without a control group tells you very little.
The question: do the edits earn citations?
Before committing to software pitched as the best tool to optimize landing pages for AI Overviews, write down the question it must answer: Do pages receiving the proposed edits earn more clickable AI Overview citations than comparable pages left untouched?
This protocol tests one kind of page-level edit. It does not test every capability in a platform or prove why Google’s retrieval system selected a particular URL. It does give you something a demo cannot: a comparison against pages exposed to the same six weeks of ordinary search volatility. If treatment and control move together, do not pay for a rewrite on the strength of a citation report.
Setup: build 10 matched pairs before touching the copy
Choose deep pages with similar starting positions
Do not start with your homepage simply because the sales deck does. A BrightEdge study found that 82.5% of AI Overview citations linked to deep content pages, two or more clicks from the homepage, while top-level landing and category pages accounted for 17%. It also found that 86% of cited URLs appeared for only one specific keyword. Test pages and queries narrow enough to have a plausible citation opportunity.
Pull 20 indexable informational or specific product pages. Following standard SEO split-testing methodology, make 10 pairs. Match each pair as closely as you can on:
- Monthly impressions in Google Search Console.
- Organic ranking positions.
- CMS template and page type.
- The specificity of the buyer question the page answers.
Assign one page from each pair to treatment and the other to control before you see any new citation results. The match will not be perfect. Write down the differences rather than quietly replacing a difficult page halfway through. You now have 10 treated pages and 10 pages that will remain untouched.
Lock three to five buyer queries per page
For each page, choose three to five questions a prospective buyer might actually search. Record the exact wording in a spreadsheet before changing any copy. Use the same number of queries for each page if you want a simple comparison between groups. The worked count below uses four locked queries per page: 80 queries in total.
Include specific sub-questions, not just broad category terms. Research reported by Search Engine Land found that ranking for fan-out sub-queries was associated with higher AI Overview citation odds. If a page covers industrial air filtration, a broad query such as “industrial air filter systems” can sit beside more precise questions about HEPA filter pressure drop or MERV requirements. Give paired pages queries of comparable specificity, even though the wording will differ.
Then stop adding queries. If a treated page misses its locked questions, searching until you find one that cites it is not a discovery. It is moving the finish line.
Test one edit, not the vendor’s entire checklist
An audit may recommend FAQ schema, new headings, shorter paragraphs, different internal anchors, and image changes in one go. Apply all of that to 10 pages and you may see a difference, but you will not know which change mattered.
For this first run, test an answer-first opening block. CXL reported that 55% of cited passages appeared within the top 30% of a document. Research by Dan Petrovic, cited by Cyrus Shepard, also found higher passage grounding for an 800-word page than for a 4,000-word page. Neither finding promises that shortening or moving text will get your pages cited. They give you a reason to test whether a clear answer near the top helps.
On each treatment page, rewrite only the top 150 words. Put the relevant buyer question in a subheading and follow it with a concise, factual three-sentence answer. Preserve the page’s supporting detail and layout. Change nothing on the control pages. If the vendor wants to test a different page-level recommendation, define that single change just as tightly before the experiment starts.
Run the protocol: baseline first, then six weeks of checks
Week 0: record what Google already cites
Before editing, check all 80 locked queries on a fixed schedule. Keep the geographic IP and time of day consistent. You can use an incognito browser or collect SERP responses through an automated fetch script, but do not switch methods mid-test.
For each query, record:
- Whether Google displayed an AI Overview.
- Whether your domain appeared anywhere in its answer.
- Whether the page URL received a direct, clickable citation.
The third field is the primary outcome. The first two help you interpret it. A query with no AI Overview cannot produce an AI Overview citation on that check; an unlinked brand mention is not a clickable URL. Week 0 is your actual starting point, not an assumed baseline from someone else’s account.
Week 1: publish the edit and hold everything else steady
At the start of Week 1, publish the answer-first blocks on the 10 treatment pages. You can submit those URLs for indexing in Google Search Console, but submission does not establish when Google will recrawl them. Note the publication and submission dates, then keep checking.
For the six-week run, avoid changes that could affect one group differently from the other: no targeted internal-link push, metadata sweep, CMS template change, or second round of copy edits. Keep the controls untouched. If a sitewide change cannot wait, log it. A holdout is useful because it makes competing explanations harder, not because it makes your website a sealed laboratory.
Weeks 1–6: check Monday, Wednesday, and Friday
Run the locked queries three times a week at the same hour, using the same collection method. Ahrefs found, in an analysis of 43,000 queries, a 70% probability that AI Overview content changed between consecutive observations. Its reported average overview persisted for 2.15 days, and 45.5% of cited URLs rotated out between refreshes. A single victory screenshot will not survive that kind of movement.
Count only a clickable citation to the specific page URL: a source card or an inline link. Keep domain mentions and links to other pages in separate columns. Some AI visibility software reports roll several kinds of appearance together. That may be useful for monitoring, but it cannot answer this test’s narrower question. Stick to the definition you set in Week 0.
Measure the gap, not the prettiest screenshot
Calculate citation presence for both groups
Six weeks at three checks a week gives you 18 observation days. At four locked queries for each of 10 pages, each group has 40 query checks per day: 720 checks for treatment and 720 for control over the run.
For each group, calculate Citation Presence Rate as clickable citations to the assigned page URLs divided by that group’s 720 checks, expressed as a percentage. Keep checks with no AI Overview in the denominator, and use the Week 0 records to see whether one group started ahead. Because the same pages and queries are checked repeatedly, those 720 observations are not 720 independent experiments. The rate is a practical scoreboard, not a statistical certificate.
Look at the weekly pattern as well as the full-run rate. Did the treated group improve relative to its own baseline while the control did not? Did the gap persist, or did one unusually favorable check create it? Compare matched pairs, too. A win spread across several pages is more useful than a group average carried by one URL.
Keep organic rank in view, but do not substitute it for citations
Traditional ranking reports cannot settle this question. In an analysis of 863,000 keyword SERPs and 4 million cited URLs, Ahrefs found that only 37.9% of cited URLs also ranked in Google’s organic top 10. A move from organic position eight to six is worth recording, but it is not itself an AI Overview citation.
Log ranking changes as context. If the treated pages climb in both organic rank and citation rate, you have more than one possible mechanism to investigate. If rank barely moves but clickable citations do, the citation measure still captures the outcome you set out to test. Do not let a vendor swap in a ranking chart when the agreed outcome was citations.
Set a decision rule without calling it proof
I would set a demanding bar before opening the results: a sustained 15-percentage-point lead for treatment over control across Weeks 3–6 warrants a wider rollout and another check. If control averages 12% and treatment 15%, that three-point gap is not enough for me to buy a claimed optimization effect. If control sits at 10% and treatment at 28%, I would take the result seriously.
Those are decision rules, not universal noise thresholds. Citation drift alone cannot tell you that every gap below 10 points is noise or that every gap above 15 points is caused by your edit. Starting differences, recrawl timing, query selection, and uneven results across pairs still matter. Record the rule now so a vendor cannot define success after seeing the chart.
Read the result: what changes next?
Treatment wins: repeat the edit before buying the story
If treatment holds a clear lead over control through Weeks 3–6, the answer-first block has passed a useful first test. It may have made a relevant passage easier for Google to select. You still cannot see inside retrieval and declare that mechanism proven.
Extend the same opening structure to the 10 control pages and keep measuring after they are recrawled. If their citation rates also rise, your case for repeating the edit gets stronger. If they do not, investigate the mismatch before rolling the change across the catalog. Only then test a second variable, such as table-formatted specifications or structured entity schema. One variable at a time keeps the next result interpretable.
No clear difference: do not pay for an unproven rewrite
If treatment and control finish close together, say 13% versus 15%, the proposed edit has not earned its place in this test. That does not prove every recommendation in the software is useless, or that Google assigned identical weights to both versions. It means the vendor cannot point to this run as evidence that these edits improved clickable citations.
If the product’s paid value proposition was precisely that rewrite, ask for a different, testable intervention or stop paying for optimization that has not shown an effect. A dashboard may still describe what happened. Description is not the improvement you were sold.
Control wins: stop and inspect the damage
If untouched pages consistently beat the edited pages, do not double down on brevity because an audit scored it highly. Inspect the openings. Did the rewrite replace a useful explanation with repetitive keywords or strip out detail a reader needed? Those are possible explanations, not a diagnosis you can make from citation rates alone.
Revert the treatment openings if the gap persists and the edits have made the pages worse. Keep logging long enough to see what happens after the revert. The control group has done its job: it stopped you from spreading an unhelpful change across the site.
Make the vendor agree to the test
Put the holdout in the trial conversation
When a sales rep says their platform gets pages into AI Overviews, offer the protocol rather than accepting a read-only demo as proof. State it plainly: 10 matched treatment pages, 10 untouched controls, locked buyer queries, six weeks of clickable-citation checks, and an agreed decision rule. Ask whether their recommended edit can be tested under those conditions before you sign an annual contract.
If you want the option to walk away when the edit fails, negotiate that term before the trial starts. Do not assume a vendor will accept zero-cost cancellation after six weeks. A refusal to test does not prove the product fails, but it tells you what evidence you will and will not get for your money.
I have little patience for paying a subscription to receive page edits your team must implement, then being handed an appearance chart with no holdout. That is the old advisory model with a newer dashboard. At groas, we take a different position: specialized models execute page-level optimizations, technical crawl fixes, and search intent work continuously, with a named senior strategist accountable for direction. Rather than asking clients to interpret audit scores and edit CMS fields, groas puts its visibility promise in writing: 40% more visibility across Google and AI search within 90 days or we work for free. I would still measure what changes. Accountability should make a control group easier to welcome, not harder.
Skip the test if you do not have enough pages
This protocol is not for a site with fewer than 20 indexable deep pages. I would also skip it if the site pulls under 1,000 monthly impressions in Google Search Console. With only three or four suitable articles, one URL moving in or out of a citation set can dominate the result. Do not buy diagnostic software to inspect an empty room.
Build a deeper library of pages that answer specific buyer questions first. If you already have the pages, lock the queries and run the holdout. If treatment wins, repeat the change. If it does not, keep your budget. Either way, the control group tells you more than a vendor’s best screenshot.