Most AI-visibility tools pass the demo and fail the trial. The demo shows monitoring: share-of-voice charts, citation counts, prompts where competitors appear and you do not. The trial should show execution: a changed page, a shipped fix, an earned mention, and an action log that ties each one to a prompt. I used to tell clients to judge software by coverage and price. I was wrong. Judge it by what got fixed while you were still allowed to cancel. Run these 18 yes/no checks during a 14–30-day trial, with one page the tool can change and one comparable page it cannot. A cleaner dashboard is not a result.

Before day one: lock the baseline

  1. Record 10 buyer prompts and their citation status before the tool starts. Choose informational and how-to prompts you want to win, then snapshot which ones trigger AI Overviews and who is cited. Check each prompt in 3–4 separate sessions before calling its status consistent. In a document the vendor cannot edit, record the trigger, cited domains, and whether your brand is cited with a link, mentioned without one, or absent. Keep these 10 prompts fixed throughout the trial. Write down the exact wording, too; a small rewrite makes a later check harder to compare. Otherwise, a report can quietly swap in easier prompts and call that progress.

  2. Choose a treated page and a comparable control page before either changes. Match them as closely as you can on clicks, impressions, age, and seasonality; assign relevant prompts to each. The tool may work on the treated page. The control stays untouched for the same period. This is not a random split of your site: a useful comparison changes one group while a comparable group stays unchanged. Save the pages’ starting versions outside the tool. Record who can publish to each page so an accidental control edit does not pass unnoticed. You will need those starting versions when a dashboard says something was fixed but the page looks suspiciously familiar.

Gap discovery: demand a prompt, a rival, and a page

  1. Get misses at prompt level, not a visibility score. For each of your 10 prompts, the tool should show cited, mentioned without a link, or absent, plus who appears instead. A score of 62 cannot tell you what to do on Monday. If the trial opens with a beautiful chart but cannot return your prompt list, fail this check. That is monitoring, not an assignment an editor or developer can act on.

  2. Make every miss point to a URL on your site. One prompt needs one candidate page and an owner for the proposed work, not a vague instruction to “improve topical authority” across the domain. If you lose “best commercial HVAC maintenance plan,” the tool should identify the service page meant to answer it. Record that pairing before edits begin; do not let the report pick a different winning page afterward. When it cannot name the page, you cannot inspect the proposed fix or later tell whether the right page won. No URL, no actionable gap.

  3. Ask why that page lost: content, structure, or authority. Content means the direct answer is missing; structure means an answer exists but is hard to extract; authority means competing sources have stronger third-party support. The distinction decides what to test next. Tools can track mentions and share of voice without rewriting content, building schema, or landing PR. That is why the monitoring-versus-execution gap matters more than another score. If the diagnosis ends at “visibility opportunity,” fail it.

Technical execution: inspect the site, not the task list

  1. Find a shipped change on the treated page that your team did not make. Export the issue list early in the trial, then compare it with the page and its source on day 14. Look for a corrected canonical, an added internal link, a revised title, or another fix the tool claimed it would make. Check the specific URL named in the issue, not merely the account-wide count of closed tasks. If the dashboard lists 47 issues and your developer has to resolve every one, you bought an audit. Crawlability, indexability, and rendering still matter for AI visibility; normal SEO applies to AI Overviews. Use this technical SEO fix checklist to inspect the work.

  2. Find each shipped fix in an action log with a URL and timestamp. The entry should say what changed, when it changed, and why that change addressed the named prompt. A recommendation, a pending ticket, and a published edit are three different things. Ask for the log by day 7; if there is none, do not wait for a week-two chart to explain what supposedly happened. Match its entries to the page itself rather than accepting a batch of completed-looking status labels. I reject before-and-after claims when I cannot connect the changed page to a dated action.

  3. Check crawler access and the rendered answer, not just robots.txt. Ask for the verdict for each relevant AI crawler and a fetch check on the pages you want cited. OAI-SearchBot matters for ChatGPT Search; GPTBot serves a different, training-related purpose. CDN and WAF protections can block AI crawlers even when robots.txt allows them. Confirm that key answers appear in accessible HTML rather than only in JS-dependent blocks, accordions, or image tables. An llms.txt file is not a substitute for that work: Google has said it will not use one for AI Overviews.

Content execution: open the published page

  1. Confirm the treated page was rewritten or a new page went live. Check the CMS revision history and the public URL. A draft inside the tool is not a shipped edit, and a brief sent to your team is not autonomous execution. Compare the published page with the saved baseline: what answer changed, which prompt was it meant to address, and when did it go live? Keep the revision alongside the action log so you can check the claimed timing later. If the words are identical to day one, mark this no even if the tool produced an impressive content plan.

  2. Look for a direct answer under the target heading. The proposed pattern is an answer-first, 40–60-word passage beneath a question-based heading: the first sentence answers the question; the rest supplies enough context to stand alone. Check the actual page, not the generated copy preview. If a reader must pass three paragraphs of preamble before reaching the answer, the rewrite has not done this job. Make the answer easy to find and lift.

  3. Verify that schema agrees with visible copy and entity names stay consistent. Check relevant FAQPage, Article, or Organization markup against what a visitor sees. A price, founder name, or service area in code should not contradict the page. Use the same entity name across the treated page and its markup. Conflicting details make the result harder to interpret: if citations do not move, you cannot tell whether the problem was the answer, the markup, or the contradiction you shipped with both.

Off-site execution: an opportunity is not an earned mention

  1. Open the URL of something earned or corrected off-site during the trial. It might be a live brand mention, a corrected directory profile, or a fixed review page. A prospect list does not count. Ask when it went live and whether the tool did the work it claims; a URL alone does not establish either. One study of 75,000 brands reported a stronger correlation between brand mentions and AI Overview visibility than between backlinks and that visibility. Correlation is not your trial result, but it is a reason to ask what happened beyond your site. If nothing happened, record that plainly.

  2. Compare your name, offer, and location across the pages that matter. Inspect your site, relevant listings, and review pages for conflicting versions. Inconsistent names make it harder to establish that those references point to the same business. The check is not whether a dashboard has discovered discrepancies; it is whether the trial corrected any it promised to correct and logged the URLs. If the diagnosis was authority, a prettier answer block alone has not addressed it.

Reporting: refuse blended scores and invented revenue

  1. Require prompt-level reporting by engine, not one combined line. Confirm in writing which engines your trial login actually covers before judging its charts. Trial tiers differ: the published comparison lists Otterly Lite at $29 a month with a seven-day no-card trial; the draft pricing for Profound, Peec, Scrunch, and AthenaHQ spans higher monthly tiers and different coverage. Check the current terms rather than assuming a demo feature is in your trial. For engines included, ask for separate weekly status for ChatGPT, Perplexity, Gemini, and AI Overviews. Keep the unavailable engines marked as unavailable, not as misses. A blended score hides where a prompt moved.

  2. Make the report label proxies as proxies, not revenue. CTR changes, branded-search lift, and referrals from AI domains can be useful directional signals. They do not establish that a particular AI Overview produced a particular sale. GSC does not separately segment AI Overview impressions and clicks, and GA4 has no distinct AIO referrer. If the report assigns “$X revenue from AI Overviews” without a click path it can show you, fail this check. A tool that cannot state what it cannot see should not get to price its guess as attribution.

Days 14–30: test the treated page against the control

  1. Rerun the same prompts, sessions, and engine breakdown against both pages. Question: did the tool’s edits coincide with a citation gain that the untouched comparison did not see? Setup: ship the logged changes on the treated page and leave the matched control alone. Controls: keep the original prompt wording and the same status definitions; record any control-page change rather than pretending the comparison still holds. Measure: on days 14 and 30, check the original prompts in 3–4 sessions per engine and record cited, mentioned, or absent as you did on day one. Count prompts that changed status, not just impressions. Use this control-group test protocol if you need the longer procedure.

  2. Judge coverage over repeated checks, then decide what the result changes. My expectation is narrow: a direct answer, matching schema, and a clean fetch give the treated page a better chance of being lifted and attributed than an unchanged page. That is an expectation, not a promised citation. Extend the same baseline method to 20–50 priority queries if you can measure them consistently through four or more weeks; a single-day win is a thin scorecard. If treated improves and control does not, inspect the log before paying. If both stay flat despite shipped fixes, investigate authority rather than declaring extraction solved. If both improve, do not credit the treated-page edit without further scrutiny. If checks 6, 9, and 12 all fail, stop shopping for a better dashboard. groas is built around shipping pages and fixes, then reporting what moved; run it through the same test.

  3. Keep the control page untouched, even when changing it feels productive. This is the check teams skip: they let the tool touch everything, then call a site-wide lift proof of its work. It is not. Before-and-after alone cannot separate an edit from demand, seasonality, or an engine update. Say you pay $399 a month without that comparison. Twelve months later, you have $4,788 in invoices and no clean way to tell whether the tool earned them. Skip the control page and you buy noise with a renewal attached.