---
title: "Thirty Days of Bot Logs: The Changelog ChatGPT Kept Fetching"
description: "I checked 30 days of groas.com edge logs. ChatGPT-User fetched a dated Google Ads changelog 398 times; a busier-looking AI Overviews post was mostly unverified bot traffic."
url: "https://groas.com/post/thirty-days-of-bot-logs-the-page-chatgpt"
image: "https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/24617c1b-e14c-47f1-bef2-dbe544cb66d0.png"
published: "2026-10-21T05:29:40.103Z"
modified: "2026-10-21T05:29:40.166Z"
---

October 21, 2026 · 9 min read

# Thirty Days of Bot Logs: The Changelog ChatGPT Kept Fetching

[Alexander PerelmanHead Of Product @ groas](https://groas.com/author/alexander-perelman)[LinkedIn](https://www.linkedin.com/in/alexander-433793253/)

![A tiny model town at night where couriers ignore glossy showrooms and queue at a humble kiosk of dated slips, the plain reference page bots actually fetch.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/24617c1b-e14c-47f1-bef2-dbe544cb66d0.png)

In this article

1. [Scene 1: The first line was /robots.txt](#scene-1-the-first-line-was-robotstxt)
2. [Scene 2: The homepage had 753 live-answer fetches](#scene-2-the-homepage-had-753-live-answer-fetches)
3. [Scene 3: The changelog nobody wrote for bots](#scene-3-the-changelog-nobody-wrote-for-bots)
4. [Scene 4: The popular post lost 387 hits](#scene-4-the-popular-post-lost-387-hits)
5. [Interlude: Test which pages a live-answer bot actually fetches](#interlude-test-which-pages-a-live-answer-bot-actually-fetches)
6. [Scene 5: The product pages stayed quiet](#scene-5-the-product-pages-stayed-quiet)
7. [Closing scene: The page I would write next](#closing-scene-the-page-i-would-write-next)

At 11:42 p.m., I had 30 days of groas.com edge logs open in a terminal, sorted by path. The top line was not a page. It was `/robots.txt`, hit thousands of times by bots checking the rules. Farther down, a dated Google Ads changelog had drawn 398 verified ChatGPT-User fetches. A post about AI Overviews looked busy too, until I checked who had actually requested it.

The product pages were barely in the conversation.

## Scene 1: The first line was `/robots.txt`

The month’s log file held 15,436 hits with an AI bot name in the user agent. After I checked the requesting IPs against vendor lists, 4,125 hits remained verified. The rest claimed a bot identity they could not back up. **A bot name in a log is a claim, not an identity.** User-agent strings are easy to spoof; the useful count starts when you [verify them against vendor IP lists](https://ooky.ai/blog/how-to-read-ai-crawler-logs).

Claude-User accounted for much of the `/robots.txt` traffic. That file tells crawlers which parts of a site they may access, so its place at the top was not surprising. It was also a warning about the rest of the screen: before treating a request as evidence of anything, I had to know which bot made it and what that bot was there to do.

OpenAI [separates three jobs](https://developers.openai.com/api/docs/bots/). GPTBot crawls for training future models. OAI-SearchBot helps surface sites in search results. ChatGPT-User fetches pages in response to user activity. Those requests can all arrive in the same log and wear similar-looking names, but they do not answer the same question. The [Ooky log guide](https://ooky.ai/blog/how-to-read-ai-crawler-logs) makes the distinction useful: training crawls, search discovery, and user-triggered fetches belong in different columns.

I stopped reading the file as one big traffic chart.

## Scene 2: The homepage had 753 live-answer fetches

ChatGPT-User fetched the homepage 753 times during those 30 days. Because [ChatGPT-User responds to user activity](https://developers.openai.com/api/docs/bots/), those verified requests were more interesting to me than an indexer’s routine crawl. They showed the homepage being fetched in the course of live answers. **They did not show me the prompts, prove that a sentence was cited, or tell me anyone clicked through.** The logs show requests, not the finished answer on somebody else’s screen.

Still, I could inspect the page the bot chose to fetch. The homepage is short on poetry and heavy on claims a reader can check: what groas runs, who it runs it for, the flat fee, and the named strategist. It gives an answer system something more usable than a polished line about growth. It gives it a description of the business.

That mattered when I reached the next path. The homepage has the obvious job of explaining groas. The changelog does not sell groas at all. It answers a narrower question with a date attached.

## Scene 3: The changelog nobody wrote for bots

Nobody on our side wrote the Google Ads updates changelog to win citations. It is a dated running list: an update, what changed, and when. In the same 30 days, it drew 651 verified bot fetches. ChatGPT-User accounted for 398 of them. That is a little more than half the homepage’s 753 ChatGPT-User fetches, and far more than any product page received.

**The reference-shaped page kept getting pulled into live-answer workflows.** I cannot tell from an edge log which answers used it or how they described it. I can tell that a user-triggered fetcher returned to that path hundreds of times.

I used to tell clients to put their best thinking into product pages. I was wrong to treat those pages as the only place the useful thinking belonged. A product page has to argue for the product. A changelog can answer _what changed?_ without asking the reader to buy anything. Dates and plain descriptions travel well outside their original layout: a model can retrieve an entry without untangling a pitch first.

The entries were not clever. I kept scrolling.

## Scene 4: The popular post lost 387 hits

The AI Overviews post looked like the winner if I counted every request that claimed an AI identity. Raw count: 405. Verified after the IP check: 18. Verified ChatGPT-User, Claude-User, or Perplexity-User fetches in that slice: zero.

Say you are spending $20k a month and someone reports all 405 hits as evidence that answer engines are using your post. The raw number is real. **The conclusion does not follow from it.** Most of those hits failed identity verification, and none of the verified hits came from the live-answer fetchers in this check.

The mechanism is dull, which is why it is easy to miss. A user-agent string is text a requesting program supplies about itself. A script can put “ChatGPT” in that field without being ChatGPT. [Checking the IP against the vendor’s published ranges](https://ooky.ai/blog/how-to-read-ai-crawler-logs) is the step that stops a costume from becoming a KPI. I could not see who was behind each unverified request, so I was not going to give those requests a story about readers, citations, or demand.

The post may still do its job for human readers. These logs just did not show live-answer fetches for it. By the time I reached that zero, the question I wanted to ask of every path had changed.

## Interlude: Test which pages a live-answer bot actually fetches

**Question:** Once you remove requests that fail identity verification, which pages do user-triggered answer bots fetch? Run this prospectively on your own site rather than borrowing my ranking. You need raw edge or origin logs with IP address, user agent, timestamp, and path. An analytics dashboard will not give you the same view. Nor should you treat a fetch as a visit: [Cloudflare’s crawl-to-refer work](https://blog.cloudflare.com/ai-search-crawl-refer-ratio-on-radar/) reports ratios from 118:1 to nearly 50,000:1 pages crawled per referred visit for AI crawlers.

1. **Set the window.** Pull 30 days of requests. Filter for GPTBot, OAI-SearchBot, ChatGPT-User, Claude-User, ClaudeBot, PerplexityBot, Perplexity-User, Applebot, and Google-Extended in the user agent. Keep the unfiltered total so you can see what the later checks remove.
2. **Verify identity.** Compare each matching request’s IP with the relevant vendor’s published ranges. OpenAI publishes [separate IP files for its bots](https://developers.openai.com/api/docs/bots/); check the corresponding lists for the other vendors too. Set aside requests whose names and IPs do not agree. Do not call those verified bot traffic just because a dashboard grouped them that way.
3. **Keep the jobs separate.** Put ChatGPT-User, Claude-User, and Perplexity-User in a user-triggered group. Track training crawlers and search indexers in separate groups. This prevents a training crawl from inflating your live-answer column. It also prevents the opposite mistake: overlooking pages that indexers can find even if they have few user-triggered fetches today.
4. **Rank paths, then read them.** Sort by verified user-triggered fetches. Put the raw named-bot count beside each path so the gap stays visible. Check which high-ranking pages give a date, a plain definition, or a direct answer, and which merely make a pitch. Before changing crawler access, read the vendors’ guidance: OpenAI notes that [ChatGPT-User is not automatic crawling and robots.txt rules may not apply to it](https://developers.openai.com/api/docs/bots/), while [Perplexity distinguishes Perplexity-User from PerplexityBot](https://nqz.ai/blog/geo-how-to-get-cited-by-perplexity).

My expectation is that a useful reference page will outrun a clever opinion post in the user-triggered column, because it is easier to retrieve for a specific question. Your logs may disagree. If they do, inspect the pages at the top before commissioning another changelog. If your busiest-looking page collapses after verification, fix the measurement before you pay for more content like it. **Either result changes what you write next.**

## Scene 5: The product pages stayed quiet

Back in the groas logs, the product pages sat there like furniture. Verified bots visited them, but ChatGPT-User almost never fetched one during a live answer. Perplexity-User barely touched them either. After 398 ChatGPT-User fetches for the changelog, that stung a little. We had rewritten those pages, tightened them, and added proof. The bot kept returning to the page we had not rewritten for it.

I do not think that makes product pages useless. They still have to explain the service to a buyer who reaches them. But their job is different. A product page argues, compares, and promises. A dated update can supply a narrow fact without requiring an answer system to adopt a seller’s opinion. That is the mechanism I can see in the page shapes; the logs alone cannot tell me what any model decided to say.

There is a useful limit here for groas, too. A plain description such as “groas runs paid and organic search for a flat monthly fee with a named strategist” is easier to inspect than “we are the best answer to search marketing.” The first says what the service offers. The second asks a stranger to take our side. If I want a sentence to stand on its own, I write it so somebody skeptical could repeat it.

That does not start with better adjectives. It starts with the [site plumbing](https://groas.com/post/before-you-write-another-page-for-ai-the), then with a page that answers a real question.

## Closing scene: The page I would write next

If I were starting with these logs tomorrow, I would write the boring page first. For each Google Ads change: a date, what changed, and who it affects. The changelog already has that shape. It gives a reader looking for an update a place to check, and it gives a fetcher a specific path to retrieve. That is a better reason to make another entry than a promise that any model will cite it.

I would also run the [access check](https://groas.com/post/can-ai-bots-actually-read-your-site-a-60) before commissioning ten more opinion posts. Can the relevant bots reach the reference page? Do the [robots rules and page blocks](https://groas.com/post/the-ai-crawler-swipe-file-copy-paste-rob) reflect which crawlers we intend to allow? The answer matters more than another round of edits to a paragraph no bot can fetch.

What I would not do is report the AI Overviews post’s 405 named-bot hits as AI visibility. Eighteen survived verification, and none of those were live-answer fetches in this window. The post can still persuade a person who reads it. It has not earned a live-answer success story from these logs. Nor would I turn the changelog’s 398 fetches into 398 citations. That would be the same mistake with a nicer number.

At 12:14 a.m., I closed the terminal window. I had come in expecting to compare pages and left with two columns I did not want anyone merging again: _verified user-triggered fetches_ and everything else.

The first line I saw had been `/robots.txt`. Below it, the homepage had 753 ChatGPT-User fetches. The dated changelog had 398. The opinion post had zero verified live-answer fetches. If I were paying someone to improve my presence in AI answers, I would want to see those columns before I saw their bot-traffic chart. I know how easy the bigger chart is to put in a report.

The log file was still open behind the terminal window. When I brought it back, the changelog’s path was still there: a date, a plain update, and 398 requests beside it.

![A server log printout with verified fetches circled and unverified requests crossed out](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/1834d7b8-0046-4bab-a1ad-7387972e796e.png)

![A dated changelog on a monitor beside product pages](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/02a7e550-5c51-4537-9e45-7533b857fb3c.png)

## Frequently Asked Questions

### How can I trust the identity of bots listed in a server log?

Bot names in a log are only a claim, not a verified identity. To verify them, you must check each request's IP address against the vendor’s published IP ranges. This process stops unverified requests from being used as a metric for performance.

### What does a high number of ChatGPT-User fetches on a homepage indicate?

High ChatGPT-User fetches show that the page is being accessed during live answers. However, these logs only show the requests; they do not reveal the prompts, whether a sentence was cited, or if a user clicked through.

### How did the dated Google Ads changelog perform in AI fetches?

The changelog drew 651 verified bot fetches in 30 days, with 398 coming specifically from ChatGPT-User. This shows that a reference page can be pulled into live-answer workflows even if it does not contain a sales pitch.

### Why should I not treat a raw bot hit count as evidence of AI visibility?

A raw count can be misleading because many requests may fail identity verification. The published logs showed that a popular AI Overviews post with 405 raw hits had zero verified fetches from live-answer fetchers.

### What is the functional difference between a product page and a reference page for AI models?

Product pages must argue, compare, and promise the service. In contrast, a changelog provides a narrow fact or dated update without requiring the AI system to adopt a seller’s opinion.

### How should I accurately measure live-answer bot traffic on my site?

To measure accurately, pull raw edge or origin logs for 30 days and verify each request’s IP address against the relevant vendor's published ranges. Separating user-triggered fetches from training crawls is also critical for accurate reporting.

## Pay For Results, Not For Hours

Businesses buy the outcome, agencies resell it, and groas answers for it either way.

[See If You Qualify](https://groas.typeform.com/to/xC1bQNUT)

## Structured data

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://graph.groas.com/entity/groas","name":"groas","url":"https://groas.com/","logo":{"@type":"ImageObject","url":"https://groas.com/icon-512.png","width":512,"height":512},"sameAs":["https://www.linkedin.com/company/groas/"]},{"@type":"WebSite","@id":"https://groas.com/#website","name":"groas","url":"https://groas.com/","inLanguage":"en-US","publisher":{"@id":"https://graph.groas.com/entity/groas"}},{"@type":"WebPage","@id":"https://groas.com/post/thirty-days-of-bot-logs-the-page-chatgpt#webpage","url":"https://groas.com/post/thirty-days-of-bot-logs-the-page-chatgpt","name":"Thirty Days of Bot Logs: The Changelog ChatGPT Kept Fetching","description":"I checked 30 days of groas.com edge logs. ChatGPT-User fetched a dated Google Ads changelog 398 times; a busier-looking AI Overviews post was mostly unverified bot traffic.","inLanguage":"en-US","isPartOf":{"@id":"https://groas.com/#website"},"about":{"@id":"https://graph.groas.com/entity/groas"},"primaryImageOfPage":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/24617c1b-e14c-47f1-bef2-dbe544cb66d0.png"},"breadcrumb":{"@id":"https://groas.com/post/thirty-days-of-bot-logs-the-page-chatgpt#breadcrumb"},"dateModified":"2026-10-21T05:29:40.166Z","mainEntity":{"@id":"https://groas.com/post/thirty-days-of-bot-logs-the-page-chatgpt#article"}},{"@type":"BreadcrumbList","@id":"https://groas.com/post/thirty-days-of-bot-logs-the-page-chatgpt#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://groas.com/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https://groas.com/blog"},{"@type":"ListItem","position":3,"name":"Thirty Days of Bot Logs: The Changelog ChatGPT Kept Fetching","item":"https://groas.com/post/thirty-days-of-bot-logs-the-page-chatgpt"}]},{"@type":"BlogPosting","@id":"https://groas.com/post/thirty-days-of-bot-logs-the-page-chatgpt#article","url":"https://groas.com/post/thirty-days-of-bot-logs-the-page-chatgpt","mainEntityOfPage":{"@id":"https://groas.com/post/thirty-days-of-bot-logs-the-page-chatgpt#webpage"},"isPartOf":{"@id":"https://groas.com/blog#blog"},"headline":"Thirty Days of Bot Logs: The Changelog ChatGPT Kept Fetching","description":"I checked 30 days of groas.com edge logs. ChatGPT-User fetched a dated Google Ads changelog 398 times; a busier-looking AI Overviews post was mostly unverified bot traffic.","datePublished":"2026-10-21T05:29:40.103Z","dateModified":"2026-10-21T05:29:40.166Z","image":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/24617c1b-e14c-47f1-bef2-dbe544cb66d0.png"},"author":{"@id":"https://groas.com/author/alexander-perelman#person"},"publisher":{"@id":"https://graph.groas.com/entity/groas"},"keywords":"bot, logs, changelog, fetches, chatgpt-user, page, pages, days","wordCount":1839,"inLanguage":"en-US"},{"@type":"Person","@id":"https://groas.com/author/alexander-perelman#person","name":"Alexander Perelman","jobTitle":"Head Of Product @ groas","description":"My background includes Goldman Sachs and Stanford Computer Science.","image":"https://groas.com/media/blog/4f17cac3a81acc224cc1b2cfab8f94bbc295a3089d13e8421125771ef8d4064f.jpg","worksFor":{"@id":"https://graph.groas.com/entity/groas"},"sameAs":["https://www.linkedin.com/in/alexander-433793253/"]},{"@type":"FAQPage","@id":"https://groas.com/post/thirty-days-of-bot-logs-the-page-chatgpt#faq","mainEntity":[{"@type":"Question","name":"How can I trust the identity of bots listed in a server log?","acceptedAnswer":{"@type":"Answer","text":"Bot names in a log are only a claim, not a verified identity. To verify them, you must check each request's IP address against the vendor’s published IP ranges. This process stops unverified requests from being used as a metric for performance."}},{"@type":"Question","name":"What does a high number of ChatGPT-User fetches on a homepage indicate?","acceptedAnswer":{"@type":"Answer","text":"High ChatGPT-User fetches show that the page is being accessed during live answers. However, these logs only show the requests; they do not reveal the prompts, whether a sentence was cited, or if a user clicked through."}},{"@type":"Question","name":"How did the dated Google Ads changelog perform in AI fetches?","acceptedAnswer":{"@type":"Answer","text":"The changelog drew 651 verified bot fetches in 30 days, with 398 coming specifically from ChatGPT-User. This shows that a reference page can be pulled into live-answer workflows even if it does not contain a sales pitch."}},{"@type":"Question","name":"Why should I not treat a raw bot hit count as evidence of AI visibility?","acceptedAnswer":{"@type":"Answer","text":"A raw count can be misleading because many requests may fail identity verification. The published logs showed that a popular AI Overviews post with 405 raw hits had zero verified fetches from live-answer fetchers."}},{"@type":"Question","name":"What is the functional difference between a product page and a reference page for AI models?","acceptedAnswer":{"@type":"Answer","text":"Product pages must argue, compare, and promise the service. In contrast, a changelog provides a narrow fact or dated update without requiring the AI system to adopt a seller’s opinion."}},{"@type":"Question","name":"How should I accurately measure live-answer bot traffic on my site?","acceptedAnswer":{"@type":"Answer","text":"To measure accurately, pull raw edge or origin logs for 30 days and verify each request’s IP address against the relevant vendor's published ranges. Separating user-triggered fetches from training crawls is also critical for accurate reporting."}}]}]}
```