How it works
An answer is assembled from pages. We publish the pages.
Search-enabled assistants retrieve web pages before composing some answers. We observe part of that process through the network sites with edge logging, recording requests by user-agent and hour. Here is what those logs can show, and what they cannot.
Best date night restaurant in Fitzroy?
A few well-reviewed options in Fitzroy for a date night:
- 1Osteria Della Rosa. Candlelit, set menu from $85, walk-ins after 9pm.
- 2Brunswick Street Wine Room. Small plates and 200 wines by the glass.
- 3Kōri. Omakase for two, usually booked a week ahead.
Sources
- Dining directory, date night in Fitzroy
- Menu record, prices and hours
- Venue website
Example answer. Venue names are illustrative.
The sequence
Question, retrieval, answer.
- 1
A person asks
A question with a place and a need in it. Best mechanic in Adelaide. A group dinner in Sydney for twelve. The place matters: the engine will look for sources that name it.
- 2
The engine retrieves
Three routes, often together. A search index. A live fetch of a handful of pages while the person waits, by a named answer-time agent. And what the model already learned from its training crawl. The first two are where a page published today can act.
- 3
It writes, and cites
A search-enabled answer may name businesses and cite retrieved pages. The number of recommendations and source selection vary by system, query and context. Being present in a retrieved page does not guarantee inclusion.
What we record
Different agents. Different kinds of evidence.
16 named AI agents do the reading. The answer-time agents are the ones that matter most: they open a page in the seconds after a person asks.
697,475
Fetches by AI systems
30-day window to 21 September 2026
137,570
Fetches by Google and Bing
Same window
5.1:1
AI reads to search reads
7,812
Fetched by an answer-time agent
ChatGPT-User, DuckAssistBot, Claude-User, Perplexity-User
The readers
What the main agents do.
Each engine publishes which agents it sends and what each is for. Search access and training access are separate decisions, and the answer-time fetchers are a third thing again. We allow the search and answer-time agents on every directory, deliberately.
OpenAI
- OAI-SearchBot
- Indexes pages for ChatGPT search. OpenAI’s documentation: sites opted out of it "will not be shown in ChatGPT search answers".
- ChatGPT-User
- Fetches a page while a person is asking. "Not used for crawling the web in an automatic fashion." This is the answer-time reader.
- GPTBot
- Reads for training. Separate from search: a site can allow OAI-SearchBot and refuse GPTBot.
Anthropic
- Claude-SearchBot
- "Navigates the web to improve search result quality." Blocking it "may reduce your site’s visibility and accuracy in user search results".
- Claude-User
- Fetches "when individuals ask questions to Claude". The answer-time reader.
- ClaudeBot
- Reads for training.
Perplexity
- PerplexityBot
- Built "to surface and link websites in search results on Perplexity". Not for training.
- Perplexity-User
- Visits a page to answer a question and "include a link to the page in its response".
Microsoft and Google
- Bingbot
- Bing’s index, which ChatGPT search and Copilot draw on. Search systems may use several providers; Bing inclusion is not a universal requirement.
- Googlebot, Google-Extended
- Google AI Overviews and AI Mode are built on the core search index. Google-Extended governs uses in Gemini training and grounding, separately from Google Search.
Meta, Amazon and others
- Meta-ExternalAgent, Amazonbot, Bytespider
- Meta AI, Alexa and ByteDance’s assistants. Large readers of our pages, and named in the logs.
- DuckAssistBot, YouBot
- Smaller assistants. Present, and counted.
Agent roles are documented by their providers. Search eligibility does not guarantee placement. See OpenAI search documentation and Anthropic crawler documentation.
What gets a business named
Specific, current, traceable.
A specific service in a specific place
Roof plumbing in Geelong, not "all your roofing needs". The engine is matching a question that names a place and a need; a page that names both gives it relevant context.
Facts it can repeat
A licence number, a rating with a count behind it, an opening hour, a price band. An answer is built from facts. Specific facts give the assistant more to check than promotional adjectives.
No contradictions between sources
The engines read Google, Bing, Yelp and the directories side by side. Conflicting contact details can confuse people and search systems. The program works to align relevant sources with your checked business record.
A request is a request. It is not a person, a citation or a ranking. We measure citations separately with a fixed monthly panel, and never add the two together. 697,475 AI fetches recorded in the thirty days to 21 September.