Measuring AI Search for Agency Clients

Start with the decision, not the dashboard

An agency should define what the client needs to decide before selecting metrics. A publisher asking whether its research is cited has a different problem from a local business asking whether AI-assisted discovery produces calls. The report should name the business question, the search surface, the audience, and the observation window before it displays a score.

• Write one measurable question for each report section.
• Name the provider or search surface instead of using AI search as a single channel.
• Separate discovery, representation, referral, and conversion questions.
• Record important limits where the client can see them.

Use a measurement ladder

The most useful model moves from technical eligibility to observed visibility and then to business outcomes. A page can be crawlable but never cited. A brand can be mentioned without a link. A visible citation can produce no visit. A small number of visits can still assist valuable conversions. Each rung answers a different question.

• Eligibility: crawl, index, snippet, and access checks.
• Visibility: mentions, citations, cited URLs, and answer position when observable.
• Engagement: referral sessions, landing pages, and meaningful on-site actions.
• Outcome: qualified leads, sales, assisted conversions, and retained clients.

Combine first-party and repeat-test evidence

Google and Bing now provide first-party reporting for some generative search experiences. Other providers may require referral analytics, server logs, or controlled prompt probes. Use official platform data when it exists, then use repeatable third-party observations to answer narrower questions that the platform does not expose.

• Keep Search Console and Bing Webmaster Tools data distinct from synthetic probes.
• Use a fixed prompt set and schedule for comparisons over time.
• Store the answer, citations, provider, date, model label, and test conditions.
• Do not describe sampled prompts as total market demand.

Report changes without inventing causation

An increase in citations after a content update is useful evidence, but it is not proof that the update caused the increase. Models, indexes, competitors, prompt interpretation, and provider interfaces can change at the same time. A trustworthy report marks direct observations, reasonable inferences, and unknowns separately.

• Use observed, inferred, and unknown labels in commentary.
• Compare more than one period before declaring a trend.
• Retain negative and unchanged results instead of showing only wins.
• Tie recommendations to evidence that a client or reviewer can inspect.

Give the client a short work list

The report should end with decisions rather than an ornamental pile of charts. Prioritize actions that also improve ordinary search and user experience: resolve access problems, strengthen weak evidence, clarify services, correct entity details, and build pages that answer real buying questions.

• Assign an owner and expected review date to each action.
• Separate quick corrections from strategy experiments.
• Do not promise a citation, ranking, or model response.
• Retest the same evidence set after meaningful implementation.

Official references

• Google: Optimizing for generative AI features: Google explains RAG, query fan-out, measurement, agentic experiences, and why ordinary SEO fundamentals still apply.
• Bing: AI Performance in Webmaster Tools: Bing defines citations, cited pages, grounding queries, and trend data while explaining what those measurements do not prove.
• OpenAI: Publishers and developers FAQ: OpenAI explains OAI-SearchBot access, ChatGPT referral tracking, model-training controls, and agent accessibility.

FAQ

Related guides

• Read Google's Generative AI Search Report
• Read Bing AI Performance Without Guessing
• Query Fan-Out: What Agencies Need to Know
• AI Citations, Traffic, and Conversions
• How to read AI Citation Monitor results
• Answer Readiness and AI search