How AI Visibility Tools Work and How Complete Their Data Is

They run your tracked prompts on each AI engine, usually once a day per country, so their data is a sample of what engines say, not what buyers saw.

Share
How AI Visibility Tools Work and How Complete Their Data Is

AI visibility tools run your tracked prompts on each AI engine, usually once a day per country, and record the answers, so their data is a sample of what engines say, never what real buyers saw.

AI visibility is how often AI engines such as ChatGPT, Gemini and Google AI Overviews name your brand when buyers ask the questions you care about. Tools measure it as visibility %: the share of answers to your visibility prompts (buyer questions that name no brand) that mention you. How that number is built decides how far you can trust it. For the whole method, from prompts to reporting, see how to measure AI search visibility.

How do AI visibility tools collect their data?

An AI visibility tool does three things:

  1. A prompt set. You, or the tool, choose the buyer prompts to track. Everything after this depends on that list. How to pick it is its own topic: choosing which prompts to track, and so is how many prompts to track.
  2. Runs. The tool sends each prompt to each engine, in each country you track, on a schedule. The usual cadence is one run a day per prompt, engine and country.
  3. Counting. It reads each stored answer for brand names and for citations. A citation is a page an AI answer shows as a source: your own page, a third-party page (a competitor's page, a listicle, etc.) or a community page (a Reddit thread, a forum post, etc.).
A four-step flow from the prompt set to runs once a day per engine and country, then stored answers, then counted mentions and citations, showing that the result is a sample and a floor, not a total.

A tool only sees the answers to the prompts you track. It can only count what it collected. If it shows 12 citations, your pages were cited at least 12 times. Treat every count as a floor, because your pages can also be cited on prompts you do not track.

Tools get their answers in one of two ways. Some capture the answer from the engine's app, the way a person sees it in a browser. Others call the engine's API, the interface built for developers. The two can give different answers.

Do AI engine API answers match what users see in the app?

Not always. API answers can differ from what the app shows, so answers captured from the app are closer to what buyers see. Treat API data as another data point (at best), not as the record of what engines show people.

Does web search change an AI engine's API answer?

Yes. In OpenAI's API, web search is a tool you turn on, and the model then decides whether to search (OpenAI docs, checked November 2026). Without that tool, the model cannot search the web.

Gemini's API works the same way. It searches Google only when the google_search tool is on, and then the model decides whether to search (Google docs, checked November 2026).

ChatGPT itself cites pages on almost every buyer prompt. Data from the Zadoosh data platform, March to October 2026, B2B brands across software and other fields: ChatGPT cited at least one page in 98% of answers to tracked prompts. An API run without web search therefore measures something else: what the model remembers, not what buyers are shown.

Do login, memory and country change an AI engine's answer?

Who asks changes the answer. Memory, login and past chats shape what one person sees, and so does the country. In ChatGPT, for example, saved memories can shape the search it runs, and the country comes from your IP address (OpenAI help, checked November 2026). A tool that asks as an anonymous user with no memory, from your buyers' country, shows your chances with a buyer who does not know you yet.

To compare a tool's answer with the app by hand, match the tool's conditions:

  • Open a fresh chat for each prompt, with one prompt per chat.
  • Log out, or use a temporary chat with memory off.
  • Use a VPN set to the country you track.
  • Run the prompt a few times, because one answer proves little.

Do Gemini, Perplexity, Claude and Grok API answers also differ from their apps?

Gemini's and Perplexity's can. Perplexity says so itself: its FAQ says results from its API, now called the Agent API, can differ from its web app (checked November 2026). Gemini's API answers can also differ from the Gemini app.

For Claude and Grok, we have no tested comparison of API and app answers, so we will not guess the size of the gap. The same hand check works on any engine: match the conditions above and compare a few answers with what your tool recorded.

Why do AI visibility numbers move between runs?

An AI engine writes each answer fresh, so two runs of one prompt rarely match. We looked at ChatGPT answers in data from the Zadoosh data platform, April to October 2026, B2B brands across software and other fields. Two runs of the same prompt on the same day shared 53% of the brands named, 31% of the cited sites and 22% of the cited pages. Brands are steadier than citations, but both move.

Real buyers likely see even more variety. Answers to different accounts overlapped less than reruns from one account. That is why a tool reads brands and citations as rates over many runs, never from one answer.

What can AI visibility data tell you, and what are its limits?

AI visibility is a real metric, but a probabilistic one. It sits between performance marketing and brand marketing. Expecting an exact dollar value for each point is the performance-marketing mistake. Calling it unmeasurable is the brand-marketing mistake.

How close the number gets depends on the runs behind it. We looked at data from the Zadoosh data platform, April to October 2026, B2B brands across software and other fields. We took the ChatGPT windows where a brand is named in some answers but not all, which is most of them. For those, this is how close a few random runs came to the brand's 30-day share:

Runs of the prompt Within 10 points of the 30-day share Typical error (median, points)
1 35% 18
3 42% 12
5 49% 11
10 64% 8

One run is often far off. At ten runs the typical error is about 8 points, which is a rough read. Each extra run helps, but some error always stays. That is why you read a trend over weeks, not a single day.

What the data can tell you depends on which of two jobs you are doing:

  • Demand. Your visibility % on visibility prompts, where no brand is named, and your share against competitors on the same prompts. You also see whether it rises or falls over weeks, and which pages engines cite in those answers, yours and other sites'.
  • Positioning. What engines say about you on branded prompts and on "vs" prompts against a competitor, and the words they use. Keep these prompts out of the visibility number.

What it cannot tell you, and why:

  • What a real buyer saw. No tool, Zadoosh included, has access to a buyer's conversation.
  • How often a prompt is asked. Only the engines have real counts, so prompt volume figures are estimates, most of them mapped from Google keyword data.
  • How much revenue one mention brings. A probabilistic metric has no fixed dollar value per point.

Is AI visibility a vanity metric?

Not if you read it for what it measures. The visibility number shows demand, not whether the answer describes you well. A mention with the wrong words does not drive demand, because visitors arrive expecting something else. That is why the positioning check matters as much as the visibility number.

Why does past AI visibility data change?

Metrics change retroactively for four common causes, and each one needs a different response.

A competitor or a name spelling was added. Engines write brand names in more than one way, such as a split-word or camel-case form. When you add a competitor, or a new spelling of one, a good tool recounts the answers it already stored, so past weeks change. A tool that does not recount leaves the new competitor at zero before the edit, and their share then looks like a sudden rise.

Prompts were added. New prompts change what the number is made of. They do not start at zero: they start at whatever your visibility is on them, so the overall number moves with how you do on the new prompts. Report two numbers: one on all prompts, and one like-for-like on the original set.

Keep adding relevant prompts whenever you find them, and remove one only if it tracked the wrong thing. The like-for-like number keeps your trend comparable while the set grows.

The engine or country mix changed. If one month covers three engines and the next covers four, the two months are not directly comparable. Mark any period whose engines or countries changed before you read a trend across it.

The capture broke. Engines change their apps without notice, and a tool's capture can break silently. A drop on one engine only, starting on one day, is a reason to spot-check the raw answers before you report a decline.

How long until a new prompt gives a usable number?

Give it one to two weeks of daily runs before you read the number. After that, look at the trend month by month. You do not need to run a prompt several times a day; once a day is enough.

Gaps cannot be filled later. An engine does not show what it answered last week, so a week without runs stays a hole in your history. Start tracking before you need the trend, and do not let it lapse.

Collect daily, but act only on a clear change. We looked at weekly ChatGPT visibility in data from the Zadoosh data platform, March to October 2026, B2B brands across software and other fields. The median weekly move was about 3 points, and most of it was noise. A weekly change was likely real from about 11 points on a typical prompt set, and from 7 points with 100 to 299 answers a week.

Over a month, a move of 3 to 6 points was already real in that same data. So with 100 or more answers a week, act on a weekly change of about ten points or more, and read smaller moves as a trend over about a month. With fewer than 100 answers a week, a weekly change cannot be told from noise: read the monthly trend only.

How do you check an AI visibility tool's numbers?

Before you trust a tool, spot-check a few prompts by hand, logged out, against what it recorded. Use only tools that show the raw answer text behind every number. For when to trust the numbers at all, see can you trust AI visibility data. Then ask the vendor these questions:

  • Prompts. Is the full prompt list behind the score open to you, and can you add to it?
  • Engines and country. Which engines and countries are tracked? For each engine, is the answer taken from the app or from the API? Does it ask as an anonymous user with no memory?
  • Runs. How many times a day does each prompt run on each engine, in each country?
  • Mentions. What counts as a mention? Does a split-word or camel-case spelling of a brand count?
  • Recounts. When you add a competitor or a spelling, are past answers recounted?

When two tools disagree, the answers to these questions usually explain why. Pick one tool and read its trend, rather than mixing the two.

Frequently asked questions

Why do more buyers mention ChatGPT than my tool's numbers suggest? Your tool only sees the prompts you track. Buyers ask many others, with their own history. So treat the tool's numbers as a floor. To see what reaches buyers, ask "How did you hear about us?" on forms and sales calls.

What is a good AI visibility score? There is no universal good number. Compare your share with your top 3 competitors on the same prompts, and look for a share that rises over about 3 months.

How often should a tool run each prompt? Once a day per prompt, engine and country. Read the result as a trend over weeks: about ten or more daily runs give a monthly read. Act on a weekly change only when it is large and the prompt set gives 100 or more answers a week; with fewer, read the monthly trend only.

Can I check my AI visibility by hand? Yes, for a first sense. Use a fresh chat, logged out, with a VPN in your buyers' country, and run each prompt a few times. A hand check shows roughly where you stand, but tracking needs a tool that shows the raw answers. For daily tracking, the Zadoosh AEO platform's free trial (7 days, 25 prompts, no card, as of November 2026) runs your prompts for you.