AI visibility tools report your brand’s share of answers across a sample of prompts the vendor picked, so the sample decides the score as much as your content does. When I tested one against my own tracking, I got 0.6% from the tool and 38% from my spreadsheet. Before you pay for one, ask the vendor who wrote the prompts behind the score.
AI visibility tools are the newest product that vendors are selling to every marketing team, but most people paying for one can’t say how the tool works out its score. That’s mostly because the vendor keeps the prompts behind the score in its own system and nobody asks to see them. At the company I work for now, Amplitude’s AI visibility report gave us a 0.6% baseline. For my own first baseline, I tracked 20 buyer prompts across five AI tools, and I wrote each prompt the way a buyer in our category would type it. For the same brand, 38% of the answers to those prompts named us and 37% cited us (the answer linked to our website).
What does an AI visibility score actually measure?
An AI visibility score is the share of AI answers that mention your brand, across a set of prompts an AI visibility tool sends to AI tools like ChatGPT or Google AI Overviews on a schedule. Share of voice is the competitive version: your mentions divided by all brand mentions in those answers, so it only exists once you’ve added competitors. Whatever the vendor calls the score, the bottom of the fraction is a prompt list the vendor usually picked. Dan Taylor, head of technical SEO (search engine optimisation) at SALT.agency, wrote in Search Engine Land in 2026:
“The problem is that these metrics rely on a hidden denominator.”
The prompt list behind most scores is smaller than the vendors’ dashboards suggest. The AI visibility vendor Profound says in its own prompt design guide (2026) that its users generally track 100 to 1,000 prompts, and a couple of hundred is typical. Entry plans go much lower, and Digiday reported in May 2026 that Ahrefs’ lite plan includes five tracked prompts. A few hundred prompts is a sample of what buyers ask, and the score only tells you how your brand does on those prompts.
Why do AI visibility tools give such different scores?
AI visibility tools give different scores because a brand can only show up in answers to prompts someone thought to write. Amplitude’s AI visibility FAQ (2026) says its prompts don’t come from actual users, because the AI companies don’t share what people search. Instead, Amplitude asks AI tools like ChatGPT to generate prompts that may interest marketers. The same FAQ says the report covers ChatGPT and Google AI Overviews. So Amplitude’s 0.6% and my 38% came from different prompts in different AI tools, two for Amplitude and five for my tracking.
I wrote my own buyer prompts on purpose, because when I started in my current role, I wanted measurement right before I worked on any content, since every later decision would rely on it. Our site analytics had already shown me what happens when nobody checks how the site analytics work: they miscounted 43% of site traffic before I rebuilt them (the rest of that work is on my results page). A generic prompt like “best tools for user research” is fair to measure, but a niche brand that sells to other businesses will rarely appear in its answers. If most of a score’s prompts are generic, the score tells you more about the category than about your content.
The scoring rule, meaning how an AI visibility tool counts each answer, changes the score too. Digital Applied’s AI share of voice framework (June 2026) gives a worked example where one brand scores 20% on mentions. On the same data, the brand scores 16.8% when the framework weights each mention by the brand’s position in the AI answer’s list of brands, and 31.4% when it counts citations instead. That’s why I report named and cited as separate figures. You can read how I track AI citations for the setup.
Why does the same brand score differently in two AI visibility tools in the same week?
The same brand scores differently in two AI visibility tools because each vendor makes its own method choices, and the dashboard shows none of them. Paul Dyer, CEO of the agency /prompt, told Digiday in 2026 that if you give three AI visibility tools the same prompts, “you get three different answers”.
| Method choice | What varies between vendors | What it does to the score |
|---|---|---|
| How many prompts does the AI visibility tool track? | Each plan includes a different prompt count. | A short prompt list’s score moves a lot when one answer changes. |
| Which AI tools does the AI visibility tool cover? | Some vendors check one or two, and others check seven or more. | If a brand is strong in one AI tool and missing from another, the score hides that by averaging the two. |
| How many times does each prompt run? | Each prompt runs anywhere from once to 100 times. | One run records one version of an answer that changes every time. |
| Does the AI visibility tool use the developer API (a way for software to query an AI tool directly) or the public app? | The vendor picks one, and the public app is what a buyer sees. | The API and the app can return different answers to the same prompt. |
How much of the change in a score is random?
A lot of the change is random, because AI answers vary even when the prompt and the model stay the same and nothing on your site changes. In a SparkToro and Gumshoe study from January 2026, about 600 volunteers ran 12 prompts through three AI tools (ChatGPT, Claude and Google’s AI) 2,961 times. The volunteers had less than a 1 in 100 chance of getting the same list of brands twice, and less than 1 in 1,000 of getting the list in the same order. In the SparkToro write-up, Rand Fishkin concluded that how often a brand appears across many runs is a reasonable metric, while AI visibility tools that report a ranking position in AI answers are “full of baloney”.
An Ahrefs study of 43,000 keywords (November 2025) found the content of Google AI Overviews changed every 2.15 days on average, and between one check and the next, 45.5% of the cited sources were new. Fixed settings don’t stop the answer changing either: in 2025, Thinking Machines Lab sent 1,000 identical requests at temperature zero, the setting that should make a model give the same answer every time, and got 80 different answers back.
Model updates change answers too: OpenAI rolled back a GPT-4o update in ChatGPT in April 2025 about a week after it launched, so anyone tracking ChatGPT that week measured a model that was gone days later. A five-point drop could come from your content, a competitor’s, a model update or chance, and a score that doesn’t come with its run count can’t tell you which.
What does an AI visibility measurement you can explain look like?
A measurement you can explain is one where you can read the prompts and rerun any of them, so you can show someone who doesn’t trust the score where any change in it came from. My measurement starts with a short list of buyer prompts I write myself, and my prompt research process covers that work in full. The list stays fixed between periods, so a change comes from the answers and not the prompts.
I run each prompt more than once in every AI tool. I also check the score against a signal that doesn’t come from AI tools at all, like searches for our brand name or what buyers say on sales calls, which is the two-signal rule I use to report AI visibility. For the visits that AI tools do send to the site, I use the AI traffic setup in GA4 (Google Analytics 4). My whole measurement setup is part of my AEO/LLMO framework, where answer engine optimisation (AEO) means structuring content so AI tools can pull it into answers and cite it, and LLMO stands for large language model optimisation.
What should you ask AI visibility tool vendors on a demo call?
The first question comes before the vendor shows you the dashboard: ask to see the full prompt list behind your score and who wrote it. If an AI tool generates the prompts, ask whether you can swap in your own.
The second question is about sampling: how many times does each prompt run in each AI tool, and what’s the margin of error? A vendor that runs each prompt once and reports a share to one decimal place is claiming more precision than the method can give.
Third, ask what happens to your past scores when the vendor changes its prompts or its scoring rule, and when an AI company updates a model. You want to hear that the vendor compares old and new periods only on the prompts they share, or at least marks the date in your past scores where something changed. The first option is what I do when I add a prompt to my own list.
The last question is easy to test during a trial: can you export every raw answer with its prompt and AI tool, plus the date? If you can’t, nobody on your team can check the score you report to leadership.
When are AI visibility tools worth paying for?
AI visibility tools are worth paying for when the work is too big to do by hand and the vendor shows you the prompts and method behind the score. Twenty prompts across five AI tools with a few runs each is a few hundred answers a month, which one person with a spreadsheet can handle. Several hundred prompts in four languages across six markets is a hell of a lot of copying and pasting, so paying for a tool makes sense there. Tracking lots of competitors, and getting an alert when a model update changes your answers overnight, are worth paying for too.
At any scale, an AI visibility tool still has to pass the four demo questions above. An agency president who spoke to Digiday for the same piece saw these tools as benchmarking, and I agree: they’re a way to watch a trend on a prompt list you already trust. If you’re still deciding where your AI search budget goes, read how I’d fund SEO and AEO/LLMO first. The content metrics I report to leadership show where an AI visibility score belongs in a report.
What I’d do differently
In my first baseline, I reported where our brand appeared in AI answers’ lists on average (position 2.3) alongside the share of answers that cited us, with no warning next to it. The SparkToro and Gumshoe study shows how random the order of AI lists is, so I’d still report position, but with a note that the position changes every time a prompt runs. That way nobody reads a move from 2.3 to 2.8 as a result.
I’d also have exported Amplitude’s prompt list on day one and added my 20 buyer prompts to it, since the tool lets you add and edit prompts. The 0.6% and the 38% would then have come from at least some of the same prompts, and I could have explained the gap to leadership in one sentence instead of reporting two separate scores.
Got a demo booked with one of these vendors? Tell me what they said about the prompt list at [email protected].
Frequently asked questions
AI share of voice is your brand’s mentions divided by all brand mentions in AI answers to a set of prompts, so it needs a list of competitors you name. Two AI visibility tools can report different shares for the same brand. Check the prompt list and the scoring rule before you compare your AI share of voice with anyone else’s.
Twenty good buyer prompts are enough for a first AI visibility baseline if each prompt runs more than once in every AI tool you track, though bigger sets make sense across several markets. Keep the list fixed between periods so changes come from the answers and not from new prompts.
An AI visibility score can change with no change on your site, because AI answers vary between runs and vendors add or remove prompts (AI companies also update their models). In 2026, SparkToro and Gumshoe found under a 1 in 100 chance of AI tools returning the same brand list twice. Check whether the prompt list or the model changed before you treat a move as a result.
You can track AI visibility without a paid tool by running a fixed list of buyer prompts through each AI tool by hand and writing down whether each answer names or cites your brand, plus which AI tool and which run each answer came from. A spreadsheet is enough for a small list, and paid tools make sense once the work spreads across several markets and languages and gets too big for one person.
Prices for AI visibility tools start at free and go up to custom enterprise deals, and prompt count mostly sets the price. Digiday reported in May 2026 that Ahrefs’ lite plan, at 129 US dollars a month, includes five tracked prompts, while dedicated vendors’ starter plans begin around 99 US dollars a month for about 50 prompts. Amplitude includes its AI visibility report in every plan and offers a limited free version.
AI rank position is where your brand appears in an AI answer’s list, and it doesn’t belong in a headline report because the order changes almost every time the same prompt runs. SparkToro and Gumshoe found under a 1 in 1,000 chance of getting the same list in the same order in their 2026 study. How often your brand appears across many runs is a much steadier metric.
