How to choose AI search prompts to track. And the method I use.

On October 20 I asked ChatGPT, Perplexity and Claude the 10 questions I've chosen to measure my business by. Nine of them don't mention my company. Some ask who to hire ("What's the best AI marketing agency for a B2B SaaS company?"), some ask how to do the work ("How do I build an AI content system for a small B2B marketing team?").

SuperMarketers was named in 0 of 27 answers. It was named in all 3 answers to the tenth question, the one with my company's name in it.

That's an honest baseline. It's also a useless weekly number. Getting onto an AI shortlist takes months, and a score that sits at zero until it doesn't can't tell me whether this week's work did anything.

So before I track more questions, I wanted to know how to choose them. I read what the tracking vendors tell their customers, what independent practitioners publish, and the new research from 2026. Then I checked every quote on the live page. This guide is what they agree on, where they don't, where good questions come from, and the method I built from it.

The short answer: track groups of real buyer questions, not single prompts. Keep questions that name your brand separate from ones that don't. Run each question several times and read trends over two to four weeks. Then split your questions by the job each number does, so the ones that move quickly show weekly progress while the slow ones show whether you're winning.

What the research agrees on

These eight points come up in at least three independent sources each. Where a source sells AI tracking, I say so.

1. A single answer is close to meaningless

Rand Fishkin and Patrick O'Donnell had about 600 volunteers run 12 prompts through ChatGPT, Claude and Google's AI, 2,961 times. Their finding: there's "<1 in 100 chance" that ChatGPT or Google's AI gives you the same list of brands twice in 100 tries.

A 2026 study of 12,933 AI answers by Andrzej Żatuchin found that "a single AI answer carries almost no brand-discriminating signal" (arXiv, Jul 2026). The fix is to report how often you appear across many runs, never a rank. Ethan Smith of Graphite puts it simply: "You can measure AI with probability distributions" (Lead with AI, Apr 2026).

2. Run each question more than once, and read trends over weeks

How many runs is the most contested number in this field (more below), but nobody says once is enough for a single question. The most specific guidance comes from a 2026 study of ChatGPT, Gemini, Google AI Mode and Perplexity:

Profound, which sells tracking, ran the same test at scale and found once a day is enough when you pool hundreds of prompts, with a warning that applies to everyone: "You can't measure your way past drift." The platforms change their models constantly (Profound, Jul 2026).

3. Judge groups of questions, not single questions

Ahrefs' advice, in its own words: don't "focus on the results of individual prompts"; "group similar prompts and analyze commonalities in aggregate results" (Ahrefs). Aleyda Solis frames her "Presence KPIs" the same way, by topic and journey stage (AI Search Optimization Checklist).

4. Keep branded and unbranded questions apart

They measure different things. Scrunch's help center: "Non-branded questions tell you where AI places you relative to competitors. Branded questions tell you how accurately AI describes you when you are directly asked for" (Scrunch). Blend them and the branded questions, which you almost always win, flatter the score. Alex Birkett of Omniscient compares one blended number to "managing a patient by their 'average lab score'" (Omniscient, Jul 2026).

5. Use the words real buyers use

AI prompts are long. Kevin Indig: "The average AI prompt is ~5x longer than classic search keywords (23 words vs. 4.2 words)" (Search Engine Journal, Feb 2026). The best source of that wording is already in your company:

Ethan Smith's version: "go where the conversation actually happens," meaning Reddit, Slack, support and sales calls. HubSpot's warning about the alternative: "Automated generation can produce grammatically valid prompts no real buyer would ask" (HubSpot).

6. Freeze the list, and version every change

If you reword questions every few weeks, you can't tell a change in AI from a change in your list. Profound's own docs: "If you need stable period-over-period comparisons, keep the prompt set consistent for the reporting window" (Profound).

7. Treat visibility as an early signal, and check it against outcomes

Wil Reynolds has an old SEO story that applies here. Back in 2014 he got Seer ranked in the top three for "SEO agency" and its variants:

Visible to the wrong searchers is still invisible to buyers. Birkett's fix is three layers: a diagnostic metric (how often you're cited), leading metrics (AI referral traffic, branded search) and lagging ones (self-reported attribution, pipeline).

8. Know what you're measuring: the app or the API

OpenAI's help center explains that ChatGPT rewrites your question into its own search queries, and can use your location and saved memories when it does (OpenAI). Tracking tools see a logged-out or API version of that. Surfer compared 1,000 prompts through the API with what users see in the app and found the brands named overlapped by only 15% to 32%:

Either can be useful. Just name which one you're measuring, and don't compare one with the other.

Where they disagree

The disagreements are as useful as the agreement. Each one is a decision you'll have to make.

The run counts look contradictory but fit together once you separate two levels. A trend across hundreds of questions needs few runs per question. A single question, or a small topic, needs many.

Wording matters more than it should. Peec analyzed 37,804 answers and found list-style prompts surface about 20% more brands than open-ended ones, and short keyword-style prompts up to 25% more:

On prompt volume, Conductor's critique is the clearest. Because AI prompts are long and personal, "MSV of any prompt = 1" (Conductor, Jul 2026). Ahrefs describes its own AI volume weighting as "a modeling choice, not a measured relationship" (Ahrefs). Use volume to compare topics, not to rank questions.

Where good questions come from

Ranked by how close each source is to what buyers actually type, and how little noise it carries. No study has validated this order yet; it's the practitioners' reasoning, and mine.

Two things about Search Console are worth knowing, because most guides miss them.

It now has a separate AI report, with limits. Since August 31, 2026, Search Console shows how often your pages appeared in AI Overviews and AI Mode. It shows pages, countries, devices and dates. It has no queries and no clicks:

Some of your "searches" are robots. When I sorted the 652 searches that found my site over 90 days, 82 were written by AI tracking tools testing their own prompts, with giveaways like "provide a definitive answer" and "I am a 25-34 year old." Every one of my 8 clicks came from people searching my name. 60% of my impressions came from people shopping for software, which I don't sell. (The full breakdown is here.) I sort the queries with simple word rules, which agreed with my own hand labels on 73% of a 60-query sample. A classification model is next.

The Shortlist Ladder: the method I use

Every source above tells you how to pick good questions. None tells you how to make the results show progress you can act on every week. That's the gap this method fills.

The idea: sort every question by the job its number does. Some questions measure whether AI describes you correctly. Some measure whether it trusts your pages on a topic. Some measure whether it recommends you. Those move at very different speeds, so they get separate lists, separate scores and separate charts. Never one blended number.

The lower rungs aren't vanity numbers. AI cites the sources it trusts on a topic, and the brands it trusts are the ones it eventually recommends. Rung 2 is where that trust gets built, one page at a time, and it's where weekly progress becomes visible.

Step 1: Write one sentence that sets the boundary

"This panel tracks how [buyer] researches [problem or purchase] in [market and language]."

Mine: "This panel tracks how lean B2B SaaS marketing teams research AI search visibility and AI implementation, and how they choose outside help, in English." Anything outside that sentence doesn't make the list, however easy it would be to win.

Step 2: Listen first, in the order above

Start with your own conversations and Search Console. Record where every question came from: a buyer's own words, a search, or your best guess. A real customer question and a guess shouldn't look equally solid in the spreadsheet.

Step 3: Score every candidate out of 8

Four tests, 0 to 2 points each:

Shortlist 6 to 8. Review 4 or 5. Drop the rest. The score compares candidates; it doesn't prove demand.

Step 4: Put each question on one rung, and tag it

One rung each, plus five tags so you can slice the results later: topic (tied to a service line), buyer stage, format (list, open question or short keyword, because format alone shifts results), answer page (exists, planned or missing) and source (from Step 2).

Then balance. Aim for 8 to 15 questions per topic, and cover more than "best X": fit, problems, alternatives, comparisons, cost and implementation are all different decisions. Fifty versions of "best agency" teach you one thing.

Step 5: Freeze it and give every question an ID

Rewording a question makes it a new question with a new ID. The old one keeps its history. If the list changes, the chart shows a visible break instead of a fake trend.

Step 6: Measure each rung its own way

• Runs: at least 5 per question per engine, read as a rolling four-week rate. More for the rungs you report publicly.
• Every rate shows its math: numerator, denominator, engine, and the number of runs that failed.
• Name the surface: ChatGPT's app, its API, Perplexity, Google AI Mode and AI Overviews are different things. Label each and never mix them.
• Score the rung's own thing: facts on rung 1, citations on rung 2, mentions on rung 3, recommendations on rung 4. A good explanation that names no agency isn't a failure on a cost question.

Step 7: Tie every shipped page to its questions

When you publish or rebuild a page, add 3 to 5 rung 2 questions it should answer. Log the change with its date and reason, and read it 28 days later. A page that starts getting cited has earned its place; one that doesn't tells you what to fix. Once a question has been read, it joins the main rung 2 list.

Step 8: Review monthly, version quarterly

Each month, promote the best new questions from Listen and note what moved and why. Each quarter, or when the offer changes, cut a new version.

My panel, as the worked example

Here's the part I got wrong first. My original 10 questions, frozen on October 18, looked like one list. Sorted by job, they're four:

So "0 of 27" blended three jobs that should never share a score. Here's the full panel, rebuilt on the ladder. Every number is a starting point, not a result.

Rung 2 is the one to watch. My 40 questions come from three places: 13 are searches people already made that found my site ("why isn't my saas showing up in chatgpt" had 26 impressions), 13 come from my question log, and 14 are the questions my recent posts and Lab experiments set out to answer. Two had no page at all: "What's a realistic 90-day plan for implementing AI in a B2B SaaS marketing team?" and "What data and system access does an AI marketing partner need?" Those are my next two pages.

On rung 4, the agencies AI named instead of me were NoGood and Omniscient Digital (8 of 27 answers each), Kalungi (7) and Single Grain (6). That tells me which pages to study: what they published that AI trusts on these questions.

Everything runs in the Lab, and every change I ship is in the Shipped log with its 28-day read.

The template

Every question gets one row:

Download the blank template (CSV), or my 40 rung 2 questions as a filled-in example.

Questions

How many prompts should I track for AI visibility?

Enough to give every topic 8 to 15 questions, split by job. For most B2B SaaS companies that's 30 to 100 to start. Sources range from 15 to over 1,000, so pick the number your team will actually act on.

How many times should I run each prompt?

At least 5 per engine, read over two to four weeks. One study recommends 7 to 8 runs a day for stable per-brand numbers. One run is a sample of one.

Should I track branded prompts?

Yes, but in their own list. They tell you whether AI describes you correctly, not whether buyers discover you.

Where do I find prompts to track?

Your sales calls and support tickets first, then long question-shaped searches in Search Console, then reviews and communities. Generate prompts with AI only to fill gaps, and label them as guesses.

Is prompt volume real?

Only roughly. AI companies don't share query data, so every volume figure is modeled from a panel or from Google data. Use it to compare topics, not single questions.

Why do my AI visibility numbers jump around?

Because AI answers change on almost every run. Read rates across many runs and weeks, and treat a single screenshot as an anecdote.

Sources

• SparkToro and Gumshoe, AI recommendation consistency, Jan 2026
• Schulte, Bleeker, Kaufmann, "Don't Measure Once," Apr 2026
• Żatuchin, "Where Does the Noise Come From?", Jul 2026
• Profound, "Is once a day enough?", Jul 2026
• Ahrefs, custom prompt tracking and Brand Radar methodology
• Aleyda Solis, AI Search Optimization Checklist
• Scrunch prompt strategy guide
• Alex Birkett, Omniscient, Jul 2026
• Kevin Indig, synthetic personas, Feb 2026
• Omnia, prompts to track, Jul 2026
• Ethan Smith interview, Apr 2026
• HubSpot, AEO prompt tracking
• Profound prompt designer docs
• seoClarity, choosing prompts
• Wil Reynolds, Seer, Mar 2026
• OpenAI, searching the web with ChatGPT
• Surfer, API vs app answers, Oct 2026
• Peec AI, prompt style research, 2026
• Conductor, AI prompt volume, Jul 2026
• Google, generative AI performance reports, Jun 2026