Skip to content
Primetra
Back to this week

Method appendix

How your numbers are made.

The detail behind your page: the panel, the minimum samples, the 95% range, the noise band, the verify windows, what each finding means and every method change. Use it to check our work.

Sample appendix for Relaykit, a fictional company. The rules and numbers here are the real ones; the panel shown is the Monitoring plan's.

Monitoring · method found-v2 · used-v4 · fix-v1

Your panel

Every week we ask the assistants your buyer questions through their official APIs, with web search on (Gemini is the one exception, see below). This is the arithmetic behind each n.

Buyer questions
30, written for your category. None of them names a brand.
Assistants
5: Claude, ChatGPT, Gemini, Grok and Perplexity.
Gemini
Gemini searches the web through Google Search, which has its own terms for how its results may be shown, kept and analysed. We run this lane only with those terms covered (Google's permission or our counsel's sign-off). Google's Search Suggestions appear next to each Gemini answer on the Answers page. If this ever changes, Gemini's history starts a new line and the numbers are marked as not comparable across the change.
Meta Model API
Meta Model API (Muse Spark + web search): A stand-in for what Meta's assistants may say: AI-generated answers from Meta's developer API with web search, not what the Meta AI app or Muse shows. Shown on its own line, not counted in Share of Agent. Its answers are not in Share of Agent, questions won or the answer counts here. Switching it on or off starts a new line for it and never changes Share of Agent.
Samples
Each question goes to each assistant 4 times every week.
Answers
Up to 30 × 5 × 4 = 600 a week. This is the n behind Share of Agent.
Per assistant
Up to 30 × 4 = 120 answers: the n behind that assistant's cited, read and named rates.
Per question
Up to 5 × 4 = 20 answers.
Question won
Cited in at least 50% of that question's answers: 10 of 20 when every answer comes back.
Competitors
Up to 8, measured on the same answers as you.
Free check, for scale
5 questions × 4 assistants, asked once: 20 answers. Directional only.

A timeout is a lost answer. It leaves n and never counts as “not cited”, so n can sit below the maximum in a week when an assistant had trouble.

Minimum samples

Below a metric's minimum we hide the number and show “too few to read”. The n still shows, so you can see how close it is.

The 95% range

Every rate uses the Wilson score interval at 95% (z = 1.96). The true value is very likely inside it. With k answers citing you out of n:

p̂ = k / n
centre = (p̂ + z²/2n) / (1 + z²/n)
half   = z · √( p̂(1 − p̂)/n + z²/4n² ) / (1 + z²/n)
range  = centre ± half, kept inside 0–100%

Unlike the textbook p̂ ± 1.96·√(p̂(1 − p̂)/n), it never runs below 0% or above 100% and it stays honest at small n. Your JSON and CSV exports carry the value, n, ci_lo and ci_hi of every rate, so you can recompute any of them (k = value × n).

Same rate, more answersSample data

n = 600 answers · 123 cite you · 20.5%

95% 17.5–23.9% · ±3.2 pts

Sample data: Relaykit's Share of Agent, held near 20.5%. With 20 answers the range runs 8%–42%. With 600 it's 17%–24%.

When a change counts

We call a move only when both weeks clear the metric's minimum sample and the two 95% ranges don't overlap. Ranges that touch count as overlapping.

Inside the band no finding fires, nothing is alerted and the memo doesn't call it a change. The number still shows on your page, with its range.

Error and empty spikes compare the day with the median of recent days, and check whether one caller or many caused the jump.

Verify windows

The window starts when you mark a fix done. When it closes, we re-check the number the fix should move, on its own measurement: the readiness check itself, the bot's own refusals, your agents' own calls.

Verify window
Kind of fix

Example: Unblock OAI-SearchBot. Window: days 2–7.

  1. Doneday 0You ship the fix and mark it done.
  2. Window opensday 2The earliest the number can move.
  3. Re-checkday 7We compare before and after.
Then, at the re-check
  • VerifiedMoved the right way, outside the noise band.
  • Not yetNo clear move yet. We keep watching, up to twice the window.
  • Too few to readToo few samples to judge, no reading to compare with, or the method changed while we watched.
  • Didn't workMoved the wrong way, or still flat after twice the window.
A bot fix shows up in the next crawls, so we judge it within days. Citations move slowly, so we wait weeks before we call it.
Re-check by fix
  • Citation lostCitation fix21 days
  • Read but not citedCitation fix28 days
  • Competitor cited insteadCitation fix28 days
  • AI search bot blockedBot or MCP fix7 days
  • Connects, never callsBot or MCP fix7 days
  • Error or empty spikeBot or MCP fix3 days
  • Unmet asksBot or MCP fix7 days
  • Spoofed bot trafficBot or MCP fix7 days
  • Readiness check failedReadiness fix3 days

What each finding means

Findings come from fixed rules over your data. No model writes or ranks them. Each one carries a label:

Verified
Backed by data we observed, like an error your MCP server returned.
Inferred
A conclusion drawn from a pattern. It says so, and it ranks lower.
Suppressed
Inside the noise band or below the minimum sample. Held back and never shown to you.
How fixes are ranked

A fixed rule, never a model: severity × confidence ÷ effort. Ties go to the fix we can verify sooner, then the more severe one.

Severity
info0
low1
med3
high5
Confidence
verified1
inferred0.6
Effort (divides)
Minutes1
Under an hour2
Half a day4
A few days8

Bot verification

A bot is verified when its IP sits in the vendor's published ranges or it carries a valid Web Bot Auth signature. One passing check is enough.

Spoofed means it claims a vendor's name and fails the vendor's own check. Unverifiable means the vendor publishes nothing to check against, so we keep the request, label it, and never guess.

Bot verificationSample requests
Example request
  1. Request
    User agent
    Googlebot/2.1
    IP
    66.249.66.1
    Claims to be Google.
  2. IP rangesIs the IP in Google’s published ranges?In range
  3. Web Bot AuthA valid signature from Google’s published keys?None sent
  4. OutcomeVerifiedCounted as Googlebot. One passing check is enough.
Illustrative requests. A user agent is a string anyone can send, so a vendor name only counts when the IP or a Web Bot Auth signature backs it up.

We keep the full IP only for requests that claim a known vendor bot, and only until the check is done. Every other IP is truncated and hashed on your side.

Noise we remove

We remove these before we count, and count what we removed. Your export lists each one under its key, so the totals add up.

Method versions

Every metric carries the version of the method that produced it. When we change a rule, a threshold, a bot table or a prompt, we bump the version and mark the break on your charts instead of drawing a trend across it.

Currentfound-v2used-v4fix-v1

On your page: found-v2 · used-v4 · fix-v1

Changelog
  • 2026-10-21 · found-v1 → found-v2A question now counts as won only when you are cited in at least half of its samples (v1 counted any single sample). Share of Agent pools every answer across assistants. Earlier question counts are not comparable.
  • 2026-10-21 · used-v1 → used-v2Two things changed in how we read agent traffic, so some numbers can step on this date without any change in your real traffic. (1) Bot recognition: Google's Google-Agent and Google-GeminiNotebook (the new name for NotebookLM) are now recognised; OpenAI's ad checker, Google-Agent and Amazon's three bots are each checked against their own published address list, so their verified counts can rise; and each Google bot is now checked only against the Google list for its own kind, so a request that borrowed another kind's Google address can now read as spoofed or unverifiable. Expect the verified, unverifiable and spoofed counts to move. (2) MCP connects: MCP servers on the 2026-07-28 spec no longer have a connect handshake, so a connect now also counts the first request from a client after it has been quiet for 30 minutes. Connect counts and the connected-but-never-called list can rise for those clients. Earlier values are not comparable, and the new numbers settle over about a week, because the look-back windows still contain events classified under the earlier version.
  • 2026-10-27 · used-v2 → used-v3We now recognise AI agents from one maintained list of agents, each fact checked against the vendor's own page. Some numbers can step on this date without any change in your real traffic. New agents are counted (Google's renamed Gemini Notebook fetcher, Kimi's fetchers, Amazon's Nova Act, Grok Bot, and Meta's and Bing's newer crawlers); names that only appear on third-party lists are no longer counted; robots.txt switches such as Google-Extended are no longer counted as visitors; and a few agents we cannot check by address (Grok Bot, Project Mariner's old name) are never marked as fake. Visits from an AI assistant now also count when the link carries ChatGPT's own tag (utm_source=chatgpt.com), and Microsoft's newer Copilot addresses count too. Agents that sign their requests are now checked against their own published keys: one we cannot tie to a company is counted as a visit from a "signed agent" of its website, never as that company's. Earlier values are not comparable, and the new numbers settle over about a week. Later additions approved by our team are recorded on each number (the agent-list revision), not as a new method.
  • 2026-10-28 · used-v3 → used-v4Every app that calls your MCP server now gets a trust level: verified, backed by a second sign, name only, or fake. Only verified and backed callers count as agents that use you; name-only callers are still listed, with their label. ChatGPT and Claude calls are now checked against the addresses OpenAI and Anthropic publish, and against the sign-in when your server shares it with us. An app that only tells us its name, even with a matching user agent, is no longer counted: this includes most desktop coding tools (Claude Code, Cursor, VS Code) unless their sign-in is shared with us. Claude's products (claude.ai, Cowork, Claude Code, Claude Design) are shown separately, about twenty more apps are recognised by name, and apps built by developers with a default name are no longer shown as ChatGPT or Claude. The agent counts and the AI agent visits will mostly step down on this date without any change in your real traffic, and can step up for ChatGPT and Claude connectors whose addresses check out. Earlier values are not comparable, and the new numbers settle over about a week.