There is a failure mode in AI visibility reporting that most dashboards are built to hide.
It happens when you do everything right. Your page is indexed, your content is solid, the model clearly retrieved something from your domain, and the answer still recommends three competitors and never says your name.
A visibility score that averages across prompts will not show you this. It will show you a number that went up slightly, or a number that held steady, and you will conclude that nothing changed. But something did change: you were in the room and you were not introduced.
This article is about how to catch that, and about the reporting layer that turns a pile of scored AI answers into something you can actually act on.
What you will finish with
A scored answer matrix where every row carries:
- A presence class: named, mentioned, or absent
- A confidence value, so low-confidence rows route to human review
- A ghost citation flag for answers where context was supplied and the brand still did not appear
- A displacement signal showing whether the shortlist is still the same incumbents as last run
- A persona split, so you can see instability instead of averaging it away
Plus the change-impact check that tells you whether your last page edit moved any of it.
Who this is for: a GEO or measurement lead who already captures AI answers and wants a better reporting layer on top.
Prerequisites:
- A captured answer set from the AI surfaces your buyers actually use, stored one row per prompt per surface per run
- The scoring schema from the AI visibility scoring workflow, or an equivalent set of Jev questions
- A Jev API key, or an agent with the TypeSafe skill installed
- A place to store rows over time, because every metric here is a comparison
Definition of done: every answer has a presence class and a confidence value, ghost citations are counted separately from ordinary absences, and you can show a run-over-run comparison without averaging away the instability.
Why averaging hides the problem
Most visibility reporting collapses to a single number: what share of prompts mention us.
That number is a trap for three reasons.
It mixes two different failures. An answer where your brand never came up and an answer where the model had your content in front of it and still chose someone else are not the same problem. The first is a reach problem. The second is a positioning or trust problem. Averaged together, they look identical.
It hides instability. If your brand appears in 40% of answers, that could mean you appear consistently in 40% of prompts, or that you appear in 80% of prompts for half your audience and never for the other half. Those are completely different situations and the aggregate cannot tell them apart.
It has no memory. A single run tells you where you stand. It cannot tell you whether the shortlist is ossifying around the same three names, which is the thing that actually determines whether you can break in.
The fix is not a better score. It is a richer row.
The three presence classes
Collapse every answer into one of three states. This is the foundation, and it needs to be decided before you score anything.
Named. The brand appears by name in the answer.
Mentioned. The brand's content or domain was used, but the brand name does not appear in the answer text. This is the ghost citation.
Absent. Neither the brand nor its content appears.
The middle class is the one nobody reports, and it is the most actionable.
Three classes, not two. The middle one is where the useful signal lives.
What a ghost citation actually tells you
A ghost citation means the model used you and did not credit you.
That can happen for a few reasons, and the reason determines the fix.
Your content was used as supporting evidence for someone else's claim. The model found your data useful but attributed the conclusion to a competitor whose page framed it better. The fix is on your page: lead with the conclusion, not the evidence.
Your brand name is not attached to the concept. The model knows the concept and does not associate it with you. The fix is entity work: consistent naming, clear about-page language, and third-party references that pair the name with the category.
Your content is extractable but not quotable. The model can pull a fact from your page but there is no clean sentence it can attribute. The fix is structural: answer-first formatting, explicit claims, and named sources.
You were retrieved as a source for a competitor's comparison. The model used your page to describe the category and then recommended the competitor. The fix is positioning: make the case for yourself in the same passage where you describe the category.
In all four cases, the ghost citation tells you that reach is not your problem. You are already being retrieved. Something downstream of retrieval is failing.
That is a completely different workstream from "we are not being found," and without the middle class you would never know which one you are in.
Scoring the classes
If you already have a scoring schema, you likely have a mention-type question. The change here is what you do with the output.
Score each answer once, then derive the class:
{
"model": "jev-latest",
"state": {
"prompt": "best project management software for small teams",
"surface": "chatgpt",
"run_id": "2026-09-01",
"answer": "...",
"brand": "Acme PM",
"context_supplied": true,
"cited_urls": ["https://example.com/best-pm-tools"]
},
"questions": {
"brand_named": {
"type": "noul",
"instructions": "Does the answer contain the brand name as a string?",
"criteria": {
"true": "The brand name appears explicitly in the answer text",
"false": "The brand name does not appear anywhere in the answer text"
}
},
"brand_content_used": {
"type": "noul",
"instructions": "Was content from the brand's domain used to construct this answer?",
"criteria": {
"true": "At least one cited URL belongs to the brand's domain, or the answer reproduces a distinctive claim from the brand's page",
"false": "No cited URL belongs to the brand's domain and no distinctive brand claim appears"
}
},
"presentation": {
"type": "choice",
"instructions": "If the brand is named, how is it presented?",
"criteria": {
"pick": "The brand is the lead or a top recommendation",
"mention": "The brand appears in a list or comparison without being recommended",
"absent": "The brand does not appear"
}
}
}
}Then derive the class in code, not in the model:
if brand_named == true and presentation in (pick, mention) -> NAMED
if brand_named == false and brand_content_used == true -> GHOST CITATION
if brand_named == false and brand_content_used == false -> ABSENTWhy derive it in code. The class is a rule, not a judgment. Letting the model decide the class adds a failure point for no benefit. Ask the model the two questions it is actually good at answering, then apply the rule yourself.
Displacement: is the shortlist moving?
The second thing a single run cannot tell you is whether you are making progress against the incumbents.
Track, per prompt, which brands appear in the answer and in what order. Then compare runs.
The metric that matters is not your own share. It is whether the same names keep appearing at the top.
Stable incumbents. The same two or three brands lead the same prompts every run. This is the hardest situation, because the model has settled. Your work has to change the underlying evidence, not just add content.
Churning shortlist. The leaders change between runs. This is an opening. A churning category means the model is uncertain, and uncertainty is where a well-structured page can enter the set.
Your entry, then exit. You appeared in one run and not the next. This usually means you were retrieved opportunistically rather than consistently. Check whether the prompt set or the retrieval changed.
Report displacement as a count of prompts where the leader changed, not as an average. Averages destroy exactly the signal you are looking for.
Three displacement patterns. Only the middle one is an opening.
Persona split: report instability, do not average it
The same question asked in different buyer framings can produce different brand lists. A question from a solo founder and a question from an enterprise procurement lead are not the same prompt, even if the underlying need is identical.
Score them separately, then report the split.
If your brand appears for the solo-founder framing and never for the enterprise framing, that is a finding. It tells you your content speaks to one buyer and not the other, and it is invisible in an aggregate score.
The rule: never average across personas. Report the per-persona presence rate. The variance is the insight.
Change-impact checks
The last piece is the one that closes the loop.
After you change a page, you need to know whether the change moved anything. That requires a before-and-after comparison on the same prompt set.
Run the scored set immediately before the edit and again after the model has had time to re-crawl. Compare:
- Presence class per prompt
- Ghost citations, specifically
- Displacement, per prompt
- Confidence distribution
If ghost citations dropped and named answers rose for the prompts your page targets, the fix worked. If nothing moved, either the edit was not the bottleneck or the model has not re-crawled yet.
Do not skip the wait. A change-impact check run too early will show no movement and you will conclude the edit failed when it simply had not been picked up.
Verify before you trust the numbers
Three checks before you act on any of this.
Hand-check the ghost citations. Read ten of them yourself. Confirm the brand's content really was used. False ghost citations usually mean your brand_content_used criteria are too loose.
Check confidence against correctness. If your low-confidence rows are not meaningfully worse than your high-confidence ones, the confidence value is not doing any work and your threshold is arbitrary.
Confirm the answer was actually captured. A missing answer is not an absence. If a surface returned an error or a rate limit, that row should be excluded, not scored as absent. This is the single most common way these reports go wrong.
Maintain it
Run the scored set on a fixed cadence, monthly is usually enough. Keep every run, because the trend is the product.
Revisit the prompt set quarterly. Buyer questions change, and a prompt set that was accurate six months ago may be measuring a market that no longer exists.
And keep the human review gate on low-confidence rows. The whole point of attaching confidence is to route the uncertain cases somewhere, and that somewhere should be a person.
What to do next
Take your last captured answer set and classify every row into named, mentioned, or absent. Count the ghost citations.
If that number is not zero, you have a positioning problem rather than a reach problem, and you now know which prompts to look at first. Start with the highest-value prompt that produced a ghost citation, read the answer, and work out which of the four causes applies.
That is a much better use of an afternoon than watching an aggregate score.
Read the rest of the series
This article is part of a ten-part series on using Jev for SEO and GEO work.
- Start here: what Jev is and how to install it
- How to build a Jev internal-linking and cannibalization workflow
- How to use Jev to audit and route your content library
- How to run AI-visibility checks with Jev as the scoring layer
- Where Jev breaks in SEO work
- How one team built a Jev internal-link analyzer for 78 yen
- How to turn Jev citation gaps into page fixes
- Is Jev actually accurate for SEO? What the evidence really shows
- How to monitor Reddit as a GEO citation source with Jev
- How to run a full Search Console export through Jev (and the three traps)
Author: Ethan Marlowe, GEO Measurement Lead Across 500+ Prompts at Auspia. Ethan writes about prompt tracking, citation reporting, visibility dashboards, and AI answer quality checks.
