Search Console has started logging things nobody typed into a search box. A visitor tells Google’s AI Mode “yes, go on,” your page shows up in the AI’s answer, and that single-word reply gets recorded as its own query, with an impression against your site attached to it, exactly like someone had searched “yes” the traditional way. One SEO who ran a 16-month analysis of their own Search Console data found 1,127 of these conversational fragments generating over 20,300 impressions. If you’ve never looked for this in your own data, it’s probably already there.
This isn’t a bug report or a glitch someone should file a ticket about. It’s a side effect of how AI Mode works, and it’s been accumulating in Search Console accounts for months before most people noticed. Once you know what to look for, the fragments are easy to spot, and understanding what they mean, and what they don’t, changes how much attention they deserve.
Why Search Console Is Recording Conversations at All
The mechanism is straightforward once you see it: Google treats every message inside an AI Mode conversation as a distinct search, not just the opening question. A follow-up question, a one-word reply, a pasted error message someone is troubleshooting: all of it gets processed the same way a typed query would, and if your page contributes to the answer Google’s AI gives in response, Search Console logs an impression against whatever text triggered that turn of the conversation. This wasn’t how Search Console worked when AI Mode launched, and the shift happened gradually enough that most site owners haven’t gone looking for it. According to the analysis, reply artifacts first started appearing in the data in December 2025, conversational activity became a persistent, ongoing pattern by March 2026, and Google didn’t formalize any of this into a dedicated report until its Generative AI performance reports launched on June 3, 2026, becoming available to all users by August 11.
For a small business tracking its own Search Console data week to week, this shows up as queries that make no sense next to a traditional keyword list: single words, half-sentences, or oddly specific phrases that clearly aren’t something a person typed cold into a search bar. Without knowing why they’re there, it’s easy to either dismiss the whole report as broken or, worse, start treating every strange fragment as a keyword worth targeting.
Seven Different Kinds of Fragment, Not One Pile of Noise
Not every conversational fragment means the same thing, and treating them as a single undifferentiated pile of noise misses real signal buried inside it. The analysis sorted what it found into seven categories:
- Reply artifacts: bare one- or two-word responses like “yes,” “sure,” or “show me.” In the sample, “yes” alone generated 110 impressions.
- Pivot follow-ups: comparison questions like “what about resend?”, where a visitor is asking the AI to check an alternative after already getting an initial answer.
- Conversational questions: natural-language questions addressed directly to the assistant rather than phrased as a search query.
- Tracker probes: synthetic prompts generated by AI-monitoring tools testing how models respond, not real user intent.
- Agent harnesses: complete machine-generated instructions that got logged by accident when an automated agent, not a person, ran a query.
- Pasted strings: error messages or spreadsheet headers someone pasted directly into an AI chat, searched verbatim.
- Long uncategorized queries: ten-plus-word strings that don’t fit neatly into any of the above but clearly reflect a real conversational exchange.
The conversational-questions bucket alone produced 559 distinct queries and 8,834 impressions in the analysis, but only 13 clicks, an extreme mismatch between visibility and traffic that’s worth sitting with before deciding this data is worth acting on.
Why Most of This Isn’t Something to Act On
It’s tempting to treat any new data source in Search Console as a keyword opportunity waiting to be mined, and for most of what’s leaking through here, that instinct is wrong. Optimizing a page for the literal phrase “yes” because it generated impressions is chasing noise, not intent. Tracker probes and agent harnesses represent monitoring tools and automated systems, not prospective customers, and mixing them into keyword research just distorts volume numbers for terms that were never searched by an actual person. The author’s own framing is blunt about this: these are query fragments, not new keywords to target, and the temptation to treat every visible number in Search Console as a signal worth chasing is precisely the mistake this data invites if you don’t filter it first.
There’s also a hard limit on how much of the picture this data even shows. More than half of all impressions, 57.7% in the analysis, carry no visible query string at all, anonymized by Google before they ever reach Search Console. What’s visible is described accurately as a “floor,” the minimum known activity, not the real total. Most AI conversations that reference your site never surface in any report you can currently see, which means any conclusion drawn from what is visible should be treated as directional rather than complete.
The Two Categories Genuinely Worth Reading
The genuinely useful part of this data sits inside the pivot follow-ups and conversational-questions categories, not the reply artifacts. When a real, specific follow-up question shows up repeatedly, like a comparison against a named competitor or alternative, that’s a content gap you can act on directly: if people are asking an AI “what about resend?” after reading about your email tool, that’s a comparison page you don’t currently have and probably should. Similarly, when a full passage from your page appears inside an AI response, that’s worth reading closely, because in a conversational AI answer, the user typically never sees your page title or URL at all. They see whatever passage got extracted and quoted back to them. That means the specific wording of the passage that got cited matters more here than it does in classic search, where a compelling title and meta description can still earn a click even if the underlying content wasn’t quoted.
A practical starting point doesn’t require enterprise tooling: open the Search Results report, filter queries by length (ten-plus words is a reasonable proxy for conversational activity), and skim for repeated pivot-style follow-ups rather than one-off phrasing. Larger sites with the technical resources to export Search Console data to BigQuery can go further and build a classifier across the full dataset, but that step is optional. The core habit, checking what specific follow-up questions and passages are showing up, works at any size and doesn’t require specialized tooling to start. A hosting company blog with a handful of comparison and how-to posts is exactly the kind of site where this matters: a repeated pivot question like “what about VPS instead?” underneath a shared-hosting article is a direct signal for the next post to write, not a metric to admire and move past.
A Limitation Worth Remembering Before Acting on Any of This
Google still doesn’t provide direct query data through its Generative AI performance report API, which means fragments like these, imperfect and partial as they are, remain one of the only ways to see what conversations are happening around your content in AI Mode. That’s a genuinely awkward position for anyone trying to measure AI search performance rigorously: the data that exists is real and occasionally useful, but it’s also a byproduct of a logging quirk rather than a deliberately designed reporting feature, and treating it as more complete or more precise than it is will lead to decisions based on a small, skewed sample of a much larger conversation that stays invisible.
*Source: The AI Conversations Leaking Into Your Search Console, Search Engine Journal*