Hover lies by omission
A struct with eleven fields shows ten, so any test value built from that answer is missing one, and the only hint is an ellipsis the agent ignored. What it takes to generate a fixture you can trust: read the declaration rather than the summary, and type-check the result before handing it over.
On this page · 4 sections
First published on 2026-09-29. The measurements reported in this article are historical and have not been re-run for this revision.
A test needs a value of a type. The type has eleven fields, the agent writing the test knows none of them, so it asks the analyzer for the shape. This is the right instinct - the analyzer is the one thing in the room that cannot be out of date - and here is what comes back:
$ prod-code hover crates/prod-code-protocol/src/messages.rs 613 12 # the declaration, by position
pub struct ShadowRunRequest {
pub client_workspace_root: String,
pub base_workspace_name: Option<String>,
pub hypotheses: Vec<ShadowHypothesis>,
pub command: Vec<String>,
pub env: Vec<(String, String)>,
pub timeout_secs: u64,
pub subdir: Option<String>,
pub parallel: usize,
pub tail_bytes: usize,
pub client_agent: Option<String>,
/* … */
}
Ten fields and an ellipsis. The eleventh is client_host. Hover is a summary meant for a human
glancing at a tooltip, so rust-analyzer stops this one after ten fields and says so with
/* … */ - a limit that is a setting, not a law, which is exactly why nothing downstream
should depend on where it falls.
A struct literal written from that answer is missing a field. It does not fail at the point of
the mistake; it fails later, in a build, with an error the agent then has to route back to the
thing it wrote three steps ago. The hover did say “there is more”, with that /* … */; the
agent built the literal from the ten fields it could see and did not, or could not, turn the
marker into the field it was missing.
Correction, 2026-10-27. As published on 2026-09-29, this paragraph ended with “Nothing about the hover said ‘there is more’”, and the deck said nothing flags the missing field. The hover does flag it:
/* … */is rust-analyzer’s truncation marker, as the paragraph above explains. The claim is now about the agent not using that marker, not about the hover hiding the omission.
Read the declaration, not the summary
code_generate_fixture asks a different question. Instead of “describe this type”, it resolves
the name to the place the type is declared and reads the declaration’s own text, then fills
each field by its type:
$ prod-code fixture ShadowRunRequest
fixture for `ShadowRunRequest` (declared in crates/prod-code-protocol/src/messages.rs)
let shadow_run_request = ShadowRunRequest {
client_workspace_root: String::new(),
base_workspace_name: None,
hypotheses: Vec::new(),
command: Vec::new(),
env: Vec::new(),
timeout_secs: 0,
subdir: None,
parallel: 0,
tail_bytes: 0,
client_agent: None,
client_host: None,
};
the analyzer accepts it: 0 errors
Eleven fields, client_host among them.
That first hover took a file and a position for a reason. ShadowRunRequest is three symbols in
this repository, and the case for
addressing a tool by symbol rather than by position
explains why a name that resolves twice is reported rather than guessed at:
{ "name": "code_hover", "arguments": { "symbol": "ShadowRunRequest" } }
`ShadowRunRequest` is ambiguous (3 candidates); qualify it (Type::name) or pass `path`:
[Struct] ShadowRunRequest crates/prod-code-protocol/src/lib.rs:13:47
[EnumMember] WireMessage::ShadowRunRequest crates/prod-code-protocol/src/messages.rs:33:5
[Struct] ShadowRunRequest crates/prod-code-protocol/src/messages.rs:613:12
(That one is the MCP form; the CLI’s hover takes a position, which is why the transcript above
does.) A re-export in the crate root, a same-named variant of the message enum, and the
declaration.
Here the ambiguity can be resolved rather than reported, because only one kind of symbol can be
built: the generator keeps structs and enums, drops re-export sites when a real declaration
exists, and falls back to asking for a path when that still leaves two.
Filling a field is mechanical; knowing you got it right is not
Integers and floats become 0 and 0.0, bool becomes false, String becomes
String::new() and PathBuf PathBuf::new(), Option becomes None, collections come back
empty, a Duration is from_secs(0), a wrapper (Box, Arc, Rc, Mutex, RwLock,
RefCell, Cell) is built around its inner value, an enum takes its first unit variant. Types declared in the workspace are built field by field, down to depth:
$ prod-code fixture DivergentBenchReport
let divergent_bench_report = DivergentBenchReport {
language: Language::Rust,
mode: WorkspaceMode::Shared,
persistent: false,
churn_percent: 0,
sessions_dropped: 0,
gateway_after: None,
workspace_name: String::new(),
target: DivergentTarget {
language: Language::Rust,
file_rel: std::path::PathBuf::new(),
symbol: String::new(),
line: 0,
},
initial_syncs: Vec::new(),
total_queries: 0,
total_errors: 0,
elapsed: std::time::Duration::from_secs(0),
qps: 0.0,
latency: LatencyStats {
count: 0,
min_ms: 0.0,
p50_ms: 0.0,
p95_ms: 0.0,
p99_ms: 0.0,
max_ms: 0.0,
},
latency_by_kind: BTreeMap::new(),
errors_by_kind: BTreeMap::new(),
verifications: Vec::new(),
all_passed: false,
};
the analyzer accepts it: 0 errors
Eighteen fields, two nested structs built field by field, two maps, a Duration, an f64, two
enums taken at their first variant. Every one of those rules is a guess about what a type’s empty value looks like, and guesses are wrong
eventually. So the last step is not a rule at all: the fixture is placed in a #[cfg(test)]
probe module inside an in-memory overlay of the file that declares the type - the same
mechanism that judges a patch before it is written
uses - and the analyzer is asked whether it compiles. Nothing is written to disk
in any mode, and depth defaults to 2.
Watch it earn that, by asking for a fixture that cannot work - --depth 0 means “do not build
nested types, fall back to Default::default()”:
$ prod-code fixture DivergentBenchReport --depth 0
`Default::default()` stands in for: DivergentTarget, Language, LatencyStats, WorkspaceMode
(not declared in this workspace, or deeper than the depth limit)
the analyzer rejects it:
the trait bound `Language: Default` is not satisfied [E0277]
(crates/prod-code-client/src/divergent_bench.rs:1757:15)
the trait bound `DivergentTarget: Default` is not satisfied [E0277]
(crates/prod-code-client/src/divergent_bench.rs:1764:13)
Default::default() is a perfectly reasonable fallback that happens to be wrong for these two
types. Without the check, that fixture looks fine, goes into a test, and fails on the next
build with an error about a trait bound, three steps from where it was written. With the check
it comes back as the analyzer’s own message, immediately, and with the list of types that fell
back so you can see which ones to fill in by hand.
What it does not do
- Rust only. The shape comes from rust-analyzer, linked into the gateway. We have not built this for the other engines, so the tool is absent for them rather than degraded.
- It writes a literal, not a constructor. If a type has a
newor a builder that enforces an invariant, the fixture goes around it. The fallback list and the type check make the literal’s limits visible, which is what an agent needs in order to decide; picking the right constructor needs argument values that are themselves fixtures, and the literal had to be right first. - The check happens where the type is declared. That file’s imports are in scope during it,
so a fixture that uses
BTreeMap::new()may still need an import when pasted into another module. The tool says so in its own description rather than pretending otherwise. Default::default()past the depth limit is a guess, named in the output as one.
What it costs
Measured the same way for each row - time prod-code … from a shell, against a warm gateway on
the LAN, with a development build of the client:
hover - a bare round trip, for reference |
0.09 s |
fixture ShadowRunRequest --no-verify (11 fields, flat) |
0.25 s |
fixture ShadowRunRequest (type-checked) |
1.04 s |
fixture DivergentBenchReport --no-verify (18 fields, nested) |
0.52 s |
fixture DivergentBenchReport (type-checked) |
1.00 s |
Every row of that table was four to nine tenths of a second slower while this post was being
written, and none of it was the analyzer. PROD_CODE_TIMING=1 on the first row prints
total=70.7ms connect=1.5ms preflight_sync=55.0ms handshake=6.7ms initialize=1.3ms did_open_sent=4.9ms query_response=1.3ms - the query itself is 1.3 ms, and the ~80 ms that
moving the analyzer off the laptop and
the round-trip breakdown measured for a one-shot query is
that total, most of it two git subprocesses. The rest of the wall clock was outside what that timer bracketed: the
two questions an invocation asks a gateway before its query, each answered by probing for
installed language servers, one probe of which starts npm. That was
issue 56, and it is fixed - the probe is taken
at startup now, not per request.
What is left is the tool’s own cost, and differences between the table’s rows suggest where it
goes. In this run’s walltimes, resolving the symbol and reading the declaration accounts for
0.16 s on the flat type and 0.43 s on the nested one, which we attribute to one document-symbol
call per type it descends into; the type check adds 0.79 s and 0.48 s. These are differences
between end-to-end walltimes of single runs, not timings of the phases themselves. Do not
read an order into those two. The measurements taken before the gateway fix put them the other
way round - 0.47 s on the flat type, 0.95 s on the nested one - so the honest statement is that
the check costs between four and nine tenths of a second, and which type pays more is not
stable between runs. That is unsurprising once you know what is being judged: not the fixture,
but the file the type is declared in, with the fixture in a probe module inside it. The check
buys the only thing that separates this from a template: in the file that declares the type, the
analyzer either accepts the literal or says why not. That is the analyzer’s verdict, not a
compiler run; it does not cover the imports the literal may need in another module, and
--no-verify skips it, so a build can still reject a fixture that came back without an error.
Correction, 2026-10-27. As published on 2026-09-29, this paragraph said the check yields an answer that is either correct or an error, never a plausible literal that a build will reject later. What the post shows is narrower: the analyzer’s verdict on the literal inside an in-memory probe module of the declaring file. No compiler run is shown, the section “What it does not do” notes that a pasted fixture may still need imports, and
--no-verifyturns the check off. The “never rejected later” guarantee is withdrawn; the timings above stand as published.
Cite this article
Alexander Panasenko (2026-09-29). Hover lies by omission. https://prod.codes/blog/hover-lies-by-omission/