Recall Is Not Evidence Use: Objective Imprinting in Retrieval-Augmented Systems
Abstract
Retrieval recall measures whether supporting evidence enters the context, while downstream behavior also depends on the policy that uses that evidence. When semantic evidence and retrieval metadata agree, evidence-reading and rank-following policies are observationally indistinguishable under ordinary accuracy. We study this ambiguity through policy identifiability and define objective imprinting: the persistence of a history-selected policy when later supervision remains task-correct while ceasing to distinguish among compatible rules. In paired histories, semantic and proxy policies remain latent behind identical agreement-set performance through 64k shared-example presentations. In source-disjoint full-corpus retrieval, this distinction has a large downstream consequence: Recall@4 reaches 99.2%, while recoverable rank errors yield an 82.8-point mean exact-match gap under a common reader. Holding retrieved passages and display order fixed, changing only rank metadata reverses proxy routing and isolates evidence use from evidence availability. Policy-identifying supervision also provides direct control: 40 cue conflicts reliably rewrite either policy while preserving agreement-set accuracy. Together, these results separate evidence availability, observed-task performance, and evidence-use policy, and show how identifying information can expose and steer the policy governing retrieved evidence.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.