Telling Reasoning Models What to Report Removes the Cost of Conflicting Sources
Abstract
Retrieval-augmented and agentic systems routinely place disagreeing sources in one context, imposing large inference costs on reasoning models. Across 23 Wikipedia contradictions and 32 constructed documents, six open model configurations use 2.0–57× more completion tokens for conflicting than matched agreeing passages (≥176× in one generation-capped case), with 89.2–99.8% of conflict tokens spent on reasoning. We show that this cost largely disappears once the model is told what to report: a single passage-selection directive removes most of the premium without requiring authority. Across 47 additional contradictions, arbitrary directives were cheaper than authoritative rules in all four models (0.72–0.93×), while permission to report either value cost 1.3–2.8× more; within that frame, adding a command reduced cost by 19–60%, whereas merely asserting which passage was operative cost 1.4–3.0× more. A factorial experiment confirms that the key factor is specifying the response decision, not grammatical form or authority: declarative reporting instructions cost 0.93–1.05× commands, while claims about which passage was operative cost 1.55–3.71× more. Blindly fixing the answer saves tokens but sacrifices accuracy, failing 66–97% of conflicts resolvable by other documents. By contrast, an evidence-conditional stopping rule removes 46–88% of the cost on 150 previously unseen, unsettleable conflicts, returns CONFLICT on 94–100%, preserves or improves accuracy in two of three models, and repays its checking overhead when just 4.1–12.9% of conflicts are unsettleable. Response policy is therefore a first-class lever on the inference cost of retrieval.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.