acceptodds
Under review as a conference paper at ICLR 2027

After Error Feedback: Continued Access to Failed Reasoning Contributes to Contextual Drag

Abstract

Iterative LLM workflows increasingly rely on previous attempts and feedback to improve subsequent solutions. However, when earlier attempts are incorrect, retaining them may paradoxically impair later reasoning, a phenomenon known as contextual drag. While prior work has documented this effect, it remains unclear whether the interference arises from the initial exposure to the failed attempt or from the model's continued access to it during subsequent reasoning. We introduce a same-text attention intervention that isolates this mechanism by preserving the full prompt, token positions, prefill computation, and first-token logits, while blocking only later generated queries from attending to the failed draft. Across controlled evaluations, Qwen3-8B on selected Game24 problems shows that accuracy drops from 87.0% in clean context to 39.1% with full access to a failed draft, but recovers to 78.3% when continued access is removed, yielding a +39.1 percentage-point effect (95% question-cluster bootstrap interval: +23.9 to +54.3). Additional models and two MATH settings show consistent trends. In contrast, blocking verified correct history reduces accuracy, indicating that the effect depends on the retained history and its feedback. These results identify continued direct access after initial processing as a causal contributor to contextual drag.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.