Contextual Drag: How Errors in the Context Affect LLM Reasoning

Many self-improvement pipelines for large language models (LLMs) reuse prior attempts as context, relying on verification or critique to extract useful reasoning from flawed attempts. We study contextual drag: failed attempts in context bias subsequent generations toward structurally similar errors, often persisting even when models recognize them as wrong. We focus on self-drag, where a model conditions on its own incorrect drafts, mirroring iterative self-refinement. Across 10 open-weight reasoning models and 8 reasoning tasks, a single incorrect draft drops average pass@1 accuracy by around 25 points on problems where direct reasoning produces both successes and failures. For GPT-OSS-20B, which has severe single-step self-drag, iterative refinement compounds this effect into self-deterioration. Explicit error signals, from either external labels or the model’s own correct verification, do not fully eliminate the effect, suggesting that correct recognition is not sufficient to reset the draft's influence. A Game of 24 case study using tree edit distance shows that subsequent generations are structurally biased toward the in-context draft, not just less accurate. Finally, filtering the draft before reuse reduces contextual drag on every model we test, by a model-dependent amount and without training or external labels; applied recursively, it turns GPT-OSS-20B’s self-deterioration on competition math into steady improvement. These results suggest that robust self-improvement requires more than reflecting on mistakes; it requires mechanisms for context reset, independent attempts, and selective reuse of prior reasoning.