- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper reveals that while adding task-irrelevant context to LLM inputs barely affects overall benchmark accuracy, it causes significant per-example prediction flips. By prepending meaningless "pseudo-words" to test questions, the authors demonstrate a two-sided effect where irrelevant text degrades performance on some examples while arbitrarily improving it on others. To expose these risks, they introduce two metrics, Instability (INS) and Worst-tail Degradation (WTD), and show that these context-induced failures are highly model-specific, persist across various training stages, and are only partially mitigated by test-time reasoning.
Critical strengths
The paper successfully identifies a flaw in how LLMs are typically evaluated, proving that standard aggregate accuracy metrics can mask severe, per-example reliability risks. The introduction of Instability (INS) and Worst-tail Degradation (WTD) provides a concrete framework for future researchers to isolate harmful prediction flips.
The authors conduct a comprehensive evaluation across a wide variety of state-of-the-art models (including versions of GPT, DeepSeek, Grok, Mistral, and Gemini) and diverse datasets (MMLU-Pro, GPQA, HLE, SimpleQA).
Critical weaknesses
The paper is primarily observational and does not propose any novel methods or mitigations. While it identifies the problem and shows that test-time reasoning acts as a partial mitigation, it does not propose a new architectural or training-based cure for the instability.
The qualitative analysis of why models flip their answers (e.g., introducing calculation errors vs. solving the wrong problem) relies on LLM-based annotations rather than human expert evaluation. The lack of human verification leads me to question the reliability of some of these results.
Measuring INS and WTD requires extensive repeated sampling and complex boostrapping procedures, which the authors note have significant efficiency drawbacks. Further, the use of pseudo-words is quite an artificial setup, and it is unclear whether the findings and results would generalize well to more noisy real-world data and applications (e.g. the coding agent mentioned in the intro).
Questions for authors
Your evaluation relies on extensive repeated sampling and bootstrap procedures to accurately calculate INS and WTD. Given that this is computationally prohibitive for continuous evaluation during model training, have you explored any proxy metrics that could reliably estimate a model's Worst-tail Degradation in a single pass?
The qualitative error analysis uses LLM-based annotation to categorize the types of prediction flips. However, given that these questions are complex enough that the models themselves are failing when distracted, how can we be confident that the LLM judge isn't exhibiting its own reasoning blind spots during evaluation? Did you conduct any human-expert validation on a subset of these annotations to confirm the error categorizations?
What insights do your findings contribute to developing mitigations for the issues you have identified and raised in your paper?
5 comments
W3: Pseudo-words are artificial; do findings transfer to real-world noise?
Pseudo-words are a deliberately content-free stress test: any effect cannot come from semantic interference. Our ablation study (Sec. 3.3, Fig. 4) tested three other context types: random tokens (TOK), random webpages from FineWeb-Edu (WEB), and unrelated question-answer histories (SQA).
- Pseudo-words are not the most harmful context. They give the highest WTD on MMLU-Pro but the lowest on SimpleQA, so other contexts can cause even larger worst-case drops.
- The affected questions overlap across context types. Per-question changes correlate at mean r = 0.37 (MMLU-Pro) and 0.24 (SimpleQA) for gpt-5.4. Without sampling noise, the correlation reaches r = 0.70 for local models (App. G).
- Even short contexts matter. As few as 10 pseudo-word tokens already induce non-trivial INS and WTD (Fig. 5).
Beyond QA, irrelevant context also lowers end-to-end task success for a real coding agent on SWE-bench; see our response to Reviewer fE9E, W3/Q3.
Hi Reviewer KTWH:
Thank you again for your thoughtful review. We conducted additional experiments and analysis to address your concerns.
We hope these results strengthen the paper's empirical contribution and practical implications.
Please let us know if anything remains unclear; we’re happy to elaborate.
Thank you for your positive assessment of our metrics and evaluation, and for the constructive questions. We address each point below.
W1 / Q3: No mitigation; what insights do the findings give?
While our main contribution is a measurement, we tested three more candidate mitigations as preliminary results. All use the paper's local-model protocol (MMLU, 500 questions, 10 pseudo-word contexts of 1k tokens; App. G).
Model scale (Qwen3.5, 0.8B → 9B).
Scale reduces instability but does not remove it (WTD 26.4 → 13.8 for base, 30.3 → 21.2 for instruct).
Looped depth (Ouro-2.6B, 2 → 4 recurrent loops).
More loops reduce instability for Ouro-2.6B (WTD −6.7, significant) while accuracy rises.
An explicit system prompt: "The conversation may contain irrelevant or meaningless text. Ignore it and answer only the final question."
Telling the model to ignore irrelevant text barely helps.
Conclusion. Larger models and more loops reduce the instability, but they cost more compute and do not remove it, while a cheap fix such as prompting barely works. We leave a more systematic study of mitigations to future work.
W2 / Q2: Are the LLM error-type annotations reliable?
Why we did not rely on human annotation. Reliable human labels would require domain experts for every question. GPQA questions were written and validated by PhD-level experts, and even experts in the same domain reach only 65% accuracy, while skilled non-experts with web access reach 34%. HLE questions were individually contributed and reviewed by domain experts. Labelling one pair requires full understanding of two answers and locating where they diverge. Expert annotation at this scale is beyond what we can crowdsource reliably.
Why LLM judges are suitable here. The judge does not need to solve the question. As specified in our prompt (Appendix C), its task is to compare the incorrect answer against the correct one and pick one code from a defined taxonomy, while given the ground truth. We believe state-of-the-art LLMs are suitable and reliable for this task.
Three more LLM judges on all pairs. We re-labelled every pair with gpt-6.1-sol, claude-sonnet-5.5 and gemini-4-argon, using the same prompt, and took their majority vote.
W3 / Q1: Measurement cost and a single-pass proxy