acceptodds
Withdrawn from ICLR 2027

How Stable Are LLM Errors? Failure-Conditioned Decision Stability in LLMs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large language models produce fluent but incorrect answers, but wrong answers need not be equally stable under small changes in query context. We study this failure-conditioned decision stability on candidate-mappable closed-answer failures using a fixed weak re-query that provides no gold label, feedback, or additional evidence. For each failure, we compare the gold candidate with the realized wrong candidate and use their final-window candidate margin as the main pre-query coordinate; full-response traces are treated descriptively, and post-query quantities only as response diagnostics. Across four tasks, recovery in the two primary models rises from near zero in the most failure-favored final-window states to 39.4-60.2% in the largest-margin bins, with the same Q5-Q1 ordering in every task; the remaining models expose weaker, sparse, re-query-sensitive, or negative boundary cases. A replay-matched audit of 607 Qwen failures removes post-answer continuation from the readout: the literal answer-token margin yields 0.794 AUROC versus 0.800 for the original final-window coordinate. On the same 607 replay-matched Qwen failures, confidence-controlled analysis yields 0.789 AUROC for the margin versus 0.661 for wrong-candidate probability. A separate response decomposition further shows that the margin is associated both with whether an erroneous answer changes and, among changed non-binary answers, with whether the new answer is gold rather than another wrong candidate. We position residual-error stability as an offline analysis axis rather than a deployment-time error detector.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, model reliability, error analysis, decision stability, candidate competition

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_jp9SOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper looks at incorrect answers of LLMs and their alternative answer options across three benchmarks (Thruthful-QA, SciFact entailment, BoolQ and ARC-challenge), and uses a "using a fixed weak re-query that provides no gold label, feedback, or additional evidence".

The paper is very difficult to follow, with many key concepts not described well and very unusual structure. Example of difficult passages are immediately in the introduction and abstract: "recovery in the two primary models"- what recovery? why primary?"; "replay-matched audit"?). Also, it is not clear what exactly is traced and how in, e.g., "for each retained failure, we trace gold-versus-realized failure candidate competition during the original response and use a final-window candidate margin did_i as the main pre-query coordinate. Full-response onset, persistence, prevalence, and path-type summaries are retained only as secondary descriptive diagnostics because the evaluated responses are predominantly short.".

There is an experimental flaw in the experiment. The prompt to "re-query" (or, in other terms, to ask the model to self-assess) contains a hint "Your previous answer may be incorrect." while it is run only on failure cases: "For each retained failure (xi , a+i , a−i ) ∈ F , we apply the same weak cautionary re-query Pmin"

The references contain hallucinated paper page numbers and mix up conference names and title. See below.

Critical strengths

Analysing errors in LLMs is in principle an interesting research direction, albeit a challenging one, as it also requires to find insights that generalize across models, to be lasting and impactful results.

Critical weaknesses

see points in my summary above. To summarize:

  • the paper suffers in terms of presentation, has an unusual structure, does not motivate and describe the problem and setup well

  • the re-query prompts with the hint ("Your previous answer may be incorrect.") seems to be confounded, and is misses a baseline. this seems one of the key contributions of the paper. It is unclear to what extent the recovery is sensitive to this particular prompt. The authors acknowledge this limitation and investigate prompt variants, but an important missing baseline is a neutral re-query without any suggestion that the previous answer was incorrect.

  • traces of hallucination in the reference list (made up page numbers), see boldface:

    • Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet. In International conference on learning representations, volume 2024,pp. 32808–32824, 2024. (https://openreview.net/forum?id=IkmD3fKBPQ)
Questions for authors

Why did you focus on valid outputs only? (line 097-098)

Reviewer_KRzKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies answers that an LLM has already gotten wrong. The same question is asked again with a fixed, information-free suffix ("Your previous answer may be incorrect. Reconsider…", Sec. 2.4), and the authors record whether the answer changes and whether it moves to gold ("recovery"). The setting is candidate-mappable closed-answer tasks (TruthfulQA-MC, SciFact, BoolQ, ARC-Challenge; 2,865 failures; six instruction-tuned models of at most 8B parameters, mainly Qwen2.5-1.5B and Gemma-2-2B). The main pre-query quantity is the final-window margin did_i between the gold label and the realized wrong label (last R+1=4R+1=4 steps, radius R=3R=3, Sec. 2.3). Claims:

  1. Recovery rises across did_i quintiles from 0.0% (Qwen) and 4.9% (Gemma) in Q1 to 39.4% and 60.2% in Q5, with Q5 above Q1 in every task (Fig. 2, Table 9).
  2. The effect survives a literal-answer-token readout (AUROC 0.794 vs 0.800) and a wrong-candidate-probability control (0.789 vs 0.661) on 607 Qwen failures (Table 2).
  3. did_i predicts whether the answer changes and, among non-binary flips, whether it lands on gold (Table 3).
  4. Residual-error stability is a complementary evaluation axis to accuracy, shown mainly by a SciFact contrast between Granite and Gemma (Table 4).
Critical strengths
  1. Careful, self-consistent bookkeeping with the controls a skeptic asks for first. Counts reconcile across Tables 1, 8, 12 and 19, and the bin counts in Fig. 2 reproduce the headline 0/137 → 54/137 (Qwen) and 6/123 → 74/123 (Gemma). The paper adds a literal-answer-token audit (Table 2), a wrong-candidate-confidence control (App. A.10), correct-case false-revision rates (Table 12), and states its own limits (Sec. 4.3, 5; App. A.11). I trust the numbers as far as they go.

  2. The change-vs-destination split is the most informative result. Adding did_i to task identity raises gold-destination AUROC among non-binary flips (Qwen 0.666 → 0.858, Gemma 0.520 → 0.745, Table 3). This goes beyond "low-confidence errors flip more often" and is the best support for the framing.

Critical weaknesses
  1. The central signal is not compared with the obvious gold-free baseline, so it cannot be separated from "close-call errors flip under any perturbation."

    • Decoding is greedy and, in all 607 audited Qwen cases, the answer is the first generated token (Table 2). The realized wrong label is then the argmax, so the literal margin is essentially minus the logit gap between the chosen and gold labels. The natural competitor is the gold-free top-1 minus top-2 candidate margin (or label-set entropy), which is identical for BoolQ. It is never reported; the only confidence control is the wrong candidate's full-vocabulary probability (Table 2).
    • The authors concede that the runner-up explanation is not ruled out (Sec. 4.2, 4.3, 5). It bites hardest on the paper's most distinctive result, gold-destination AUROC in Table 3: if gold is usually the runner-up, "the flip lands on gold" follows from "the flip goes to the runner-up."
    • The audits are Qwen-only. For Gemma, which has the larger effect, the curve is not monotone at the low end (6, 0, 10, 22 of about 123 in Q1–Q4) before jumping to 74/123 (Fig. 2), and 23 of 24 readout-edge cases recover (App. A.11); the authors note that Q5 recovery is still at least 51/99 = 51.5% after removing all 24.

    This is decisive because, without this baseline, the reading of did_i as showing where an unstable decision moves cannot be separated from a generic runner-up effect. The paper does not claim a gold-specific mechanism (Secs. 4.3, 5), so my concern is about interpretation, not an overclaim. Since did_i needs the gold label, a gold-free equivalent would also matter for how widely the method can be used.

  2. The "complementary evaluation axis" claim is thin, and the scope is narrow.

    • It rests on one contrast: SciFact, Granite vs Gemma (Table 4). The two models leave different residual sets (108 vs 155 failures on the same 340 items, Table 8), and a more accurate model's remaining errors are plausibly harder. No matched-item analysis is attempted.
    • Trajectory descriptors carry little information because responses are very short (about 2 tokens for TruthfulQA-MC, BoolQ and ARC and 7.2 for SciFact in the Qwen audit set, Table 17; descriptor-only AUROC 0.551, Table 14), and post-query diagnostics are near tautological (Table 19).
    • Evidence covers closed-answer tasks, six models of at most 8B parameters and one suffix-style re-query (four prompt variants, Table 13). Sec. 5 states it does not establish applicability to open-ended or multi-token answers, and the method needs the gold label, so it is an offline analysis tool only.

    The empirical takeaway is narrower than the title and introduction suggest.

Questions for authors
  1. [Clarification] Can you report the gold-free baseline (top-1 minus top-2 logit margin, or label-set entropy), alone and with task identity, on the 607 Qwen failures and on Gemma? Does did_i add AUROC on top of it, for both "any flip" and "gold destination" (Table 3)? A clear increment would strengthen the main claim and could change my rating. If the gold-free version performs as well, please say so, since it would remove the need for gold labels.
  2. [Clarification] In what fraction of non-BoolQ flips is gold the runner-up at the answer token? What is the gold-destination AUROC of did_i within the subset where gold is not the runner-up?
  3. [Clarification] Table 2 reports 0.661 for Task + wrong-candidate probability on the 607 replay-matched cases, below the univariate wrong-candidate-probability AUROC of 0.728 on all 682 Qwen failures (Table 16). Is the difference due to the subset or to the task-identity control, and what is the univariate value on the 607 cases? Also, Table 14 reports 373 recovered cases for n=2,167n=2{,}167 (the retained counts of Qwen, Gemma, Granite and Llama), but the recovered counts of these four models in Table 12 sum to 371. Which records account for the difference?
  4. [Suggestion / future work] For the SciFact contrast (Table 4), please add recovery on items that both Granite and Gemma get wrong, and a breakdown by realized-wrong label.
  5. [Suggestion / future work] Please repeat the literal-token audit for Gemma, and test a larger model or a structurally different re-query. Is there a sequence-level version of did_i for open-ended or multi-token answers?
Reviewer_rrEoOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper investigates for a LLM how wrong answers can be changed by a non-informed re-query operation. For a correct and incorrect answer trace and for each position in the trace, the difference between the logits of correct and incorrect tokens, the so-called margin trajectory, can be summarized by different statistics and then used to predict the results of the re-query. This is predictive in 4% to 21% of the cases, depending on model and dataset.

Critical strengths

None

Critical weaknesses
  • The paper seems to be written entirely by an LLM, is therefore extremely hard to follow und understand, especially due to discussing terms that have not been defined before and a very information-dense writing style.
  • Multiple important baselines are missing such as a study and comparison on decoding methods (the paper only uses greedy decoding), or other entropy based works that take decisions based on various forms of token-wise entropy.
  • The results in this paper are in itself not sufficient for publication and show no practical way forward to being used.

I strongly recommend rejection in its current form.

Questions for authors
  • What if instead of greedy decoding one simply uses some decoding method?
  • How do these results compare to identifying correct and incorrect answers with entropy-based approaches?
  • How is this result practically relevant or can be used in practice?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.