acceptodds
Under review as a conference paper at ICLR 2027

How LLMs Detect and Correct Their Own Errors: The Role of Internal Confidence Signals

Abstract

Large language models can detect their own errors and sometimes correct them without external feedback, but the underlying mechanisms remain unknown. We investigate this through the lens of second-order models of confidence from decision neuroscience. In a first-order system, confidence derives from the generation signal itself and is therefore maximal for the chosen response, precluding error detection. Second-order models posit a partially independent evaluative signal that can disagree with the committed response, providing the basis for error detection. Kumaran et al. (2026) showed that LLMs cache a confidence representation at a token immediately following the answer (i.e. post-answer newline: PANL)—that causally drives verbal confidence and dissociates from log-probabilities. Here we test whether this signal supports error detection—the core prediction of the second-order framework—and a hypothesized extension to self-correction. Using a verify-then-correct paradigm, we show that: (i) verbal confidence predicts error detection far beyond token log-probabilities, ruling out a first-order account; (ii) PANL activations predict error detection beyond verbal confidence itself; and (iii) PANL predicts which answers the model will revise, and critically which errors the model can correct—beyond all behavioural signals. Causal interventions establish that PANL signals rescue error detection when answer information is corrupted, and that ablating answer-adjacent activations including PANL disrupts self-correction. All findings replicate across models (Gemma 3 27B and Qwen 2.5 7B) and datasets (TriviaQA, PopQA, and MNLI). These results reveal that LLMs naturally implement a second-order confidence architecture whose internal evaluative signal encodes not only whether an answer is likely wrong but whether the model has the knowledge to fix it.

Primary area:
foundation or frontier models, including LLMs
Keywords:
confidence, LLMs, error detection, metacognition

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_iMJhOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Self-correction in LLMs refers to the ability of the LLM to correct an previously incorrect answer without any external feedback. The authors want to explore if the model's confidence can be used in self-correction to predict if a model can identify an error and successfully correct it. Through their experiments, they revealed that using the post-answer newline token (PANL) achieves the best performance in predicting the model's verification behavior compared to using the model's answer token logprobs and the verbalized confidence scores. They also showed how PANL is also correlated with if a model's self-correction is successful. Lastly, they performed activation patching to causally verify their conclusions that PANL can be used for error detection (i.e. verification behavior) and PANL plays an important role, along with the last answer token, in self-correction.

Critical strengths
  1. Thorough analysis: I liked how the authors incrementally demonstrated how PANL is correlational with verification behavior prediction and self-correction success prediction, and proved their conclusions causally.
  2. Cross-dataset performance: the results showed how probes on similar datasets transfer well, whereas probes on substantially different datasets don't. This is interesting because it suggests that the verification signal could be a global, model-specific signal.
Critical weaknesses
  1. Limited experiments: most experiments were only tested on Gemma 3 27B, and the results showed that it had a strong Y bias, so it is unsure if there are any confounding variables.
  2. PANL for error detection: in section 4, one paragraph says that PANL is causally sufficient for error detection, but in the next one, it says that it is not necessary. The two experiments used two different ablation methods; then would the conclusions change if the ablations were kept the same?
Questions for authors
  1. There are quite a few works saying how LLMs are overconfident when prompted to generate confidence scores. But, in the experiments, you discovered that adding the verbalized confidence scores improved the verification response prediction performance. This then means that the confidence scores does carry some important signals, which would contradict previous works. Why is that?
  2. Do all models exhibit the strong Y bias as Gemma 27B? Would that affect the conclusions made in the experiments section?
Reviewer_F2vPOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates wether internal confidence representations help explain how LLMs recognize and corret their mistakes, focusing on analyzing post-answer newline token(PANL). The authors split experiments into first answer, verification and self-correction three stages to study PANL wheter can predict model's judgement, whether modify answers as well as whether correction is successful. Experiments mainly use Gemma3 27b and test in qwen2.5 7b. datasets including TriviaQA, PopQA and MNLI. The experiments find PANL's correction success AUROC is 0.614, higher than behaviral baseline(0.475).

Critical strengths
  1. This psper addresses a meaningful qeustion.The problem clearly distinguishs from error detection, answer modification and correction success. This paper studies these situations seperately and analyzes data samples that having initially wrong but changing answer later.
  2. This paper conducts extensive experiments using Gemma 3 27B and Qwen 2.5 7B and across TriviaQA, PopQA, and MNLI. It not only reports probe accuracy, but also conducts activation patching, single postion ablation and LAT+PANL joint ablation.
  3. This paper connects internal representation with actual decision making. selective self-correction experiments show that applying correction only on half samples could have the same accuracy as all samples.
Critical weaknesses
  1. Ruling out a first-order account is a bit overclaimed.This paper compares PANL with chosen-token log-probability statistics. These statistics cannot represent all useful information in the generation process. consider adding generation-time hidden states as baselines.
  2. Causal experiments cannot isolate confidence-specific information. Intervention on whole residual-stream vector impact answer semantics, identity and evaluation information. These experiment results show relative token positions participate in the generation process, but cannot specify confidence signal.
Questions for authors
  1. Which experiment can distinguish second-order mechanism and same model see complete answer and do additional computation?
  2. How can we design intervention that distinguish confidence-specific information and general answer representation?
  3. how to explain correctability's performance difference in various tasks and weak transfer?
Reviewer_ggLPOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates whether large language models use internal evaluative signals to detect and correct their own errors. Building on second-order models of confidence from decision neuroscience and prior work by Kumaran et al. (2026), which identified confidence representations at the post-answer newline (PANL), the authors examine whether PANL activations predict verification decisions, answer revisions, and successful self-correction.

The experiments employ a verify-then-correct paradigm, using Gemma 3 27B on TriviaQA as the primary experimental setting, with additional evaluations on Qwen 2.5 7B, PopQA, and MNLI.

The authors present the following findings:

  1. Verification decisions: Verbal confidence predicts verification decisions beyond answer log-probabilities.
  2. Internal representations: Linear probes trained on PANL activations predict verification decisions beyond all behavioral signals.
  3. Correctability: PANL probes predict which revisions will succeed (correctability), whereas behavioral signals perform at chance level.
  4. Causal evidence: Activation patching shows that PANL activations are sufficient to restore error detection after answer corruption, while mean ablation suggests that PANL and the last answer token (LAT) carry redundant signals. A noise-injection experiment extends these findings to self-correction.
Critical strengths
  1. The combination of behavioral analysis, linear probing, and causal interventions provides a comprehensive approach to investigating the role of internal signals and hidden states in error detection and self-correction.

  2. The paper extends prior research on PANL confidence representations by distinguishing among error detection, answer revision, and successful correction. Although the main correctability result on TriviaQA is modest, the finding is supported by additional experiments on PopQA, MNLI, and Qwen, as well as foil-based analyses and cross-dataset probe transfer. The selective-correction experiments further demonstrate potential practical relevance by achieving nearly the same accuracy as unconditional correction while reducing the number of correction calls. This combination of conceptual novelty, replication across datasets and models, and application-oriented evaluation makes the contribution significant.

Critical weaknesses
  1. The paper shows that verbal confidence and PANL activations predict verification decisions better than token log-probabilities. However, it does not evaluate stronger first-order alternatives, such as entropy or probability margins. Therefore, the results support a second-order explanation but do not conclusively establish it.

  2. Restoring PANL activations improves error detection, but PANL may also contain information about the original answer. It remains unclear whether the improvement results from restoring an evaluative signal or merely recovering information about the original answer.

  3. The additional effect of jointly ablating LAT and PANL does not necessarily demonstrate that they carry redundant information.

Questions for authors

How much do the PANL results change when the verification prompt is worded or formatted differently, such as by modifying the newline placement or replacing “Y/N” with longer responses? Have the authors tested different prompt templates?

How does PANL prediction performance compare when activations are extracted during the initial answer phase (Phase 0) versus the verification phase (Phase 1)?

Comments:

The paper contains several inconsistencies that should be clarified. Appendix A.3 defines P(IK) based on agreement with the ground truth, whereas Appendix G.4 describes it in terms of agreement with the modal sampled response. The timing of PANL extraction is also described inconsistently between the experimental methodology and the caption of Figure 3. Clarifying these issues would improve reproducibility and confidence in the reported results.

Figure A5 is missing and displays a placeholder path instead of the intended figure.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.