- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
Self-correction in LLMs refers to the ability of the LLM to correct an previously incorrect answer without any external feedback. The authors want to explore if the model's confidence can be used in self-correction to predict if a model can identify an error and successfully correct it. Through their experiments, they revealed that using the post-answer newline token (PANL) achieves the best performance in predicting the model's verification behavior compared to using the model's answer token logprobs and the verbalized confidence scores. They also showed how PANL is also correlated with if a model's self-correction is successful. Lastly, they performed activation patching to causally verify their conclusions that PANL can be used for error detection (i.e. verification behavior) and PANL plays an important role, along with the last answer token, in self-correction.
Critical strengths
- Thorough analysis: I liked how the authors incrementally demonstrated how PANL is correlational with verification behavior prediction and self-correction success prediction, and proved their conclusions causally.
- Cross-dataset performance: the results showed how probes on similar datasets transfer well, whereas probes on substantially different datasets don't. This is interesting because it suggests that the verification signal could be a global, model-specific signal.
Critical weaknesses
- Limited experiments: most experiments were only tested on Gemma 3 27B, and the results showed that it had a strong Y bias, so it is unsure if there are any confounding variables.
- PANL for error detection: in section 4, one paragraph says that PANL is causally sufficient for error detection, but in the next one, it says that it is not necessary. The two experiments used two different ablation methods; then would the conclusions change if the ablations were kept the same?
Questions for authors
- There are quite a few works saying how LLMs are overconfident when prompted to generate confidence scores. But, in the experiments, you discovered that adding the verbalized confidence scores improved the verification response prediction performance. This then means that the confidence scores does carry some important signals, which would contradict previous works. Why is that?
- Do all models exhibit the strong Y bias as Gemma 27B? Would that affect the conclusions made in the experiments section?