- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a two-stage self-awareness framework for LLM reliability estimation. It trains two lightweight probes on a frozen LLM's hidden states to predict pre-generation competence and post-generation answer correctness, respectively. Sample-derived correctness and consistency signals are used as auxiliary supervision. Experiments across multiple models and QA benchmarks demonstrate improvements in confidence estimation and calibration over the evaluated baselines.
Critical strengths
- Important problem and encouraging empirical results. Reliable confidence estimation is important for LLM deployment. The paper evaluates multiple model families across in-distribution and out-of-distribution settings, with generally promising improvements in confidence estimation and calibration.
- Useful empirical analyses. The paper includes ablations on auxiliary supervision, feature selection, and stage-specific confidence signals, providing some insights into the factors affecting reliability estimation.
Critical weaknesses
- Limited novelty and insufficient baseline comparisons. Prior work already uses pre- and post-generation hidden-state probes for reliability estimation. This paper distinguishes stage-specific prediction targets and adds sample-derived auxiliary supervision, but its methodological advance over existing probing approaches remains modest. Moreover, SA and the KB baseline use different probe architectures and optimization settings. Although the supervision ablations isolate the benefit of auxiliary targets within the SA configuration, they do not explain the substantial gap between KB and hard-only Post. Thus, the sources of the reported gains over KB remain insufficiently resolved.
- Unclear benefit of the two-stage design and limited evaluation. The main SA results rely only on the Post head, while combining Pre and Post does not consistently improve performance. This raises questions about the necessity of the two-stage framework. Moreover, the evaluation is limited to static QA tasks, without considering more realistic agentic scenarios involving tool use and multi-step decisions.
I would be open to raising my score if the authors can satisfactorily address these concerns and questions.
Questions for authors
- Given the existing supervision ablations, could the authors explain the substantial AUROC gap between KB and hard-only Post? A controlled comparison varying probe architecture and auxiliary supervision under a common optimization and model-selection protocol would help distinguish architectural and training effects from the benefit of the proposed supervision.
- What is the actual benefit of the Pre head if the main results only use Post?
- Could the authors evaluate the method on agentic tasks to demonstrate its practical value for reliability estimation?