- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces introspective verdict head (LVH), an approach for evaluating reasoning-step correctness using LLM hidden representations rather than generating explicit critiques or using expensive PRMs. The authors propose combining a final-layer linear readout with an intermediate-layer (70% of model depth) MLP readout. The backbone is adapted via LoRA using MATH-derived labeled conversations. On ProcessBench, LVH achieves mean first-error localization F1 scores of 58.1 (1.5B) and 68.5 (7B) with a single forward pass, without text generation or code execution. Combining LVH with a generative process reward model further improves mean F1 from 75.2 to 77.4 at 7B.
Critical strengths
- Empirical evidence shows competitive performance of the proposed LVH approach compared to other PRMs - it can achieve similar accuracy at lower cost (Figure 1)
Critical weaknesses
- Limited novelty relative to prior work.
The main contribution of the paper appears to be the use of LLM hidden representations for step-level correctness verification.
However, this direction has already been explored in prior work, particularly ReProbe (Ni et al., 2026), which is also cited in the paper. ReProbe investigates step-level correctness prediction from LLM hidden states and compares its performance against process reward models (PRMs). It also examines several aspects presented as contributions of LVH, including linear probing, generalization across datasets, and fusion with generative PRMs. Given this huge overlap, the specific novelty of LVH is not clear. A direct comparison with ReProbe would also be valuable, especially since ReProbe does not require a separate backbone inference pass and may therefore be more computationally efficient while achieving competitive verification performance.
- Unclear benefits of the proposed architecture.
Seems like another contribution is the architecture combining an intermediate-layer MLP readout with a final-layer linear readout, along with LoRA adaptation of the backbone. However, the ablation results in Table 3 show only modest improvements over simpler single-layer alternatives. This raises questions about whether the additional architectural complexity is sufficiently justified by the observed performance gains.
Questions for authors
-
The paper would benefit from a direct comparison with ReProbe, as well as a more careful discussion of its relationship to other relevant prior work, to better clarify its contributions.
-
More evidence would be helpful to support the motivation for the proposed two-layer architecture. In particular, the claim that "the final state could be misled by a fluent but wrong step, while an earlier state may align more closely with whether the step follows from its premises" does not seem sufficiently supported by the current experiments. The ablations also provide limited evidence that combining the two layers is preferable to simpler alternatives.
Additional suggestion: The writing could be clearer in several places. E.g., the abstract describes the approach as using the "verifier's own hidden states," which initially led me to believe that the method uses hidden states from some PRM, but (if I understand correctly) it instead uses the backbone LLM's hidden states.
I am open to further discussion and willing to reconsider my assessment if I have misunderstood any aspects of the paper, particularly given some ambiguities in the presentation.
I request the reviewers to respond to my and the other reviewers' comments! Please share any results you'd like to show us (better if shared as a general comment for other reviewers). I believe I have suggested which experiments to run quite clearly and looking forward to it.