acceptodds
Under review as a conference paper at ICLR 2027

Introspective Verification: Reading Step Correctness at Intermediate Depth

Abstract

Process reward models score the correctness of individual reasoning steps, and these scores drive selection and search in multi step reasoning. The most accurate of these models work through generation, decoding a critique for every step and often calling an interpreter, which makes checking a solution expensive. Whether the same judgment can instead be read directly from the verifier's own hidden states has not been resolved. We use a linear probe that locates the signal at an intermediate layer near seventy percent of the depth, where step correctness is more separable than at the final layer. We call this introspection, since the verdict is decoded from the backbone's own representation rather than generated as text. Once each readout is trained rather than probed, however, the final layer becomes the stronger readout on its own. Our introspective verdict head sums a trained intermediate readout and a trained final readout into a per step reward in a single forward pass, with no generation and no code execution. We adapt the backbone with LoRA on about eight thousand conversations from one released label set, and both training and inference run on a single 32GB V100 at the 1.5B and the 7B scale. At 1.5B the fusion hyperparameters are tuned only on MATH and applied unchanged to three domains never touched during tuning, which shows cross-domain generalization. On ProcessBench the 1.5B head reaches 58.1 mean F1, above most trained process reward models including ones at 7B and 14B scale, at about one quarter the token cost of a single generative pass. Fused with a 7B generative verifier it raises mean F1 from 75.2 to 77.4, without lowering any subset. Code is available at https://anonymous.4open.science/r/LLM_CALM-48B7.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
mathematical reasoning, step-level verification, hidden state probing

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_q3h2OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces introspective verdict head (LVH), an approach for evaluating reasoning-step correctness using LLM hidden representations rather than generating explicit critiques or using expensive PRMs. The authors propose combining a final-layer linear readout with an intermediate-layer (70% of model depth) MLP readout. The backbone is adapted via LoRA using MATH-derived labeled conversations. On ProcessBench, LVH achieves mean first-error localization F1 scores of 58.1 (1.5B) and 68.5 (7B) with a single forward pass, without text generation or code execution. Combining LVH with a generative process reward model further improves mean F1 from 75.2 to 77.4 at 7B.

Critical strengths
  1. Empirical evidence shows competitive performance of the proposed LVH approach compared to other PRMs - it can achieve similar accuracy at lower cost (Figure 1)
Critical weaknesses
  1. Limited novelty relative to prior work.

The main contribution of the paper appears to be the use of LLM hidden representations for step-level correctness verification.

However, this direction has already been explored in prior work, particularly ReProbe (Ni et al., 2026), which is also cited in the paper. ReProbe investigates step-level correctness prediction from LLM hidden states and compares its performance against process reward models (PRMs). It also examines several aspects presented as contributions of LVH, including linear probing, generalization across datasets, and fusion with generative PRMs. Given this huge overlap, the specific novelty of LVH is not clear. A direct comparison with ReProbe would also be valuable, especially since ReProbe does not require a separate backbone inference pass and may therefore be more computationally efficient while achieving competitive verification performance.

  1. Unclear benefits of the proposed architecture.

Seems like another contribution is the architecture combining an intermediate-layer MLP readout with a final-layer linear readout, along with LoRA adaptation of the backbone. However, the ablation results in Table 3 show only modest improvements over simpler single-layer alternatives. This raises questions about whether the additional architectural complexity is sufficiently justified by the observed performance gains.

Questions for authors
  1. The paper would benefit from a direct comparison with ReProbe, as well as a more careful discussion of its relationship to other relevant prior work, to better clarify its contributions.

  2. More evidence would be helpful to support the motivation for the proposed two-layer architecture. In particular, the claim that "the final state could be misled by a fluent but wrong step, while an earlier state may align more closely with whether the step follows from its premises" does not seem sufficiently supported by the current experiments. The ablations also provide limited evidence that combining the two layers is preferable to simpler alternatives.

Additional suggestion: The writing could be clearer in several places. E.g., the abstract describes the approach as using the "verifier's own hidden states," which initially led me to believe that the method uses hidden states from some PRM, but (if I understand correctly) it instead uses the backbone LLM's hidden states.

I am open to further discussion and willing to reconsider my assessment if I have misunderstood any aspects of the paper, particularly given some ambiguities in the presentation.

Reviewer_SnMuOpenReview ↗
Rating
1
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a simple classifier that reads hidden states to find errors in math reasoning traces and reduce verification cost. The authors claim an advance over earlier verifiers by combining signals from an intermediate layer and the final layer. Experiments on 1.5B and 7B models on ProcessBench’s four math subsets show gains in scores with a lower verification cost.

Critical strengths

The main strength of the paper is that the motivation is very clear for anyone working in post-training. In this sense, (1) low verification cost is an important point. It's impressive that an intermediate layer can provide such reliable signals while also reducing the verification cost compared to baselines (e.g., 31s vs. 1s). I also appreciated the detailed case study in the appendix, especially Appendix C.

Critical weaknesses

Below are the most critical weaknesses that I consider important and should be addressed

  1. Robustness of the results: I could not see mentions of multi-seed runs, nor any error bars in any of the tables or the figures. I'm not convinced that the reported results could be robust. It would be good to see the same experiments repeated in different seeds. Furthermore, the full method is tested at two sizes of one model family. From my experience, algorithms or frameworks that access intermediate levels of the architecture tend to be strongly influenced by the model family. It would be good to see other models tested, at least the main results. The two sizes also use different calibration procedures. Overall, I'm not sure if we can clearly understand where the improvements come from.

And a slightly minor issue (minor because it can be done and requires less thinking. I have not considered the limited rebuttal timeline). 2. Baseline comparisons: I'm curious if the authors could provide more baseline comparisons for the 7B panel of Tab 1. The 7B panel of Tab 1 reports only GenPRM as a baseline. Other 7B baselines appear in the upper panel. I'm also concerned about the method's overlap with Ni et al., (2026).

Questions for authors
  • I've mentioned this above, but could the authors report variation across train seeds or data splits?
  • What gains remain when both sizes use the same calibration proecdure and have access to target-domain scores?
  • I'd like the authors' opinion on the paper's overlap with Ni et al., (2026) (the ReProbe paper) that also uses a small model to verify step correctness from internal states. Would this be concurrent work? From my understanding, the difference seems to come from the selection of layers to pull the internal states (vs. Reprobe uses all layers). To show strength over the baseline, I think it would be necessary to ablate over the combination of selected layers. I have a feeling that the paper does not give enough credit to the ReProbe paper in the related works section even though it explicitly studies the same topic: step verification from internal states.
1 comment

I request the reviewers to respond to my and the other reviewers' comments! Please share any results you'd like to show us (better if shared as a general comment for other reviewers). I believe I have suggested which experiments to run quite clearly and looking forward to it.

Reviewer_uhftOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes LVH, a process verifier combining intermediate- and final-layer readouts, trained with LoRA on approximately 8,000 labeled conversations. It reports 58.1 mean ProcessBench F1 at 1.5B without generation, and improved accuracy when combined with GenPRM.

Critical strengths
  • Addresses a practical problem: reducing the cost of step verification.
  • Simple architecture with promising reported latency savings and competitive small-model performance. Good ablations
Critical weaknesses
  • The proposed architecture’s advantage is unclear. Table 3’s final-layer-only baseline achieves 58.6 mean F1, exceeding LVH’s 58.1. LVH improves the two harder subsets slightly, but uncertainty is unreported. Capacity matched comparison needed.
  • Table 1 mixes released baseline scores with locally measured results, including fusion evaluated on 120 cases per subset. The claimed 7B improvement needs an explicitly reported GenPRM baseline on those exact cases and a paired confidence interval for the gain
  • Generality remains limited. Evaluation covers one mathematical benchmark, without downstream selection/search experiments
  • Lack of novelty, experiments done with very old work and datasets
Questions for authors

NA

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.