- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes PAIR, a phase-aligned within-question framework for analyzing hidden states along reasoning trajectories. Instead of comparing states across different questions or aligning trajectories by absolute step indices, PAIR compares successful and unsuccessful trajectories for the same question at similar relative phases. The method learns phase-specific path-quality directions and evaluates them on trajectory ranking, Best-of-N selection, and activation steering. The results suggest that standard across-question probes partly capture question-level information, while PAIR provides more informative trajectory-level signals.
Critical strengths
-
The paper identifies an important limitation of across-question correctness probing, showing that probe performance may partly reflect question-level information rather than reasoning quality.
-
PAIR is well motivated and shows consistent gains in trajectory ranking, Best-of-N selection, and steering.
Critical weaknesses
-
The paper is difficult to follow, and the main motivation and method could be explained more clearly.
-
The contribution of phase alignment is not fully isolated. The strong normalized-step baseline suggests that part of the gain may come from relative-position normalization rather than meaningful phase structure.
Questions for authors
- Could the authors better isolate the benefit of phase alignment from simple relative-position normalization? The normalized-step baseline is already strong in several settings.
4 comments
Response to Reviewer ksno (2/2): W2 and Q1
W2 & Q1. Phase Alignment and Relative-Position Normalization
Thank you for this question. We would like to clarify that PAIR does use relative position, and the strong Normalized-step baseline confirms that relative position matters. Our results show that (1) PAIR's gain comes from combining relative position with within-question comparison, not from relative position alone, and (2) the resulting phases correspond to meaningful reasoning stages. Normalized-step takes the step closest to 80% progress in one greedy trajectory per question and trains a pointwise probe on it. PAIR instead compares correct and incorrect trajectories of the same question at each phase. To separate these components, we compare four methods from the revised Table 2 (Section 4.3):
| Model | Method | WQ-AUC (GSM8K) | BoN@32 (GSM8K) | WQ-AUC (MATH) | BoN@32 (MATH) | WQ-AUC (BBH) | BoN@32 (BBH) |
|---|---|---|---|---|---|---|---|
| Llama-3.1-8B | Normalized-step | 0.67 | 85.3 | 0.71 | 40.0 | 0.58 | 68.0 |
| Fixed-pos. pairwise | 0.74 | 82.0 | 0.69 | 42.7 | 0.72 | 86.7 | |
| Stage-aligned † | 0.73 | 88.0 | 0.83 | 44.0 | 0.74 | 89.3 | |
| PAIR | 0.76 | 87.3 | 0.77 | 46.7 | 0.74 | 92.0 | |
| Qwen3-4B | Normalized-step | 0.68 | 80.7 | 0.60 | 76.0 | 0.62 | 94.6 |
| Fixed-pos. pairwise | 0.81 | 80.7 | 0.77 | 79.3 | 0.69 | 92.0 | |
| Stage-aligned † | 0.78 | 84.0 | 0.86 | 85.3 | 0.90 | 96.0 | |
| PAIR | 0.91 | 80.7 | 0.89 | 84.7 | 0.86 | 97.3 | |
| Gemma3-4B | Normalized-step | 0.57 | 88.0 | 0.54 | 66.7 | 0.62 | 96.0 |
| Fixed-pos. pairwise | 0.56 | 90.7 | 0.61 | 73.3 | 0.66 | 94.6 | |
| Stage-aligned † | 0.66 | 90.0 | 0.60 | 76.7 | 0.78 | 97.3 | |
| PAIR | 0.61 | 95.3 | 0.74 | 75.3 | 0.75 | 96.0 | |
| R1-Distill-8B | Normalized-step | 0.57 | 86.6 | 0.66 | 80.0 | 0.84 | 100.0 |
| Fixed-pos. pairwise | 0.65 | 88.7 | 0.73 | 84.0 | 0.77 | 94.6 | |
| Stage-aligned † | 0.79 | 90.7 | 0.89 | 86.7 | 0.88 | 100.0 | |
| PAIR | 0.75 | 89.3 | 0.87 | 86.7 | 0.94 | 100.0 |
† Stage-aligned uses step-level LLM annotation and serves as a reference. It is excluded from bold highlighting.
Normalized-step uses relative position without within-question comparison, and Fixed-pos. pairwise uses within-question comparison without relative position. PAIR combines both. It is never below Normalized-step (better in 21 of 24 cells, tied in 3) and outperforms Fixed-pos. pairwise in 23 of 24 cells (tied in 1). Each component therefore contributes, and their combination gives the strongest ranking signal. PAIR also performs close to Stage-aligned, which aligns steps by LLM-annotated functional stages, without needing any labels.
Normalizing by step count captures functional structure. Relative position can be measured in different ways, so we also checked which one best groups steps with similar roles. We labeled every reasoning step as Setup, Calculation, Verification, or Answer using Gemma-3-27B-IT with a fixed rubric (92% agreement with human labels on 200 sampled steps). For pairs of steps from two trajectories of the same question with different numbers of steps, we measure how often two steps in the same slot have the same stage. We compare phase alignment (normalized by step count) with Token-position normalization (normalized by token position) and absolute steps.
| Model | Dataset | Phase (PAIR) | Absolute step (No normalization) | Token-position normalization | Chance |
|---|---|---|---|---|---|
| Llama-3.1-8B | GSM8K | 0.792 | 0.740 | 0.788 | 0.527 |
| MATH-500 | 0.655 | 0.592 | 0.610 | 0.470 | |
| BBH | 0.695 | 0.726 | 0.778 | 0.455 | |
| Qwen3-4B | GSM8K | 0.659 | 0.696 | 0.620 | 0.387 |
| MATH-500 | 0.563 | 0.474 | 0.491 | 0.378 | |
| BBH | 0.692 | 0.678 | 0.600 | 0.377 | |
| Gemma3-4B | GSM8K | 0.817 | 0.748 | 0.722 | 0.527 |
| MATH-500 | 0.672 | 0.644 | 0.565 | 0.441 | |
| BBH | 0.716 | 0.663 | 0.611 | 0.410 | |
| R1-Distill-8B | GSM8K | 0.661 | 0.630 | 0.543 | 0.434 |
| MATH-500 | 0.574 | 0.504 | 0.481 | 0.412 | |
| BBH | 0.635 | 0.598 | 0.487 | 0.347 |
Normalizing by step count matches the annotated stages better than normalizing by token position in 11 of 12 settings. Steps are the units in which the model organizes its reasoning, so aligning by step count groups steps that play similar roles more reliably. Results hold for to (Appendix A.8).
Thank you for your detailed responses and clarifications. I appreciate the authors' efforts in addressing my concerns. I will maintain my positive rating.
Thank you for your positive assessment. We are glad our responses addressed your concerns.
Response to Reviewer ksno (1/2): W1
Thank you for your constructive feedback. We hope our responses will strengthen confidence in supporting our submission.
W1. Clarity of motivation and method
Thank you for this feedback. To make the main idea easier to follow, we summarize it here and have revised the paper to make it clearer in the same way.
Motivation. Prior work probes hidden states along a reasoning trajectory by labeling every state with the final-answer correctness and training a probe across many questions. This has two problems. First, the probe can rely on how hard a question is rather than on the quality of a particular trajectory. Second, states at the same absolute step index can play different roles in different trajectories.
Method. PAIR addresses both problems. It samples many trajectories for each question and compares correct and incorrect trajectories only within the same question, which removes question-level differences. It also maps each trajectory onto a fixed number of phases by relative progress, so that compared states sit at similar points in their trajectories. Within each phase, it learns a direction that ranks correct trajectories above incorrect ones. We use these directions for trajectory ranking and Best-of-N selection, and test their causal effect with steering.
Revisions made in the updated manuscript.
If there are any other parts that remain unclear, we would be grateful for any pointers and will revise them further.