- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper studies whether step-level process supervision can be recovered from heterogeneous free-form reasoning trajectories when only final answers are verified. It first aligns trajectories to a task-specific latent consensus structure using fused unbalanced Gromov–Wasserstein transport, then applies structured variational inference with worker-specific transition and outcome parameters to estimate step validity. Experiments cover five reasoning benchmarks, downstream fine-tuning of two language models, and a smaller human-crowd study.
Critical strengths
-
Important problem and thoughtful formulation. Recovering process supervision without annotating every reasoning step is valuable, and the proposed alignment framework accommodates variable-length trajectories, different solution paths, and error correction.
-
Broad empirical scope. The paper evaluates five reasoning domains, two downstream model backbones, several ablations, and human as well as LLM crowdworkers. This is considerably more comprehensive than a single-benchmark evaluation.
Critical weaknesses
-
The evaluated method is not actually free of step-level supervision. Appendix D.2 states that expert step-validity labels from 10 validation questions per domain are used for hyperparameter selection. This is consequential because the method is materially sensitive to these choices: for example, Table 8 shows GSM8K AUPRC changing from 61.8 to 80.7 when changes from 4 to 12. Therefore, the experiments do not establish the paper’s central claim that process supervision can be recovered without external step-level annotations.
-
The latent state is not convincingly identified as logical validity. The only direct correctness signal for an entire latent sequence is a binary terminal outcome. The monotonicity constraints in Equations 6–7 orient the latent states but do not show that they correspond to logical correctness rather than merely an outcome-predictive hidden state. This concern is amplified by the conditional-independence assumption: multiple synthetic “workers” may be prompt variants of the same underlying LLM and therefore share systematic errors. No identifiability result or stress test with correlated, duplicated, or majority-wrong workers is provided. This undermines the main interpretation of the inferred probabilities.
Questions for authors
-
What completely label-free procedure would select in a new domain? Please report results using a configuration selected without any target-domain step-validity labels.
-
Under what assumptions is logical step validity identifiable from terminal outcomes alone? Can the authors provide either a formal recovery argument or controlled experiments with known latent states, including correlated and majority-wrong worker populations?