- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Prototype-Mediated Process Supervision (PMPS), which extends a CODI-style framework with soft, many-to-many alignment between explicit CoT representations and latent states. A gradually relaxed positional prior encourages sequential alignment during training. Experiments across multiple model families and mathematical benchmarks show accuracy improvements over the evaluated latent reasoning baselines with comparable output lengths.
Critical strengths
- The motivation is clear, and the method provides a coherent approach to process supervision without requiring predefined CoT-step segmentation or an additional language-model decoder.
- The evaluation includes multiple model families, a strong CODI-based SIM-CoT baseline, and MATH experiments. Component ablations provide useful evidence for the proposed design.
- The paper is generally well written, with clear diagrams and detailed methodological descriptions.
Critical weaknesses
- The contribution appears relatively incremental. PMPS adds prototype-based alignment and a scheduled positional prior to the CODI framework, with evaluation largely following existing CODI/SIM-CoT settings. Average improvements over SIM-CoT are modest—approximately 1.2 percentage points on both 1B and 3B—leaving the significance of the methodological advance insufficiently demonstrated.
- The interpretability comparison does not adequately evaluate SIM-CoT’s capabilities. Appendix D projects SIM-CoT’s latent states through the base LM head instead of its trained auxiliary decoder, which is designed to reconstruct intermediate reasoning steps. This may underestimate the information recoverable from SIM-CoT’s latents and limits conclusions about relative interpretability. A comparison using its intended decoder would be more informative.
Questions for authors
- Could the authors include Coconut + SIM-CoT under matched settings, particularly for the GPT-2 comparison in Figure 1, and explicitly label the SIM-CoT variants?
- Could the interpretability comparison include SIM-CoT’s trained auxiliary decoder? Do the observed differences persist when using this decoder instead of the base LM head?
- Could the authors report training GPU-hours and peak GPU memory for PMPS, CODI, and SIM-CoT under comparable hardware and training settings? This would clarify the actual training savings from replacing SIM-CoT’s auxiliary decoder with prototype-based supervision.