- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes SLPO, a training method for autoregressive latent reasoning models that use continuous vectors instead of discrete tokens for intermediate reasoning. It introduces a stopping gate and uses final-answer rewards to optimize a joint objective with three terms: a Gaussian surrogate score for latent transitions, stopping-time log-probabilities, and answer-token log-probabilities.
Critical strengths
-
The Gaussian surrogate offers a way to train continuous latent transitions using final-answer rewards. The method can be applied to existing COCONUT and CODI models.
-
The learned stopping gate lets the model adjust its reasoning length for each input. The extra forward passes used to estimate the Gaussian are only needed during training.
-
Pass@8 and Pass@16 improve across all tested combinations of latent models, backbones, and datasets. The method also works with different policy optimizers and soft-token inference.
Critical weaknesses
-
The paper does not show how much the latent Gaussian surrogate contributes beyond answer-token RL and stopping-gate training. I would like to see an ablation that removes only the surrogate term while keeping the rest of the setup unchanged.
-
The argument in Appendix E relies on the surrogate and true policy score gradients being sufficiently close, but the paper neither establishes this condition nor checks it empirically. It is therefore unclear why the proposed isotropic Gaussian is a suitable surrogate for latent policy optimization.
-
The COCONUT/CODI baselines use 6 latent steps, while SLPO allows up to 12 and sometimes uses nearly all of them. Without comparisons at matched inference budgets, it is difficult to tell how much of the improvement comes from the method itself and how much comes from additional computation.
-
Appendix D.2 reports per-step training costs but leaves out the stopping-gate cold start, which requires decoding answers at multiple prefix lengths. Total training GPU-hours, including this stage, would give a clearer picture of the method’s computational cost.
-
All evaluated backbones are small (<= 3B parameters). Could the authors also evaluate the method on a larger LLM?
Questions for authors
See the weaknesses above.
Dear Reviewer SK4F,
As the discussion phase nears its end, we wanted to gently check in to see if our response has resolved your concerns. Specifically, following your constructive suggestions, we added the component ablations (removing the stopping head vs. removing the surrogate likelihood), clarified CODI's test-time scaling and compute-efficiency gains, and provided an instance-level horizon analysis showing why adaptive stopping improves over a fixed 6-step budget.
We would love to hear if these results address your questions, and we remain happy to provide any further clarifications before the discussion period concludes. Thank you again for your time and support!
Best regards,
The Authors