- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Using 126M/325M models, the paper studies how outcome-reward RL selects terse or state-summary trajectories and changes inference cost. Independently trained terse policies and prefix edits distinguish alternative training paths from a frozen policy's dependence on summaries. Two profiled hybrids lose their measured speed advantage; shortening incurs architecture- and task-dependent accuracy costs, while the proposed style-stratified normalization (SSN) prevents the transition from terse reasoning to long trajectories with repeated state summaries in all nine tested runs.
Critical strengths
The paper addresses an important efficiency question through shared recipes within each scale, several recurrent mixers and additional initialization seeds. Wall-clock measurements at three batch sizes demonstrate a concrete consequence of post-training trajectory choices (Table 1).
Separating independently trained terse policies from frozen-policy prefix interventions is a valuable design choice. FORBID, SFT-style changes and SSN provide complementary evidence of training-path dependence, with failed non-inferiority tests and seed-sensitive results reported transparently.
Critical weaknesses
The distinctive mechanism remains insufficiently isolated. Figure 2 pools VARTRACK and SORT, despite state-style accuracy being 8–26 pp worse on SORT (Appendix C); SSN changes both advantage scaling and gradient sparsity, and positive advantages also occur without switching (§5). These findings do not yet separate the proposed feedback from task composition and strategy selection already studied in Krishnamurthy et al., Select and Improve (2026), which is missing from the related work.
The practical efficiency evidence remains narrow. The 126M/325M results are useful controlled observations, but pretraining histories differ across scales (§2; Appendix E). Higher-accuracy SFT bases at 325M remain terse and attain 94–96% accuracy (§3.1), leaving unresolved whether switching persists beyond a particular competence/difficulty regime. The authors acknowledge that generalization to production models is untested (§8); this limits the deployment significance of the cost reversal without invalidating the small-model findings.
Questions for authors
Task and capability controls: Does the accuracy/advantage reversal persist within each task/difficulty when comparing styles on the same prompts? After matching starting SFT accuracy and relative task difficulty, does switching still differ across architectures/scales? The current cross-scale comparison also changes pretraining history.
Causal validation: Does style-wise centering with the original prompt-level standard deviation still prevent switching? A random-partition control with comparable gradient sparsity and update magnitude would help distinguish the proposed mechanism from reduced or rescaled updates.
Comparative validation: How do L1/LCPO or the solvable-problem second stage of Concise Reasoning compare with SSN/FORBID on the same held-out items, reporting accuracy and wall time with graph setup amortized? Equation (3) does not implement either method.