- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper focuses on diversity collapse in LLM post-training, particularly the loss of diversity among correct reasoning trajectories. It proposes STRIDE, which measures the novelty of individual reasoning steps through regularized reconstruction and adds step-specific advantages to GRPO. Experiments on five mathematical reasoning benchmarks report improvements in Pass@32 and reasoning diversity, while maintaining comparable Pass@1.
Critical strengths
The paper examines an important problem concerning the diversity of reasoning paths. STRIDE also achieves competitive results against the evaluated baselines, particularly on Pass@32 and the reported diversity measures.
Critical weaknesses
1.The paper discusses diversity collapse in existing post-training methods and reviews approaches to promoting diversity, but does not sufficiently explain why assigning diversity credit to individual reasoning steps would help address this problem. The experiments also do not clearly establish how much of the improvement comes from the step-level design. In Table 3, both Token-uniform and Single-step use only correct rollouts as the novelty reference set, whereas STRIDE includes both correct and incorrect rollouts. Since the reference set also affects the novelty scores, these comparisons do not isolate the contribution of the step-level design.
2.The paper discusses representation-based exploration and existing step-level credit assignment methods, but the experimental baselines are limited to GRPO and its likelihood- and entropy-based variants. Without direct comparisons with these closely related methods, it remains unclear what additional benefits STRIDE provides over existing approaches.
Questions for authors
Equation (5) measures step novelty using a reconstruction cost that combines reconstruction error with an L2 penalty on the coefficients. What motivates including the coefficient penalty in the novelty measure, and how does this reconstruction-based novelty relate to meaningful differences in reasoning strategies?
Response to the Reviewer.
Thank you for your time and constructive feedback.
Matched reference set. We reran both variants with the same all-eligible reference set and settings as STRIDE:
STRIDE outperforms both variants on every metric. Token-uniform, which has step scores but no local allocation, does no better than Single-step. The gain therefore comes from local allocation. We have updated Table 3 with these results in the revised version.
Questions For Authors
Thanks for the question. Without the penalty, any target in the span of the comparison steps is reconstructed with zero error, even if that takes large coefficients that cancel each other out. A step could then look "not novel" just because some odd combination of other steps happens to reproduce it. The L2 penalty rules out these fragile fits, so the score reflects how well the comparison steps actually support the target. It also keeps the problem well-posed: for >0, the regularized normal matrix is positive definite and the solution is unique. From Eq. (6),
c(ϕ∣Z)=λϕ⊤(Z⊤Z+λI)−1ϕ,
so directions the comparison steps cover well get low scores, and directions they barely cover get high scores. This is closely related to the regularized elliptical bonuses used in exploration.
On the second part, the score measures novelty in the frozen model's representation space, and we do not claim it guarantees a different mathematical strategy. What we want it to pick up is a step that brings in a new reduction or argument compared to the other rollouts and the earlier steps of the same response. A step that only restates something already there is easy to reconstruct and scores low. Our evidence for this is indirect but consistent. With the reference set matched, local allocation gives higher S@32 (point 1 above), and the S@32 judge agrees with blinded human labels on 78.9% of pools. We have made this distinction clear in the revision (line 202, 214).
We hope this answers your questions and would be grateful if you would kindly reconsider your rating.