- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes OASIS, which uses the length statistics of correct and incorrect rollouts for the same prompt to adjust the penalty threshold for correct responses during length-aware GRPO training. The authors construct candidates from the overall incorrect distribution and soft short/long failure components, select a threshold through scoring and curvature checks, and apply a batch-level iso-pressure gain to compensate for the penalty reduction caused by upward threshold shifts.
The authors evaluate OASIS on four reasoning models and five mathematical benchmarks. Relative to the base models, average generation length decreases by 32.7% and average accuracy increases by approximately 1.7 percentage points, with gains varying across models. The paper also includes SmartThinker retrained under matched settings and component ablations.
Critical strengths
-
The paper provides a concrete analysis of existing length estimation. Under single-Gaussian moment matching and the stated variance conditions, the authors show how marginal pooling attenuates the threshold correction derived from failures. This offers a useful diagnosis of existing probabilistic length estimation; the directional candidates further seek to preserve distinct short- and long-failure structures.
-
The matched comparison with the closest method supports some incremental value. Table 2 retrains SmartThinker on all four models with matched rollout settings and training steps. The final OASIS variant improves average accuracy and AE over this baseline on each model. Although it uses more tokens on two models, these results support an improved accuracy–length trade-off.
Critical weaknesses
-
The method definition and ablation attribution are unclear. Section 4.1, Eqs. (9)–(12), uses a common mixture objective, whereas Algorithm 1, lines 43–46, uses different references by candidate type. The regular branch also leaves the subsequently used overall incorrect reference undefined. Table 3 does not specify whether adding directional candidates also changes the scoring density, making their independent contribution difficult to verify. These issues hinder reproduction and ablation attribution, but do not establish that the implementation is incorrect.
-
The relative benefits of the added mechanisms remain insufficiently established. SmartThinker already provides Gaussian target lengths, penalties for overlong correct rollouts, and dynamic scaling. This paper adds incorrect-conditioned and directional candidates, along with batch-level restoration; the introduction's success-only characterization of SmartThinker is inaccurate. The matched comparison is valuable, but comparisons with cited representative methods such as ARLCP and DRPO are missing; ARLCP also shares the backbones and training data. Although iso has an on/off ablation, its comparator fixes g=1, leaving stronger penalties as an alternative explanation for its gains. The specific value of the restoration rule therefore needs a control comparison.
Questions for authors
-
Does Table 3 keep the scoring density unchanged when adding directional candidates? Please reconcile the common mixture score with Algorithm 1's branch-specific scores and specify the implementation used.
-
Could you compare iso against several moderately higher fixed gains under the same threshold-selection rule, to distinguish restoration from general penalty strengthening, and report variability across training seeds?
-
Could you add comparable evaluations of representative methods such as ARLCP or DRPO and SmartThinker accuracy–length curves across penalty strengths? Please use a consistent evaluation protocol and clarify training-budget and optimizer differences.
I would reconsider my current score if the implementation rule is clarified and the added mechanisms demonstrate consistent gains at comparable cost.
4 comments
Question2: Iso-Pressure Restoration vs. Fixed Gain (Ablation Control)
Table: Accuracy and reasoning efficiency across iso-pressure settings. All experiments use DeepSeek-R1-Distill-Qwen-1.5B as the base model. We report accuracy (Acc.) and accuracy-efficiency score (AE) with the mean and standard deviation () evaluated across multiple trials (incorporating an extra trial with a different seed to ensure robustness) on MATH500, AIME 24--26, and AMC23. Average reasoning length (Len.) is reported as the mean token count rounded to the nearest integer.
Notation: M500 = MATH500; AIME = AIME 24–26; Avg = average across benchmarks. Fixed 1.25/1.50 = Ours () with fixed pressure 1.25/1.50.
| Method | M500 Acc↑ | M500 Len↓ | M500 AE↑ | AIME Acc↑ | AIME Len↓ | AIME AE↑ | AMC23 Acc↑ | AMC23 Len↓ | AMC23 AE↑ | Avg Acc↑ | Avg Len↓ | Avg AE↑ |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Base | 76.2 | 9,708 | 28.9 | 36,102 | 80.0 | 20,944 | 61.7 | 22,251 | ||||
| Fixed 1.25 | 2,077 | 8,605 | 3,958 | 4,880 | ||||||||
| Fixed 1.50 | 7,957 | 3,566 | 4,537 | |||||||||
| Ours (+iso) | 2,205 |
To address this question, we conducted controlled experiments on DeepSeek-R1-Distill-Qwen-1.5B (150 steps) across multiple seeds and compared our method against both modest stronger fixed pressure (1.25, 1.50).
Intuitively, a higher, moderate fixed pressure (1.25, 1.50) might be expected to shorten reasoning lengths. Although the fixed gains of 1.25 and 1.50 exceed the maximum observed iso-pressure of 1.166 in Deepseek-1.5b, they do not guarantee shorter final reasoning length. Even with the same threshold-selection rule, pressure can indirectly affect subsequent threshold estimates through policy updates (Eq.16, Eq.12, Eq.5). The selected threshold need not decrease with increasing gain, so final reasoning length is not guaranteed to decrease monotonically.
And to evaluate the optimization stability of the proposed method, we ran additional trials under an independent random seed. Across seeds, Math-500 accuracy remains highly concentrated for all. In case of AMC23, 1.50 fixed pressure achieves the highest ae score. While a high fixed pressure (1.50) can occasionally yield strong performance on moderately challenging tasks like AMC23, such static configurations fail to generalize to more complex reasoning domains. Indeed, as reasoning complexity increases on AIME, the advantage of our dynamic calibration becomes clear. In contrast, the fixed pressure settings suffer from a severe performance drop on this benchmark, as clearly reflected in their significantly lower AE scores.
This consistent trend suggests that our dynamic thresholding and iso-pressure restoration provide a robust learning signal that effectively prevents accuracy degradation while reducing length.
Question 3. Additional Comparison with Related Methods.
-
We have retrained QLPO, RoRecomp, and ARLCP on DeepSeek-R1-Distill-Qwen-1.5B using the same training and evaluation settings as OASIS. The results are reported in the table in common response. As detailed in our common response above, the results demonstrate that OASIS remains more effective than these closely related methods.
-
SmartThinker's penalty strength is coupled to its calibration. Varying it independently would modify the original formulation and could violate its intended advantage-sign condition. We therefore retain its original calibration for the baseline comparison. Table 3 in the paper shows the contributions of the threshold candidates selection and iso-pressure restoration. The fixed-pressure ablation in our preceding response further supports adaptive pressure. With the same threshold-selection rule, iso-pressure achieves higher average accuracy and AE than the fixed gains.
Thanks for clarifying the scoring rule and adding the baseline comparisons. I’m raising my rating to 3. Please revise Algorithm 1 in the manuscript to show that all candidates use the same scoring reference.
Dear Reviewer,
Thank you again for your careful review and constructive suggestions. We have incorporated the revisions discussed in our response into the updated paper. Your thoughtful feedback has helped us improve the paper, and we sincerely appreciate the time and effort you devoted to reviewing our work.
We thank for your careful review and constructive feedback. We address each concern below.
Weakness1: Clarification on Scoring Density and Score Reconciliation (Algorithm 1)
We appreciate the reviewer for catching this apparent discrepancy. While transcribing the method into pseudocode, the distinction between the derivation nulls (for all candidates) was mistakenly carried over into the J evaluation. The actual implementation, as the reviewer notes, and as stated in the body, scores all candidates against one common reference: the two-component incorrect mixture when the soft split is defined, a single Gaussian otherwise. We have corrected Algorithm 1 accordingly.
Consequently, the concern that adding directional candidates might alter the scoring density is resolved, as the scoring density remains strictly unchanged across all candidates. The gains reported in Table 3 therefore reflect the isolated effect of candidate expansion, confirming its clear independent contribution to identifying an elaborate threshold position.
Weakness2 : Misleading explanation about other methods and insufficient comparison.
We appreciate the reviewer's observation that characterizing SmartThinker simply as "success-only" in the Introduction was somewhat coarse. SmartThinker derives its target length from both correct rollouts and the marginal (all-rollout) distribution , which implicitly incorporates incorrect rollouts. Our intended distinction, which is formally analyzed in Section 4.1 (Eq. 4), is not between success-only vs. failure-aware, but between marginal pooling (SmartThinker) and explicit failure conditioning (OASIS). As derived in Eq. (4), fitting a single Gaussian to the pooled marginal distribution p(L ∣ x) introduces the attenuation factor κ = R/(R + π_C D²), which structurally absorbs the class separation between correct and incorrect rollouts into inflated variance.
We explicitly conditions on the incorrect distribution and models directional failure modes (Eqs. 5–9), eliminating this structural attenuation.
While this distinction is already rigorous in Section 4.1, we could slightly polish the introductory phrasing to prevent any potential ambiguity.
The comparison and iso pressure rule ablation will be further answered below.
Question1: Clarification on Scoring Density (Table 3) and Score Reconciliation (Algorithm 1)
Regarding the pseudocode in algorithm1, we appreciate the reviewer for catching this discrepancy. While transcribing the method into pseudocode, the distinction between the derivation nulls (for all candidates) was mistakenly carried over into the J evaluation. The actual implementation, as the reviewer notes, and as stated in the body, scores all candidates against one common reference: the two-component incorrect mixture when the soft split is defined, a single Gaussian otherwise. We have corrected Algorithm 1 accordingly.
Consequently, the concern that adding directional candidates might alter the scoring density is resolved, as the scoring density remains strictly unchanged across all candidates. The gains reported in Table 3 therefore reflect the isolated effect of candidate expansion, confirming its clear independent contribution to identifying an elaborate threshold position.