- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes the Optimal Reasoning Interval (ORI) and Length-Interval-based Reward Shaping (LIAR). The authors map success probabilities at a fixed reasoning depth to reliability under repeated sampling, then define the ORI as the set of lengths satisfying a reliability threshold. In training, they group problems by current accuracy, estimate target intervals from the cumulative distribution of the shortest correct monitoring trajectories, and apply piecewise-linear penalties to correct trajectories outside the interval and overly long incorrect trajectories.
The experiments collectively cover four models, eight mathematical tasks, and four out-of-domain tasks. Across the five main mathematical tasks on Qwen-1.5B, LIAR improves average accuracy over LASER-D by 1.7 pp, while average generation length increases from 3740 to 5472 tokens. The paper provides unified evaluation, comparisons under identical generation caps, and an analysis of decoding uncertainty.
Critical strengths
-
Clear presentation and reasonably broad experimental coverage. The authors evaluate interval-based shaping across multiple models and tasks beyond mathematics, with unified sampling, some component ablations, and decoding bootstrap analysis. Average accuracy gains over LASER-D across the four models are 1.7/2.5/1.2/1.8 pp, and the main-table averages can be reproduced from the reported entries, supporting the method's empirical value.
-
The ORI formalization and endpoint rules offer identifiable methodological contributions. The authors construct a cumulative curve from whether each problem has a correct trajectory within a given budget, then determine the two endpoints using a success-rate threshold and residual success mass.
Critical weaknesses
-
The guarantees for the estimated intervals have gaps. Proposition D.4(ii) overlooks the constraint L≥L_l in Eq.2: kη≤ε does not rule out L_l exceeding the tail cutoff, so it cannot guarantee that L_r stays below that cutoff. Eq.4 on p.21 also contains an algebraic error. The 56 unsuccessful longest samples in Appendix E do not establish the per-query condition η≤0.625% required for the high-difficulty group. Guarantees for the theoretical ORI, empirical cumulative curve, and reward interval need to be distinguished. Additional assumptions or narrower claims may resolve these issues; they do not directly invalidate the main-table results.
-
Cost comparisons and ablations do not isolate the contribution. LIAR gains 1.7/2.5 pp over LASER-D on 1.5B/7B while using approximately 46%/56% more tokens; Table 7 only matches caps. ARLCP (ICLR 2026) uses the same backbone family and DeepScaleR data and reports higher mean accuracy than LASER in its own evaluation, yet is neither cited nor compared under a common protocol. Setting L_r=L_max in Table 6 also changes L_0 and the overlength penalties for correct and incorrect trajectories; removing tiers also unifies τ. Controls isolating the penalty on short correct trajectories, online updates, and new endpoint rules are missing, as are training-run uncertainty estimates for Table 6.
-
Related work and citations need fuller coverage. Relevant precedents include ALP's continuous penalty adjustment based on online accuracy, LAPO's quantile intervals and two-sided rewards, DA-DLER's difficulty-grouped truncation, and DRER's dynamic interval-based advantage adjustment. ALP, DRER, and Video-R1 are not cited, while the relevant mechanisms in LAPO and DA-DLER are insufficiently discussed. Accuracy-based difficulty adaptation and interval rewards should not themselves be presented as new contributions. Minor corrections include classifying AutoThink and Thinkless as adaptive reasoning methods learned through training, and citing the original LiveCodeBench and DeepSeekMath, which introduced GRPO.
Questions for authors
-
Theoretical guarantees: Please correct D.4(ii) and Eq.4, distinguish the guarantees for the theoretical ORI, empirical estimates, and intervals after stabilizer updates, and explain how Eq.3 handles selection effects from grouping the same rollouts used for estimation.
-
Cost and ablations: Can you compare with LASER-D at similar actual token usage and report monitoring overhead? Please isolate the penalty on short correct trajectories and compare dynamic endpoints with frozen or simple quantile intervals, controlling the penalty scale and incorrect-trajectory branch. Clarify how ε is set without tiers and report training-run variability for key ablations.
-
Baselines and related work: Can you include ARLCP accuracy and token comparisons under a common protocol, or explain specific factors that prevent a comparable evaluation? Please clarify the contribution beyond ALP's accuracy-adaptive penalties and LAPO/DRER's interval mechanisms, and add the relevant citations and discussion.
Correcting the theory and providing key controls at comparable cost could change my score.
5 comments
W2 & Q2. Cost, the ablation table, and the controls
1. Cost. The arithmetic is right and it settles the claim it tests: 5,472 / 3,740 = +46.3% on the 1.5B pair and 6,006 / 3,855 = +55.8% on the 7B pair. §5.2 of the submitted version described the method as reaching comparable accuracy at lower cost; the revision withdraws that description. In its place Appendix H reports 24 accuracy–cost points — two models, three benchmarks, four budgets, avg@16 throughout — with measured generated tokens on the x-axis. A shared cap constrains generation without equalising it, so the submitted table is not a cost comparison, and we report no interpolated points: adjacent cap settings overlap by 283 tokens on MATH500, by 92 on OlympiadBench, and not at all on AIME 2026. We did not retrain at matched realized cost, and latency is not compared.
2. Monitoring overhead. k × |D^M| = 8 × 500 = 4,000 sequences per endpoint update against 20 × 128 × G = 20,480 between updates, i.e. 19.5% of the generation budget, independent of the training horizon. That is a share of generation, not of total training cost.
3. The ablation table. The rows of the submitted Table 6 bundle changes: setting L_r = L_max also moves L_0 = λ L_r and rescales the overlength penalties on both branches, and removing the tiers collapses three τ_D and three ε_D into single values. Neither row isolates the factor in its label, so the sentence in §5.2 attributing the no-tiers drop to difficulty adaptivity is withdrawn and the rows are reported as component sensitivities. The same applies to ε: the value used there was not recorded in a reconstructable form, so the row is a joint intervention, and the rule to use is tier-size-weighted, ε = Σ_D |D^M_D| ε_D / Σ_D |D^M_D|, which is 0.0771 on the 233/94/173 sizes.
4. Controls. Three variants, each changing one component with tiers, τ_D, the online curve, the penalty scale and the update schedule held fixed.
Table 2 — single-factor component ablations (1.5B). Δ is relative to LIAR under the same evaluation protocol; the parenthesised change is in generated tokens against LIAR's 5,472.
| Variant | Change | Macro acc (%) | Avg Tok | Δ (pp) |
|---|---|---|---|---|
| LIAR | — | 50.5 | 5,472 | — |
| Adaptive single point | L_l = L_r, tiers and online curve kept | 49.1 | 6,317 (+15%) | −1.4 |
| Frozen interval | endpoints fixed before training | 49.8 | 6,196 (+13%) | −0.7 |
| Upper boundary only | short-correct penalty removed | 49.0 | 4,244 (−22%) | −1.5 |
The three rows ask, in order, whether two endpoints are needed, whether they must be re-estimated, and whether the short-correct penalty contributes — removing it saves a fifth of the tokens at a cost of 1.5 points. Every removal costs accuracy, and the token directions differ: interval variants +13–15%, short-correct penalty −22%. A LAPO-style quantile interval was not run, because it replaces the endpoint definition rather than removing one component, so it is not a control.
5. Variance. The submitted table is one training run. We add one more seed of the same recipe (s1) to the one used in the paper (s0), so the comparison rests on two LIAR seeds against the frozen LASER-D checkpoint.
Table 3 — seeds and question-level intervals. Avg Tok is the macro average generated length; the CI is a two-level hierarchical bootstrap (questions, then 16 decoding samples) on the macro gap against LASER-D.
| Run | Macro acc (%) | Macro Avg Tok | Two-level 95% CI (macro Δ) |
|---|---|---|---|
s0 | 50.50 | 5,472 | [+0.9, +2.7] |
s1 | 50.50 | 5,361 | [+0.7, +2.5] |
| LASER-D (frozen, one checkpoint) | 48.8 | 3,740 | — |
Both seeds agree on macro accuracy and differ by 111 tokens in macro length; the macro lower bound is positive for both. With the baseline frozen, this is a one-sided lower bound on total variability.
ARLCP is answered with Question 3 below.
W3 & Q3. Prior work, novelty, and ARLCP
1. What is not new. Difficulty adaptation from online accuracy and rewards shaped on an interval of lengths are both prior art; the introduction now says so and places the contribution elsewhere. We have added ALP, DRER, Video-R1, the original LiveCodeBench paper, DeepSeekMath, and ARLCP (ICLR 2026), and corrected AutoThink and Thinkless to adaptive reasoning methods learned through training.
2. What the contribution is. (i) The endpoints come from a tail condition on the success-ratio curve — L_l is the first budget reaching τ_D, L_r the first after which further depth adds less than ε_D of success mass — not from a length percentile or a difficulty statistic. (ii) The criterion is measurable from one online batch of k = 8 rollouts, needs no question with several correct traces, and stays defined for questions that no rollout solves. (iii) S_D(L_r) ≥ S_D(L_max) − ε_D follows from the selection rule, and L_r ≤ γ_θ n̂_D only under both premises of the corrected proposition.
3. The closest precedent. DRER, not LASER-D: it is already bucket-wise, two-sided and online. The targets differ in what they are statistics of — percentiles of the lengths of correct traces within a bucket (DRER P5/P95; LAPO P30/P70 per question) versus the tail of the success-ratio curve with a declared tolerable loss ε_D. ALP adapts a penalty to online accuracy, DA-DLER groups truncation by difficulty, ARLCP estimates complexity; none places endpoints this way.
4. ARLCP. No official checkpoint is public, and the published results do not cover all five benchmarks under our protocol, so we train a 1.5B model to the same recipe and evaluate it in our harness.
Table 4 — ARLCP against the two reference points. avg@16, same harness, macro over the five main benchmarks.
| Model | Macro acc (%) | Macro Avg Tok |
|---|---|---|
| LIAR | 50.50 | 5,472 |
| LASER-D | 48.8 | 3,740 |
| ARLCP (1.5B) | 49.34 | 3,401 |
ARLCP spends 62% of LIAR's tokens and 91% of LASER-D's, with accuracy in between — a third point on the same curve. Appendix K lists the protocol axes: backbone, corpus, evaluation protocol, parser, prompt, sampling, training schedule and training L_max all match; only the reflection detector and its corpus statistics are ARLCP-specific, reproduced from the paper. Axes that differ are marked "not directly comparable on that axis"; we make no equivalence claim to the published numbers.
We sincerely hope that our rebuttal could properly address your concerns. If so, we would deeply appreciate it if you could raise your score. If not, please let us know your further concerns, and we will continue actively responding to your comments and improving our submission.
Response to Reviewer PcVS
We thank the reviewer for the careful and constructive comments. We respond to each comment as follows.
W1 & Q1. The defective bound, Eq. 4, and what each object guarantees
1. The statement. Proposition D.4(ii) overlooked
L ≥ L_l: the right-endpoint rule minimises over a feasible set whose lower bound isL_l, sok η_D ≤ ε_Dalone does not keepL_rbelow the tail cutoff. The revised proposition carries both premises — (a)k η_D ≤ ε_Dand (b)S_D(L_max) ≥ τ_D + ε_D— and concludesL_r ≤ γ_θ n̂_D, henceL_l ≤ γ_θ n̂_DsinceL_l ≤ L_rby construction. Premise (b) cannot be dropped: a remark gives a configuration satisfying (a) in whichL_lalready lies past the cutoff, so the rule returnsL_r = L_l.2. Eq. 4. The decomposition did contain an algebraic error; the correct tail term is
(1−p_q(L))^k [1 − (1 − x_q(L)/(1−p_q(L)))^k], from1 − p_q(L_max) = (1 − p_q(L)) − x_q(L). Since1 − (1−u)^k ≤ k u, the chaink x (1−p)^{k−1} ≤ k xand every downstream statement are unchanged.3. What each object carries. (i) The theoretical ORI is a depth-space object; we do not estimate it and claim no recovery result. (ii) The curve
S_Dis an estimator conditional on the current policy, with a curve-level band rather than pointwise intervals. (iii) The freshly estimated interval is the only object carrying a bound:S_D(L_r) ≥ S_D(L_max) − ε_Dholds unconditionally by construction, whileL_r ≤ γ_θ n̂_Dneeds both premises. (iv) The interval used in training is the stabilized one, and the bound does not transfer: the stabilizer delays contraction but never accelerates it, so only the direction is guaranteed. Appendix D tabulates what is established for each.4. Eq. 3. It is a conditional identity for a fixed monitoring batch,
S_D(L) = (1/|D^M_D|) Σ_q [1 − (1 − p_q(L))^k]; we never estimatep_q(L)---we count each question's earliest correct rollout. The samekrollouts tier a question and build the curve, so the statistic and the quantity it defines share sampling noise, and we do not claim to have removed that coupling. Three things keep the procedure well defined: the endpoint rule is a deterministic functional with no hypothesis test; all guarantees are conditional on the batch, so endpoints are operating parameters rather than population estimates; and the monitoring set is disjoint from the gradient minibatches and supplies no reward samples.5. The evidence. A zero count over 56 rollouts does not resolve a tail rate down to
η_D ≤ ε_D/k. We did not relax the tolerance; we rebuilt the test.Table 1 — the rebuilt tail test, LIAR 1.5B, 5,001 questions,
k = 8.n̂_D= tier cutoff in tokens;n_tail= rollouts at or beyond it;ok= the correct ones;CP95= one-sided 95% Clopper–Pearson bound on the tail correct rate; budget =ε_D / k.All three tiers meet their budget. Cutoffs are fixed per tier by a length-only criterion before correctness is seen, and cap hits count as failures.