acceptodds
Under review as a conference paper at ICLR 2027

Fixed Length Penalties: Accuracy–Length Trade-offs in Mathematical Reasoning

Abstract

Length-aware reinforcement learning can shorten reasoning, but assessing adaptive rewards requires a strong account of what a simple fixed cost already achieves. We study a constant penalty on capped, normalized completion length in reinforcement learning with verifiable rewards. Across eight paired training seeds of DeepSeek-R1-Distill-Qwen-1.5B, a coefficient of 0.3 reduces mean generated tokens by 26.9%, 12.0%, 2.8%, and 7.8% on GSM8K, MATH-500, AIME, and AMC, with mean accuracy changes of −0.80, +0.63, +0.21, and +2.11 percentage points. Every seed pair yields shorter generations on every benchmark, and tokens per correct response fall by 26.2%, 12.7%, 3.8%, and 11.1%. Without problem-specific budgets, difficulty estimates, or online penalty adaptation, the fixed cost captures 84.9–97.3% of the largest mean token saving achieved by the tested adaptive controls. The largest absolute savings occur in intermediate difficulty groups, while difficult AIME problems dominate the remaining generation volume. These results establish a simple and competitive accuracy–efficiency reference for evaluating adaptive length control.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Efficient Reasoning, Mathematical Reasoning, Reinforcement Learning with Verifiable Rewards, Length Regularization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_JqAuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the impact of fixed length penalties on accuracy and reasoning efficiency in reinforcement learning for mathematical reasoning. The authors introduce a normalized and clipped length penalty into the correctness reward, without requiring additional difficulty estimation or dynamic parameter adjustment. Using DeepSeek-R1-Distill-Qwen-1.5B, the paper compares the proposed method against several adaptive length control approaches across four mathematical reasoning benchmarks. The results show that the method reduces generated tokens by 2.8%–26.9% while largely maintaining accuracy, suggesting that simple fixed penalties can serve as an effective baseline for evaluating more sophisticated length control methods.

Critical strengths

The proposed method is simple and easy to implement, requiring neither additional difficulty estimation nor dynamic reward adjustment mechanisms, which gives it practical value. Experimentally, the authors compare their approach with several adaptive length control methods across four mathematical reasoning benchmarks and evaluate the stability of the results using multiple random seeds. In addition, the paper analyzes the effects of different penalty coefficients, problem difficulty levels, and model scales on reasoning compression, providing empirical insights into both the effectiveness and limitations of fixed length penalties.

Critical weaknesses

W1. Limited methodological novelty and substantial overlap with prior work.

This paper reduces reasoning token consumption by introducing a fixed length penalty into the correctness reward. However, similar ideas have already been explored in prior studies, such as Arora & Zanette (2025) and L1 (2025). In particular, LEASH (arXiv:2512.21540) has already introduced a fixed-penalty variant, LEASH-C, and compared the effectiveness of fixed and adaptive length penalties.

Therefore, although this paper provides some empirical value, its core idea and main findings are closely related to existing research. The authors should further clarify the unique contributions of this work and provide more comprehensive discussions and comparisons with directly relevant studies.

Relevant references:

  • Arora & Zanette (2025). Training Language Models to Reason Efficiently
  • Aggarwal & Welleck (2025). L1: Controlling How Long a Reasoning Model Thinks With Reinforcement Learning
  • LEASH (2025). Adaptive Length Penalty and Reward Shaping for Efficient Large Reasoning Model

W2. Limited model scale and outdated evaluation benchmarks.

The main experiments are conducted only on DeepSeek-R1-Distill-Qwen-1.5B, with reinforcement learning training data primarily drawn from GSM8K. Moreover, the evaluation benchmarks are relatively outdated, especially considering that newer benchmarks, such as AIME 2026, are now available.

Questions for authors

see above

3 comments

Thank you for highlighting the connection to LEASH-C and for suggesting broader evaluation across models, tasks, and benchmarks. Your feedback helped us sharpen the empirical contribution: eight-seed controls identify how normalization and decoding budgets shape accuracy–length trade-offs, while direct prior-method comparisons and the 7B, AIME2026, and local MBPP extensions broaden the evidence. Together, these results clarify the conditions under which a simple fixed length cost reduces token use.

  1. Prior work and the empirical contribution (W1)

We agree that fixed length penalties and fixed-versus-adaptive comparisons have precedents. The Introduction and Related Work now acknowledge LEASH-C explicitly. Arora and Zanette study length-aware RL; L1 learns prompt-specified length constraints; LEASH uses a target-excess cost with dual adaptation, and LEASH-C fixes its coefficient. Our capped completion cost charges length from the first token, including incorrect responses. This reward difference defines the experiment, not priority for the general principle.

The added controls supply the empirical contribution. An eight-training-seed Correctness/Fixed × group-SD/no-SD factorial gives MATH/AIME/AMC token savings of 14.64%/9.05%/10.85% with scaling versus 3.31%/2.12%/5.10% without at 32K (Table 10). Five of six token contrasts, but none of the accuracy contrasts, pass the corresponding Holm correction. The lambda × SD sweep shows further normalization dependence (.2/group six seeds; other cells eight). Saved-checkpoint long-budget evaluation shows that short-budget accuracy advantages disappear or reverse while token savings persist (Appendix I). These findings specify the conditions under which a fixed reference is informative; they do not establish accuracy equivalence or universal superiority.

Appendix J adds Arora's released-recipe RLOO comparison (one training seed, 100 rollout rounds/200 Adam updates per arm) and controlled LEASH/LEASH-C adaptations (two seeds/400 updates). The latter are not full original-compute reproductions. Arora's asymmetric recovery histories and unrestored private generation RNG further limit method ranking; unfavorable comparisons are retained. The finite seven-configuration, two-seed tuning grid yields no candidates satisfying its specified accuracy constraints, so there is no selected independent-test Pareto frontier. We acknowledge that a complete tuned comparison and broader methodological novelty are not established by these results.

  1. Model scale, training-domain coverage, and newer benchmarks (W2)

Healthy DeepSeek-R1-Distill-Qwen-7B runs use two seeds per arm, BF16 full-parameter training, no group-SD, and a matched initialization (Appendix L, Table 15). At 32K, Fixed saves 4.31%/0.86%/3.19% tokens on MATH/AIME/AMC, with accuracy changes of +0.013/−0.069/+1.280 pp. Savings vary across tasks. A comparison with the primary 1.5B group-SD results changes both size and normalization and does not isolate a scale effect; at matched no-SD, MATH savings are 3.31% for 1.5B versus 4.31% for 7B. The old approximately 2% anomaly predates RL and is retained only as diagnostic background. Two seeds and one model family do not establish broad model generalization.

All 30 AIME2026 questions are evaluated with eight paired training seeds and eight responses per question at 8K completion and 32K total context (Appendix M, Table 26). Accuracy changes are −0.677/+0.729 pp, with token savings of 0.76%/0.73%. Neither metric passes its two-budget Holm correction. This adds current benchmark coverage, with small uncertain effects; 30 questions and multiple samples do not establish strict decontamination or population-wide performance. The initialization and complete results are included.

The second training-domain check uses MBPP: 120 training problems, 100 updates, one seed, and 64 disjoint test problems × four responses (Appendix N). Correctness/Gated/Ungated yield 125/126/0 successes out of 256 at about 686/349/4 tokens. Gating is worth investigating locally, but this is not a full multi-seed non-math study and does not erase historical gated HumanEval degradation or ungated collapse. Primary training still uses one prompt × eight responses/update for about 400 distinct problems/run; data/compute scaling and independent model families remain outside the completed scope.

The revision therefore contributes a controlled empirical reference with measured optimization, budget, and task boundaries, rather than a claim that fixed penalties are new or reliably outperform adaptive methods.

Thank you sincerely for asking us to clarify what this study adds relative to LEASH-C and other established length-control work. Following your suggestions, we completed additional cross-model training and expanded the empirical questions. The added content concerns three areas.

First, we broadened model coverage and tested which observations transfer. We completed Correctness/Fixed × group-SD/no-SD training on Qwen2.5-Math-1.5B-Instruct and DeepSeek-R1-Distill-Llama-8B, with 400 updates per arm, four mathematical benchmarks, and two total-context budgets per model. This adds evidence from a Llama backbone. For example, Qwen/MATH/4K/group saves 4.26% of tokens, and Llama/MATH/8K/no-SD saves 6.38%. We also retain Llama/group’s quality deterioration and the changes in difficulty profiles and tail-compression relationships across the new models. These outcomes help delineate which observations transfer and which depend on the configuration.

Second, we examined whether compression training remains compatible with inference-time answer selection. In the original eight-seed GSM8K/group cohort with a 3,072-token completion cap, the advantage of shortest over random selection is +2.93 percentage points for Correctness but −2.46 points for Fixed; every seed exhibits this sign reversal. Five-fold question-held-out calibration on the same Fixed answer pool achieves 83.98% selection accuracy, compared with 81.99% when transferring ranks calibrated on Correctness outputs. Equally calibrated Correctness remains higher at 87.04%, so the additional finding concerns selection-rule adaptation. Calibration uses correctness labels only on its training folds, and all generation costs have already been incurred. The new models provide local support and counterexamples, showing that failure of shortest selection should not be treated as universal.

Third, we examined how saved tokens translate into finite-sampling utility. At equal expected generation-token cost, oracle coverage in the original eight-seed GSM8K/group cohort increases from 92.69% to 93.95%. The new models also provide local gains—for example, +0.69 percentage points for Llama/no-SD on GSM8K at 32K—alongside counterexamples such as −0.62 points on MATH-500 at 32K. This measures whether at least one correct answer is generated and still requires a verifier. It addresses whether cheaper responses can support greater problem coverage under a matched expected budget.

These extensions develop the empirical content around model transfer, compatibility between training objectives and inference-time selection, and sampling-budget reallocation. We fully recognize precedents for fixed penalties, length selection, calibration and pass@k; the added value lies in the specific controlled evidence and applicability boundaries. The new model runs remain single-training-seed GSM8K experiments, and the extended analyses are post hoc. Together with the previously reported AIME2026 evaluation, they supplement the original eight-seed results. Thank you for helping us make the empirical contribution more concrete and testable.

Reviewer_JqAuOpenReview ↗

Response to rebuttal

Thank you for the detailed response. The authors have generalized the model to 7B and also tested it on AIME26. However, looking at the overall results, the method performs significantly better on the relatively simpler MATH dataset, while its impact on more challenging datasets appears less pronounced. I will maintain my current score.

Reviewer_wbcUOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies a constant length penalty in RLVR, training with GRPO on DeepSeek-R1-Distill_Qwen-1.5B on GSM8K data for 400 updates. It reports a real token reduction with a mix of positive and negative changes in accuracy.

Critical strengths

One main strength is transparency on the work, there are 8 paired training seeds and each pair shortens the number of tokens generated on every benchmark. Accuracy and tokens per correct response make the trade offs clear.

Also, the negative results are helpful, it compares different reward designs and penalty strengths, and shows that the same undated objective can collapse code post training. This makes the paper very robust.

Critical weaknesses

One big concern is that the accuracy claims are a result of the truncation. In particular, answers over the token cap are marked as wrong, and when the token cap was increased it increased the correctness score by over 8 points. Splitting the questions into ones that are truncated vs not, the penalty gives at best no accuracy improvement on ones that are not truncated. So it seems like we should think of higher accuracy not as reasoning better or doing better work, but just that now more answers finish before truncation.

Also, the std normalization is basically adaptive, if all responses are wrong or right then the advantage does not depend on lambda, and these groups probably dominate. So the “fixed” cost is actually strong in all right or all wrong groups, and weak in ones with some of each. This seems like it explains the flat lambda sweep. The key ablation we need to see is turning off std scaling.

Also, length penalties in RLVR are pretty well established, this seems like it serves more as a baseline, and is only studied on one small model and 400 GSM8K prompts with a 2048 token cap.

Two of the four adaptive baselines are author-created, rather than using published methods like L1, ShorterBetter, or ThinkPrune. And most differences can potentially be accounted for by seed noise.

It would be useful to see a row on the base model as well.

Finally, the 7B run seems entirely broken. A 4x larger model scores only 2% but the small model scores 84%, this doesn’t seem to make sense, at least this should be well explained before drawing any conclusions.

Questions for authors

Do the accuracy differences still show up when MATH-500, AMC, AIME are evaled at 32k?

What fraction of the training groups were all correct or all wrong?

What happens if we turn off std normalization?

Which 7B model is used, and how can you explain 2%?

Can you add a base model row to Table 1?

Can you compare against at least one well-known published adaptive method using their hyperparams and compute?

5 comments

Thank you for the detailed questions on decoding budgets, normalization, initialization, and the 7B results. Your feedback helped us strengthen the paper with eight-seed controls and new 7B runs with matched initialization controls. The added evidence shows persistent token savings at 32K, quantifies their dependence on normalization, and clarifies the role of the decoding budget in the observed accuracy–length trade-offs.

  1. Accuracy at 32K and truncation (W1, Q1)

Saved-checkpoint evaluation, eight paired training seeds, 32K total context:

BenchmarkFixed−Correctness accuracy (pp)Token saving
MATH-500−0.27215.48%
AIME−0.3999.13%
AMC−0.15111.74%

Table 12 gives pointwise intervals; Tables 19–20 include AIME/AMC at 16K. MATH's effect changes from +0.963 pp at 8K completion to −0.272 pp at 32K context (budget-interaction Holm p=.0391). This supports budget-dependent completion, not improved intrinsic reasoning. The verifier scores extracted answers even from length-stopped outputs; truncation is not automatically incorrect. Selected untruncated subsets are noncausal. Context includes the prompt; this queue's 8K anchors remain separate from historical scores.

  1. Group composition and removing SD normalization (W2, Q2–Q3)

Original eight-seed all-correct/all-wrong/mixed fractions are 41.66%/5.88%/52.47% for Vanilla and 47.84%/4.81%/47.34% for Fixed (Table 14). Section 3 derives approximate lambda cancellation in equal-correctness groups: fixed coefficients do not imply constant effective pressure after SD scaling.

Correctness/Fixed × group-SD/no-SD uses eight training seeds/cell, 400 updates (Appendix H). At 32K, group/no-SD savings are 14.64/3.31% on MATH, 9.05/2.12% on AIME, and 10.85/5.10% on AMC. Five token contrasts pass six-cell Holm; no-SD AIME and all accuracy contrasts do not. Tables 16–17 retain standard budgets. The lambda × SD sweep (.2/group six seeds, others eight; Figure 6) shows further sensitivity. Full adaptive no-SD retraining/tuning and a learning-rate grid remain undone; we do not dismiss adaptive rewards.

  1. The 7B model and the old approximately 2% result (W6, Q4)

DeepSeek-R1-Distill-Qwen-7B's old initialization already had 1.98% saved GSM8K-training accuracy and 99.85% truncation before RL. The scale-failure claim is removed; the exact cause remains unresolved.

Healthy reruns use BF16 full parameters, no group-SD, 400 updates, two seeds/arm and matched initialization (Table 15), unlike the old LoRA diagnostic. At 32K, MATH/AIME/AMC accuracy deltas are +0.013/−0.069/+1.280 pp; savings are 4.31/0.86/3.19%. Two seeds are descriptive, not a scale law or noninferiority result. MATH initialization scores 86.675% versus Fixed's 86.288%; unfavorable seed pairs remain.

  1. Initialization rows (W5, Q5)

Tables 11/15 and complete tables provide the exact RL initialization under matched prompts, budgets and verifier. We do not insert a different queue's base score into historical Table 1 or synthesize its missing row. Cross-training-seed SD is undefined for a single initialization checkpoint.

  1. Published adaptive method and scope (W3–W4, Q6)

Appendix J adds Correctness/Fixed/Arora with released RLOO (commit e3611e4b37e792a65e78a2d21fa3202a01a38306): seed42, 100 rollout rounds/200 Adam updates per arm, 32 prompts × 8 responses/round; lr=5e-6, warmup=.03, temperature/top-p=1/1, prompt/completion caps512/32000, BF16/ZeRO2, initial KL=.001, clip=.2. Each arm normally uses two shared actor/reference GPUs plus two generation GPUs. Fixed's denominator is 32000 here, versus 2048 in main GRPO. Appendix G gives the execution settings.

At 32K, Fixed−Arora accuracy is +0.875/+2.639/+1.657/+4.167 pp on MATH/AIME/AMC/AIME2026; savings are 3.98/−0.10/3.40/0.09% (Table 13). Neither metric passes question-cluster Holm. One training seed, asymmetric recovery and unrestored private generation RNG preclude definitive ranking. Table 21 retains unfavorable comparisons.

Controlled LEASH/LEASH-C uses two seeds/400 updates, not full original compute (Table 22). Released ShorterBetter is an unpaired checkpoint reference (Table 23); L1/ThinkPrune were not retrained. The seven-config, two-seed grid yields no Fixed/LEASH candidate within 0/1/2 pp accuracy tolerances (Table 24), hence no selected test frontier or general-superiority claim.

Section 4.1 clarifies one prompt × eight responses/update, about 400 distinct problems/run from a 7473-problem pool. Coverage is one family, 1.5B/7B, without scaling studies. Local MBPP uses one seed, 120 training problems/100 updates and 64 disjoint test problems × four responses (Appendix N); ungated collapse remains. This is not broad non-math validation. Close accuracy means do not establish equivalence.

Wow, thanks for these extensive additional experiments, they resolve my soundness questions and I appreciate how candidly the results are reported.

So at 32K the accuracy differences are small and slightly negative, while the token savings persist and are actually larger. I agree with your reading that the 8K accuracy gains reflect budget-dependent completion, not better reasoning. I feel that the abstract, contributions and conclusion should no longer present "positive accuracy changes on three of four benchmarks" as a finding.

On std normalization, the ablation is informative, removing it cuts the savings from roughly 15/9/11 percent to 3/2/5 pct, and ~half of training groups have no correctness variance. So most of the effect comes from the interaction between the fixed penalty and group-std scaling, not from the fixed coefficient itself. I feel that this should be stated in the abstract and introduction, since it changes what "fixed" means in the title and in the fixed-vs-adaptive comparison

My remaining reservation is significance rather than soundness, with the accuracy effect gone and the saving largely attributable to std normalization, the contribution is a well-controlled baseline result on one 1.5B model trained on GSM8K. That is useful for the community but incremental.

We sincerely thank you for your thoughtful follow-up. Your comments have helped us clarify the paper’s central empirical contribution: consistent token compression across paired training seeds, together with its dependence on decoding budget and group-SD scaling.

We agree that the positive accuracy point estimates should not be presented as established improvements or evidence of stronger intrinsic reasoning. Accordingly, we have revised the Abstract (p. 1), the concluding paragraph of the Introduction and contribution statements (pp. 1–2), and the Discussion and Conclusion (Section 5, p. 10). These passages now emphasize token savings, normalization dependence, and the difficulty-resolved compression profile. The abstract explicitly states that the positive primary-budget accuracy estimates do not survive Holm correction and that the 32K estimates on MATH-500, AIME, and AMC are slightly negative. All accuracy results and uncertainty estimates remain reported. We also distinguish the saved-checkpoint budget evaluations from the separately trained normalization factorial and report their compression magnitudes explicitly.

Regarding normalization, Section 3.2 (p. 4) now clarifies that fixed λ and B can coexist with normalization-dependent effective optimization pressure. We agree that this dependence is a central finding. One clarification is that the no-SD conditions still show observed token savings of 3.31%, 2.12%, and 5.10% on MATH-500, AIME, and AMC, respectively (Appendix H, Table 10). We therefore characterize normalization as a key condition governing compression magnitude; we also explicitly limit the Holm-corrected interaction finding to MATH-500.

Finally, we acknowledge the limited model and training-domain coverage and make the primary 1.5B/GSM8K training scope explicit in Section 5. Within that scope, the four-benchmark, eight-paired-seed evaluation and the normalization and budget controls provide a systematic empirical characterization of when and how much compression occurs. The 7B, AIME 2026, and code extensions retain their stated limitations and unfavorable outcomes. We appreciate your guidance in making this contribution and its boundaries more precise. The corresponding manuscript changes are highlighted in blue.

Reviewer_AZJUOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies a fixed-coefficient length penalty for GRPO-based RLVR and think it as a simple but competitive baseline against adaptive length controls. Experiments are conducted on GSM8K, MATH-500, and AIME/AMC and DeepSeek-R1-Distill-Qwen-1.5B. When training on GSM8K with penalty coefficient 0.3 across eight paired seeds, this method reduces output length by 2.8–26.9% while largely preserving comparable accuracy. Despite its simplicity, the fixed penalty recovers 84.9–97.3% of the peak token reduction among adaptive baselines. Additional analyses explore coefficient sensitivity, model reasoning patterns, and the author also extend the method to code generation, and scaling to 7B parameters.

Critical strengths

1.The question studied in paper is simple but well motivated. Indeed, we need to know how much benefit we can get out of a simple fixed penalty. This provides a practical view for efficient reasoning.

2.The empirical evaluation is comprehensive and solid (eight paired seeds, nested paired bootstrap, Holm correction). The authors are also upfront about statistical nuances like regression to the mean.

3.The length reductions hold consistently across seeds and benchmarks, and the generalization to code generation.

4.The paper writing is good. All experimental and implementation setups are clearly laid out, making the work easy to follow and reproduce.

Critical weaknesses

1.In solve all or solve none groups, the advantage collapses to −(si−sˉ)/σs-(s_i - \bar{s})/\sigma_s. As noted in Sec. 4.3, λ\lambda cancels out, and so do the group-level coefficients of ALP and dual-gate. Since solve all groups likely dominate GSM8K training, the fixed, ALP, and dual-gate methods make nearly identical updates there, leaving Table 1 to only reflect mixed groups (which also explains the flat λ\lambda sweep). The comparison may be not convinced without SD scaling.

2.As shown in Table 1, it does not show that adaptivity brings little gain. Inter-method differences may fall within seed noise, the fixed penalty never achieves best in any single column, and the claimed 84.9–97.3% token savings ignore accuracy trade-offs. Moreover, two adaptive baselines are untuned variants, while method you cite in paper [1] could also be considered.

3.Training is confined to a single 1.5B model on GSM8K (2k length cap) with seemingly one prompt per step (~400 problems total). At evaluation, the 8k context truncates 65–70% of AIME outputs, heavily right-censoring the harder benchmarks.

4.The ungated fixed penalty actually collapses code pass@1 to $<$0.5%, which is a failure omitted from the abstract. Meanwhile, the 7B run hits $\le$2.3% accuracy with $>$99.5% truncation, pointing to a broken training/eval pipeline rather than a meaningful boundary result.

5.Overstated accuracy claims: None of the reported accuracy gains survive Holm correction, whereas the accuracy drop on GSM8K has a confidence interval strictly below zero.

[1] Arora & Zanette, "Training Language Models to Reason Efficiently", NeurIPS 2025

Questions for authors

1.How sensitive are the main results and the λ\lambda sweep to removing group-SD normalization? And how often do you encounter solve all, solve none or solve partial rollout groups during training?

2.Can you run inference on the saved checkpoints with a 16k–32k context limit on AIME and AMC, just to make sure the observed trends aren't confounded by length truncation?

3.Could you compare against properly tuned adaptive baselines, ideally plotting the accuracy–length Pareto frontier with paired error bars? (Respond based on the your situation. Exactly, it is resource-intensive.)

4.What is the effective batch size in terms of unique problems seen?

2 comments

Thank you for your careful review and concrete suggestions on normalization, longer budgets, and adaptive baselines. The added eight-seed controls quantify normalization effects and show persistent token savings at 32K. Together with direct baseline comparisons, these results strengthen our contribution: a controlled empirical account of accuracy–length trade-offs under fixed length costs.

  1. Normalization, group composition, and lambda (W1, Q1)

Section 3 derives approximate lambda cancellation in equal-correctness groups when reward-SD scaling dominates the stabilizer. Correctness/Fixed × group-SD/no-SD uses eight training seeds/cell and 400 updates. At 32K context, MATH/AIME/AMC token savings are 14.64/9.05/10.85% with SD versus 3.31/2.12/5.10% without (Table 10). Five of six token contrasts pass per-metric Holm correction; no-SD AIME and all six accuracy contrasts do not. Tables 16–17 give standard-budget results.

The lambda × SD sweep (Figure 6) shows normalization-dependent sensitivity: .2/group has six seeds; other cells have eight. We did not retrain/tune all adaptive arms without SD or search learning rates; no fully tuned no-SD ranking is established.

Original eight-seed all-correct/all-wrong/mixed fractions are 41.66/5.88/52.47% for Vanilla and 47.84/4.81/47.34% for Fixed (Table 14). Equal-correctness groups are substantial, but all-correct groups do not dominate. Gating and differing trajectories prevent cancellation from implying identical updates.

  1. Longer budgets (W3, Q2)

Saved checkpoints were re-evaluated with eight paired training seeds: AIME/AMC at 16K/32K total context and MATH at 32K, with this queue's own 8K-completion anchors. At 32K, Fixed-minus-Correctness accuracy is −0.272/−0.399/−0.151 pp on MATH/AIME/AMC; token savings are 15.48/9.13/11.74% (Table 12; full results Tables 19–20). Short-budget accuracy advantages disappear/reverse while compression persists. Section 4.4 emphasizes budget completion, not improved intrinsic reasoning. Context includes prompts; scores are separate from historical results. Budgets remain finite; selected untruncated subsets are descriptive.

  1. Adaptive comparisons and tuning (W2, Q3)

Appendix J adds Correctness/Fixed/Arora under the released RLOO recipe: one training seed, 100 rollout rounds/200 Adam updates/arm, 32 prompts × eight responses/round, and denominator 32000 rather than main GRPO's 2048. At 32K, Fixed-minus-Arora accuracy is +0.875/+2.639/+1.657/+4.167 pp on MATH/AIME/AMC/AIME2026; token savings are 3.98/−0.10/3.40/0.09%. Neither metric passes question-cluster Holm correction. Unfavorable comparisons remain. Asymmetric recovery and unrestored private generation RNG further limit this single-seed comparison; it cannot establish a method ranking.

Controlled LEASH/LEASH-C uses two seeds/400 updates (Table 22), not full original compute. The grid covers Fixed lambda=.1/.3/.8, LEASH targets128/256/512, and a reference: seven configurations × two seeds on 512 validation questions. Neither family meets 0/1/2 pp accuracy tolerances (Table 24); no selected independent-test frontier exists. As you note, broader tuning is resource-intensive; this is a partial response, not a full tuned Pareto study. We claim neither column-wise wins nor little value from adaptation. The saving-share ratio is removed from the abstract and retained only descriptively at unequal accuracies.

  1. Effective batch and training coverage (W3, Q4)

Section 4.1 specifies one unique prompt × eight responses/update, 400 updates: about 400 distinct problems and 3200 responses/run from a 7473-problem pool. Appendix K distinguishes original sampler reconstruction from directly logged new trajectories. Arora has 3200 prompt draws/arm, not necessarily distinct questions. There is no data/compute scaling study; 7B remains in the same family and code training is local.

  1. Code, 7B, and accuracy claims (W4–W5)

Ungated historical code pass@1 remains below 0.5%; gated HumanEval scores 28.82% versus Vanilla's 44.66%. A local one-seed MBPP check uses 120 training problems/100 updates and 64 disjoint test problems × four samples: Correctness/Gated/Ungated yield 125/126/0 successes out of 256 at about 686/349/4 tokens (Appendix N). It does not erase historical failures.

The old 7B anomaly predates RL; its cause is unresolved. Healthy DeepSeek-R1-Distill-Qwen-7B reruns use BF16 full parameters, no group-SD, 400 updates and two seeds/arm (Table 15), unlike the old LoRA diagnostic. MATH/AIME/AMC 32K token savings are 4.31/0.86/3.19%: descriptive, not a scale law or noninferiority result.

Original positive accuracy changes do not survive Holm; GSM8K's −0.80 pp remains. Its nested-bootstrap interval is just below zero; seed-t/crossed-bootstrap intervals cross zero. Analyses are post hoc. Close accuracy is not equivalence; we claim conditional token savings, not accuracy preservation, universal transfer, or measured latency/cost gains.

Thank you sincerely for your detailed analysis of normalization and comparison fairness. Your feedback helped us extend the empirical contribution toward a more specific question: what compression a fixed length cost produces across models, normalization settings and budgets, and how that compression is distributed across cost components and outputs.

First, we added controlled cross-model training to characterize the practical outcomes and boundaries of normalization. We completed Correctness/Fixed × group-SD/no-SD comparisons on Qwen2.5-Math-1.5B-Instruct and DeepSeek-R1-Distill-Llama-8B, with 400 updates per arm and evaluation on four mathematical benchmarks at two total-context budgets per model. Mean token counts decrease in 15/16 Qwen conditions. Llama also shows reductions in 7/8 conditions without SD scaling, including a 6.38% saving on MATH-500 at 8K. Llama/group instead exhibits stronger compression accompanied by quality deterioration. Reporting these outcomes together distinguishes transferable compression effects from model- and normalization-dependent failures. The additional evidence characterizes the actual outcomes associated with the known normalization mechanism under different configurations.

Second, we investigated where the tokens are saved and whether average savings represent other cost improvements. In the original model’s AIME/32K/group evaluation, reduced expenditure on incorrect responses corresponds descriptively to 90.17% of net savings; the corresponding figure for Qwen/AIME/4K/group is 95.88%. These are accounting decompositions rather than causal attributions. The relationship between mean and tail compression also changes: in the original-model MATH/32K/group normalization-control cohort, mean length and the upper-decile mean decrease by 14.64% and 10.38%, respectively, whereas Qwen/MATH/4K/group gives 4.26% and 6.40%. Thus, the sources and distribution of savings need separate measurement; their proportional relationships cannot simply be transferred from one checkpoint to another.

We also extended the analysis to the use of compressed outputs. Both new models exhibit local instances where correct short completions become more frequent while correctness within the short-completion set decreases. Under some protocols, recalibrating length-based selection on Fixed outputs also performs better than transferring the rule calibrated on Correctness outputs. These analyses connect compression training to inference-time answer selection.

We believe these additions advance the original multi-seed baseline evidence into a more specific account of cost distributions and applicability boundaries, providing dimensions that evaluations of fixed and adaptive length controls should jointly examine. Each new model configuration has one training seed, training remains on GSM8K, and the extensions are reported as post-hoc analyses with unfavorable results retained. Thank you again for motivating these more informative controlled tests.

Comments on the paper

1 comment

We thank the reviewers for their thoughtful feedback. During the rebuttal period, we carefully addressed their comments and conducted extensive additional experiments. We have uploaded a revised paper PDF, with newly added content relative to the original submission highlighted in blue for ease of review. We have also uploaded a new, complete supplementary materials ZIP archive.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.