acceptodds
Under review as a conference paper at ICLR 2027

ROAST: Reducing Overthinking in Reasoning via Advantage Reshaping and State-aware Termination

11 views

Abstract

Large reasoning models trained via Reinforcement Learning with Verifiable Rewards (RLVR) exhibit exceptional capabilities in complex tasks, yet they increasingly suffer from overthinking. This phenomenon traps them in redundant cycles and excessive self-denial. Existing methods that apply scalar penalties to reasoning length or steps struggle to balance efficiency and accuracy. Furthermore, the absence of structural metrics and the sparsity of binary rewards in Group Relative Policy Optimization (GRPO) severely hinder effective policy updates. To address this, we propose ROAST (Reducing Overthinking via Advantage reshaping and State-aware Termination). First, we introduce a structural prior that utilizes a suffix automaton to detect repeated substrings and high local repetition density. This mechanism safely terminates the reasoning process, significantly reducing generation length without compromising accuracy. Using the detected repetition patterns, we reshape the GRPO advantage for precise credit assignment: it rewards the shortest on-policy correct trajectories while applying relative down-weighting exclusively to the truncated redundant suffixes. This guarantees a positive net advantage for correct answers, mitigating gradient noise. Experiments demonstrate that ROAST effectively mitigates endless self-denial and path-switching, slashing reasoning lengths by 46-56% while preserving overall performance.

Primary area:
reinforcement learning
Keywords:
Overthinking, Large Reasoning Models, RLVR

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_2RPfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ROAST, a reinforcement-learning framework for reducing excessive reasoning. ROAST uses an online suffix automaton during training rollouts to detect severe structural repetition and terminate the reasoning phase, and combines this with token-level advantage reshaping that favors efficient correct responses while selectively down-weighting detected redundant suffixes. The method is evaluated on several mathematical reasoning benchmarks across multiple Qwen3 and DeepSeek-R1-Distill models, with additional OOD evaluation on GPQA. Importantly, the SAM intervention is disabled at evaluation time, so the reported test-time shortening reflects behavior learned by the post-trained policy rather than inference-time truncation.

Critical strengths

The paper identifies a real and potentially expensive failure mode in reasoning models: some trajectories enter severe repetitive loops and consume many tokens after useful reasoning has effectively finished. The idea of localizing the efficiency penalty to a detected redundant suffix is more compelling than applying a generic length penalty to all reasoning, since legitimate difficult problems may genuinely require long solutions.

I also appreciate that SAM is disabled during evaluation while the trained models remain shorter. This provides useful evidence that the method changes the learned policy rather than obtaining its gains simply through a test-time truncation rule.

Critical weaknesses

My main concern is that the headline compression relative to the base model obscures the smaller incremental gain over the more relevant GRPO baseline. For example, on Qwen3-4B, GRPO already substantially reduces reasoning length, and ROAST provides roughly a further 18--20% reduction with only a small accuracy change. This can still be practically useful, but it makes the incremental contribution considerably more modest than the base-model-relative compression numbers suggest.

More importantly, the ablation raises a question about what actually drives this gain. SAM alone provides only a small average length reduction, while the efficiency anchor accounts for much more of the compression. This suggests that the novel structural-repetition mechanism may function mainly as a stabilizer for more conventional efficiency shaping rather than being the primary source of the efficiency improvement. In addition, SAM detects a relatively narrow form of overthinking based on structural repetition, and the intervention rate appears lower for the stronger models in the study. Since the largest evaluated model is only 7B, I am not yet convinced that this failure mode remains sufficiently prevalent at larger scales or that the method covers overthinking more generally.

Questions for authors
  1. Could the authors report results on substantially larger reasoning models and analyze SAM intervention frequency as a function of model capability and problem difficulty?

  2. Could the authors clarify the causal contribution of SAM relative to the efficiency anchors? In particular, given the relatively small compression from SAM alone, is the main role of SAM better understood as stabilizing efficiency-oriented training rather than directly removing most redundant reasoning?

  3. Do the authors have an estimate of SAM precision and recall against human-labeled overthinking, including semantically redundant but non-repetitive reasoning? This would help establish how much of the broader overthinking problem is actually captured by the proposed detector.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 2RPf (1/2)

W1: Additional gains over GRPO-8k

ROAST improves accuracy over GRPO-8k while using substantially fewer tokens: mean math output length fell by 21.82% on Qwen3-1.7B and 18.50% on Qwen3-4B, with accuracy gains of 4.46 and 0.60 pp, respectively. Both trained models used standard decoding with SAM off (Table 1).

BenchmarkQwen3-1.7B tokens savedAcc. Δ (pp)Qwen3-4B tokens savedAcc. Δ (pp)
AIME2529.91%+10.8415.80%+1.66
AIME2620.15%−0.8411.07%+1.67
AMC2317.36%+6.2525.47%+0.62
MinervaMath5.94%+1.5736.12%−1.56
Math average21.82% (6,856 → 5,360)+4.4618.50% (4,930 → 4,018)+0.60
GPQA-Diamond15.66%0.008.88%0.00

The math results come from Table 1 and the GPQA-Diamond results from Appendix D.

On Qwen3-4B, ROAST reduced end-to-end training time on two A800 GPUs from 66.5 h (GRPO-8k) to 59.8 h (10.1% less).

Q1: Model scale and problem difficulty

Preliminary 14B training results.

Qwen3-14B training has reached step 60. At this checkpoint, ROAST used 22.43–40.72% fewer tokens than the base model and achieved higher accuracy on all four math benchmarks. Both were evaluated using standard decoding with SAM off.

BenchmarkBase acc / tokensROAST step 60 acc / tokensTokens savedAccuracy Δ
AIME2565.83% / 12,27366.67% / 9,52022.43%+0.84 pp
AIME2665.00% / 11,44466.67% / 8,59424.90%+1.67 pp
AMC2392.50% / 6,44397.50% / 3,84440.34%+5.00 pp
MinervaMath63.60% / 4,89964.34% / 2,90440.72%+0.74 pp

The early training trend is similar at 4B and 14B. From step 0 to 60, AIME24 reasoning length fell by 10.68% and 12.13%, with accuracy gains of 36.67 and 40.00 pp, respectively.

Trigger frequency depends on the task.

Larger models triggered less often on GPQA, while Qwen3-4B and 14B triggered similarly on code. We compare counts over equally long reasoning prefixes for each question within a model family.

ModelGPQA, 16k (198 questions)BigCodeBench, 32k (142 tasks)
Qwen3-1.7B17—
Qwen3-4B518
Qwen3-14B319
DeepSeek-R1-Distill-Qwen-1.5B3—
DeepSeek-R1-Distill-Qwen-7B39
DeepSeek-R1-Distill-Qwen-14B15

On the 81 GPQA questions with expert difficulty ratings, trigger frequency showed no consistent trend with difficulty.

SAM applied to 14B base models.

SAM also shortened outputs when applied directly to the Qwen3-14B and DeepSeek-R1-Distill-Qwen-14B base checkpoints. The table combines both models and compares standard decoding with SAM enabled. Code uses 142 tasks with validated unit tests per model and threshold 120; math uses threshold 60 for higher coverage (Q3). Both keep the window at 200 tokens and density at 0.85.

Benchmark (repeat threshold)Cuts / totalTokens saved, allTokens saved, cutsPasses or correct, SAM off → on
BigCodeBench-Hard (120)42/2842.50%11.08%72 → 72
AIME26, AMC23, MinervaMath (60)207/1,4884.93%16.13%898 → 906

Fixed-cap comparison.

Against an 8,192-token reasoning cap on the same math pairs, SAM used 2.55% more output tokens and confirmed correct counts went from 904 to 906.

AuthorsOpenReview ↗

Response to Reviewer 2RPf (2/2)

Q2: Contributions of SAM and the efficiency anchors

SAM makes training with the efficiency anchors more effective. It stops repetitive rollouts and identifies the suffix to penalize, while the anchors favor correct responses that are shorter or use fewer reflection steps.

Training with and without SAM.

Both Qwen3-4B runs enable both anchors; only Full ROAST uses SAM during training rollouts. Validation uses SAM off, one response per question, and an 8k cap. We report checkpoints through step 320 on a doubling grid; cells show accuracy (%) / mean reasoning tokens.

StepAIME24 w/o SAMAIME24 Full ROASTGSM8K w/o SAMGSM8K Full ROAST
06.67 / 7,304.23.33 / 6,818.090.37 / 1,020.990.07 / 904.3
4050.00 / 6,144.653.33 / 6,112.792.87 / 673.393.78 / 665.7
8053.33 / 5,838.853.33 / 5,206.392.27 / 515.593.33 / 457.6
16053.33 / 5,244.053.33 / 4,867.192.80 / 473.394.62 / 392.2
32050.00 / 5,468.050.00 / 4,897.694.62 / 455.994.62 / 455.9

On AIME24, SAM saves 7.19–10.83% of reasoning tokens at equal accuracy at steps 80, 160, and 320. On GSM8K, step 80 saves 11.23% with a 1.06 pp accuracy gain; step 160 saves 17.14% with a 1.82 pp gain. Both GSM8K metrics match at step 320.

Contribution of the suffix penalty.

The w/o Penalty control keeps SAM and both anchors on (1TaD W1). At step 100, adding the penalty saves 17.95% of GSM8K reasoning tokens with an accuracy change of −0.08 pp, and 1.18% of AIME24 tokens at equal accuracy.

Single-anchor ablations.

In Table 3, adding SAM and its suffix penalty to either anchor improves accuracy and shortens outputs. SAM Only includes the penalty and neither anchor.

Qwen3-4B comparisonMean token changeAccuracy Δ (pp)
Length anchor vs GRPO-8k−633−0.96
SAM Only vs GRPO-8k−87+0.48
Length anchor + SAM vs length anchor−190+1.28
Step+SAM vs Step−327+1.25

Q3: SAM precision and recall

Human annotations.

We use three sets:

  • AMC23 and MinervaMath: 1,872 responses from six base models, manually annotated by two experts for redundant steps, to evaluate threshold sensitivity.
  • DeltaBench: 1,235 error-selected responses with human annotations of unhelpful reasoning; 755 have positive or negative labels and 480 are undetermined.
  • ReasoningFlow: 31 traces manually annotated by two experts for restatements, including nonverbatim redundancy.

Threshold sensitivity.

The default threshold of 120 gives the highest precision in this sweep, 93.14%–96.08%. Lower thresholds increase recall and F1. We vary only the repeat threshold on the 1,872 math responses, keeping the window at 200 tokens and density at 0.85. A response is positive if it contains at least one human-labeled redundant step. The ranges account for 24 undetermined labels (Appendix J.5).

Repeat thresholdResponses with an alarmSample precisionSample recallSample F1
12010293.14%–96.08%8.32%–8.72%0.153–0.160
9015891.14%–93.67%12.62%–13.16%0.222–0.231
6035987.74%–90.25%27.73%–28.67%0.421–0.435
4561783.95%–86.71%45.92%–47.01%0.594–0.610
301,03074.56%–76.60%68.33%–69.09%0.713–0.727

On 240 AIME26 pairs from two 14B models (window 200), threshold 120 changed 2 correct answers to wrong, versus 19 at threshold 30. This supports our default's emphasis on precision and avoiding premature termination. The paper's Table 4 shows the same trade-off: (120, 200) cuts 19.5% of responses with 0 correct-to-wrong changes; (60, 100) cuts 58.1% with 41.

DeltaBench and nonverbatim redundancy.

At threshold 120, DeltaBench gives 84.3–85.2% response-level precision and 7.7–9.3% recall on the 755 labeled responses. Both tokenizers process the same traces; about half of the triggered responses have undetermined labels. On ReasoningFlow, SAM detects 2 of 22 positive traces. Among 18 traces with nonverbatim restatements of at least 11 tokens, its first window overlaps none of those spans. Appendix J.1 reports the full detection and window-overlap results.

References

[1] He, Y., et al. (2025). Can Large Language Models Detect Errors in Long Chain-of-Thought Reasoning? arXiv:2502.19361.

[2] Lee, J., et al. (2026). ReasoningFlow: Discourse Structures for Understanding LLM Reasoning Traces. arXiv:2606.05402.

AuthorsOpenReview ↗

Inquiry Regarding Discussion Phase Follow-Up

Dear Reviewer 2RPf,

Thank you again for your constructive feedback. As the discussion period draws to a close, we would be grateful if you could revisit our responses.

To address your concerns, we provided preliminary Qwen3-14B training results at step 60, additional controls examining the contributions of SAM and the suffix penalty with both efficiency anchors enabled, and sample-level precision, recall, and F1 measured against human annotations across repetition thresholds. We also clarified the incremental gains over GRPO-8k and SAM’s limited coverage of nonverbatim redundancy.

We would particularly appreciate your assessment of the additional evidence on model scale, SAM’s contribution during training, and detector reliability. Please let us know if any concerns remain or require further clarification.

Thank you again for your time and consideration.

Best regards,

The Authors of Paper 25398

Reviewer_1TaDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies overthinking in reasoning models trained with reinforcement learning from verifiable rewards. It argues that incorporating length penalties directly into GRPO rewards can assign negative advantages to longer but correct responses, while severe textual repetition can provide a structural signal of redundant reasoning. During training rollouts, ROAST uses a suffix automaton to detect long repeated substrings and high local repetition density, forces an end-of-thought marker when triggered, and then continues generating the answer. Training retains the correctness advantage, adds advantage bonuses to the shortest and fewest-step correct responses within each group, and applies relative down-weighting only to repeated suffixes of qualifying, correctly answered truncated trajectories. The paper reports accuracy and generation length on four models and four mathematical benchmarks, disables SAM at evaluation, and examines component and threshold ablations.

Critical strengths
  1. The structural detector and advantage reshaping are concretely specified and provide an operational alternative to direct length penalties. SAM uses only the generated token sequence, without accessing internal model activations or invoking an auxiliary judge. The two triggers, efficiency anchors, and suffix down-weighting region have explicit formulas. Separating correctness advantages from efficiency incentives also allows the reasoning prefix and the repetition-triggering suffix to receive different training signals. (Sections 3.1 and 4.1–4.2, Eqs. 1–14)

  2. The accuracy–length trade-off remains favorable with SAM disabled at evaluation, showing that the gains are not restricted to inference-time forced truncation. Table 1 covers four model sizes from the Qwen3 and DeepSeek-R1-Distill-Qwen families and four mathematical benchmarks. For example, on Qwen3-4B, average accuracy improves from 62.93% to 63.53% over GRPO-8k while average length decreases from 4930 to 4018 tokens. The paper also compares length-reward, truncation, and state-signal methods and reports component, shaping-strength, and threshold ablations, rather than only comparing against unmodified base models. (Section 5.1, lines 306–326; Table 1; Section 6, Tables 3–4)

Critical weaknesses

W1. The independent contribution of advantage shaping and its accuracy–length trade-off are insufficiently validated Section 4.2 defines shortest-length anchors, fewest-step anchors, and local suffix penalties as distinct training signals, but Table 3 does not specify which signals are enabled in Anchor Only, Anchor+SAM, and full ROAST. The text describes Anchor Only as length bonuses and links the full method's gains to using both anchor types, leaving it unclear whether the difference between full ROAST and Anchor+SAM comes from the step anchor, the suffix penalty, or both. The results support the combined method, but the increment attributable specifically to precise suffix credit assignment requires a more direct control. (Section 4.2, lines 254–300; Section 6, Table 3, lines 438–454) Table 3 also shows that increasing α further shortens outputs but harms accuracy. Since α controls both anchor bonuses and suffix penalties, this comparison does not disentangle their effects. Comparing only the default and one aggressive value does not establish the parameter range over which performance deteriorates. (Section 4.2, Eqs. 9 and 13; Table 3)

W2. The treatment of forced end-of-thought tokens in the policy-gradient and KL computations is unclear Eq. 5 defines the policy as deterministically emitting EoT after a repetition trigger, and the importance ratio in Eq. 15 also uses this SAM-intervened policy. However, the main text does not specify whether the forced end marker counts toward generation length, enters the policy-gradient and KL losses, or has its log probability recorded under the raw model or the intervened policy. Under the deterministic intervention distribution, that position itself provides no direct gradient on the raw model's stopping probability. If training instead uses raw-model probabilities, the correspondence to the written objective needs explanation. These choices affect the precise meaning of the on-policy update and the interpretation of autonomous termination with SAM disabled at evaluation. (Section 4.1, lines 201–210, Eq. 5; Section 4.2, lines 285–300, Eq. 15)

W3. The detector's semantic validity, scope, and realized computational benefits are insufficiently validated SAM triggers termination using fixed thresholds on token-substring repetition and local repetition density. It does not directly distinguish necessary repeated derivations from unproductive loops and may miss semantic stagnation expressed with different wording. Section 5.3 supports semantic improvement through fewer self-doubt and approach-switching markers, but the step reward already depends on separators such as “wait”; overlap between these vocabularies is unspecified, and lower marker frequency may reflect changes in expression. The main text also mentions GPQA transfer results, but SAM is disabled at evaluation, so transfer of the trained model does not directly validate the detector's ability to identify harmful loops across tasks. (Sections 4.1–4.2; Section 5.1, lines 311–316; Section 5.2, lines 366–369; Section 5.3, Table 2) Reducing generated tokens does not imply a proportional reduction in training compute; the cost of detector–generation-engine interaction must be included to establish the realized efficiency benefit. (Section 4.1, lines 176–210; Table 1)

Questions for authors
  1. Regarding W1: Which anchors and suffix penalties are enabled in each Table 3 variant, and is there a control retaining SAM and both anchors while disabling only the suffix penalty? Are finer-grained α sensitivity results or results varying bonus and penalty strengths separately available to characterize the accuracy–length trade-off?

  2. Regarding W2: How are log probabilities and loss masks constructed at forced EoT positions? Do these positions contribute to the policy-gradient loss, KL loss, and length statistics? Does the implemented importance ratio use the raw-model or SAM-intervened distribution, and how does this correspond to Eqs. 5 and 15?

  3. Regarding W3: How much overlap is there between the step-separator phrases and semantic diagnostic markers? Are there checks independent of these markers that examine premature truncation of necessary verification or missed paraphrased loops? How do rollout throughput and end-to-end training time change after accounting for SAM state maintenance and its interaction with the generation engine?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 1TaD (1/2)

W1: Ablation settings and the suffix penalty

We added a Qwen3-4B control that sets the suffix penalty to zero while keeping SAM in the training rollouts and both anchors enabled. The table shows the training settings (✓ enabled, ✗ disabled). Checkpoint evaluation and in-training validation both use standard decoding with SAM off.

Qwen3-4B settingRollout SAMLength bonusStep bonusSuffix penaltyMean tokensMean accuracy
GRPO-8k✗✗✗✗4,93062.93%
SAM Only✓✗✗✓4,84363.41%
Anchor Only✗✓✗✗4,29761.97%
Anchor+SAM✓✓✗✓4,10763.25%
Step✗✗✓✗4,65861.74%
Step+SAM✓✗✓✓4,33162.99%
Full, α = 0.1✓✓✓✓4,01863.53%
Full, α = 0.5✓✓✓✓3,27059.48%

The anchors add positive bonuses to correct-response advantages, favoring shorter reasoning and fewer reflection steps. With SAM and the suffix penalty enabled, adding the step anchor saves 89 tokens and gains 0.28 pp in Table 3. The new control compares suffix penalties of 0 and 0.1, with both anchor bonuses fixed at 0.1.

The no-penalty run has reached step 100, the latest validation step shared by all three runs. Cells report accuracy (%) and mean reasoning tokens (one response per question, 8k cap).

StepBenchmarkw/o online SAMw/o PenaltyFull ROAST
0AIME246.67 / 7,3043.33 / 6,9213.33 / 6,818
0GSM8K90.37 / 1,02190.45 / 94190.07 / 904
50AIME2436.67 / 6,43343.33 / 5,97853.33 / 6,020
50GSM8K92.65 / 63592.49 / 62492.80 / 599
100AIME2456.67 / 5,42060.00 / 5,14560.00 / 5,084
100GSM8K93.03 / 47992.72 / 49592.65 / 406

At step 100, adding the suffix penalty reduced GSM8K reasoning length by 17.95% with an accuracy change of −0.08 pp. AIME24 length fell 1.18% at equal accuracy. Table 3's α = 0.5 setting increases both anchor bonuses and the penalty. See our reply to 2RPf Q2 for the w/o SAM comparison through step 320.

W2: Policy-gradient and KL terms at forced </think>

We do not modify the model probabilities used in the training loss, including at forced </think>. The old and current actors recompute log-probs on stored responses using their original, unmasked distributions. Equation (5) specifies the SAM-intervened rollout rule, whereas the implemented importance ratio divides the current actor's unmasked probability by the old actor's unmasked probability on those responses. KL uses the unmasked current actor and reference distributions. The forced token counts toward response length and has mask value 1 in both losses, so its original log-probability provides a policy-gradient signal for stopping.

During rollout, SAM forces </think> at a repetition trigger and then continues answer generation. Each update uses fresh rollouts from the pre-update actor with SAM enabled. We clarified the rollout rule around Eq. (5). In the revised Eq. (15), the token-level ratio uses these unmasked actor probabilities; the SAM-intervened policy specifies the rollout distribution in the expectation.

Training uses equal train and PPO mini-batch sizes (32 prompts) and one PPO epoch per rollout batch. Each batch therefore receives one full-batch actor update, with the policy gradient evaluated at the pre-update actor.

AuthorsOpenReview ↗

Response to Reviewer 1TaD (2/2)

W3: Keyword overlap, verification, and efficiency

Keyword overlap.

There are no exact matches between the 14 step-anchor keywords and the eight diagnostic phrases. The only substring overlap is wait within wait, no. In correct traces, Table 2 reports 1.09 → 0.70 → 0.62 occurrences per 1,000 tokens for the length anchor, the length anchor with SAM, and Full ROAST. We revised §5.3 to describe the change in phrase frequency.

In DeltaBench, reflection labels were more frequent in sections with a Table 2 phrase (9.3%, 77/824) than in sections without one (2.8%, 278/10,017). Comparing sections of similar length in the same trace gave a gap of 4.75 pp (95% CI 1.50 to 8.31). Removing wait, no from the phrase list gave a gap of 3.46 pp (95% CI −0.47 to 7.96).

In ReasoningFlow, 18 of the 31 manually annotated traces contained a nonverbatim restatement of at least 11 tokens. SAM's first trigger window overlapped none of those restatements.

Verification after a trigger.

On BigCodeBench, we compared forcing EoT with continuing from the same reasoning prefix, seed and budget. In 101 of the 112 triggered pairs, the first SAM window contained neither keyword list.

BigCodeBench pairsOutput tokens saved [95% CI]Fail → passPass → fail
All 112 pairs18.84% [13.79, 24.89]20
101 pairs without either keyword list14.27% [10.86, 17.93]20

On the same 112 pairs, YfBb Q2 compares SAM with the original decode (24 → 24 passes); here we compare with a new continuation from the trigger prefix (22 → 24 passes).

On GPQA, the same-prefix test found six wrong-to-correct and three correct-to-wrong changes. In one of the three correct-to-wrong cases, the forced branch skipped an option check that the continued branch performed (Appendix J.4).

Measured efficiency on Qwen3-4B.

Generation timing includes SAM state updates and interaction with vLLM. Both training jobs used two NVIDIA A800 GPUs.

WorkloadBaselineWith SAM or ROASTWall time saved
BigCodeBench rollout (100 tasks, 27 cuts)2,509 s, SAM off2,058 s, inline SAM17.99% [6.59, 30.04]
End-to-end training66.5 h, GRPO-8k59.8 h, ROAST10.1%

The rollout interval is a 95% CI. Throughput increased by 21.94% in this test. In this timing cohort, passes among the 25 scored truncations went from 3 to 2. The end-to-end measurements correspond to 133.0 → 119.6 GPU-hours.

After training, Qwen3-4B ROAST reduced measured decoding time by 58% relative to the base model, with SAM off for both. Mean output length fell from 9,141 to 4,018 tokens (Table 1).

References

[1] He, Y., et al. (2025). Can Large Language Models Detect Errors in Long Chain-of-Thought Reasoning? arXiv:2502.19361. [2] Lee, J., et al. (2026). ReasoningFlow: Discourse Structures for Understanding LLM Reasoning Traces. arXiv:2606.05402.

I thank the authors for their thorough responses and extensive experimental results. However, these results do not fully resolve my remaining concerns. In particular, the idea of “detectable structural redundancy” still feels somewhat heuristic, and the SAM-based detection of redundant suffixes remains relatively coarse. I would therefore prefer to keep my current score.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Response to Reviewer YfBb (1/2)

Q1: Redundancy beyond exact repetition

63.5% of annotated restatement tokens fell outside later verbatim copies in the 31 ReasoningFlow traces. In the 102 SAM-truncated math trajectories (out of 1,872 responses), 68.82% of the omitted reasoning tokens were later verbatim copies. Two experts manually annotated each trace in both sets. We identify later verbatim copies using exact matches of at least 11 consecutive tokens.

DatasetTokens countedCountShare95% CI
ReasoningFlowRestatements outside later copies880/1,38663.5%52.5–80.2%
AMC23 + MinervaMathLater copies in omitted reasoning306,342/445,14168.82%63.50–74.38%

In the 18 ReasoningFlow traces with a nonverbatim restatement of at least 11 tokens, SAM's first trigger window overlapped none of those restatements. See Appendix J.1.

Q2: Transfer to code and long-document tasks

The main experiments used the same SAM thresholds and 14 step-anchor keywords across models (Appendices A and B). The code and long-document transfer tests retained the paper's default SAM settings (120, 200, 0.85) across all four base models (Appendix J.2).

With a 32k output cap, we report each base model's SAM transfer results on 142 BigCodeBench-Hard tasks with validated unit tests and 38 English LongBench v2 questions with full input documents. Arrows show standard decoding → SAM-enabled decoding.

BenchmarkTruncated / totalTokens saved, allMean tokens on cuts (saved)Passes or confirmed correct, allPasses or confirmed correct, cuts
BigCodeBench-⁠Hard112/5685.67%6,984 → 5,727 (17.99%)116 → 11624 → 24
LongBench v211/15214.21%7,730 → 3,991 (48.37%)42 → 413 → 2

On LongBench v2, one answer changed from correct to wrong. Token savings were 4.41% after removing the question with the largest savings. On BigCodeBench, pass/fail outcomes agreed in all 568 pairs.

With a fixed reasoning cap as the comparator, SAM used 3.05% fewer output tokens on code (8,192-token cap, passes 114 → 116) and 1.78% more on long documents (4,096-token cap, confirmed correct 42 → 41).

Step-anchor diagnostic.

Using the original 14 keywords, we compared passing code candidates within 25 model–task groups. Selecting a candidate with the fewest keyword matches used 11.30% fewer reasoning tokens than random selection (95% CI 6.65–16.65%). Appendix J.2 gives the per-model and fixed-cap results.

Trained-policy transfer.

We evaluated the base and ROAST-trained DeepSeek-R1-Distill-Qwen-7B checkpoints with SAM off for both.

BenchmarkBase acc (%) / tokensROAST acc (%) / tokensTokens saved
BigCodeBench-Hard5.54 / 5,7385.57 / 2,76851.76%
LongBench v215.13 / 1,11415.13 / 68938.15%

References

[1] Lee, J., et al. (2026). ReasoningFlow: Discourse Structures for Understanding LLM Reasoning Traces. arXiv:2606.05402.

AuthorsOpenReview ↗

Response to Reviewer YfBb (2/2)

Q3: Advantage bound with multiple truncations

With rewards ±1\pm 1 normalized using the sample standard deviation, the lower bound for each token of a correct response in a mixed group of size GG is

1G+Gϵ/2−α.\frac{1}{\sqrt{G}+G\epsilon/2}-\alpha.

The base advantage is smallest with one wrong response. The two anchors can only add to it, and the penalty on each token is capped at α\alpha regardless of how many responses SAM truncates.

At G=8G = 8, α=0.1\alpha = 0.1, and ϵ=10−6\epsilon = 10^{-6}, the bound is about 0.254. In all-correct groups the penalty is disabled and the bound is zero. We checked 2,304 groups, including 1,044 with multiple penalized correct responses. See §4.2 and Appendix J.3.

Q4: Answer quality after forced termination

Forced termination reduced empty answers caused by budget exhaustion from 8 to 0 across 75 paired continuations from the same prefix and seed. On GPQA, six wrong answers became correct and three correct answers became wrong. Empty answers count as wrong.

BenchmarkPairsEmpty, continued → forcedWrong to correctCorrect to wrong
GPQA-Diamond647 → 063
LongBench v2111 → 000
All758 → 063

In one of the three GPQA losses, the forced branch derived the expression for option A and selected C. The continued branch checked the options and selected A. All nine resolved LongBench v2 pairs agreed, and two continued answers had unresolved choices. See Appendix J.4.

AuthorsOpenReview ↗

Inquiry Regarding Discussion Phase Follow-Up

Dear Reviewer YfBb,

Thank you again for your thoughtful review. As the discussion period draws to a close, we would be grateful if you could revisit our responses.

Following your suggestions, we added analyses based on human annotations clarifying SAM’s coverage of nonverbatim redundancy, code and long-document evaluations using the original SAM settings, and paired continuation experiments examining answer quality after forced termination. We also clarified the advantage bound when multiple responses within a group are truncated. These additions are detailed in Appendix J.1–J.4.

We would greatly appreciate your feedback on whether these additions address your questions. If any points would benefit from further explanation, we would welcome the opportunity to clarify them.

Thank you again for your time and consideration.

Best regards,

The Authors of Paper 25398

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.