acceptodds
Under review as a conference paper at ICLR 2027

Giving Credit Where It’s Due: Redundancy-Aware Learning for Efficient Reasoning

Abstract

Large reasoning models can produce correct yet unnecessarily long reasoning traces. Existing methods improve reasoning efficiency with trajectory-level objectives or local token- and step-level signals, but rarely model inter-step semantic dependencies. This limits their ability to distinguish redundant steps from those that support later deductions, making it harder to shorten reasoning without sacrificing accuracy. We introduce **RECAP** (REdundancy-aware Credit Assignment via Propagation), which addresses this limitation by **assigning credit where it is due** based on both a step's downstream role in the reasoning structure and its contribution to solving the problem correctly. We define ***structural responsibility*** to capture the step’s downstream role by measuring how strongly later reasoning depends on it, using credit propagated backward from the final-answer node through an outcome-independent, LLM-annotated semantic dependency graph. **However, a step can have high structural responsibility yet steer the reasoning away from the correct solution.** RECAP therefore introduces ***step efficacy*** to measure answer-directed progress through changes in gold-answer log-likelihood as each step is added. Together, these signals reshape rollout-level GRPO advantages into step-specific updates. RECAP requires neither a separately trained process reward model nor preconstructed concise trajectories. Across two 7B models and four mathematical reasoning benchmarks, RECAP improves the accuracy–efficiency trade-off. On Qwen2.5-Math-7B, it improves pass@1 by 2.0-3.7 percentage points while reducing reasoning tokens by 8%–31% relative to GRPO across all four benchmarks. Analysis suggests these savings reflect fewer reasoning operations and less dead-end reasoning, rather than more compact expression.

Primary area:
reinforcement learning
Keywords:
LLM Reasoning, Efficienct Reasoning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_nBR9OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how to reduce unnecessary computation and redundant steps in large language model reasoning while maintaining reasoning accuracy. The authors point out that existing GRPO-based reinforcement learning methods typically assign the same advantage to all tokens within a reasoning trajectory, making it difficult to distinguish essential reasoning steps that contribute to the final answer from redundant ones. Moreover, existing fine-grained optimization methods often focus on the local contributions of individual tokens or steps, without explicitly modeling semantic dependencies among reasoning steps.

To address these challenges, the authors propose RECAP (Redundancy-aware Credit Assignment via Propagation), a step-level credit assignment method built upon GRPO. Specifically, the authors first use an LLM to construct a semantic dependency graph among reasoning steps and compute each step's Structural Responsibility by propagating credit backward from the final-answer node. They then measure Step Efficacy through changes in the gold answer's log-likelihood before and after incorporating each reasoning step. Finally, the two signals are combined to construct step-specific weights that redistribute the original GRPO advantages, thereby reinforcing important reasoning steps while downweighting redundant ones.

Experiments are conducted on Qwen2.5-Math-7B and DeepSeek-R1-Distill-Qwen-7B across four mathematical reasoning benchmarks: GSM8K, MATH-500, AIME 2024, and AIME 2025. The method is compared against GRPO, length-penalty approaches, and other reasoning-efficiency optimization methods. On Qwen2.5-Math-7B, RECAP improves pass@1 by 2.0–3.7 percentage points while reducing reasoning tokens by 8%–31% compared with GRPO. On DeepSeek-R1-Distill-Qwen-7B, RECAP also achieves a competitive accuracy–efficiency trade-off. Additionally, component ablations, semantic dependency annotation quality evaluations, and reasoning trajectory analyses examine the contributions of different components and show that RECAP reduces redundant steps and unproductive reasoning branches in generated trajectories.

Critical strengths
  • To address the difficulty of distinguishing essential reasoning steps from redundant ones in existing GRPO methods, this paper proposes a step-level credit assignment approach based on semantic dependencies. By combining Structural Responsibility and Step Efficacy, the method enables fine-grained reweighting of the original GRPO advantages. The overall approach is clear and provides an alternative to direct length penalties for improving reasoning efficiency.

  • The paper conducts experiments on two 7B models with different reasoning characteristics across four mathematical reasoning benchmarks, comparing RECAP against several relevant methods, including GRPO, DDCA, LEASH, BINGO, and ThoughtFold. On Qwen2.5-Math-7B, RECAP improves reasoning accuracy while reducing the number of generated tokens. On DeepSeek-R1-Distill-Qwen-7B, it also demonstrates a competitive accuracy–efficiency trade-off. These results provide substantial empirical evidence for the practical effectiveness of the proposed method.

  • The experiments include separate ablations of Structural Responsibility and Step Efficacy, as well as random and shuffled structural-weight experiments to further examine the influence of semantic dependency information on training effectiveness. In addition, the paper analyzes dependency annotation quality, the number of reasoning operations, and unproductive reasoning branches, while also reporting additional training overhead. These analyses provide further evidence supporting the effectiveness of the method and offer insights into the potential sources of its reasoning-efficiency improvements.

Critical weaknesses
  • The paper estimates the contributions of individual reasoning steps using semantic dependencies among steps and changes in the gold answer's log-likelihood. However, these two signals do not necessarily accurately reflect the actual contribution of each step to the final answer. For example, a step may be heavily relied upon by subsequent reasoning without positively contributing to the correct solution, and an increase in the gold answer's likelihood does not necessarily indicate that the step is mathematically correct or necessary. Although the authors demonstrate the effectiveness of their approach through random and shuffled structural-weight experiments, these results primarily show that the proposed advantage reweighting strategy can improve training outcomes, rather than sufficiently establishing the reliability of its credit assignment.

  • According to Table 2, introducing Step Efficacy improves pass@1 only from 81.32% to 81.90% and reduces the average number of generated tokens from 370 to 352, while pass@8 slightly decreases. Given that Step Efficacy is a key component designed to distinguish structural dependencies from contributions toward the correct answer, these relatively modest gains do not sufficiently establish its necessity. Moreover, the current component ablations are mainly conducted on Qwen2.5-Math-7B, without corresponding validation on the second backbone.

  • The authors analyze the number of reasoning operations and dead-end reasoning to show that RECAP reduces redundant content in generated trajectories. However, shorter reasoning trajectories may naturally contain fewer unproductive steps and exhibit simpler dependency structures. The current experiments lack structural comparisons under matched reasoning lengths or computational budgets. Therefore, it remains unclear to what extent the observed improvements result from more accurate step-level credit assignment rather than simply shortening the overall reasoning trajectories.

Questions for authors

I'm open to raise my score if the authors could adequately address the following concerns. Given the substantial engineering effort involved in the experiments, I believe that a few lightweight additional experiments and clarifications would be sufficient.

  • Could the authors further validate whether Structural Responsibility and Step Efficacy genuinely reflect the contributions of individual reasoning steps to the final answer? For example, could a small-scale counterfactual experiment be conducted by removing or perturbing selected reasoning steps and measuring the resulting changes in subsequent reasoning accuracy or the gold answer's likelihood? Examining the correlation between the assigned credit and the actual contribution of each step would help establish whether RECAP achieves reliable credit assignment rather than merely providing an effective advantage reweighting strategy.

  • According to Table 2, the additional performance gains from Step Efficacy are relatively modest. Could the authors further clarify whether these improvements are stable and statistically significant? For example, providing corresponding component ablations on DeepSeek-R1-Distill-Qwen-7B or reporting the variability of the existing experimental results would help establish the independent contribution and necessity of Step Efficacy across different models.

  • Could the authors further investigate whether the changes in reasoning trajectory length influence the dead-end reasoning analysis? For example, comparing RECAP with other methods under matched generated-token counts, numbers of reasoning steps, and answer correctness could help determine whether RECAP achieves lower dead-end step ratios or more effective dependency structures independently of trajectory length. Such an analysis would provide stronger evidence that RECAP genuinely improves the effectiveness of reasoning steps rather than simply generating shorter reasoning trajectories.

6 comments
AuthorsOpenReview ↗

Rebuttal by authors (Part 1)

We thank the reviewer for the thoughtful feedback and for recognizing RECAP's semantic dependency-aware credit assignment framework, its effectiveness in improving the accuracy–efficiency trade-off across two model backbones and four benchmarks, and the supporting ablation and reasoning trajectory analyses.

Following the reviewer's suggestions, we add (i) counterfactual step-corruption experiments validating both credit signals (Tables R8–R12), (ii) a Step Efficacy ablation on DeepSeek-R1-Distill-Qwen-7B (Table R13), and (iii) a rollout-level dead-end analysis controlling for length, step count, and correctness (Tables R14-R15).

W1 & Q1. Reliability of step-Level credit assignment.

We thank the reviewer for raising the question of whether RECAP's assigned credits meaningfully reflect the contributions of individual reasoning steps.

Counterfactual corruption shows that RECAP's credit reflects meaningful step-level contributions to downstream answer accuracy: corrupting answer-reaching steps causes ~3× the accuracy drop of dead-end steps, and among structurally important steps, those with positive Step Efficacy cause a 13.2-point larger drop.

As the reviewer notes, a step may be heavily relied upon downstream without advancing the correct solution. This limitation of Structural Responsibility alone is what motivates Step Efficacy (lines 067–069 & Section 4). We therefore test both signals counterfactually.

Setup. We generate 8 rollouts per problem with the RECAP-trained Qwen2.5-Math-7B model on the 512-problem validation set and retain 234 problems with both correct and incorrect rollouts, where group-relative advantages are nonzero and step-level credit assignment affects training. This yielded 1850 validly annotated rollouts. Excluding final-answer steps, steps without numerical values, and steps lacking valid Structural Responsibility scores leaves 6393 reasoning steps. For each selected step, we corrupt all numerical values and independently sample 16 continuations from both the original and corrupted prefixes to measure changes in final-answer accuracy. Numerical corruption preserves the step's wording and position while altering its content. (In pilot deletion experiments, the model often re-derived missing steps, yielding negligible accuracy changes, so we chose corruption as the counterfactual intervention)

1) Structural Responsibility identifies steps that downstream reasoning depends on.

We compare the accuracy drops caused by corrupting dead-end steps (resp = 0) and answer-reaching steps (resp > 0).

As shown in Table R8, corrupting answer-reaching steps causes approximately three times the accuracy drop of corrupting dead-end steps (23.9 vs. 7.6 pp). This substantial difference suggests that Structural Responsibility captures meaningful downstream dependencies: the answer-reaching steps identified by Structural Responsibility have a much greater influence on subsequent reasoning than dead-end steps, supporting the use of dependency structure to distinguish reasoning steps that warrant stronger credit assignment.

Table R8. Accuracy change after corrupting dead-end vs. answer-reaching steps.

Step type# StepsAccuracy before → afterDrop95% CI
Dead-end (resp=0)40236.8% → 29.2%7.6%[5.0, 10.2]
Answer-reaching (resp>0)5,99150.6% → 26.7%23.9%[22.8, 24.9]

This difference persists across all step-length and numerical-count groups (Table R9). Even among steps of similar length or numerical count, corrupting answer-reaching steps consistently causes substantially larger accuracy drops than corrupting dead-end steps, indicating that Structural Responsibility captures meaningful downstream dependencies beyond these potential confounding factors.

Table R9. Counterfactual accuracy drops after numerical corruption, stratified by step length and numerical count. Each value is the mean decrease in downstream answer accuracy (percentage points, pp) obtained by individually corrupting n reasoning steps in the corresponding group and averaging their accuracy drops.

StratificationGroupAccuracy drop after corrupting a dead-end step (pp)Accuracy drop after corrupting an answer-reaching step (pp)
Step lengthShort (≤66 tokens)7.0 (n=184)18.0 (n=1948)
Medium (67–117)7.7 (n=121)25.0 (n=2020)
Long (>117)8.5 (n=97)28.4 (n=2023)
Numerical countFew (≤6)4.4 (n=212)17.1 (n=2362)
Medium (7–12)12.4 (n=113)26.0 (n=1600)
Many (>12)9.3 (n=77)30.0 (n=2029)
AuthorsOpenReview ↗

Rebuttal by authors (Part 2)

W1 & Q1 - (2) Step Efficacy tracks answer-directed contribution, including among structurally important steps

Step Efficacy further distinguishes answer-reaching steps by their impact on answer accuracy (Table R10). We consider steps with positive Structural Responsibility (resp>0), divide them into two groups based on Step Efficacy (Δ > 0 vs. Δ <= 0), and compare the accuracy drops caused by numerical corruption. Both groups have downstream dependencies and similar length and numerical content, yet corrupting steps with Δ > 0 causes a substantially larger accuracy drop than corrupting steps with Δ ≤ 0 (30.4 vs. 17.1 pp), a difference of 13.2 pp (95% CI [11.3, 15.1]). Thus, Step Efficacy helps distinguish steps with different answer-directed contributions even when they are structurally important, supporting its complementary role in RECAP's credit assignment.

Table R10. Step Efficacy distinguishes answer-reaching steps with different counterfactual Impacts on answer accuracy.

Step Efficacy# StepsAvg. lengthAvg. numerical countAccuracy drop (pp)95% CI
(Δ>0)3,04412714.930.4[28.3, 32.6]
(Δ<=0)2,94712013.217.1[15.3, 18.9]

Step Efficacy is positively associated with downstream answer accuracy. Among answer-advancing steps (Δ>0\Delta >0) in correct rollouts, higher original Step Efficacy corresponds to larger accuracy drops after numerical corruption. As shown in Table R11, steps in the highest-efficacy group experience a 63.0 pp accuracy drop, compared with 45.0 and 34.6 points in the middle- and lowest-efficacy groups, despite similar initial accuracy (82–90%). Moreover, original Step Efficacy is positively correlated with the subsequent accuracy drop (Spearman's ρ = 0.30, 95% CI [0.25, 0.34]). Corruption lowers Δ in all terciles, but Δ remains positive for the highest tercile (1.69) even though accuracy drops by 63 points, consistent with the reviewer's observation that answer likelihood alone does not fully reflect correctness. RECAP therefore uses Step Efficacy to calibrate structural credit rather than as a standalone measure of correctness.

In conclusion, these results provide independent counterfactual evidence that Step Efficacy reflects meaningful information about a step's influence on the final answer.

Table R11. Step efficacy and downstream accuracy before and after numeral corruption (answer-advancing steps, Δ>0\Delta >0, in correct rollouts; grouped into terciles of original Δ\Delta).

Original Δ\Delta# StepsΔ\Delta: before → afterAccuracy: before → after (change)
Low6490.06 → −0.3889.8% → 55.2% (−34.6)
Middle6480.65 → 0.3982.3% → 37.4% (−45.0)
High6472.94 → 1.6989.5% → 26.5% (−63.0)
AuthorsOpenReview ↗

Rebuttal by authors (Part 3)

W1 & Q1 - (3) RECAP's credit signals identify steps with greater influence on answer accuracy.

Within each rollout, we corrupt the highest- and lowest-scored step under each criterion (Table R12).

As shown in Table R12, for all three RECAP signals (structure responsibility, step efficacy, and their combination), corrupting the highest-scored step causes a significantly larger drop in accuracy (drop of 8.9–10.9 pp) than corrupting the lowest-scored step. In contrast, random selection shows no difference (+0.1 points), and position-based selection shows the opposite pattern, with earlier steps being slightly more influential (−2.8 points). Step length is also predictive (+8.3 points), likely because corrupting a longer step perturbs more content; the length-controlled analyses above (Tables R9–R10) show that both Structural Responsibility and Step Efficacy remain predictive beyond length.

Note.Table R12 evaluates whether each signal identifies causally important steps in a fixed trained policy, whereas the ablation in Table 2 measures each signal’s contribution to training. The two are complementary rather than comparable. Structural Responsibility has the larger training effect (Table 2) because it sets the score of dead-end steps to zero, whereas Step Efficacy only rescales the credit of the remaining steps within a bounded range.

Table R12. Accuracy drop (pp) after corrupting the highest- vs. lowest-scored step within the same rollout. # Rollouts reports how many rollouts contain at least two perturbable reasoning steps with distinct scores under each selection criterion, allowing a valid highest- vs. lowest-scored step comparison. Extra drop denotes the difference in accuracy reduction between the highest- and lowest-scored steps.

Selection criterion# RolloutsHigh-score drop (pp) (Accuracy: before → after)Low-score drop (pp) (Accuracy: before → after)Extra drop (pp)95% CI
Structural Responsibility87027.6 (50.0→22.3)18.7 (49.6→30.9)+8.9[6.0, 11.8]
Step Efficacy1,31031.7 (53.0→21.3)20.9 (46.6→25.7)+10.8[8.4, 13.2]
Combined RECAP Weight1,26832.1 (52.2→20.1)21.2 (50.6→29.4)+10.9[8.5, 13.3]
Random1,31024.6 (50.3→25.7)24.5 (50.3→25.8)+0.1[−2.1, 2.4]
Position1,31025.9 (50.6→24.7)28.7 (50.4→21.7)−2.8[−5.7, −0.2]
Step Length1,30928.3 (50.3→22.0)20.0 (50.6→30.6)+8.3[5.9, 10.7]

W2 & Q2. Stability and consistency of step efficacy’s contribution

We thank the reviewer for raising the question of whether Step Efficacy provides stable and consistent improvements. Following the reviewer's suggestion, we conducted an additional ablation on DeepSeek-R1-Distill-Qwen-7B (Table R13), comparing full RECAP against the variant without Step Efficacy.

Across both backbones, Step Efficacy improves pass@1 and reduces reasoning tokens while maintaining comparable pass@8, with larger gains on the long-thinking model (+1.20 pp pass@1, −18.8% tokens). At the step level, it further distinguishes answer-directed contributions that Structural Responsibility alone cannot differentiate (Table R10).

These results suggest that Structural Responsibility provides the primary credit signal, while Step Efficacy adds a complementary, answer-directed calibration. Its aggregate effect is larger on the long-thinking model.

Table R13. Ablation of Step Efficacy (pooled over GSM8K, MATH-500, AIME 2024, and AIME 2025). pass@1 is reported as mean ± std over 8 sampled runs. Δ is full RECAP minus w/o step efficacy, with 95% confidence intervals from a paired bootstrap over problems (5,000 resamples, stratified by benchmark).

ModelMethodpass@1 ↑pass@8 ↑tokens ↓
Qwen2.5-Math-7BRECAP (full)81.90± 0.4091.80352
w/o step efficacy81.32± 0.6092.18370
Δ [95% CI]+0.59
+0.01,+1.15+0.01, +1.15

| −0.37

−1.12,+0.37−1.12, +0.37

| −4.7% [−7.2, −2.0] | | DeepSeek-R1-Distill-Qwen-7B | RECAP (full) | 89.11± 0.47 | 95.65 | 976 | | | w/o step efficacy | 87.91± 0.37 | 95.53 | 1202 | | | Δ [95% CI] | +1.2

+0.63,+1.76+0.63, +1.76

| 0.12

−0.49,+0.73−0.49, +0.73

| −18.8% [−20.3, −17.5] |

Reviewer_ovUtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper addresses the inefficiency of long chain-of-thought reasoning in LLMs and the limitation of GRPO in effiicient reasoning, which assigns a uniform advantage to all tokens in a rollout regardless of their contribution. The authors propose RECAP, a method that constructs a typed semantic dependency graph over each rollout's reasoning steps (with support, context, and restate edges annotated by an LLM judge), propagates structural responsibility backward from the final answer, and combines it with a step efficacy signal measured as the change in gold-answer log-likelihood when each step is added, using this to rescale the rollout-level GRPO advantage into step-specific advantages. They train Qwen2.5-Math-7B and DeepSeek-R1-Distill-Qwen-7B on 2048 DAPO-Math problems and evaluate on GSM8K, MATH-500, AIME 2024, and AIME 2025, reporting pass@1, pass@k, and mean generated tokens, along with ablations of the two signals and analyses of reasoning-operation counts and dead-end steps.

Critical strengths
  1. The research problem the paper addresses is impactful. Efficient reasoning is an important topic in the field of LLM reasoning.
  2. The paper presents a reasonable experimental design.
  3. The analysis of the sources of token savings is relatively in-depth: it distinguishes between the two possibilities of "more compact expression" and "doing less," and further supports this with the proportion of dead-end steps.
Critical weaknesses
  1. Dependency annotation relies on an external strong model (Claude Haiku 4.5), which limits cost and reproducibility. The method targets token savings at inference time, but the additional annotation and computation steps it introduces raise training-time cost.
  2. There are many hyperparameters, which increases the cost of transferring the method across tasks and models and raises the barrier to practical application.
  3. On DeepSeek-R1-Distill-Qwen-7B, the model trained with RECAP produces more output tokens than GRPO on all four benchmarks, which is in tension with the motivation (e.g., the last sentence of the first paragraph, trajectory-level GRPO "permitting redundant reasoning to persist" actually produces less output tokens).
  4. There is no evaluation on tasks outside mathematics, such as science and code. Alternatively, could the authors discuss whether the annotation and step efficacy computation procedures in RECAP can still be applied to more open-ended tasks with freer reasoning structures?
  5. The policy models used for evaluation are relatively old. Results on newer model families such as Qwen3 would be more convincing. Alternatively, the authors need to explain why older models were used.
  6. The paper reports a maximum response length of 8192 tokens for training but does not state the maximum generation length used at evaluation time. Was the evaluation length set to 32768, aligned with the DeepSeek-R1 technical report? If not, could the authors clarify this setting and discuss whether the difference affects comparability with the R1-Distill baselines?
Questions for authors

Please see Weaknesses.

7 comments
AuthorsOpenReview ↗

Rebuttal by authors (Part 1)

We thank the reviewer for recognizing the importance of the research problem, the reasonable experimental design, and the in-depth analysis of RECAP's token savings. We address the concerns below.

Summary of new results: (i) a full training-cost breakdown (Tables R1–R2); (ii) evidence that RECAP's gains do not hinge on precise hyperparameter tuning (Table R6); (iii) cross-domain transfer to GPQA (Table R4); and (iv) truncation statistics under the 8192-token budget (Table R7).

W1. Training cost and dependence on a strong external annotator

RECAP's overhead is confined to training, and RECAP is not tied to a specific annotator. We quantify the measured costs below (Tables R1–R2) and report results with an open-weight Qwen3-32B annotator under the same training configuration.

1) RECAP adds a one-time training cost but no annotation or scoring overhead at inference.

Our primary objective is to improve the inference-time accuracy–efficiency trade-off of the learned policy. RECAP achieves this through more informative step-level credit assignment during RL training, with no annotation or scoring overhead at inference time.
Importantly, the additional training computation yields clear inference-time benefits. On Qwen2.5-Math-7B, RECAP increases training time from 5 to 8.5 hours on 8×H100 relative to GRPO, but improves pass@1 by 2.0–3.7 percentage points (pp) while reducing inference tokens by 8%–31%. Moreover, compared with ThoughtFold on Qwen2.5-Math-7B, RECAP trains faster under the same hardware configuration (8.5 vs. 10 hours), while achieving higher accuracy and using fewer inference tokens across all four benchmarks. This overhead is incurred once, whereas the resulting accuracy–efficiency benefits apply to every query at inference time.

Table R1. Training time (8×H100, 60 training steps).

ModelMethodTotalTotal Time / training stepTime of step efficacy computation / training stepTime of dependency annotation / training step
Qwen2.5-Math-7BGRPO5.0h~300s––
RECAP8.5h~510s~149s~61s
DeepSeek-R1-Distill-Qwen-7BGRPO15.2h~912s––
RECAP22.0h~1320s~250s~158s

Table R2. Annotation API usage per training run (Claude Haiku 4.5; 2048 annotation calls per step).

ModelInput tokensOutput tokensAPI cost
Qwen2.5-Math-7B328M40.6M$531
DeepSeek-R1-Distill-Qwen-7B816M126M$1447
AuthorsOpenReview ↗

Rebuttal by authors (Part 2)

W1-(2) Dependence on a strong external annotator.

Haiku 4.5 was selected through a prior empirical annotator selection study (Appendix A.3), where it had higher agreement with an independent judge panel and better downstream performance than Qwen3-32B, rather than because RECAP fundamentally depends on this particular annotator. RECAP benefits from higher-quality annotation, but its main accuracy gains can be largely preserved using the more reliably identified support dependencies, and it can also operate with an open-weight annotator. Our experimental results further support this point.

  • RECAP's main accuracy gains can be preserved through the more reliably identified support dependencies, without requiring equally accurate annotation of all three relation types. Table R3 below shows that using support edges alone stays within 0.7 pass@1 points of Full RECAP, and fully swapping context and restatement still outperforms GRPO on all four benchmarks. This suggests that reliable identification of the main support backbone is particularly important.
  • RECAP remains effective with a weaker open-weight annotator, although higher annotation quality yields better performance. Under the same hyperparameters (Appendix Table 5), replacing Haiku 4.5 with Qwen3-32B reduces pass@1 by 1.2–4.0 percentage points and increases response length by approximately 1%–9%. This decline is moderate given its much lower annotation agreement (41.9% vs. 77.0%), and is consistent with its weaker support-edge annotation (F1 62.8 vs. 83.0).
  • Practicality and reproducibility. Annotation runs only during training, accounts for approximately 12% of training wall-clock time and costs approximately \531 per run on Qwen2.5-Math-7B and \1447 on DeepSeek-R1-Distill-Qwen-7B with Haiku 4.5. An open-weight annotator can replace it at a moderate accuracy cost. For reproducibility, we have provided the annotation prompts and corresponding settings in Appendix A.4.1 and Figure 9.

Table R3. Ablation on dependency-edge types in structural responsibility on Qwen2.5-Math-7B. Each entry reports pass@1 (%) / average generated tokens.

Method (S/C/R)GSM8KMATH-500AIME24AIME25
GRPO84.4 / 35472.2 / 64124.2 / 15298.8 / 1405
Full RECAP (1/0.5/0)87.7 / 24674.2 / 52426.7 / 140112.5 / 1098
Support-only (1/0/0)87.1 / 22873.5 / 51426.2 / 134012.5 / 1424
C↔R Swapped (1/0/0.5)86.7 / 23674.0 / 54725.0 / 14809.2 / 1149
Uniform (1/1/1)80.7 / 40369.9 / 70421.2 / 163711.2 / 1575

W2. Hyperparameter complexity.

RECAP adds no optimizer or rollout hyperparameters, and its gains do not hinge on precise tuning.

RECAP-specific hyperparameters fall into three groups (Appendix A.4.1):

Table R6. RECAP-specific hyperparameters.

GroupHyperparameterQwen2.5-Math-7BDeepSeek-R1-DistillHow set
Fixed by designγsupportγ_{support} / γrestateγ_{restate}1 / 01 / 0Semantic role; only edge-weight ratios matter
Rule-based (no RL tuning)τΔ\tau_\Delta24Clips ~2 steps per trace (offline Δ\Delta distribution)
τw\tau_w510~99th percentile of offline step weights
Max semantic steps– (raw split)50Only for long traces; annotation-reliability screen + short pilot
Validation-tunedγcontextγ_{context} / α\alpha / β\beta0.5 / 0.5 / 0.30.75 / 1.0 / 0.3Small grid on held-out validation

Good performance without precise hyperparameter tuning. As shown in Appendix Table 5 (Haiku row), a configuration not tuned for Qwen2.5-Math-7B already matches or exceeds GRPO (Table 1) in both accuracy and tokens on all four benchmarks. We agree that the validation-tuned coefficients differ between our two backbones, so a new model may require a small search. We recommend starting from β=0.3\beta= 0.3, α∈\alpha\in{0.5, 1.0}, and γcontext∈γ_{context}\in{0.5, 0.75}, with thresholds set by the rules in Table R6.

AuthorsOpenReview ↗

Rebuttal by authors (Part 3)

W3. RECAP vs. GRPO length on DeepSeek-R1-Distill-Qwen-7B

We thank the reviewer for pointing out a potential ambiguity in our motivation. Our primary goal is to improve the inference-time accuracy–efficiency trade-off, rather than simply minimizing output tokens. Our claim concerns redundancy rather than length. The limitation of trajectory-level GRPO is that it cannot distinguish redundant reasoning from steps that contribute to the final answer, which does not imply that GRPO necessarily produces longer responses. To make this distinction clearer, we will revise the sentence as follows: "Standard outcome-based RL provides no step-level signal for distinguishing low-contribution steps from the deductions that contribute to the final answer. Thus, even if it produces shorter responses (e.g., under a fixed generation budget), the reduction is not targeted at redundant steps."

On all four DeepSeek benchmarks, no evaluated baseline achieves both higher pass@1 and fewer tokens than RECAP: GRPO is shorter but 2.0–5.4 points less accurate, while every baseline with equal or higher pass@1 uses 9.7%–36.2% more tokens.

Why GRPO is shorter on DeepSeek. The base model frequently exceeds the 8192-token limit (54.6% and 65.4% of its AIME 2024 / 2025 responses are truncated; Table R7). Since truncated responses receive no reward, this limit imposes an implicit length pressure on all methods, and GRPO and RECAP shorten at nearly the same rate during training (Figure 7). Under this shared pressure, RECAP shortens more selectively: it ends training with fewer reasoning steps and higher training reward than GRPO (Figure 7), and at test time it is 2.0–5.4 points more accurate with 1%–16% more tokens. This accuracy gain is not explained by truncation: on MATH-500 and AIME 2025, RECAP is truncated slightly more often than GRPO (2.4% vs. 2.1%; 29.6% vs. 27.9%), yet it is 5.1 and 5.4 points more accurate.

Reviewer_6EFkOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RECAP (REdundancy-aware Credit Assignment via Propagation), a step-level credit assignment framework for improving the efficiency of LLM reasoning. RECAP constructs a semantic dependency graph over reasoning steps, distinguishing support, context, and restatement relations. It then propagates structural responsibility backward from the final answer and combines this signal with step efficacy, measured by changes in gold-answer log-likelihood, to derive step-specific GRPO advantages. Experiments on two 7B models across four mathematical reasoning benchmarks demonstrate improved accuracy–efficiency trade-offs. Ablation studies investigate the contributions of structural responsibility and step efficacy, while additional analyses suggest that the reduction in reasoning tokens primarily comes from eliminating unnecessary reasoning operations and dead-end branches.

Critical strengths
  1. A meaningful approach to structure-aware credit assignment. The paper addresses an important limitation of existing reasoning-efficiency methods: trajectory-level rewards and local importance estimates cannot explicitly capture dependencies between reasoning steps. The proposed semantic dependency graph provides a principled mechanism for identifying steps that support downstream deductions, distinguishing them from redundant or unused reasoning. Combining structural responsibility with answer-directed efficacy is a technically interesting approach that does not require training a separate process reward model.

  2. Promising empirical improvements supported by targeted analyses. RECAP demonstrates substantial improvements over GRPO on Qwen2.5-Math-7B, increasing pass@1 by 2.0–3.7 percentage points while reducing generated tokens by 8%–31% across four benchmarks. The ablation studies provide evidence that semantically aligned structural responsibility contributes meaningfully to performance. Furthermore, the analysis of reasoning operations and dead-end branches provides useful evidence that the efficiency improvements reflect changes in reasoning behavior rather than merely shorter textual expressions.

Critical weaknesses
  1. Substantial training overhead and dependence on a strong external annotator limit practical efficiency and scalability.

RECAP requires an external LLM to construct semantic dependency graphs and additional forward computations to estimate step efficacy. According to Table 14, training time increases from approximately 5 to 8.5 hours on Qwen2.5-Math-7B and from 15.2 to 22 hours on DeepSeek-R1-Distill-Qwen-7B compared with GRPO.

Moreover, the method relies on Claude Haiku 4.5, and Appendix A.3 shows considerable performance degradation when replacing it with Qwen3-32B. Even the selected annotator achieves relatively modest agreement with the judge consensus for context and restatement relations.

Although the inference-time token reductions are promising, the paper does not sufficiently quantify the overall training and annotation costs or demonstrate whether the benefits remain competitive under matched computational budgets. This limitation is consequential because efficiency is a central motivation of the work.

  1. Limited evidence of robustness and generalization across reasoning settings.

The evaluation is restricted to two 7B models and mathematical reasoning benchmarks. While the improvements on Qwen2.5-Math-7B are convincing, results on DeepSeek-R1-Distill-Qwen-7B are less consistent. RECAP offers a competitive trade-off but does not consistently outperform the strongest baselines in either accuracy or token consumption individually.

Additionally, the AIME benchmarks contain only 30 problems each, resulting in substantial evaluation variance. The training setup reports a single random seed, and independent multi-seed training results are not provided. Consequently, it remains unclear how reliably the observed improvements generalize across training runs, model scales, and reasoning domains.

Questions for authors
  1. Could the authors provide a more comprehensive accounting of computational and API costs, including dependency annotation and step-efficacy estimation? How does RECAP compare against the strongest baselines under matched training-time or computational budgets?

  2. How sensitive is RECAP to errors in the semantic dependency graph, particularly context and restatement edges, where annotator agreement is relatively low? Could the authors clarify whether the gains primarily arise from high-quality support edges or require all three relation types?

  3. Could the authors provide results from multiple independent training seeds, particularly for the smaller AIME benchmarks? This would help determine whether the observed accuracy improvements are statistically reliable.

3 comments
AuthorsOpenReview ↗

Rebuttal by authors (Part 1)

We thank the reviewer for their feedback and for recognizing RECAP's structure-aware credit assignment framework, its ability to capture dependencies between reasoning steps, and its empirical accuracy–efficiency improvements. We address the concerns and questions below.

Summary of new results: (i) a full training-cost breakdown (Tables R1–R2); (ii) edge-type ablations testing sensitivity to annotation errors (Table R3); (iii) cross-domain transfer to GPQA (Table R4); and (iv) three-seed training runs (Table R5).

W1 & Q1. Training overhead and dependence on an external annotator

(a) Training cost

Our primary objective is to improve the inference-time accuracy–efficiency trade-off of the learned policy. RECAP achieves this through more informative step-level credit assignment during RL training, with no annotation or scoring overhead at inference time.

Importantly, the additional training computation yields clear inference-time benefits. On On Qwen2.5-Math-7B, RECAP increases training time from 5 to 8.5 hours on 8×H100 relative to GRPO, but improves pass@1 by 2.0–3.7 percentage points while reducing inference tokens by 8%–31%. Moreover, compared with ThoughtFold on Qwen2.5-Math-7B, RECAP trains faster (8.5 vs. 10 hours), while achieving higher accuracy and using fewer inference tokens across all four benchmarks.

Matched computational budgets. Our experimental protocol controls for training exposure to isolate the effectiveness of different training methods. Specifically, all methods share the same training exposure: the same prompts, 16 rollouts per prompt, batch size, and 60 policy updates. Matching wall-clock time would generally result in different numbers of rollouts or policy updates due to method-specific auxiliary computations, making it harder to isolate the effects of the training methods from differences in training exposure. We therefore report the measured wall-clock training times and annotation API costs separately. On Qwen2.5-Math-7B, RECAP requires 8.5 hours, compared with 5 hours for GRPO and 10 hours for ThoughtFold on the same 8×H100 configuration. Notably, RECAP requires less training time than ThoughtFold while achieving higher pass@1 and fewer inference tokens across all four benchmarks, showing that its advantage over ThoughtFold cannot be explained by a larger training-time budget.

  • Training time. The training-time breakdown is presented in Table R1, as also reported in Appendix A.4.3.

    Table R1. Training time (8×H100, 60 training steps). | Model | Method | Total | Total Time / training step | Time of step efficacy computation / training step | Time of dependency annotation / training step | |---|---|---|---|---|---| | Qwen2.5-Math-7B | GRPO | 5.0h | ~300s | – | – | | | RECAP | 8.5h | ~510s | ~149s | ~61s | | DeepSeek-R1-Distill-Qwen-7B | GRPO | 15.2h | ~912s | – | – | | | RECAP | 22.0h | ~1320s | ~250s | ~158s |

  • API costs. Table R2 summarizes the measured API token usage and dependency annotation costs with Haiku 4.5 for a single training run.

    Table R2. Annotation API usage per training run (Claude Haiku 4.5; 2048 annotation calls per step). | Model | Input tokens | Output tokens | API cost | |---|---:|---:|---:| | Qwen2.5-Math-7B | 328M | 40.6M | $531 | | DeepSeek-R1-Distill-Qwen-7B | 816M | 126M | $1447 |

AuthorsOpenReview ↗

Rebuttal by authors (Part 2)

W1 & Q1. (b) Dependency on a strong external annotator

RECAP benefits from higher-quality annotation, but its main accuracy gains can be largely preserved using the more reliably identified support dependencies, and it can also operate with an open-weight annotator. Our experimental results further support this point.

  • RECAP's main accuracy gains can be preserved through the more reliably identified support dependencies, without requiring equally accurate annotation of all three relation types. Table R3 for Q2 below shows that using support edges alone stays within 0.7 pass@1 points of Full RECAP, and fully swapping context and restatement still outperforms GRPO on all four benchmarks. This suggests that reliable identification of the main support backbone is particularly important.
  • Fine-grained dependency annotation is inherently ambiguous and challenging, but imperfect context and restatement labels do not necessarily undermine RECAP's effectiveness. Even the three independent judges exhibit only 75–85% pairwise agreement (Appendix A.3). Despite Haiku 4.5's relatively modest context and restatement F1 scores (35.4 and 51.3), our new ablations in Table R3 show that RECAP tolerates even complete confusion between these two relation types.
  • RECAP remains effective with a weaker open-weight annotator, although higher annotation quality yields better performance. Under the same hyperparameters (Appendix Table 5), replacing Haiku 4.5 with Qwen3-32B reduces pass@1 by 1.2–4.0 percentage points and increases response length by approximately 1%–9%. This decline is moderate given its much lower annotation agreement (41.9% vs. 77.0%), and is consistent with its weaker support-edge annotation (F1 62.8 vs. 83.0).
  • Practicality. Annotation runs only during training, accounts for approximately 12% of training wall-clock time and costs approximately \531 per run on Qwen2.5-Math-7B and \1447 on DeepSeek-R1-Distill-Qwen-7B with Haiku 4.5. An open-weight annotator can replace it at a moderate accuracy cost.

Q2 Sensitivity to dependency annotation errors and contributions of different edge types

We thank the reviewer for raising this question. Our results show that semantically aligned structural credit assignment is important, while RECAP remains robust to confusion between context and restatement edges.

Existing evidence (Table 2). Shuffling structural responsibility while preserving its score distribution reduces pass@1 by 2.80 pp, while random responsibility causes a 3.45-point drop, demonstrating the importance of semantically aligned credit assignment.

Additional edge-type ablations (Table R3). We ablate the default propagation weights (1/0.5/0) for support (essential), context (complementary), and restatement (redundant) edges on Qwen2.5-Math-7B to assess their contributions and sensitivity to misclassification. Three key findings emerge:

  • Support edges preserve most accuracy gains, which represent the main reasoning dependencies. Using only support edges reduces accuracy by at most 0.7 percentage points across all four benchmarks. Support edges are also the most reliably annotated (Appendix Table 4). Context edges provide complementary benefits, including a 22.9% reduction in token usage on AIME25 at unchanged accuracy. This efficiency gap is partly driven by support-only responses reaching the generation-length cap. Excluding these rollouts, the average length is 1099 tokens, close to Full RECAP (1098). This suggests that context edges may help limit excessive exploration on challenging problems.
  • Context–restatement distinction is beneficial, but confusion is tolerable. Although swapping these edge types reduces accuracy relative to Full RECAP, the swapped variant still outperforms GRPO in both accuracy and token efficiency across all four benchmarks.
  • Distinguishing support from weaker relations matters. Treating all edges equally reduces accuracy by 7.0 and 4.3 percentage points on GSM8K and MATH-500, respectively, while increasing token usage across all four benchmarks.

Table R3. Ablation on dependency-edge types in structural responsibility on Qwen2.5-Math-7B. Each entry reports pass@1 (%) / average generated tokens.

Method (S/C/R)GSM8KMATH-500AIME24AIME25
GRPO84.4 / 35472.2 / 64124.2 / 15298.8 / 1405
Full RECAP (1/0.5/0)87.7 / 24674.2 / 52426.7 / 140112.5 / 1098
Support-only (1/0/0)87.1 / 22873.5 / 51426.2 / 134012.5 / 1424
C↔R Swapped (1/0/0.5)86.7 / 23674.0 / 54725.0 / 14809.2 / 1149
Uniform (1/1/1)80.7 / 40369.9 / 70421.2 / 163711.2 / 1575

Overall, RECAP does not require a perfect distinction between context and restatement, but relies on separating support dependencies from weaker or redundant relations, which is also where annotation is most reliable (Appendix Table 4).

AuthorsOpenReview ↗

Rebuttal by authors (Part 3)

W2 & Q3. Robustness and generalization

Our additional experiments show that RECAP's accuracy–efficiency gains extend beyond mathematics. Moreover, RECAP is not dominated by any baseline on the four DeepSeek benchmarks, and its gains over GRPO hold across three training seeds.

(a) Cross-domain generalization

RECAP's accuracy–efficiency gains transfer from mathematical to scientific reasoning without any further training or domain-specific adaptation. We evaluated math-trained DeepSeek-R1-Distill-Qwen-7B models on GPQA-Diamond, a challenging scientific reasoning benchmark with 198 questions covering physics, chemistry, and biology (Table R4). RECAP achieves the highest mean pass@1 (47.9%) and the fewest generated tokens (3277) among all evaluated methods. Compared with GRPO, it improves accuracy by 5.0 percentage points while reducing tokens by 24.3%. It also achieves comparable accuracy to ThoughtFold with 25.2% fewer tokens. The new results also strengthen the evidence for RECAP's effectiveness on DeepSeek-R1-Distill-Qwen-7B.

Table R4. Evaluation on GPQA-Diamond using DeepSeek-R1-Distill-Qwen-7B.

Methodpass@1 ↑ (mean ± std)Avg. Tokens ↓ (mean ± std)
Base36.0 ± 2.45841 ± 97
Prompt-guided38.8 ± 2.55516 ± 71
GRPO42.9 ± 1.94330 ± 113
SimpleLength39.4 ± 1.25359 ± 90
DDCA46.1 ± 2.84412 ± 74
LEASH45.8 ± 2.34491 ± 137
Bingo41.3 ± 1.55210 ± 47
ThoughtFold47.5 ± 3.14380 ± 104
RECAP (ours)47.9 ± 2.03277 ± 53

(b) Consistency on DeepSeek-R1-Distill-Qwen-7B

We agree that RECAP is not individually best in accuracy or in token count on every DeepSeek benchmark. However, on all four DeepSeek benchmarks, no evaluated baseline achieves both higher pass@1 and fewer tokens than RECAP: GRPO is shorter but 2.0–5.4 points less accurate, while every baseline with equal or higher pass@1 uses 9.7%–36.2% more tokens.

(c) Robustness across training seeds

RECAP's accuracy–efficiency improvements remain consistent across independent training runs. To assess training-run robustness, we conducted three independent training seeds on Qwen2.5-Math-7B (Table R5). RECAP achieves higher mean pass@1 (+1.4–3.9 percentage points) and fewer generated tokens (8.0–25.1%) than GRPO across all four benchmarks. Notably, RECAP also exhibits lower cross-seed variability in pass@1 than GRPO on both AIME benchmarks, providing further evidence that the improvements are not specific to a single training seed.

Table R5. Robustness of RECAP across 3 training seeds (Qwen2.5-Math-7B). Values are mean ± standard deviation across three independent training seeds. Pass@1 is reported in %.

MethodGSM8KMATH-500AIME24AIME25
pass@1 ↑tokens ↓pass@1 ↑tokens ↓pass@1 ↑tokens ↓pass@1 ↑tokens ↓
GRPO83.5 ± 1.4374 ± 1871.3 ± 2.0678 ± 3222.6 ± 1.31579 ± 5310.3 ± 2.01456 ± 152
RECAP86.2 ± 1.3280 ± 2974.2 ± 0.3532 ± 926.5 ± 0.61452 ± 33111.7 ± 0.81135 ± 137

(d) Model scale

We agree that evaluating additional model scales would further strengthen our conclusions. We focus on two 7B models to compare non-thinking and long-thinking reasoning regimes while keeping the underlying model family comparable (DeepSeek-R1-Distill-Qwen-7B is distilled from Qwen2.5-Math-7B). Extending RECAP to larger model scales remains future work.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Responses to Reviewer Concerns

We thank the reviewers for their feedback. We would like to summarize RECAP's contributions and how our rebuttal addresses their concerns.

Core contribution. RECAP introduces semantic dependency-aware step-level credit assignment to reduce redundant reasoning and improve the inference-time accuracy–efficiency trade-off.

Addressing all concerns raised by Reviewer 6EFk and overlapping concerns from Reviewer ovUt:

  • Training overhead and computational fairness (Reviewer 6EFk W1/Q1; Reviewer ovUt W1) : We clarify that RECAP targets the inference-time accuracy–efficiency trade-off, with additional computation confined to training and no inference-time overhead. We report GPU/API costs (Table R1–R2). Matching training exposure rather than wall-clock time better isolates method effectiveness. RECAP also outperforms ThoughtFold across all four benchmarks on Qwen2.5-Math-7B with less training time (8.5 vs. 10h).
  • Dependence on a strong annotator and annotation errors (Reviewer 6EFk W1/Q2; Reviewer ovUt W1): We demonstrate that RECAP does not inherently depend on a strong proprietary annotator or precise annotation of all relation types. It remains effective with an open-weight annotator model, support-only edges preserve nearly all accuracy gains, and context/restatement confusion is tolerated (Table R3; Appendix Table 5).
  • Robustness and generalization (Reviewer 6EFk W2/Q3; Reviewer ovUt W4): Three training seeds confirm consistent accuracy–efficiency gains over GRPO across all four benchmarks (Table R5). On DeepSeek-R1-Distill-Qwen-7B, no baseline dominates RECAP in both accuracy and tokens on any benchmark, while results on scientific reasoning benchmark GPQA further demonstrates cross-domain transfer with higher accuracy and fewer tokens than all baselines, without retraining (Table R4). The two related 7B backbones also provide a controlled comparison across distinct reasoning regimes. We further clarify RECAP's applicability to broader reasoning tasks: Structural Responsibility does not require canonical answers, while the current Step Efficacy formulation is suited to tasks with short reference answers and can be adapted for open-ended tasks.

Reviewer ovUt

  • (W2) Hyperparameter complexity: Only three hyperparameters require validation tuning, while the rest are fixed by design or set by statistical rules. RECAP's gains do not depend on precise model-specific tuning: a configuration not tuned for Qwen2.5-Math-7B already matches or outperforms GRPO in both accuracy and token efficiency across all four benchmarks (Table R6; Appendix Table 5).
  • (W3) Interpretation of the accuracy–efficiency trade-off on DeepSeek-R1-Distill-Qwen-7B: We clarify that GRPO's limitation is its inability to distinguish useful from redundant reasoning, which does not imply that GRPO necessarily produces longer outputs. Shorter outputs may also remove useful deductions and hurt accuracy. GRPO uses fewer tokens but achieves 2.0–5.4 pp lower accuracy than RECAP on DeepSeek-R1-Distill-Qwen-7B. Our goal is to improve the inference-time accuracy–efficiency trade-off, not simply minimize output length.
  • (W5) Policy models choice: The related 7B models enable a controlled comparison between non-thinking and long-thinking regimes, while both predate AIME 2025, reducing contamination concerns.
  • (W6) Evaluation length and comparability: All methods use the same 8192-token evaluation limit, ensuring fair within-study comparisons.

Reviewer nBR9

  • (W1/Q1) Reliability of step-level credit assignment: Counterfactual experiments on 6393 reasoning steps show that Structural Responsibility identifies steps with greater downstream impact, while Step Efficacy further distinguishes their contributions to answer accuracy, providing independent evidence for the validity of both credit signals (Tables R8–R12).
  • (W2/Q2) Whether Step Efficacy is necessary and effective: Ablations on both backbones show additional accuracy and token-efficiency gains, with particularly strong improvements on DeepSeek (+1.20 pp pass@1, −18.8% tokens; Table R13), while counterfactual results confirm its complementary information (Table R10).
  • (W3/Q3) Whether fewer dead ends merely reflect shorter trajectories: After matching problem, correctness, token count, and step count, RECAP still achieves a significantly lower dead-end token ratio than GRPO (−4.5 pp; Tables R14–R15), demonstrating that the reduction in dead-end reasoning cannot be explained by shorter trajectories alone.

In the follow-up discussion, the reviewers did not identify any specific remaining concerns or flaws in the new evidence. We believe our additional experiments and clarifications directly address the concerns raised and substantially strengthen the evidence supporting RECAP's central claims. We respectfully ask the Area Chair to consider the full rebuttal in the final assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.