- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes DeCoR, an efficient reasoning framework that separates exploration and termination using a base model and a pruned auxiliary model. It combines preference-guided layer pruning with uncertainty- and consistency-based gating to determine when to continue reasoning, correct the trajectory, or terminate. Experiments on three 8B reasoning models and five benchmarks show reduced output length and improved accuracy.
Critical strengths
- The shared-parameter and KV-cache implementation introduces almost no additional memory overhead. The analysis of cache reuse, auxiliary-model performance, and inference latency is also useful from a systems perspective.
Critical weaknesses
-
Insufficient justification/validation of the core assumptions. DeCoR assumes that high answer entropy indicates insufficient reasoning and that low-entropy, consistent answers indicate reliable convergence. However, the paper does not analyze correct-but-high-entropy predictions on challenging problems or incorrect-but-low-entropy predictions arising from model overconfidence. It also does not demonstrate that the pruned auxiliary model performs more semantically diverse or effective exploration rather than simply continuing the same reasoning trajectory.
-
Method complexity relative to the reported gains. DeCoR requires per-model offline calibration, a model-specific pruning mask, multiple gating mechanisms, and several hyperparameters. Given this added implementation complexity, the reported accuracy gains are relatively modest, and their reliability is unclear because no standard deviations or confidence intervals are provided. Moreover, DeCoR increases total FLOPs despite reducing output tokens and latency.
Questions for authors
- DeCoR uses a single fixed pruning mask per base model across all tasks. Is the selected pruning mask stable across different calibration subsets or random seeds, and do task-specific masks result in materially different performance?
3 comments
Q2: Method complexity relative to reported gains. DeCoR requires per-model offline calibration, a model-specific pruning mask, multiple gating mechanisms, and several hyperparameters. Given this added implementation complexity, the reported accuracy gains are relatively modest, and their reliability is unclear because no standard deviations or confidence intervals are provided. Moreover, DeCoR increases total FLOPs despite reducing output tokens and latency.
A2: Many thanks for this comment. First, DeCoR requires only a one-time calibration for each base model on 400 samples, with an average cost of approximately 5 A100 GPU-hours. The resulting pruning mask is reused across all five benchmarks without additional training or task-specific recalibration. Moreover, the auxiliary pathway is implemented through in-place layer masking with shared parameters and a shared KV cache, introducing at most 0.02 GB of additional peak memory (Appendix L). Regarding gating complexity, DeCoR employs two complementary indicators based on simple uncertainty and consistency checks, with a unified hyperparameter configuration across all three base models and five benchmarks. Appendix D further validates parameter robustness. Thus, while DeCoR introduces additional calibration and inference control, it avoids repeated task-specific optimization and the overhead of maintaining two independently loaded models. Please refer to the response to reviewer 15U3's Q2 for further analysis of pruning generalizability.
To address the concern about statistical reliability, we additionally report 95% confidence intervals for accuracy across eight independent runs. As shown below, DeCoR achieves higher mean accuracy than Vanilla across all three base models and five benchmarks, with non-overlapping confidence intervals in 10 of the 15 comparisons. The remaining overlaps occur on AIME26 across all three base models, as well as on MATH500 and GPQA-D for Ministral-3-8B, indicating greater statistical uncertainty in these comparisons. These results provide further evidence for the reliability of accuracy improvements achieved by DeCoR across different models and benchmarks. DeCoR should be evaluated in terms of the overall trade-off between accuracy and efficiency, as it consistently improves accuracy while substantially reducing reasoning tokens and inference latency across diverse models and benchmarks (see Table 2).
| Base Model | Method | GSM8K | MATH500 | AIME26 | GPQA-D | BBEH |
|---|---|---|---|---|---|---|
| R1-Distill-Llama-8B | Vanilla | 81.5 [81.2, 81.8] | 75.8 [74.4, 77.2] | 42.1 [38.7, 45.5] | 44.5 [43.4, 45.6] | 15.2 [15.0, 15.4] |
| DeCoR | 85.0 [83.9, 86.1] | 82.3 [79.6, 85.0] | 46.1 [44.3, 47.9] | 48.3 [46.4, 50.2] | 19.1 [18.3, 19.9] | |
| Qwen3-8B | Vanilla | 84.1 [82.9, 85.3] | 79.5 [79.0, 80.0] | 45.2 [43.3, 47.1] | 46.1 [43.7, 48.5] | 16.6 [15.7, 17.5] |
| DeCoR | 88.2 [85.9, 90.5] | 82.3 [81.0, 83.6] | 49.1 [45.9, 52.3] | 49.7 [48.6, 50.8] | 20.8 [19.7, 21.9] | |
| Ministral-3-8B | Vanilla | 91.2 [90.5, 91.9] | 84.1 [82.2, 86.0] | 41.2 [38.4, 44.0] | 50.2 [48.6, 51.8] | 18.9 [18.4, 19.4] |
| DeCoR | 94.0 [92.4, 95.6] | 87.2 [85.2, 89.2] | 45.6 [43.3, 47.9] | 53.0 [50.9, 55.1] | 24.3 [22.9, 25.7] |
We acknowledge that DeCoR incurs additional FLOPs, while total FLOPs and wall-clock latency measure different aspects of computational efficiency. Conceptually, DeCoR resembles test-time scaling: rather than extending the reasoning chain, it allocates additional computation to intermediate answer assessment and selective correction, enabling more effective reasoning within a reduced token budget. By leveraging the pruned auxiliary model for exploration and reducing redundant reasoning through dual indicator gating, DeCoR consistently achieves lower output token counts and inference latency despite the additional arithmetic operations. Our latency measurements include intermediate answer generation, corrective interventions, and context prefilling overhead. Furthermore, Appendix K (Table 18) shows that shared KV caching achieves an 88.5%--91.6% reuse rate and reduces FLOPs by 76.2%--78.3% over the unshared variant. Thus, DeCoR improves practical reasoning efficiency and accuracy while accepting a moderate increase in total FLOPs.
We have incorporated confidence interval statistics into Appendix C.4.
Q3: DeCoR uses a single fixed pruning mask per base model across all tasks. Is the selected pruning mask stable across different calibration subsets or random seeds, and do task-specific masks result in materially different performance?
A3: Thank you for this valuable question. We conduct an additional calibration robustness analysis, detailed in our response to Reviewer 15U3's Q2. The results demonstrate that pruning masks are relatively stable across calibration subsets, while task-specific and out-of-domain calibration yield comparable performance, supporting the robustness and cross-task generalizability of DeCoR.
Dear Reviewer 6BDr,
We hope this message finds you well. As the discussion period is nearing its end, we wanted to ensure that we have addressed all of your concerns satisfactorily. If there is any additional feedback or points you would like us to consider, please let us know. Your insights are invaluable to us, and we are eager to resolve any remaining issues to improve our work.
Thank you for your time and effort in reviewing our paper.
We sincerely thank the reviewer for the professional comments, as well as the positive recognition of our work. We hope you might reconsider the rating in light of our clarifications. Detailed responses are provided point by point as follows:
Q1: Insufficient justification and empirical validation of core assumptions. DeCoR assumes that high answer entropy indicates insufficient reasoning and that low-entropy, consistent answers indicate reliable convergence. However, the paper analyzes neither correct yet high-entropy predictions on challenging problems nor incorrect yet low-entropy predictions arising from model overconfidence. It also does not demonstrate that the pruned auxiliary model performs more semantically diverse or effective exploration, rather than simply continuing the same reasoning trajectory.
A1: Many thanks for this insightful comment. To further investigate potential factors that may compromise final reasoning quality and accuracy, we quantify the proportion of challenging gating cases encountered by DeCoR across all datasets. We analyze five distinct cases: (1) Low-U Wrong: the proportion of incorrect answers among low-entropy intermediate answers. (2) Wrong Acceptance: the proportion of completed storage windows with an incorrect latest answer that still satisfy the consistency criterion and trigger termination. (3) Wrong-to-Correct: the proportion of trajectories containing low-entropy incorrect answers that ultimately achieve correct final answers. (4) High-U Correct: the proportion of correct answers among high-entropy intermediate answers. (5) Final Correct: the proportion of trajectories containing high-entropy correct answers that ultimately terminate with correct answers.
The results show that predictive uncertainty is informative but imperfect, as both low-uncertainty errors and high-uncertainty correct answers occur. Importantly, the consistency gate filters out most incorrect candidates within completed low-uncertainty windows, while continued reasoning allows many trajectories with incorrect intermediate answers to reach the correct final answer. These findings support the value of combining uncertainty and consistency for more reliable reasoning control.
Our existing results already support the auxiliary model's semantic diversity and efficacy during reasoning. Figure 4 shows its higher-uncertainty exploration patterns, while Figure 7 demonstrates the gains of preference-guided pruning over ShortGPT and the unpruned variant. Moreover, as discussed for reviewer ieBf (Q2, Q4), our analysis of uncertainty and intermediate answer dynamics further supports DeCoR's collaborative reasoning. Nevertheless, we conduct an additional experiment to examine the auxiliary model's semantic dynamics. Specifically, from shared prefixes in DeCoR trajectories, the pruned auxiliary model and full base model independently generate segments under identical decoding settings and token budgets without gating. The same base model then generates intermediate answers before and after each continuation, isolating trajectory differences from evaluation. We define three metrics: (1) exploration gain (EG), the average increase in correct intermediate answer proportion before and after continuation, (2) answer discovery rate (ADR, %), calculated as the proportion of prefixes with no correct answers before exploration that yield at least one correct answer afterward, and (3) semantic exploration novelty (SEN, %), the manually audited proportion of continuations introducing novel and useful reasoning directions relative to their prefixes. We sample 120 problems across MATH500, AIME26, GPQA-D, and BBEH (30 each), extracting the first 15 segments of each DeCoR trajectory as shared prefixes. For each prefix, both models independently generate four 512-token continuations, followed by four intermediate answer generations with the base model. Results are shown below.
It is clear that the auxiliary model achieves EG values comparable to those of the base model, while consistently improving ADR and SEN. This indicates that the auxiliary model not only maintains effective reasoning progress but also more frequently discovers correct answers and introduces novel reasoning directions, rather than merely extending existing reasoning trajectories.
We have incorporated these results into Appendix H and Appendix J.