acceptodds
Under review as a conference paper at ICLR 2027

Trust-Region On-Policy Sparse Optimization for Pruning Reasoning Language Models

Abstract

Reasoning language models incur high inference costs because they generate long chains of thought before producing an answer, making model compression techniques such as pruning increasingly important. However, current pruning approaches typically cause severe performance degradation at high sparsity, leaving effective reasoning model pruning an open problem. In this work, we identify a key challenge in pruning reasoning models: pruning alters the prefixes—the prompts and partially generated sequences—encountered during generation, causing the fixed, off-policy calibration data to become increasingly mismatched with the evolving sparse model. To tackle this challenge, we introduce SCOUT, a novel sparse optimization framework that alleviates the distributional mismatch by leveraging on-policy data from the evolving sparse model, effectively curbing reasoning performance drop. At its core, SCOUT dynamically refines the sparsity mask through iterative pruning and reviving, while constraining each update within a KL divergence-based trust region to bound policy drift. We further establish a stationarity bound for an idealized form of this alternating procedure, where the trust-region budget and rollout staleness govern the resulting convergence neighborhood. Empirically, we showed that SCOUT substantially outperforms existing pruning methods at high sparsity across different Qwen3 models in various scales.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
LLM pruning, model compression, reasoning language models, sparse optimization, on-policy learning, trust-region optimization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Hz6hOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an iterative pruning method for reasoning language models. The main idea is to use trajectories generated by the evolving sparse model to guide optimization, rather than relying entirely on fixed calibration data. SCOUT alternates between weight updates and mask updates. The mask is determined using Adam-based saliency scores, while a KL-based trust region controls how aggressively the mask can change. The authors evaluate SCOUT on Qwen3 models from 1.7B to 8B parameters at 50–80% unstructured sparsity. The results show substantial improvements over existing methods at high sparsity. The paper also provides a theoretical analysis of an idealized optimization procedure and reports decoding speedups using sparse kernels.

Critical strengths

Strong results at high sparsity. The performance improvements are substantial, particularly at 70–80% sparsity. For example, at 80% sparsity, SCOUT improves the four-task average from 18.6% to 33.0% on Qwen3-4B and from 30.4% to 41.2% on Qwen3-8B compared with ALPS+training. The inclusion of post-pruning trained baselines makes these results more convincing than comparisons with one-shot methods alone.

An interesting approach with useful empirical analysis. Combining on-policy distillation with adaptive mask refinement is a reasonable way to address the distribution shift caused by pruning. I also appreciate the additional analysis of the sparse model's own rollouts and the decoding experiments. These provide useful evidence beyond downstream accuracy alone.

Critical weaknesses
  1. The contribution of the KL-based trust region could be better isolated: According to Appendix A, the "w/o TR" variant directly applies the target mask at the first projection event, whereas SCOUT gradually increases sparsity. This comparison therefore changes the sparsification trajectory as well as the use of the trust-region constraint. Since gradual pruning with continued training is already known to be effective, it is not entirely clear how much of the improvement comes specifically from KL-based mask selection. A comparison with a gradual pruning baseline using a predefined sparsity schedule and comparable training settings would make the contribution more convincing.

  2. The connection between the trust-region formulation and its implementation could be clarified more: Equation (2) constrains the divergence between consecutive sparse models, accounting for changes in both weights and masks. However, the empirical KL constraint in Equation (13) is applied to the mask transition with the updated weights held fixed. The weight updates themselves are not explicitly checked against the same constraint. The KL screening also uses only four sequences of 512 tokens, sampled from a rollout cache that is refreshed periodically. Although Appendix C.4 provides additional analysis on longer rollouts, it remains unclear how sensitive the mask selection is to the screening data and its staleness. These issues do not invalidate the empirical results, but they limit how directly the theoretical motivation supports the implemented procedure.

Questions for authors
  1. Could the authors compare SCOUT against gradual pruning with a predefined sparsity schedule, using the same training objective and a comparable computational budget? This would help isolate the benefit of KL-based mask selection.

  2. How sensitive is the method to the size and length of the KL screening set, as well as the rollout refresh interval? Also, how much additional KL drift is introduced by the weight updates between mask projections?

  3. In the "w/o on-policy" ablation, the authors remove the OPD objective and replace the rollout-based KL screening data with fixed-corpus data. Could these two effects be evaluated separately? It would also be interesting to understand the contribution of mask revival after the target sparsity has been reached.

3 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We appreciate the reviewer’s careful assessment of our work and the positive evaluation of our high-sparsity results and empirical analyses. We understand the concerns regarding the contribution of the KL trust region, its implementation, and the role of on-policy information. To address these concerns, we conducted additional controlled experiments.

Following Reviewer Q4RB's feedback, we extended the on-policy training rollout length from 512 to 2,048 tokens to better reflect the long-generation setting of reasoning models. Unless otherwise specified, the following experiments use Qwen3-1.7B at 70% sparsity, with 2,048 optimization steps, the NTP+KD+OPD objective, and mask updates every 8 steps. We report Avg4, the average accuracy across MATH-500, GSM8K, LiveCodeBench, and IFEval, and describe the specific changes made in each ablation below. For the default SCOUT configuration, we report the mean over three training seeds (38.63%); all other variants and baselines use a single training seed unless otherwise noted.

Comparison with predefined gradual pruning schedules

Could the authors compare SCOUT against gradual pruning with a predefined sparsity schedule, using the same training objective and a comparable computational budget?

To distinguish the contribution of KL-based mask updates from conventional gradual pruning, we compared SCOUT with cubic and linear sparsity schedules. Both schedules were configured to reach the target sparsity at step 200, matching SCOUT. We report results across three training seeds.

Comparison of sparsity schedules (Qwen3-1.7B, 70% sparsity, Avg4, %)

ScheduleSteps to 70%Avg4 (3 seeds)Max KL per update (3 seeds)Updates with KL > 0.02
SCOUT (δ=0.02\delta=0.02)200 / 200 / 20038.73 / 38.66 / 38.50.020 / 0.020 / 0.0200 / 0 / 0
Cubic200 / 200 / 20037.83 / 38.02 / 37.750.039 / 0.035 / 0.04112 / 9 / 11
Linear200 / 200 / 20036.84 / 37.53 / 37.650.068 / 0.078 / 0.0679 / 9 / 8

SCOUT achieves the highest accuracy across all three seeds, although cubic pruning remains competitive. Although cubic pruning exceeds the KL threshold slightly more often, it induces substantially smaller maximum KL divergence than linear pruning, consistent with our motivation to limit abrupt distributional changes during pruning.

However, both predefined schedules produce mask updates that exceed the KL threshold. While such schedules implicitly control distributional changes through their pruning rates, SCOUT explicitly measures and constrains KL divergence at each mask update. Although SCOUT still requires choosing δ\delta, this parameter directly specifies the allowable distributional change rather than relying on a predefined sparsity trajectory. We will incorporate these comparisons into the revised paper.

AuthorsOpenReview ↗

Rebuttal by Authors

Sensitivity of KL screening and KL drift from weight updates

How sensitive is the method to the size and length of the KL screening set, as well as the rollout refresh interval? Also, how much additional KL drift is introduced by the weight updates between mask projections? The connection between the trust-region formulation and its implementation could be clarified more

We examined the sensitivity of SCOUT to screening set size, sequence length, and rollout refresh interval. We also measured the KL drift induced by weight updates between mask projections.

1. Sensitivity to screening set size and sequence length

Along with extending the training rollout length to 2,048 tokens, we increased the default KL screening sequence length from 512 to 2,048 tokens. We then varied the screening set size and sequence length to evaluate their effects.

Screening set size (Qwen3-1.7B, 70% sparsity, Avg4, %)

Screening sequences4 (default)816
Avg438.6338.5138.52

Screening sequence length (Qwen3-1.7B, 70% sparsity, Avg4, %)

Sequence length2565122,048 (default)
Avg438.7038.8638.63

Performance varies only slightly across the evaluated screening set sizes and sequence lengths, suggesting that SCOUT is relatively insensitive to these choices within the tested ranges.

2. Sensitivity to rollout refresh interval

Rollout refresh interval (Qwen3-1.7B, 70% sparsity, Avg4, %)

Refresh interval (steps)832 (default)1282,048 (no refresh)
Avg439.0238.6337.5434.93

Unlike screening set size and length, rollout freshness has a noticeable effect. Accuracy decreases from 39.0% with an 8-step refresh interval to 34.9% without intermediate refreshes, highlighting the importance of periodically updating on-policy information.

3. KL drift from weight updates

SCOUT explicitly screens mask transitions, assuming that individual continuous weight updates induce relatively small distributional changes. To examine this assumption, we measured KL divergence after individual weight updates and cumulatively over the eight updates between mask projections.

KL drift from weight updates (Qwen3-1.7B, 70% sparsity)

MeasurementMean KLMedian KL95th percentileMax KL
Single weight update0.00140.00110.00290.0053
Between mask projections (8 steps)0.0150.00760.0470.086

Individual weight updates induce small KL divergence (mean 0.0014, maximum 0.0053), below the threshold of δ=0.02\delta=0.02. Although the accumulated KL over eight updates can be larger (up to 0.086), gradients are recomputed at the current weights after each step, allowing the optimization direction to adapt as the model changes. Thus, a larger cumulative KL does not necessarily violate the locality assumption underlying individual weight updates.

This also helps clarify the relationship between Eq. (2) and Eq. (13). Eq. (2) presents an idealized formulation that jointly considers weight and mask updates, whereas Eq. (13) explicitly constrains mask transitions with the updated weights held fixed. Our implementation focuses on controlling these discrete transitions while relying on individual continuous weight updates to remain local, as supported by the measurements above. Importantly, the theoretical analysis in Appendix E is formulated specifically for mask transitions, consistent with Eq. (13).

We acknowledge that Eq. (2) may obscure this distinction, as the implemented constraint does not guarantee that the combined change in weights and masks remains within the same KL threshold. We will revise the manuscript to clarify the relationship between the idealized formulation, the implemented constraint, and the scope of the theoretical analysis.

AuthorsOpenReview ↗

Rebuttal by Authors

Separating the effects of OPD and KL screening data

In the "w/o on-policy" ablation, the authors remove the OPD objective and replace the rollout-based KL screening data with fixed-corpus data. Could these two effects be evaluated separately?

We agree that our original "w/o on-policy" ablation simultaneously removed OPD and replaced on-policy KL screening data with a fixed corpus, making their individual contributions difficult to distinguish.

To separate these effects, we conducted additional controlled experiments. First, we replaced on-policy KL screening data with a fixed corpus while retaining OPD. Second, we excluded OPD gradients from saliency estimation while keeping the full NTP+KD+OPD objective for weight optimization.

Contributions of OPD, saliency estimation, and KL screening data (Qwen3-1.7B, 70% sparsity, Avg4, %)

Training objectiveSaliencyKL screening dataAvg4
NTP+KD+OPDFull objectiveOn-policy rollout38.63
NTP+KD+OPDFull objectiveFixed corpus38.33
NTP+KD+OPDExcluding OPDOn-policy rollout37.07
NTP+KDExcluding OPDFixed corpus29.50

Replacing on-policy KL screening data with a fixed corpus has negligible effect on accuracy (38.63% vs. 38.33%). In contrast, excluding OPD gradients from saliency estimation reduces accuracy by 1.56 percentage points (38.63% to 37.07%), despite keeping the training objective unchanged. Removing OPD entirely leads to a larger degradation (38.63% to 29.50%).

These results suggest that on-policy information contributes more to mask selection through saliency estimation than through KL screening data, while removing OPD entirely leads to a substantial degradation in overall performance. We will include these ablations in the revised paper.

Contribution of mask revival after reaching target sparsity

It would also be interesting to understand the contribution of mask revival after the target sparsity has been reached.

SCOUT continues updating the mask after reaching the target sparsity, allowing previously pruned weights to be revived. To evaluate its contribution, we compared SCOUT with a variant that freezes the mask upon reaching 70% sparsity. Both variants follow the same procedure before reaching the target and continue weight optimization under identical conditions.

Effect of mask refinement after reaching target sparsity (Qwen3-1.7B, 70% sparsity, Avg4, %)

MethodSeed 42Seed 1Seed 2Mean
SCOUT (revival enabled)38.7338.6638.5038.63
Fixed mask after reaching 70%37.8237.1337.8437.60

Allowing continued mask refinement improves average accuracy by 1.03 percentage points. We also measured the frequency of revival events after reaching the target sparsity.

Revival events per interval

Steps201–512513–1,0241,025–1,5361,537–2,048Total
Revival events (%)0.0110.0460.0300.0050.092

Percentages are relative to the number of active weights at the target sparsity.

Cumulative revival events account for only 0.092% of active weights, indicating that SCOUT does not rely on extensive mask changes after reaching the target sparsity. Continued mask refinement provides a modest additional benefit rather than being the primary source of SCOUT's performance gains. We will include this analysis in the revised paper.

Reviewer_SWVrOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies the problem of pruning LLMs using a distillation-type loss function. In particular, the authors consider finding a sparse model that minimizes the on-policy KD loss. As this problem is difficult to solve, they propose an alternating optimization framework, that progressively updates the model weight with a distillation loss and selects a sparse mask. The mask selection uses a saliency-based selection within a trust region. The authors provide numerical experiments on several LLMs and benchmarks, and compare with baseline pruning methods.

Critical strengths

The mixing of pruning and distillation is a clever and useful idea.

Critical weaknesses

1- I think the algorithmic design needs to be investigated more carefully. The motivation for using on-policy data is clear, but the resulting method combines several ingredients whose individual contributions are difficult to disentangle. At a high level, SCOUT performs iterative pruning interleaved with weight updates, where the weight-update objective includes on-policy distillation and the amount of pruning at each projection step is controlled by a KL-based trust region.

This raises the question of how much of the improvement comes specifically from the proposed on-policy/trust-region formulation, as opposed to more general benefits of gradual pruning and retraining. For example, the on-policy distillation objective could be replaced by standard SFT or offline distillation; the saliency-based support update could be replaced by an existing pruning criterion such as SparseGPT or ALPS; and the trust region can also be viewed as an adaptive mechanism for controlling the sparsity trajectory. Gradual pruning schedules themselves are well established and have also been studied for LLM pruning, e.g., arXiv:2401.02938.

I would therefore like to see more controlled experiments that separately study the importance of: (i) the on-policy distillation objective, relative to standard SFT/offline KD; (ii) the proposed saliency-based mask update, relative to established pruning criteria; and (iii) the adaptive trust-region sparsity schedule, relative to simple predetermined gradual-pruning schedules. This is particularly important for interpreting the current post-pruning training experiment: the paper applies additional training to a fixed ALPS mask, whereas SCOUT benefits from pruning gradually while training throughout. A stronger comparison would use an iterative ALPS/SparseGPT-style baseline that progressively increases sparsity while retraining or distilling between pruning stages under a comparable training budget.

2- The scalability of SCOUT also needs to be discussed more thoroughly. The method requires 2,048 optimization steps, repeated on-policy rollout generation, teacher-model evaluations for distillation, and frequent mask updates and trust-region screening. It is therefore unclear how practical the method would be for substantially larger models, e.g., models above 30B parameters. Reporting wall-clock time, GPU-hours, peak memory, and the relative cost of rollout generation, teacher inference, and optimization would make the practical tradeoff much clearer. Ideally, scalability should also be demonstrated empirically on a larger model.

3- More of a minor issue, but the authors are not using the standard implementation of ALPS and SparseGPT (which use C4 for calibration). In fact, what they report is closer to RAC, as it partially uses the problem solution given by the OpenThoughts dataset. This distinction is important, because the nominal SparseGPT/ALPS baselines already benefit from task-relevant reasoning trajectories and are therefore closer in spirit to reasoning-aware calibration than the standard implementations of these methods. As currently presented, this detail is easy to miss because the exact calibration construction is deferred to the appendix, and it may make the comparison with RAC appear less favorable than it would under the standard calibration protocol. I would recommend reporting the standard C4-calibrated versions of SparseGPT/ALPS as well, or at minimum making this calibration choice explicit in the main experimental section.

Questions for authors

Please see my 3 points above.

5 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We thank the reviewer for the thorough evaluation of our work. We understand the concerns regarding the individual contributions of SCOUT's components, its scalability to larger models, and the calibration protocol used for baseline comparisons. We agree that these concerns warrant further investigation and have conducted additional experiments to address them.

Following Reviewer Q4RB's feedback on the relatively short rollout length in our original experiments, we extended the on-policy rollout length from 512 to 2,048 tokens. We believe this setting better reflects the long-generation nature of reasoning models and allows us to evaluate SCOUT's central motivation more directly. Unless otherwise specified, the following controlled ablations (reporting Avg4, the average accuracy across MATH-500, GSM8K, LiveCodeBench, and IFEval) use Qwen3-1.7B at 70% sparsity, with 2,048 optimization steps and mask updates every 8 steps. For the default SCOUT configuration, we report the mean over three training seeds (38.63%); all other variants and baselines use a single training seed unless otherwise noted. We present the additional results and clarifications below.

Contribution of on-policy distillation

I would therefore like to see [...] the importance of: (i) the on-policy distillation objective, relative to standard SFT/offline KD

To isolate the contribution of on-policy distillation (OPD), we conducted controlled ablations on Qwen3-1.7B at 70% sparsity. First, to test if OPD's benefit stems merely from additional training data, we replaced OPD with an equal number of sequences (2,048 tokens each) drawn from an unused subset of OpenThoughts3 [1] (offline KD). Second, to test if on-policy information aids mask selection, we computed the saliency score using gradients that excluded the OPD term, while keeping the full NTP+KD+OPD objective for weight optimization.

Contribution of on-policy distillation (Qwen3-1.7B, 70% sparsity, Avg4, %)

ConfigurationAvg4
SCOUT (NTP+KD+OPD)38.63
NTP+KD + additional offline KD31.53
NTP+KD (without OPD)29.50
SCOUT (OPD excluded from saliency)37.07

SCOUT outperforms additional offline KD by 7.1 percentage points, confirming OPD's benefit goes beyond simply providing more distillation data. Moreover, excluding OPD gradients from saliency estimation reduces accuracy by 1.56 points, indicating that on-policy information directly improves mask selection. We will include these ablations in the revised paper.

Comparison of saliency criteria

(ii) the proposed saliency-based mask update, relative to established pruning criteria

To evaluate the proposed saliency criterion against established alternatives, we compared our Adam second-moment-based saliency with magnitude, Wanda [2], and SNIP [3]. All methods used global ranking on Qwen3-1.7B at 70% sparsity, keeping the training objective (NTP+KD+OPD) and optimization schedule identical. We report results for two KL thresholds (δ=0.02\delta=0.02 and 0.050.05).

Comparison of saliency criteria (Global ranking, Qwen3-1.7B, 70% sparsity, Avg4, %)

SaliencySteps to 70% (δ=0.02\delta=0.02)Avg4 (δ=0.02\delta=0.02)Steps to 70% (δ=0.05\delta=0.05)Avg4 (δ=0.05\delta=0.05)
Ours20038.6312036.73
SNIP [3]4644.942325.26

Our criterion achieves the target sparsity quickly and maintains stable reasoning performance. SNIP also reaches 70% sparsity but yields substantially lower accuracy.

Magnitude and Wanda are excluded from the table because they failed to produce a functional sparse model under global ranking. Magnitude pruning caused the training loss to continuously explode before reaching the target sparsity, failing to recover meaningful performance. Wanda disproportionately pruned specific layers, completely collapsing certain projection matrices and failing to reach the target sparsity entirely.

Crucially, SCOUT reuses the Adam second-moment estimates inherently tracked during weight optimization. It imposes virtually zero additional cost for collecting saliency statistics, whereas Wanda requires collecting and storing additional input activation statistics. We will include this comparison in the revised paper.

AuthorsOpenReview ↗

Rebuttal by Authors

Trust-region vs. predetermined gradual-pruning schedules

(iii) the adaptive trust-region sparsity schedule, relative to simple predetermined gradual-pruning schedules

We agree that gradual pruning with predefined sparsity schedules can itself contribute to performance improvements and that our original experiments did not provide a sufficiently direct comparison with such approaches. To address this, we additionally compared SCOUT with cubic and linear sparsity schedules.

Under the default ablation setting, we isolated the effect of the sparsity progression by keeping the saliency criterion and all other hyperparameters identical. For a fair comparison, both predefined schedules were configured to reach the target sparsity at step 200, matching SCOUT. We report the final accuracy and the maximum KL divergence induced by mask updates across three training seeds.

Comparison of sparsity schedules (Qwen3-1.7B, 70% sparsity, Avg4, %)

ScheduleSteps to 70%Avg4 (3 seeds)Max KL per update (3 seeds)Updates with KL > 0.02
SCOUT (δ=0.02\delta=0.02)200 / 200 / 20038.73 / 38.66 / 38.50.020 / 0.020 / 0.0200 / 0 / 0
Cubic200 / 200 / 20037.83 / 38.02 / 37.750.039 / 0.035 / 0.04112 / 9 / 11
Linear200 / 200 / 20036.84 / 37.53 / 37.650.068 / 0.078 / 0.0679 / 9 / 8

SCOUT achieves the highest accuracy across all three seeds. Cubic pruning also achieves competitive performance and consistently outperforms linear pruning. Notably, although cubic pruning exceeds the KL threshold slightly more often, its maximum KL divergence per update is substantially smaller than that of linear pruning, suggesting that the larger distributional shifts induced by the linear schedule may contribute to its lower performance. This observation is consistent with SCOUT's motivation to limit abrupt changes in the model's predictive distribution during pruning.

However, predefined schedules determine how many weights to prune without explicitly considering the resulting distributional change. Consequently, even gradual schedules can produce mask updates that substantially exceed the KL threshold, as shown in the table. In contrast, SCOUT directly measures and constrains the KL divergence, providing a more principled approach to determining mask updates based on their distributional impact rather than relying on a heuristic sparsity trajectory.

Although SCOUT still requires selecting the KL threshold δ\delta, this parameter directly specifies the allowable distributional change, rather than indirectly controlling it through a predefined pruning duration or sparsity schedule. We will include these additional results in the revised paper.

Comparison with iterative ALPS/SparseGPT

A more realistic baseline would be to apply existing pruning methods iteratively, with gradual sparsity increases and retraining between pruning stages.

To address this, we extended ALPS and SparseGPT to a five-stage iterative pruning setting on Qwen3-1.7B at 70% sparsity, using both cubic and linear sparsity schedules (configured to reach the target at step 200, matching SCOUT). All methods used the same NTP+KD+OPD objective and 2,048 total optimization steps.

Comparison with iterative pruning baselines (Qwen3-1.7B, 70% sparsity, Avg4, %)

MethodAvg4Mask selection time (h)Total time (h)
SCOUT38.630.117.0
Iterative ALPS (cubic)34.330.67.8
Iterative ALPS (linear)32.440.67.9
Iterative SparseGPT (cubic)22.750.207.5
Iterative SparseGPT (linear)22.440.197.2

Despite requiring more total runtime than SCOUT (7.8h vs. 7.0h), the strongest iterative baseline (ALPS cubic) still falls 4.3 percentage points short.

AuthorsOpenReview ↗

Rebuttal by Authors

Scalability and computational cost

The scalability of SCOUT also needs to be discussed more thoroughly. Reporting wall-clock time, GPU-hours, peak memory, and the relative cost of rollout generation, teacher inference, and optimization would make the practical tradeoff much clearer. Ideally, scalability should also be demonstrated empirically on a larger model.

To evaluate scalability, we applied SCOUT to Qwen3-32B on 4×B300 GPUs. We reduced the training budget to 512 steps. We also tested SCOUT-lite (256 steps, approximate KL screening).

Results on Qwen3-32B at 70% sparsity (Dense: 84.3%, Avg4, %)

MethodStepsAvg4
ALPS one-shot–58.3
ALPS + training51266.2
SCOUT51273.3
SCOUT-lite25669.3

SCOUT achieves 73.3%, outperforming ALPS+training by 7.1 percentage points. Even the 256-step SCOUT-lite outperforms the 512-step ALPS+training baseline.

Regarding computational trade-offs, we measured wall-clock time, GPU-hours, and peak memory:

Computational cost across model sizes (ALPS vs. SCOUT)

ModelHardwareStepsPruning cost (h)GPU-hoursPeak memory / GPU
1.7B1×GH2002,048ALPS: 0.11 / SCOUT: 0.11ALPS: 6.9 / SCOUT: 7.0ALPS: 47.7G / SCOUT: 47.9G
4B1×B2002,048ALPS: 0.19 / SCOUT: 0.16ALPS: 7.4 / SCOUT: 7.5ALPS: 71.9G / SCOUT: 72.4G
8B1×B2002,048ALPS: 0.32 / SCOUT: 0.37ALPS: 9.4 / SCOUT: 9.6ALPS: 115.0G / SCOUT: 116.0G
32B4×B300512ALPS: 1.94 (1 GPU) / SCOUT: 0.34 (4 GPUs)ALPS: 43.1 / SCOUT: 43.6ALPS: 140.3G / SCOUT: 144.2G

Across all scales, SCOUT requires comparable total training time and GPU-hours to ALPS+training, with small peak memory overhead.

Finally, a runtime breakdown of SCOUT on 32B reveals that student optimization (60.6%), rollout generation (19.8%), and teacher inference (12.6%) consume ~93% of the time. Because ALPS+training uses the same NTP+KD+OPD objective, it incurs these exact same costs. The remaining ~7% includes SCOUT-specific operations (mask updates and KL screening) as well as other overheads common to both methods, such as data loading and checkpointing. We will include these scalability and cost metrics in the revision.

Calibration dataset for SparseGPT and ALPS

The authors are not using the standard implementation of ALPS and SparseGPT (which use C4 for calibration). What they report is closer to RAC, as it partially uses the problem solutions given by OpenThoughts.

We agree that our calibration of SparseGPT and ALPS (80% OpenThoughts3, 20% FineWeb) is closer in spirit to RAC, as it leverages reasoning traces from larger models rather than general C4 text. We originally chose this mixture to ensure the baselines could benefit from task-relevant reasoning data.

To clarify the impact of this choice, we additionally evaluated SparseGPT and ALPS using the standard C4 calibration dataset.

Effect of calibration dataset (50% sparsity, Avg4, %)

Calibration dataset1.7B SparseGPT1.7B ALPS4B SparseGPT4B ALPS
C43.5217.8143.5450.63
OpenThoughts3[1] + FineWeb[4]39.5752.7765.771.08

As shown above, using standard C4 severely degrades pruning performance compared to our reasoning-aware mixture. For instance, ALPS accuracy drops from 52.77% to 17.81% on Qwen3-1.7B and from 71.08% to 50.63% on Qwen3-4B.

We will make this calibration choice explicit in the main experimental section and include the C4-calibrated results to ensure a transparent comparison with standard implementations.

References
[1] Guha et al., OpenThoughts: Data Recipes for Reasoning Models, ICLR 2026.
[2] Sun et al., A Simple and Effective Pruning Approach for Large Language Models, ICLR 2024.
[3] Lee et al., SNIP: Single-Shot Network Pruning based on Connection Sensitivity, ICLR 2019.
[4] Penedo et al., The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale, NeurIPS 2024.

Reviewer_Q4RBOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents SCOUT, an iterative on-policy pruning method for reasoning models. The authors alternate between updating the weights of the model and selecting the pruning mask for the parameters, with each mask update within a KL trust region. The authors also provide theoretical bounds for their method. SCOUT shows better performance compared to pruning baselines on Qwen3 1.7B–8B, especially at high sparsity (70–80%), highlighting the benefits of being on-policy and a modest improvement in inference costs.

Critical strengths
  • The paper is remarkably easy to follow and well written. The notation is clear, without unnecessary over-complications, and the method section is easy to understand in details.
  • The method is simple and interesting and results are positive, especially compared to the baselines, and the pruning/reviving mechanism is quite smart.
Critical weaknesses
  • The experimental setup could have been richer. Would have been better to test with at least another family of models besides Qwen3, to have more evidence for robustness of the method. Also, self-generated answers use at most 512 tokens during training, which is very short by today's standards (and given the paper premise on the special needs due to longer rollouts).
  • In practice, speed-ups are limited given the additional tokens generated with high pruning (and only measured at batch size of 1). Furthermore, while pruning reduces storage for the parameters it does not reduce memory requirements for long-context (and even increases them because of longer generations), which can eventually outweigh parameters memory.
  • It could have been nicer to see more focus on which parameters should be masked from an inference cost perspective, as, especially given the current infrastructure ecosystem, they affect costs differently.
Questions for authors
  • Do you think there are alternative good choices for the saliency score?
  • Did you tune the hyperparameters for the experiments in Figure 3?
  • How many iterations do you use for your method? Do you have a sense of how this changes performance?
  • Are masked parameters frequently revived? How does pruning size evolve throughout training?
  • Can you discuss the relation with alternative research directions to make reasoning chains more efficient, including natural language compression, KV cache compression for reasoning, or even length-aware post-training methods?
4 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We thank the reviewer for the thoughtful and constructive review, and for recognizing the contributions of our work. We address the reviewer’s comments in detail below, and welcome any further discussion.

Unless otherwise specified, the following evaluations and controlled ablations (reporting Avg4, the average accuracy across MATH-500, GSM8K, LiveCodeBench, and IFEval) use Qwen3-1.7B at 70% sparsity with 2,048 optimization steps, 2,048-token rollouts (following the suggestion), and mask updates every 8 steps. For the default SCOUT configuration, we report the mean over three training seeds (38.63%); all other variants and baselines use a single training seed unless otherwise noted.

Additional experiments on different model family

Would have been better to test with at least another family of models besides Qwen3, to have more evidence for robustness of the method.

To verify that SCOUT generalizes beyond Qwen3, we evaluated it on DeepSeek-R1-Distill-Llama-8B (using 2048-token rollouts).

Results on R1-Distill-Llama-8B model (dense: 65.71, Avg4, %)

SparsityALPS+trainingSCOUTGap
50%59.1459.75+0.61
60%51.6155.15+3.54
70%39.6542.33+2.68
80%19.7430.42+10.68

Consistent with our Qwen3 results, SCOUT’s advantage grows with sparsity: the two methods are comparable at 50% sparsity, while SCOUT leads by 2.7–3.5 percentage points at 60–70% and by 10.68 points at 80%. This suggests that SCOUT’s effectiveness is not restricted to the Qwen3 family. We will include these results in the revision.

Additional experiments with longer rollout length

Also, self-generated answers use at most 512 tokens during training, which is very short by today's standards (and given the paper premise on the special needs due to longer rollouts).

We originally used 512 tokens to maximize training efficiency. To address the concern regarding long-horizon reasoning, we repeated our experiments using 2048-token rollouts.

Results at 70% sparsity across model scales (Avg4, %)

ModelALPS+training (512)SCOUT (512)ALPS+training (2048)SCOUT (2048)
Qwen3-1.7B21.5133.6728.3738.63*
Qwen3-4B49.3653.4448.6256.15
Qwen3-8B57.2961.2959.0361.04

* 1.7B SCOUT (2048): mean over 3 training seeds; other entries: single seed.

SCOUT continues to outperform ALPS+training across all scales. Extending the rollout length to 2048 further improves SCOUT's absolute performance on the 1.7B and 4B models, while maintaining comparable performance on the 8B model. We agree that 2048 tokens better reflects the central motivation of our paper, and we will update the main results accordingly.

Practical speedup and memory efficiency

Speed-ups are limited given the additional tokens generated with high pruning (and only measured at batch size of 1). Longer generations [...] can eventually outweigh parameter memory.

Our initial batch-size-1 limitation stemmed from the MACKO [1] kernel. To evaluate batched inference, we conducted additional experiments using SpInfer [2] on an L40S GPU.

Speedup and memory reduction (Qwen3-8B, 60% sparsity)

BatchPer-token speedupMATH-500GSM8KLCBMemory (Dense →\rightarrow SCOUT)
11.65x1.45x1.50x1.24x19.7 →\rightarrow 12.8 GiB (-35%)
81.70x1.49x1.55x1.28x25.0 →\rightarrow 17.7 GiB (-29%)
321.46x1.28x1.33x1.10x28.1 →\rightarrow 25.1 GiB (-11%)

Even after accounting for the increase in generated tokens, SCOUT achieves effective sequence-level speedups and net memory reductions (where weight memory savings offset KV cache increases) across all batch sizes. We will add these SpInfer benchmarks to the revised version.

AuthorsOpenReview ↗

Rebuttal by Authors

Hardware-aware parameter selection for inference efficiency

It could have been nicer to see more focus on which parameters should be masked from an inference cost perspective, as, especially given the current infrastructure ecosystem, they affect costs differently.

We agree that hardware-aware sparsity allocation is a promising direction. Sparse kernels have specific constraints: MACKO compresses only layers with ≤0.5\le 0.5 density, while SpInfer yields diminishing latency returns in already highly sparse layers at large batch sizes. SCOUT's framework is flexible enough to incorporate these kernel-specific density constraints into its saliency-based mask selection while retaining the KL trust-region bound. We will discuss this in future work.

Alternative choices for saliency score

Do you think there are alternative good choices for the saliency score?

To evaluate the proposed saliency criterion against established alternatives, we compared our Adam second-moment-based saliency with magnitude, Wanda [3] and SNIP [4]. All methods used global ranking on Qwen3-1.7B at 70% sparsity, keeping the training objective (NTP+KD+OPD) and optimization schedule identical. We report results for two KL thresholds (δ=0.02\delta=0.02 and 0.050.05).

Comparison of saliency criteria (Global ranking, Qwen3-1.7B, 70% sparsity, Avg4, %)

SaliencySteps to 70% (δ=0.02\delta=0.02)Avg4 (δ=0.02\delta=0.02)Steps to 70% (δ=0.05\delta=0.05)Avg4 (δ=0.05\delta=0.05)
Ours20038.6312036.73
SNIP [4]4644.942325.26

Our criterion achieves the target sparsity quickly and maintains stable reasoning performance under both KL thresholds. While SNIP also reaches 70% sparsity, it yields substantially lower accuracy.

Magnitude and Wanda are excluded from the table because they failed to produce a functional sparse model under global ranking. Magnitude pruning caused the training loss to continuously explode before reaching the target sparsity, failing to recover meaningful performance. Wanda disproportionately pruned specific layers, completely collapsing certain projection matrices and failing to reach the target sparsity entirely.

Crucially, SCOUT reuses the Adam second-moment estimates inherently tracked during weight optimization. It imposes virtually zero additional cost for collecting saliency statistics, whereas Wanda requires collecting and storing additional input activation statistics. We will include this comparison in the revised paper.

Hyperparameter tuning for Figure 3

Did you tune the hyperparameters for the experiments in Figure 3?

For Figure 3 (512-token rollouts), we tuned the learning rate for both ALPS+training and SCOUT, selecting 1×10−41\times10^{-4} over 5×10−55\times10^{-5} for 70–80% sparsity. We also evaluated multiple KL thresholds for SCOUT. For example, on Qwen3-8B at 70% sparsity, δ=0.03\delta=0.03 achieved slightly higher accuracy than δ=0.02\delta=0.02 (61.29% vs. 60.0%). Nevertheless, SCOUT with δ=0.02\delta=0.02 still outperformed ALPS+training (60.0% vs. 57.3%), suggesting that the performance advantage is not highly sensitive to this choice in this setting. We will clarify the hyperparameter selection procedure in the revised paper.

AuthorsOpenReview ↗

Rebuttal by Authors

Number of iterations and performance

How many iterations do you use for your method? Do you have a sense of how this changes performance?

SCOUT performs 2,048 optimization steps with mask updates every 8 steps. To analyze convergence, we evaluated intermediate checkpoints of Qwen3-1.7B at 70% sparsity.

Performance of intermediate checkpoints (Qwen3-1.7B, 70% sparsity, seed 42)

Steps200 (Target Reached)5121,0241,5362,048 (Final)
Avg4 (%)22.5831.8636.9837.7438.73

The model reaches the 70% target sparsity rapidly at step 200. It attains ~95% of its final performance by step 1,024, with diminishing returns thereafter. This demonstrates that SCOUT does not strictly require the full 2,048 steps to converge effectively.

Parameter revival and sparsity evolution

Are masked parameters frequently revived? How does pruning size evolve throughout training?

SCOUT's pruning trajectory consists of a rapid initial pruning phase followed by conservative mask refinement. On Qwen3-1.7B (70% sparsity), sparsity increases rapidly at first, with progressively smaller increments as the model approaches the target.

Sparsity evolution during initial pruning

Optimization step32486496128160192200
Sparsity (%)5.432.143.453.559.864.468.770.0

The model reaches 70% sparsity at step 200. During this initial phase, 986 million weights are pruned and only 457 are revived.

Once the target is reached, SCOUT maintains 70% sparsity and performs small-scale mask swaps. Over the remaining 1,848 steps, the cumulative number of revived weights is 388K, accounting for 0.092% of the active weights.

To test the impact of this refinement, we froze the mask at step 200 and continued training. Across three training seeds, freezing the mask reduced the mean final accuracy from 38.63% to 37.6%. These results indicate that SCOUT relies primarily on rapid initial pruning, with limited subsequent mask refinement providing a modest additional benefit. We will clarify this trajectory in the revised paper.

Relation to alternative approaches for efficient reasoning

Can you discuss the relation with alternative research directions to make reasoning chains more efficient, including natural language compression, KV cache compression for reasoning, or even length-aware post-training methods?

Methods like natural language compression or length-aware post-training reduce the number of generated tokens, while KV cache compression reduces memory footprint over long contexts. SCOUT is orthogonal to these approaches, as it reduces the computational cost per token via weight compression. Combining SCOUT with these methods could yield multiplicative speedups and memory savings. We will add a discussion detailing these complementary relationships.

References
[1] Macko and Boža, MACKO: Sparse Matrix-Vector Multiplication for Low Sparsity, ICML 2026.
[2] Fan et al., SpInfer: Leveraging Low-Level Sparsity for Efficient Large Language Model Inference on GPUs, EuroSys 2025.
[3] Sun et al., A Simple and Effective Pruning Approach for Large Language Models, ICLR 2024.
[4] Lee et al., SNIP: Single-Shot Network Pruning based on Connection Sensitivity, ICLR 2019.

Thank you for the detailed response.

I appreciate the positive longer-context results, although 2k tokens is still far from real-world context lengths (but I understand the compute constraints).

I find the sparsity dynamics results very interesting, although it is a bit disappointing that reviving helps very little. Thanks for sharing them.

Overall, I maintain my current positive score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of rebuttal findings and revisions

We sincerely thank all reviewers for their time and constructive feedback. Below, we briefly summarize the key findings from our rebuttal responses and the corresponding revisions to the manuscript. We have incorporated part of the feedback into the revised PDF, with changes marked in red, and will incorporate the remaining feedback and experimental results into the final manuscript.

1. Generalization Across Model Families and Scales (Q4RB, SWVr)
On Qwen3-32B at 70% sparsity, SCOUT achieves 73.3 vs. 66.2 for ALPS+training, a 7.1-point improvement. On DeepSeek-R1-Distill-Llama-8B, the improvement over ALPS+training reaches +10.7 points at 80% sparsity. With 2,048-token rollouts, SCOUT outperforms ALPS+training at 60% sparsity and above across all three Qwen3 scales.

2. Contribution of On-Policy Information (SWVr, Hz6h)
On Qwen3-1.7B at 70% sparsity with 2,048-token rollouts, replacing on-policy distillation with an equal amount of offline distillation data reduces accuracy from 38.6 to 31.5. Excluding the on-policy term from saliency alone reduces accuracy to 37.1, while disabling rollout refresh reduces it to 34.9.

3. Computational Efficiency and Practical Speedup (Q4RB, SWVr)
Training cost is comparable to ALPS+training across model scales. With batched sparse inference on Qwen3-8B at 60% sparsity (batch sizes 1–32), SCOUT achieves a 1.10–1.55x sequence-level speedup after accounting for longer generations, while reducing total memory by 11–35%.

4. Benefits of Trust-Region-Guided Pruning (SWVr, Hz6h)
Under the same objective, training budget, and 2,048-token rollouts, SCOUT outperforms cubic and linear sparsity schedules on each of three training seeds on Qwen3-1.7B at 70% sparsity (38.6 vs. 37.9 and 37.3 on average). Iterative ALPS reaches 34.3 under the same setting.

Changes in the Revised PDF (Marked in Red)

  • Sec. 3.1–3.2, Sec. 4, Appendix F: Clarified that the trust region constrains mask transitions with weights held fixed, and that the theoretical analysis relies on this condition (Hz6h).
  • Appendix A, Table 1: Added measurements of KL drift from weight updates (Hz6h).
  • Sec. 5.1: Clarified that the one-shot baselines are calibrated on our fixed corpus rather than C4 (SWVr).
  • Appendix C.1, Table 10: Added comparisons between 512- and 2,048-token rollouts across all models and sparsity levels (Q4RB).

Planned Revisions for the Final Manuscript
We will incorporate the remaining experiments reported in our individual responses, covering iterative and schedule-based pruning baselines, alternative saliency criteria, C4-calibrated baselines, on-policy ablations, cross-model scalability, computational efficiency, and sensitivity analyses. We will also update the main results to reflect the 2,048-token rollout setting and discuss complementary approaches to efficient reasoning.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.