- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SURCO, a post-training method for pruning the recurrent states of SSMs. Instead of directly retaining the original readout gains of surviving states, SURCO adopts a response Gram matrix constructed from a calibration set to refit these gains through closed-form least squares, compensating for the output loss caused by pruning. It also introduces layer-wise pruning budget allocation based on end-to-end KL divergence. Experiments on Mamba-3 SISO/MIMO and Mamba-2 show that SURCO improves accuracy by 2–6% on average over LAST, AIRE-Prune, GHOST, and other baselines at 50% state removal.
Critical strengths
- The paper identifies a practical problem overlooked by previous state pruning methods: selecting a pruning mask without optimizing the remaining states' readout gains can lead to unnecessary accuracy loss.
- The proposed method is simple and effective. Its training-free, closed-form formulation also makes it practical to implement.
- The experimental design is reasonable. Table 4 decouples compensation from mask selection and demonstrates that the mechanism can generalize across different pruning methods.
Critical weaknesses
-
Limited novelty. Post-pruning refitting of surviving parameters via least-squares reconstruction is a classical technique in neural network compression (e.g., channel pruning, structured sparsity). SURCO's application of this paradigm to SSM recurrent states is not sufficiently differentiated from these prior formulations. Moreover, the proposed conditional necessity criterion, derived from output-space Gram matrices, closely parallels existing importance metrics based on response correlations or Hessian sensitivities, leaving the core methodological innovation unclear.
-
Narrow experimental scope. Results are restricted to two pruning ratios (37.5% and 50%), with no systematic characterization across the full operating range (e.g., 10%–90%). The largest model evaluated is 1.5B, leaving scalability unverified. More critically, Table 4 shows that SURCO's compensation alone substantially improves masks produced by LAST and AIRE-Prune, suggesting the observed gains may primarily stem from the post-hoc refitting step. The paper must therefore more rigorously establish the incremental value of its full pipeline (joint mask selection + compensation) over simply applying its compensation as a plug-in to existing pruning masks.
Questions for authors
- SURCO's compensation mechanism relies heavily on the additive decomposition of the Mamba-3 readout in Eq. (4). Can this approach be generalized to SSMs whose readout does not have a simple additive structure?
- Could the authors provide results across a wider range of pruning ratios (e.g., 10%–90%) and direct comparisons of end-to-end inference efficiency against coarse-grained pruning methods such as Mamba-Shedder?
4 comments
Q2. Pruning Ratio Sweep
Pruning-ratio sweep. On Mamba-3 SISO-443M, we evaluated full SURCO from 0–87.5% state removal; LAST, AIRE-Prune, and GHOST were compared at matched budgets through 75%. FineWeb-Edu PPL (lower is better):
| Removed | SURCO | LAST | AIRE | GHOST |
|---|---|---|---|---|
| 0% | 13.570 | 13.570 | 13.570 | 13.570 |
| 12.5% | 13.688 | 13.651 | 13.649 | 13.698 |
| 25% | 13.846 | 13.947 | 13.850 | 14.039 |
| 37.5% | 14.049 | 14.410 | 14.367 | 14.722 |
| 50% | 14.327 | 15.887 | 15.306 | 18.441 |
| 62.5% | 15.093 | 20.000 | 19.367 | 22.509 |
| 75% | 18.250 | 27.063 | 30.771 | 29.384 |
| 87.5% | 22.380 | NA | NA | NA |
NOTE: NA in the table means very high PPL (unstable).
LAMBADA accuracy (higher is better):
| Removed | SURCO | LAST | AIRE | GHOST |
|---|---|---|---|---|
| 0% | 43.39 | 43.39 | 43.39 | 43.39 |
| 12.5% | 42.79 | 42.37 | 42.63 | 38.81 |
| 25% | 42.65 | 42.30 | 42.51 | 34.66 |
| 37.5% | 42.35 | 42.02 | 42.57 | 31.24 |
| 50% | 40.3 | 35.42 | 33.8 | 17.58 |
| 62.5% | 30.80 | 22.57 | 19.77 | 8.11 |
| 75% | 17.54 | 11.12 | 7.92 | 3.32 |
SURCO leads clearly at across the complete sweep window at given pruning budget. Its seven-task average (LM-harness) is 49.99 dense, 49.15 at 50%, 44.45 at 75%, and 42.42 at 87.5%.
Q2. Comparison with Mamba-Shedder: Complementary pruning strategies
Mamba-Shedder and SURCO are complementary pruning methods. Mamba-Shedder removes entire Mamba blocks, reducing model parameters and computation, whereas SURCO reduces recurrent-state dimensions within surviving blocks. Thus, SURCO can be applied on top of Mamba-Shedder to further reduce inference cost. We evaluate their composition in both model accuracy and hardware efficiency.
(a) Accuracy: Dense → Mamba-Shedder → +SURCO
Experimental setup. We reproduced Mamba-Shedder's cumulative, Alpaca-calibrated, PPL-greedy block removal and subsequently applied SURCO to surviving layers. We evaluate Mamba-2-780M and Mamba-3 SISO-443M using the same seven-task LM suite (Avg7).
Mamba-2-780M accuracy (Avg7 ↑):
| Blocks removed | Shedder only | +SURCO25 | +SURCO50 |
|---|---|---|---|
| 0 (dense) | 51.91 | 50.66 | 49.11 |
| 4 | 49.83 | 48.77 | 46.80 |
| 8 | 46.20 | 45.55 | 43.13 |
| 12 | 42.78 | 42.11 | 40.26 |
Observations: These results show that state pruning remains effective after coarse-grained block removal, with a relatively consistent marginal accuracy cost. Here, we have opperated with uniform allocation, and bf16 evaluation variation reaches approximately 0.33 Avg7 points.
(b) Hardware: Dense → Mamba-Shedder → +SURCO
Experimental setup. On one NVIDIA H100, we physically remove Mamba-3 SISO-443M blocks and materialize SURCO-pruned surviving layers (FoldedMamba3), without hook-simulated decoding. We measure batch-256 throughput and peak memory.
| Configuration | Avg7 ↑ | Speedup ↑ | Peak memory ↓ |
|---|---|---|---|
| Dense | 49.99 | 1.00× | 7.92 GB |
| Shedder: 4 blocks | 45.57 | 1.20× | 7.83 GB |
| +SURCO25 | 44.62 | 1.55× | 5.39 GB |
| +SURCO50 | 43.73 | 1.99× | 3.97 GB |
| Shedder: 8 blocks | 40.28 | 1.48× | 7.22 GB |
| +SURCO25 | 39.72 | 1.92× | 4.53 GB |
| +SURCO50 | 38.68 | 2.47× | 3.36 GB |
Observations: We can observe that SUCO provides real hardware benefits over Mamba-sheddar. The SURCO performance to speedup tradeoff is better than mamba-sheddar
Cross-architecture verification. On Mamba-2-780M, 12-block removal provides 1.33× speedup, SURCO50 alone 1.88×, and their combination 2.51×, reducing peak memory from 12.00 GB (dense) to 5.48 GB (−54%). These Mamba-2 hardware measurements use a materialized state-dimension surrogate; accuracy is measured separately on the actual pruning path.
(c) Matched-state quality against Mamba-Shedder
We also compare SURCO against Mamba-Shedder's coarse SSM removal at identical recurrent-state budgets on Mamba-2-780M.
| State removed | Shedder Avg7 | SURCO Avg7 | Shedder PPL | SURCO PPL |
|---|---|---|---|---|
| 25% | 48.66 | 51.68 | 14.584 | 13.658 |
| 50% | 38.65 | 50.40 | 21.711 | 13.850 |
At matched 50% state removal, SURCO retains +11.75 Avg7 over Shedder's SSM removal, confirming the advantage of fine-grained recurrent-state reduction over eliminating entire SSM components.
Conclusion. These experiments demonstrate two complementary benefits: (i) SURCO preserves substantially more accuracy than coarse SSM removal at matched state budgets; (ii) applying SURCO on top of Mamba-Shedder block removal yields substantial additional memory and latency reductions, with a relatively consistent marginal accuracy cost.
W1. Novelty
Prior state-pruning methods primarily focus on identifying and removing redundant states, while leaving the surviving states unchanged. We in SURCO take a complementary perspective: rather than only deciding which states to remove, it also compensates and re-optimizes the surviving states to recover the lost output response. This motivates our reduced-realization formulation, which combines:
- Response geometry: Constructs Gram matrices from realized, post-projection state-to-output contributions, capturing functional overlap between recurrent-state units.
- Conditional necessity: Measures the residual response energy of a state unit after projecting it onto the span of other units, using a Schur-complement formulation and normalization by physical state cost (including indivisible rotary pairs).
- Survivor compensation: Jointly refits surviving readouts after fixing the complete pruning mask and folds the resulting gains into existing parameters, introducing no additional decode-time operations.
- Global allocation: Uses measured end-to-end predictive KL distortion to distribute the state budget across layers, rather than relying solely on local importance scores.
Our contribution is unified, recurrent-state-specific formulation and integration compensating pruned states for selective SSM compression, rather than least-squares refitting in isolation.
W2. Added Experiments
(i) Disentangling compensation from the complete SURCO pipeline
Experimental setup. On Mamba-3 MIMO-1.5B at 50% state removal, we evaluate seven-task LM accuracy (Avg7) and FineWeb-Edu PPL. All runs retain the same total budget (mean 64/128 coordinates per layer). We vary selection, allocation, and compensation independently and refit gains for each mask.
(a) Compensation: fixed mask and allocation.
| Selector + allocation | No comp. Avg7 | +Comp. Avg7 | Gain |
|---|---|---|---|
| SURCO + KL allocation | 51.99 | 55.01 | +3.02 |
| LAST + own allocation | 49.33 | 51.78 | +2.45 |
| AIRE-Prune + own allocation | 48.97 | 52.04 | +3.07 |
| GHOST + uniform allocation | 49.58 | 52.59 | +3.01 |
Observation: Compensation improves all four fixed masks by 2.45–3.07 points.
(b) Selection: fixed uniform allocation, compensation on.
| Selector | FW-Edu PPL ↓ | Avg7 ↑ |
|---|---|---|
| SURCO (conditional necessity) | 11.661 | 52.93 |
| GHOST | 11.962 | 52.59 |
| LAST | 12.053 | 52.11 |
| AIRE-Prune | 12.213 | 51.83 |
Observation: With allocation and compensation fixed, selectors span 0.80 points; SURCO leads GHOST by 0.34.
(c) Allocation: fixed SURCO selector and compensation. Uniform allocation yields 52.63 Avg7, whereas measured KL-distortion allocation yields 55.01 (+2.38). This isolates the benefit of KL allocation.
Conclusion (Q1). Along a matched-budget controlled path,
GHOST/uniform/no compensation 49.58 → +compensation 52.59 (+3.01) → SURCO selection 52.93 (+0.34) → KL allocation 55.01 (+2.08).
The substantial MIMO gain is driven primarily by survivor compensation and KL-guided allocation with selector having its contribution as well..
(ii) Model scalability.
We evaluated all available released Mamba-3 checkpoints, spanning 187M–1.5B parameters, and additionally evaluated Mamba-2-780M. The 1.5B checkpoint is the largest publicly released Mamba-3 checkpoint used in our study. Evaluating substantially larger models would require access to additional pretrained checkpoints or training resources beyond the computational scope of this work. We therefore leave evaluation at larger scales as an important direction for future work, subject to checkpoint availability and computational resources.
We will incorporate these additional experiments and profiling results into the revised appendix. Hope we have addressed your concerns. We would appreciate any further feedback on whether these points and the requested experiments sufficiently resolve the open issues. We are happy to clarify anything further.
We thank the reviewer for their valuable feedback. We added pruning sweeps and comparisons with Mamba-Shedder, including combined accuracy, latency, and memory.
Q1. Generalization beyond Mamba-3
SURCO's exact Gram reconstruction requires an additive decomposition of the monitored state-dependent readout into pruning-unit responses (possibly followed by a fixed linear projection). It applies to compatible additive-readout SSMs beyond Mamba-3. On Mamba-2-780M, SURCO retains 50.40 Avg7 at 50% state removal versus 51.91 dense. For genuinely non-additive readouts, local linearization is a possible extension, but the exact fixed-mask reconstruction guarantee does not carry over; we do not claim to have evaluated that setting. Our contribution is recurrent-state-specific refitting, rotary-safe units, gain folding without decode operations, and KL-based allocation. But addressing the non additive readout SSM to be considered as future work. One possible approach could be to construct a local linear approximation of the readout around calibration trajectories and use the resulting Jacobian-based state-response representations to estimate inter-state dependencies and survivor compensation.