acceptodds
Under review as a conference paper at ICLR 2027

Stability-Aware Structured Pruning of Large Language Models with Dynamic Search Spaces

Abstract

Structured pruning reduces the memory and computational costs of large language models (LLMs) by removing complete attention heads and multilayer perceptron (MLP) neurons. However, importance estimates derived from limited calibration data can vary across samples, and fixed search boundaries restrict adaptation to optimization feedback. We propose DSASP, a structured pruning framework that combines cross-sample stability assessment with dynamic search spaces. DSASP augments first-order Taylor and empirical-Fisher importance estimates with two complementary statistics: Importance Variance, which measures fluctuations in score magnitude, and Cross-sample Ranking Consistency, which measures stability in relative ordering. A Tree-structured Parzen Estimator optimizes the four scoring weights using proxy perplexity within an adaptive search region. Improvements shift and contract this region, whereas sustained stagnation expands it. Across LLaMA-7B and Vicuna-7B at pruning ratios of 20% and 40%, DSASP achieves the lowest perplexity among the evaluated pruning methods on both WikiText-2 and PTB and the highest mean zero-shot accuracy in all four settings. Ablation studies support the contribution of both the combined stability signals and the dynamic search mechanism. These results establish cross-sample stability and adaptive configuration search as complementary tools for preserving model performance under structured pruning.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Structured Pruning, Multidimensional Stability, Dynamic Search Spaces, Bayesian Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_tsq5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DSASP combines Taylor and empirical-Fisher importance scores with cross-sample magnitude dispersion and ranking consistency, then optimizes their weights using TPE with adaptive search boundaries. Experiments on LLaMA-7B and Vicuna-7B report improved perplexity and mean zero-shot accuracy at two pruning ratios. The intended contribution is that stability information and adaptive configuration search jointly improve structural selection.

Critical strengths
  • Distinguishing average importance from its consistency across calibration samples is a reasonable direction for improving pruning under limited data.
  • Consistent experimental improvements. Tables 2–5 show lower perplexity in all four settings. Recalculating the seven-task averages also confirms the abstract’s claim of the highest mean accuracy in all four settings.
Critical weaknesses
  • The methodological novelty beyond existing adaptive pruning frameworks is insufficiently established. AdaPruner[1] already formulates calibration and importance-estimation configuration as a Bayesian optimization problem, while FLAP[2] already exploits cross-sample activation fluctuations for structured pruning.
  • The experimental results does not yet establish the claimed complementary contributions. Tables 7–8 remove RC and dispersion together, so neither their individual value nor their complementarity is demonstrated; the combined stability gain on Vicuna is only 0.11 percentage points, without repeated-run uncertainty. The ablations also use a different search budget and omit their pruning ratios. Factorial ablations with matched settings, multiple calibration/search seeds, and convergence-versus-cost curves are needed.
  • The motivation for dynamic search conflates adapting exploration with enlarging the feasible configuration space. The introduction presents fixed search boundaries as a limitation, yet DSASP starts with the entire domain [0,1]4[0,1]^4, and Equation (9) confines every subsequent region to that same domain. Consequently, expansion only revisits configurations already admissible to the original fixed-domain optimizer; its potential benefit is a better allocation of evaluations, which requires evidence from matched-budget convergence comparisons. Adaptive search regions also have substantial precedent in Bayesian optimization, including TuRBO[3].

References

[1] Kong et al. Sample-aware Adaptive Structured Pruning for Large Language Models

[2] An et al. Fluctuation-Based Adaptive Structured Pruning for Large Language Models

[3] Eriksson et al. Scalable Global Optimization via Local Bayesian Optimization

Questions for authors
  1. What are the sample identifiers and splits for calibration selection, proxy optimization, and final perplexity evaluation? Are they strictly disjoint?
  2. What pruning ratios were used in Tables 7–8, and what happens when RC and dispersion are ablated separately?
4 comments

Comment 1. What is the methodological novelty of DSASP compared with AdaPruner and FLAP?

Response. Thank you for encouraging us to sharpen the distinction from AdaPruner and FLAP.

Evidence. The submitted manuscript already describes the relevant components separately: Section 2.1 identifies FLAP as an activation-fluctuation method for sparsity allocation, Section 2.3 describes AdaPruner as searching calibration data and importance-estimation configurations, and Sections 3.2-3.3 define DSASP’s stability-aware score and dynamic region. The distinction is that DSASP combines two mechanisms not used jointly by these methods. First, it augments Taylor and empirical-Fisher contribution evidence with both magnitude dispersion and relative-rank consistency computed from sample-wise structural evidence. Second, it reallocates the finite TPE search budget by shifting, contracting, and expanding the admissible local region according to optimization feedback. AdaPruner searches different objects within fixed global bounds, while FLAP uses activation fluctuation rather than ranking consistency of gradient-based structural evidence.

Comment 2. Do RC and Importance Variance make separate and complementary contributions to DSASP?

Response. Thank you for requesting component-wise evidence for the two stability statistics.

Evidence. We added separate removal ablations under the same model, pruning ratio, search budget, and evaluation protocol as Tables 7 and 8. Full DSASP achieves 49.69% on LLaMA-7B at 40% pruning, compared with 48.50% without RC, 48.52% without Var, and 48.47% without both. On Vicuna-7B at 20% pruning, the corresponding values are 59.43%, 59.19%, 59.12%, and 59.32%. Full DSASP is best in both settings, and removing either statistic reduces mean accuracy. The effects need not be additive because the remaining weights are re-optimized and each variant induces a discrete pruning mask.

\hline \mathbf{Setting} & \mathbf{Full} & \mathbf{w/o\\ RC} & \mathbf{w/o\\ Var} & \mathbf{w/o\\ RC\\ and\\ Var} \\\\ \hline \text{LLaMA-7B 40\\%} & 49.69 & 48.50 & 48.52 & 48.47 \\\\ \text{Vicuna-7B 20\\%} & 59.43 & 59.19 & 59.12 & 59.32 \\\\ \hline \end{array}$$ Revision place. We have added the norc and novar variants to Tables 7 and 8 and expanded the component-wise analysis at the end of Section 4.5. Comment 3. If the initial search region already covers $[0,1]^4$, how does the dynamic search region improve optimization under a finite evaluation budget? Response. Thank you for asking us to clarify the role of expansion after the initial global region and for pointing out TuRBO as a relevant precedent. Evidence. TuRBO performs Bayesian optimization using local Gaussian-process trust regions, expanding a region after consecutive successes and shrinking it after consecutive failures (Eriksson et al., 2019). DSASP differs in its search variables, optimization framework, and region-update rule. DSASP uses TPE to search the four fusion weights of the Taylor, empirical-Fisher, RC, and Importance Variance terms within the fixed global domain $[0,1]^4$. As specified in Section 3.3, Eq. (9) initializes the admissible region over the entire domain and constrains every subsequent region to remain within these global bounds. When an improvement is found, Eqs. (10)–(11) shift the center toward the improving configuration and contract the radius, increasing the search resolution around a promising region. When progress stalls, Eq. (12) re-expands the previously contracted region, thereby broadening exploration within the global domain. Expansion therefore does not create configurations outside the original feasible space. Its role is to adapt the focus and resolution of the search under a finite evaluation budget: DSASP begins with full-domain exploration, concentrates evaluations around promising configurations, and restores broader exploration when the concentrated search no longer improves. Under the matched search budget reported in Tables 7–8, the dynamic region improves mean zero-shot accuracy from 48.69% to 49.69% on LLaMA-7B at 40% pruning and from 58.34% to 59.43% on Vicuna-7B at 20% pruning. Revision place. We have added TuRBO to Section 2.3 and the reference list, and clarified that DSASP dynamically adjusts the resolution and allocation of search within the fixed global domain rather than enlarging the feasible configuration space.

Comment 4. What are the sample identifiers and splits for calibration selection, proxy optimization, and final perplexity evaluation? Are they strictly disjoint?

Response. Thank you for requesting a clearer account of the three stages. The stages are procedurally separated and serve different roles.

Evidence. First, DSASP selects 10 calibration samples from 300 BookCorpus candidates and uses only these samples to compute the sample-wise structural statistics. Second, TPE evaluates each candidate weight configuration on a fixed 16-sample WikiText-2 proxy. This small proxy is used only as a low-cost surrogate for comparing candidate masks; WikiText-2 is not required by the method, and another small proxy corpus could be used. Third, after the search terminates, the selected weights and pruning mask are fixed, and the model is evaluated using the 128-sample WikiText-2 and PTB protocols and seven zero-shot tasks. PTB and all zero-shot tasks are never used by the optimizer. This follows the AdaPruner-style calibration-proxy-evaluation pipeline while keeping the final downstream evaluation outside the search loop.

Comment 5. What pruning ratios were used in Tables 7–8, and what happens when RC and dispersion are ablated separately?

Response. Thank you for asking us to make the ablation settings explicit.

Evidence. Table 7 reports LLaMA-7B at 40% pruning, while Table 8 reports Vicuna-7B at 20% pruning. We have added these ratios to the table captions. The newly added component-wise rows in Tables 7 and 8

Revision place. The captions of Tables 7 and 8 in Section 4.5 now include the corresponding model and pruning ratio.

Reviewer_tsq5OpenReview ↗

Response to Author Rebuttal

Thank you for the clarifications and additional ablations. The separate RC/Var ablations, specified pruning ratios, and revised explanation of dynamic search address part of my concerns. However, procedural separation does not establish that the WikiText-2 proxy and final evaluation samples are disjoint. The small Vicuna gain also remains difficult to assess without repeated-run uncertainty. While the positioning is clearer, I remain unconvinced that the methodological advance is sufficiently substantial. I therefore maintain my original score.

Response. Thank you for your follow-up. We address the remaining concerns below.

Our previous response distinguished the roles of calibration, configuration search, and final evaluation but did not directly state the relationship between the corresponding WikiText-2 samples. The 16 WikiText-2 examples used by TPE are contained in the 128-example WikiText-2 pool used for PPL reporting. Following AdaPruner’s evaluation-guided Bayesian optimization procedure , DSASP uses perplexity on this small evaluation set as a low-cost signal for comparing candidate pruning configurations.

BookCorpus supplies the calibration statistics, while PTB and all seven zero-shot tasks are never accessed by TPE or used for configuration selection. Across all four model–pruning-ratio settings, DSASP achieves the lowest PTB perplexity and the highest mean zero-shot accuracy among the evaluated pruning methods. These results provide performance evidence beyond the WikiText-2 signal used during configuration search.

The repeated-run question concerns variation across calibration and search seeds. Tables 7–8 serve as controlled component comparisons: within each setting, all variants use the same model, pruning ratio, calibration data, proxy set, search budget, seed, and evaluation protocol. On Vicuna-7B, the mean accuracies for full, w/o RC, w/o Var, and w/o both are 59.43, 59.19, 59.12, and 59.32, respectively. On LLaMA-7B, the corresponding values are 49.69, 48.50, 48.52, and 48.47. The 0.11-point value is one full-versus-w/o-both comparison, while removing RC or Importance Variance produces the same directional effect in both settings. The fixed-region results, 58.34 on Vicuna-7B and 48.69 on LLaMA-7B, further isolate the contribution of dynamic search. Because the remaining weights are re-optimized and each variant produces a discrete pruning mask, the component effects are not expected to be additive. The evidentiary role of these ablations is to isolate the observed component effects under matched conditions; the four main settings and the out-of-proxy benchmarks provide the broader performance evidence.

DSASP’s contribution is the coupled pruning formulation defined by Equations (8)–(12). Equation (8) integrates Taylor and empirical-Fisher structural evidence with magnitude dispersion and relative-rank consistency. Equations (9)–(12) allocate a finite TPE evaluation budget between local refinement and broader exploration within the fixed global domain. AdaPruner searches calibration samples and importance-estimation configurations, FLAP uses activation fluctuation for sparsity allocation, and TuRBO addresses general Gaussian-process trust-region optimization. DSASP instead models two forms of stability in sample-wise gradient-based structural evidence and jointly optimizes their fusion through feedback-controlled TPE search.

The complete source code and experimental scripts are included in the supplementary attachment for direct verification and reproduction of the reported experiments.

Reviewer_pzptOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DSASP for structured pruning of LLMs by removing complete attention heads and MLP neurons. Importance estimated on a small calibration set can fluctuate across samples, so an average score hides that instability. Bayesian optimization of the scoring weights is also limited when the search bounds stay fixed. DSASP adds importance variance and cross-sample ranking consistency to first-order Taylor sensitivity and an empirical-Fisher curvature term: high variance is penalized, stable ranking is rewarded, and the four terms are combined by a set of weights. Those weights are searched with TPE using proxy perplexity. The search region contracts toward an improved configuration and expands after consecutive failures. The selected weights then determine the pruning mask.

Critical strengths

The idea is clear and the method is well laid out. The paper treats two sources of uncertainty: importance that is unstable across calibration samples, and search bounds that do not respond to feedback. Variance and ranking consistency describe the first; the search region contracts after improvement and expands after stagnation to address the second.

Critical weaknesses
  1. Two related-work subsections share the same title. Sections 2.1 and 2.2 are both titled “Structured Pruning and Redundancy in Transformers.” However, their contents are different;

  2. The pruning ratios and model coverage are narrow. Experiments compare only 20% and 40% pruning, with no lower or higher ratios. LLaMA-7B and Vicuna-7B belong to the same LLaMA-7B family; Vicuna is an instruction-tuned version of LLaMA, so neither architecture nor scale is varied. The current results therefore do not show whether the method remains effective at other compression ratios or on other model families;

  3. It is necessary to report the computational cost.

Questions for authors

The same as the weakness part.

3 comments

Comment 1. Why do Sections 2.1 and 2.2 have the same title, and what is the correct title of Section 2.2?

Response. Thank you for identifying this typographical error.

Evidence. The content of Section 2.2 concerns importance estimation rather than repeating the discussion of Transformer redundancy. We have therefore corrected its title to “Importance Estimation for Pruning”; the section content and organization are otherwise unchanged.

Revision place. The heading of Section 2.2 has been corrected from “Structured Pruning and Redundancy in Transformers” to “Importance Estimation for Pruning.”

Comment 2. What evidence supports the generalization of DSASP across model architectures, model scales, and pruning ratios?

Response. Thank you for highlighting the value of broader model and pruning-ratio coverage.

Evidence. The current evaluation includes a base checkpoint, LLaMA-7B, and an instruction-tuned checkpoint, Vicuna-7B, each evaluated at 20% and 40% pruning. As reported in Tables 2-5, DSASP obtains the lowest WikiText-2 and PTB perplexity and the highest mean zero-shot accuracy among the evaluated pruning methods across these four settings.This consistency provides evidence that the observed gains are not specific to a single checkpoint or pruning ratio and supports the stability of DSASP across the evaluated settings. We agree that additional model families and pruning ratios would provide a broader assessment of generalization. Because each extension requires a complete set of matched baseline runs, the available rebuttal period does not allow us to complete and verify these experiments to the same standard for the current version. Should the subsequent review invite a major revision, we will extend the evaluation to additional model families or pruning ratios under the same experimental protocol.

Comment 3. What is the computational cost of DSASP?

Response. Thank you for requesting a more complete computational-cost report.

Evidence. Under the main setting on one NVIDIA RTX 6000D, DSASP requires approximately 26-30 minutes, or 0.43-0.50 GPU-hours, to produce the final mask for each model-pruning-ratio configuration. This includes structural-score precomputation, the complete 150 × 4 TPE search with 16 proxy samples, and final mask construction. Final downstream evaluation is reported separately because it is shared by all pruning methods. The search is a one-time offline cost, requires no fine-tuning or post-pruning recovery. The resulting 20% and 40% masks correspond to estimated projection-FLOP reductions of approximately 16% and 32%, respectively.

Revision place. We have added the measured offline cost and its scope to the final paragraph of Section 4.1.

Thanks for the detailed response. The heading correction and the reported offline cost resolve part of my concern. One experimental result still needs explanation.

The main table and the ablation “full” variant disagree on the same zero-shot tasks: 47.81 versus 49.69 on LLaMA-7B at 40%, and 57.24 versus 59.43 on Vicuna-7B at 20%. The two runs also use different numbers of search trials and proxy samples. Please state which settings changed, and why the main run, despite its larger search budget, has lower accuracy.

Response. Thank you for the follow-up question. The term “full” indicates that all DSASP components, including Taylor, empirical Fisher, RC, Importance Variance, and dynamic search, are enabled; it does not mean that the main-table result and the ablation-table result come from the same optimization run. The main and ablation experiments independently search the fusion coefficients under their respective settings. Each search therefore selects its own coefficient configuration and generates its own structural pruning mask.

Importantly, the two protocols differ in proxy-set size, trial structure, and TPE search history. The main run is therefore not a continuation of the ablation run, and the two experiments do not constitute a controlled comparison in which only the number of search trials changes. The reported values reflect the downstream performance of two independently selected masks, rather than inconsistent evaluations of the same mask. Accordingly, Tables 7 and 8 provide controlled comparisons among full, norc, novar, norcv, and nodyn under a shared ablation protocol.

Evidence. Table 1 specifies that the main experiment uses 150 TPE trials with four candidates per trial and 16 proxy samples, whereas the ablation experiment uses 25 TPE trials and eight proxy samples. Section 3.3 and Appendix B specify that each run evaluates candidate configurations using proxy perplexity on its corresponding proxy set and retains the best configuration from its own search history. Section 4.5 further states that the model, pruning ratio, search budget, and evaluation procedure are held constant across all ablation variants. Accordingly, the main tables evaluate DSASP against the baseline pruning methods under the main protocol, while Tables 7 and 8 provide controlled component comparisons under the common ablation protocol.

Reviewer_9eWyOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper considers the importance stability problem in the LLM structured pruning. It incorporates importance variance and cross-sample ranking consistency into the importance score, then uses Bayesian optimization to find suitable weights for all these scores. Experiments on two models show improvements in perplexity and zero-shot performance over the compared baselines.

Critical strengths
  • The paper focuses on an interesting problem in structured pruning by considering the stability of importance estimates across different calibration samples.
  • The paper is clear and easy to follow. The expereiments show consistent improvements on several models and benchmarks.
Critical weaknesses
  • The biggest concern I have for this paper is the assumption behind the motivation. The paper assumes that structures with more stable importance scores should be preserved. This is just an reasonable assumption. For example, some neurons may be important only for specific domains or functions, such as long-context processing or medical reasoning. In that case, their importance could vary substantially across samples, yet they may still be critical, and therefore high variance may be a reason to preserve these neurons rather than penalize them. Both assumptions are plausible, and the author should provide more evidence behind the assumption this paper takes.
  • Another concern is that even if a neuron is consistently unimportant, the method can still assign it a positive stability score. Although the coefficients can control the contribution of each component, the method does not inherently distinguish this case. Also, according to Eq. (13), the stability measures are calculated from the unweighted sum of Taylor and Fisher importance, although their weights are optimized separately in the final score. This creates a mismatch between the stability measures and the optimized importance score.
  • Some discussion of closely related work is missing. AdaPruner[1] already uses Bayesian optimization for structured pruning. GPrune-LLM[2] explores importance ranking stability across calibration distributions. The paper should better clarify its contribution from these methods.

[1] Kong J, Ma X, Wang J, et al. Sample-aware adaptive structured pruning for large language models

[2] Liu X, Saxena D, Cao J, et al. GPrune-LLM: Generalization-Aware Structured Pruning for Large Language Models

Questions for authors

Can the authors report the computational cost of the proposed method, like search time and actual inference speedup?

3 comments

Comment 1. How does DSASP distinguish stability from importance, especially for specialized neurons?

Response. Thank you for raising this important concern about the motivation of DSASP. The submitted manuscript does not use stability as a stand-alone pruning criterion.

Evidence. As defined in Section 3.2.3 and Eq. (8), the final structural score jointly combines normalized Taylor and empirical-Fisher contribution estimates with RC and Importance Variance, and the four coefficients are optimized together. Taylor and empirical-Fisher scores provide direct evidence of structural contribution, while RC and Importance Variance characterize the reliability of that evidence across calibration samples. A structure with strong contribution evidence can therefore remain highly ranked even when its sample-wise importance varies, including a structure specialized to a subset of inputs. Accordingly, DSASP does not equate stability with importance; stability is used as complementary reliability evidence rather than as a substitute for structural contribution.

Comment 2. How does DSASP prevent stable but low-importance structures from receiving inflated scores, and why are RC and Importance Variance computed from fixed Taylor-Fisher evidence?

Response. Thank you for this careful question about the interaction between contribution and stability. The submitted formulation does not use the stability terms as an independent retention criterion.

Evidence. Section 3.2.2 and Eq. (7) define RC as a one-sided reward that is positive only when a structure’s ranking consistency exceeds the within-layer median. Section 3.2.3 and Eq. (8) then rank structures using the combined Taylor, empirical-Fisher, RC, and Importance Variance score. Appendix A defines the canonical sample-wise evidence as the sum of the Taylor and empirical-Fisher terms. RC and Importance Variance are computed from this fixed evidence, whereas the four optimized coefficients determine how contribution and reliability are fused. Keeping the canonical evidence fixed ensures that the stability statistics describe the same sample-wise signal across candidate fusion weights; the optimization then controls how strongly each contribution and reliability term affects the final ranking.

Comment 3. How does DSASP differ from AdaPruner and GPrune-LLM?

Response. Thank you for pointing out this relevant connection and for encouraging us to clarify the positioning of DSASP.

Evidence. The submitted manuscript discusses AdaPruner because it also uses TPE-based search, but the two methods search different objects. AdaPruner searches calibration samples and importance-estimation configurations, whereas DSASP retains sample-wise structural evidence and optimizes the fusion weights of Taylor, empirical-Fisher, RC, and Importance Variance within a feedback-driven dynamic region. We also thank the reviewer for pointing us to GPrune-LLM and will include its citation and discussion in the revised version. GPrune-LLM addresses cross-distribution calibration bias by adapting the neuron-ranking space, scoring source, and sparsity allocation. DSASP instead evaluates the dispersion and relative-rank consistency of structural importance across individual samples within a calibration setting. The methods therefore share an interest in ranking reliability but address different levels of calibration variation.

Revision place. We have added GPrune-LLM to Section 2.2 after the discussion of calibration-sensitive pruning methods and to the reference list.

Comment 4 . Can the authors report the computational cost of the proposed method, like search time and actual inference speedup?

Response. Thank you for requesting a clearer account of the offline cost and inference-side computation. DSASP performs a one-time offline search.

Evidence. On one NVIDIA RTX 6000D, the main 150 × 4 search requires approximately 26-30 minutes, or 0.43-0.50 GPU-hours, for each model-pruning-ratio configuration. This measurement includes score precomputation, proxy optimization with 16 samples, and final mask construction. DSASP requires no fine-tuning or post-pruning recovery. Because pruning is applied to 26 of the 32 Transformer layers, the 20% and 40% settings correspond to estimated reductions of approximately 16% and 32% in Transformer projection FLOPs.

Revision place. We have added a “Computational cost” paragraph at the end of Section 4.1 .

Thank you for the detailed response. The clarification of the scoring formulation and the additional discussion of related work and computational cost are helpful.

However, my main concern remains the assumption behind stability-aware pruning. While the authors clarify that stability is used as complementary evidence rather than an independent criterion, this does not explain why higher variance should lead to lower preservation priority. The alternative interpretation that high variance may reflect domain-specific importance remains plausible. Similarly, the formulation still rewards consistently unimportant structures, and the response does not fully justify this design choice. I would like to see more direct evidence supporting the relationship between stability and actual pruning utility.

Given these remaining concerns, I tend to maintain my original score.

Response. Thank you for clarifying the remaining concern. DSASP determines preservation through the complete structural score rather than through either stability statistic in isolation. In Eq. (8), the normalized Taylor and empirical-Fisher terms represent structural contribution, Importance Variance quantifies projection-relative magnitude dispersion, and RC quantifies cross-sample consistency in relative rank.

The use of Importance Variance follows from the shared-mask setting of structured pruning. For structures with comparable contribution evidence, lower cross-sample dispersion provides more reliable support for a single pruning mask applied across inputs. This preference is relative rather than decisive. For structures whose sample-wise importance magnitudes vary, the Taylor and empirical-Fisher terms continue to represent their direct contribution, while RC separately captures whether their relative ranks remain consistent across samples. The jointly optimized coefficients determine how strongly magnitude dispersion and ranking consistency affect the final ranking.

Importantly, a positive stability component does not translate directly into preservation under Eq. (8). A positive RC value contributes only the weighted term γRC(g)\gamma \mathrm{RC}(g), while below-average magnitude dispersion affects the score only through −δVar(g)-\delta \mathrm{Var}(g). Neither term defines an independent retention threshold. Every structure is ranked by the complete score, including αI^T(g)\alpha \hat{I}_T(g) and βI^F(g)\beta \hat{I}_F(g), relative to competing structures. Eq. (22) then removes the structures with the lowest complete scores.

Evidence. Eqs. (2)-(3) define projection-relative magnitude dispersion, Eqs. (6)-(7) define the bounded and median-thresholded RC adjustment, and Eq. (8) combines these statistics with the normalized Taylor and empirical-Fisher contribution terms. Eq. (22) constructs the pruning mask from this complete ranking. Consistent with this formulation, Tables 7 and 8 show that the full score achieves the highest mean accuracy on both evaluated checkpoints under matched ablation settings, while removing either RC or Importance Variance lowers the corresponding mean accuracy after the remaining coefficients are re-optimized. These comparisons provide direct mask-level support for the role of each statistic in the two evaluated settings. We hope this resolves the remaining concern.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.