acceptodds
Withdrawn from ICLR 2027

Compression order in spiking neural networks: normalization and model construction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Compression-order comparisons depend on how models are constructed and calibrated. We study channel pruning, unfolded weight-only quantization and threshold scaling in spiking neural networks (SNNs), without intervening training. Matched-state controls distinguish batch-normalization (BN) history from model construction: aggressive clipping without rounding reproduces large history-dependent gaps, including in ReLU networks, and common recalibration yields identical logits. Separately, a prespecified twelve-run CIFAR-100 ResNet-SNN comparison at two time steps gives a mean absolute accuracy gap of 0.17 percentage points (pp) on reused test images, despite 6.0% prediction disagreement under the specified construction. Both orders satisfy the prespecified operational 5-pp budget on evaluated validation and test sets, with 10.08% fewer convolution multiply–accumulate operations. Descriptive extensions include matched T=4 retraining, additional image and event-frame datasets, and Transformer MLPs. Calibration improves both-order budget passage in the complete ImageNet-100 comparison; full reconstruction changes one finite-split qualification decision that final-only recalibration preserves. In a descriptive comparison of twelve T=2 ResNets at =1.02, the mean between-order relative synaptic-operation range contracts from 12.48% to 0.031% after common recalibration, with all orders remaining budget-feasible. Retrospective candidate policies on reused validation splits quantify compression choices and test-budget violations. These observations distinguish normalization history, model behavior, budget feasibility and activity-dependent work in compression-order evaluation.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Interpreting Compression-Order Effects in Spiking Neural Networks

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_PFSUOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

This paper studies pruning–quantization order effects in SNNs, mainly comparing P→Q and Q→P. It uses matched-state controls to separate BN/calibration history from differences in the final model construction, and further examines prediction disagreement, compression-budget feasibility, and modeled synaptic operations. Across several SNN settings and a ReLU control, the paper finds that some large order-dependent accuracy gaps can disappear after common recalibration, while different orderings may still produce different predictions or computational behavior even when their final accuracies are similar.

Critical strengths
  1. The paper includes a fairly careful set of control experiments, especially for separating BN/calibration history from changes in model construction.

  2. The empirical coverage is broad, and the authors are generally careful about the scope of their claims and limitations.

Critical weaknesses
  1. The paper does not establish a sufficiently clear, novel, or SNN-specific scientific contribution.
    While the general insights are understandable and potentially useful in practice, I am not convinced that the paper currently demonstrates a sufficiently new scientific finding. Many of the individual observations are fairly expected—for example, stale BN statistics can hurt performance, and similar accuracy does not necessarily imply similar predictions or identical model states. Combining these observations into a more systematic evaluation may be practically valuable, but the resulting contribution currently reads more like a diagnostic or best-practice framework than a substantive new result in SNN compression. This concern is further reinforced by the ReLU control: the core narrative regarding normalization history, calibration, and model states appears largely agnostic to spiking dynamics, and the ReLU experiments already demonstrate the main phenomenon outside the SNN setting. The SNN-specific experiments introduced later therefore feel more like establishing domain relevance than advancing or changing the central argument. The theoretical component does not substantially strengthen the novelty claim either. In particular, Proposition 1 appears almost immediate: if two orderings arrive at the same non-BN state zz and are then subjected to the same deterministic recalibration,
R(z,r;C)=(z,B(z;C)),R(z,r;C)=(z,B(z;C)),

they naturally produce the same final state. I find this useful as a control or clarification, but it does not seem to provide meaningful theoretical depth. Overall, the final takeaway—that compression-order studies should examine construction, calibration, predictions, feasibility, and work in addition to accuracy—is sensible advice, but it makes the contribution feel closer to an evaluation checklist or methodological guideline than a substantial scientific discovery. This is consequential to my assessment because I remain unclear about what the paper's central nontrivial advance is, and in particular what makes that advance specifically an SNN compression contribution rather than a more general observation about normalization, model state, and evaluation methodology.

  1. The paper lacks a sufficiently focused central research question and a clear hierarchy between the main claim, controls, and secondary analyses.
    The Abstract and Introduction currently read somewhat like a lab notebook: one issue is identified, then one control is added, followed by another issue and another analysis. This makes it difficult to determine early on what the paper's primary research question or hypothesis actually is, and the main contribution becomes buried under many details, controls, and side observations. The same problem extends to the overall argument: observations about construction order, BN calibration, prediction disagreement, feasibility, and computational work are all potentially relevant, but the paper does not clearly distinguish which of these constitutes the central scientific finding and which are supporting diagnostics or methodological recommendations. As a result, the reader has to reconstruct the main message from a sequence of increasingly refined analyses rather than seeing a clearly stated claim followed by evidence designed to test that claim. This is more than a presentation issue, because it directly affects how the scientific contribution can be evaluated: when the paper's main result, controls, and practical recommendations are not clearly separated, it becomes difficult to judge which evidence is essential to the central claim and whether that claim itself is sufficiently strong and novel. In its current form, the paper therefore comes across more as a careful diagnostic investigation of compression-order evaluation than as a sharply focused study built around a clearly identifiable scientific result.

Additional formatting note: The provided template also appears to have been compiled with XeLaTeX rather than pdfLaTeX.

Questions for authors
  1. What is the practical motivation for comparing P→Q and Q→P, particularly in the SNN setting?
    SNN pruning itself is already a relatively small topic, and I am not sufficiently familiar with the latest ANN pruning literature to judge how important this ordering question is there. From this paper alone, I still do not understand why Q→P is an important practical pipeline, especially under the no-intervening-training setting. A convincing answer here would affect my view of the significance of the problem.

  2. How should the importance of the reported "order effect" be interpreted given the very different magnitudes across settings?
    The very large gaps mainly appear in specially constructed cases and largely disappear after common recalibration, which makes them look more like stale-BN failures. In the main dynamic setting, the accuracy difference is very small. I would like to understand what practical or scientific significance the authors believe remains for compression ordering itself.

  3. What is the intended role of the threshold/SOP analysis in the main story?
    This is one of the few parts that feels genuinely SNN-specific, and I actually find it interesting because different orderings can change spike activity and modeled synaptic work. However, it appears relatively late and is not developed into a central result. I would like the authors to clarify whether this is meant to be a core SNN-specific finding or only an additional extension.

Reviewer_xYKKOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates how compression order affects spiking neural networks (SNNs) and attempts to explain the performance differences between prune-then-quantize and quantize-then-prune. This paper finds that the effectiveness of the compression order depends on the model-building and calibration processes. Significant differences in accuracy may stem from mismatched BN statistics. Small differences in accuracy may mask substantial variations in prediction.

Critical strengths

None

Critical weaknesses
  • This paper is poorly written. Neither the research motivation nor the contributions are clearly outlined. The introduction section is too brief and fails to explain why the compression order problem is worthy of study. This paper reads more like an experimental report than a well-developed research paper.

  • The conclusions apply only to a limited set of compression processes. This paper investigates compression without training or fine-tuning between operations, primarily employing channel pruning, weight quantization, and non-folding BN. It does not cover quantization-aware training, pruning while training, post-pruning recovery training, etc. Incorporating these steps would alter the weights, activations, and calibration methods. Therefore, this paper cannot directly determine the optimal order for these processes, nor can it rule out the advantages of the order used in other training-based compression methods.

  • The font used in this paper appears to differ from that in the ICLR template. The styles of the title and section headings also differ from those in the template. The authors may have used the wrong template or LaTeX compiler.

  • The title in the submitted PDF is different from the one in OpenReview. The title in OpenReview is "Compression order in spiking neural networks: normalization and model construction," while the one in the PDF is "Interpreting Compression-Order Effects in Spiking Neural Networks."

Questions for authors
  1. Please clarify the motivation and contribution of this paper.
  2. Please explain the scope of the paper's conclusions.
Reviewer_Niw9OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the effects of compression order in SNNs, focusing on channel pruning and weight quantization, with additional experiments on neuron-threshold scaling. The authors seek to disentangle the effects of BN calibration history from those arising from differences in model construction. In matched-state controls, they show that clipping-induced, order-dependent accuracy gaps disappear after common BN recalibration when the underlying model states are identical. In a prespecified comparison of CIFAR-100 ResNet-SNNs, they report a small mean absolute accuracy gap of 0.17 pp between pruning-then-quantization and quantization-then-pruning, despite a prediction disagreement rate of approximately 5.97%. Both orders satisfy a 5-percentage-point accuracy-loss budget, while the 10% physical channel-removal setting reduces convolutional MACs by 10.08%. Further experiments across different architectures, datasets, calibration procedures, and threshold-scaling settings examine the generality of these observations and their implications for prediction behavior, budget feasibility, and modeled synaptic operations.

Critical strengths
  1. The paper investigates how BN calibration history can confound comparisons of compression orders. Its matched-state controls help distinguish calibration-related effects from differences in model construction, highlighting a potentially useful consideration for evaluating compressed models.

  2. The authors conduct experiments across multiple architectures, datasets, quantization settings, and calibration procedures. Beyond aggregate accuracy, they examine prediction disagreement, accuracy-budget feasibility, and modeled computational costs. This provides a relatively comprehensive empirical characterization of compression-order effects under the studied settings.

Critical weaknesses
  1. Limited novelty and insufficient technical contributions. The paper does not introduce a new compression algorithm, a general framework for compression-order selection, or a substantive theoretical result. Instead, it primarily presents extensive empirical comparisons across compression orders, quantization settings, pruning configurations, and calibration procedures. Its observations overlap with existing work on BN recalibration for pruning evaluation and pruning–quantization ordering, which the paper itself cites. The theoretical proposition is also relatively straightforward, establishing that identical model states remain functionally identical after the same deterministic recalibration. More importantly, the empirical findings are not distilled into a principled or generalizable rule for choosing compression orders. The paper neither provides a method for predicting which order will perform better nor demonstrates an improved joint compression method. Consequently, its incremental technical contribution appears limited, making it difficult to identify a sufficiently substantial advance for ICLR.

  2. Limited practical utility beyond evaluating both candidate orders. In the primary setting, the pruning–quantization comparison considers only two possible orders: pruning followed by quantization and quantization followed by pruning. In practice, evaluating both pipelines and selecting one based on validation performance is a straightforward baseline. Since the paper does not propose a principled order-selection strategy, demonstrate a reliable way to predict the better order in advance, or show that its analysis reduces the cost of selecting an order, it remains unclear what practical advantage it offers over simply trying both alternatives. The additional experiments broaden the empirical coverage but do not resolve this central issue.

  3. Limited evidence for practically significant compression-order effects. The primary experiment reports a mean absolute accuracy gap of only 0.17 pp between the two compression orders, while the 95% confidence interval for the signed PQ–QP accuracy difference includes zero. Although the authors evaluate 12 independently trained and calibrated models and provide statistical analyses, the results do not establish a consistent accuracy advantage for either order. Furthermore, the run-level statistical analyses condition on a reused test set, while image-resampling analyses are presented only as descriptive sensitivity checks. The reported prediction disagreement of approximately 5.97% is potentially interesting, but the paper does not establish whether it has meaningful downstream consequences. Overall, the empirical results provide limited evidence for practically significant compression-order effects in the studied settings.

Questions for authors

Given existing work on BN recalibration for pruning evaluation and prior studies of pruning–quantization ordering, could the authors clarify what fundamentally new insight this work establishes? In particular, does the proposed analysis yield a generalizable principle for understanding or selecting compression orders, beyond demonstrating the effects of mismatched BN statistics in the studied pipelines?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.