- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
This paper studies pruning–quantization order effects in SNNs, mainly comparing P→Q and Q→P. It uses matched-state controls to separate BN/calibration history from differences in the final model construction, and further examines prediction disagreement, compression-budget feasibility, and modeled synaptic operations. Across several SNN settings and a ReLU control, the paper finds that some large order-dependent accuracy gaps can disappear after common recalibration, while different orderings may still produce different predictions or computational behavior even when their final accuracies are similar.
Critical strengths
-
The paper includes a fairly careful set of control experiments, especially for separating BN/calibration history from changes in model construction.
-
The empirical coverage is broad, and the authors are generally careful about the scope of their claims and limitations.
Critical weaknesses
- The paper does not establish a sufficiently clear, novel, or SNN-specific scientific contribution.
While the general insights are understandable and potentially useful in practice, I am not convinced that the paper currently demonstrates a sufficiently new scientific finding. Many of the individual observations are fairly expected—for example, stale BN statistics can hurt performance, and similar accuracy does not necessarily imply similar predictions or identical model states. Combining these observations into a more systematic evaluation may be practically valuable, but the resulting contribution currently reads more like a diagnostic or best-practice framework than a substantive new result in SNN compression. This concern is further reinforced by the ReLU control: the core narrative regarding normalization history, calibration, and model states appears largely agnostic to spiking dynamics, and the ReLU experiments already demonstrate the main phenomenon outside the SNN setting. The SNN-specific experiments introduced later therefore feel more like establishing domain relevance than advancing or changing the central argument. The theoretical component does not substantially strengthen the novelty claim either. In particular, Proposition 1 appears almost immediate: if two orderings arrive at the same non-BN state and are then subjected to the same deterministic recalibration,
they naturally produce the same final state. I find this useful as a control or clarification, but it does not seem to provide meaningful theoretical depth. Overall, the final takeaway—that compression-order studies should examine construction, calibration, predictions, feasibility, and work in addition to accuracy—is sensible advice, but it makes the contribution feel closer to an evaluation checklist or methodological guideline than a substantial scientific discovery. This is consequential to my assessment because I remain unclear about what the paper's central nontrivial advance is, and in particular what makes that advance specifically an SNN compression contribution rather than a more general observation about normalization, model state, and evaluation methodology.
- The paper lacks a sufficiently focused central research question and a clear hierarchy between the main claim, controls, and secondary analyses.
The Abstract and Introduction currently read somewhat like a lab notebook: one issue is identified, then one control is added, followed by another issue and another analysis. This makes it difficult to determine early on what the paper's primary research question or hypothesis actually is, and the main contribution becomes buried under many details, controls, and side observations. The same problem extends to the overall argument: observations about construction order, BN calibration, prediction disagreement, feasibility, and computational work are all potentially relevant, but the paper does not clearly distinguish which of these constitutes the central scientific finding and which are supporting diagnostics or methodological recommendations. As a result, the reader has to reconstruct the main message from a sequence of increasingly refined analyses rather than seeing a clearly stated claim followed by evidence designed to test that claim. This is more than a presentation issue, because it directly affects how the scientific contribution can be evaluated: when the paper's main result, controls, and practical recommendations are not clearly separated, it becomes difficult to judge which evidence is essential to the central claim and whether that claim itself is sufficiently strong and novel. In its current form, the paper therefore comes across more as a careful diagnostic investigation of compression-order evaluation than as a sharply focused study built around a clearly identifiable scientific result.
Additional formatting note: The provided template also appears to have been compiled with XeLaTeX rather than pdfLaTeX.
Questions for authors
-
What is the practical motivation for comparing P→Q and Q→P, particularly in the SNN setting?
SNN pruning itself is already a relatively small topic, and I am not sufficiently familiar with the latest ANN pruning literature to judge how important this ordering question is there. From this paper alone, I still do not understand why Q→P is an important practical pipeline, especially under the no-intervening-training setting. A convincing answer here would affect my view of the significance of the problem. -
How should the importance of the reported "order effect" be interpreted given the very different magnitudes across settings?
The very large gaps mainly appear in specially constructed cases and largely disappear after common recalibration, which makes them look more like stale-BN failures. In the main dynamic setting, the accuracy difference is very small. I would like to understand what practical or scientific significance the authors believe remains for compression ordering itself. -
What is the intended role of the threshold/SOP analysis in the main story?
This is one of the few parts that feels genuinely SNN-specific, and I actually find it interesting because different orderings can change spike activity and modeled synaptic work. However, it appears relatively late and is not developed into a central result. I would like the authors to clarify whether this is meant to be a core SNN-specific finding or only an additional extension.