- W1. Gradient-based relevance signal reliability is insufficiently justified. The core signal A=U×∂M/∂U relies on gradient computation, but SNNs typically use surrogate gradients to handle the non-differentiable spiking process, meaning the gradient signal is not the exact gradient of the original discrete network but an approximation [1][2]. Therefore, interpreting this signal directly as the true task contribution of internal neuronal representations lacks sufficient basis. The paper does not discuss or experimentally verify whether gradient estimation errors affect task relevance ranking and, consequently, the reliability of the entire threshold selection mechanism.
- W2. Mathematical definitions and theoretical assumptions are inconsistent. The task relevance defined in Equation (2) includes negative values, but subsequent normalization formulas and theoretical proofs (e.g., Proposition 1) assume non-negative weights that sum to 1, which are not necessarily consistent. This undermines the justification for the boundedness of spatiotemporal support and raises further questions about the mathematical validity of subsequent semantic retention, redundancy suppression, and threshold optimization objectives. The authors do not explain how negative gradient contributions are handled or prove that the actual computations satisfy the relevant theoretical assumptions.
- W3. Training costs may be underestimated, and efficiency claims lack sufficient support. The method involves additional task gradient extraction, pairwise similarity computation across spatiotemporal positions, and periodic threshold search, while SNN gradient backpropagation itself already incurs significant computational and memory overhead [3][4]. In particular, pairwise similarity computation has complexity that grows quadratically with the number of spatiotemporal positions. However, the paper primarily relies on small FLOPs differences to indicate computational efficiency, lacking real end-to-end training time, peak memory, and scaling experiments. Therefore, the authors have not sufficiently demonstrated that the additional gradient analysis and threshold calibration do not introduce significant practical training overhead.
- W4. Core innovation claims are exaggerated, and the actual mechanism does not fully align with the research motivation. The paper's title and motivation emphasize selecting which internal neuronal representations should be converted into spikes based on task relevance, but the actual mechanism only learns shared thresholds at the modality and stage levels, which cannot distinguish between neuronal responses with the same membrane potential but different task relevance. Therefore, the claimed fine-grained task-relevant representation selection is not directly realized in the actual spike generation process. The authors also do not sufficiently explain why the fine-grained relevance estimation is ultimately only used for global threshold calibration and how this design achieves the claimed selective spike generation mechanism.
- W5. The true source of performance improvement is unclear (mechanism vs. architecture contributions are confounded). Ablation experiments in the appendix show that adjusting only the Stage 3 threshold can achieve the same performance as the full multi-stage ST-MAT, indicating that the necessity of the complete threshold adaptation mechanism has not been sufficiently validated. Furthermore, the paper primarily conducts experiments based on the authors' designed QK/QK/QKV multimodal architecture, lacking fair validation on other standard SNN backbones. Therefore, it remains unclear whether the performance improvement comes from the proposed ST-MAT mechanism or is influenced by specific architecture, cross-modal attention placement, and training configurations, lacking clear experimental attribution.
- W6. Theoretical analysis lacks direct connection to the core task objective. The theoretical proofs in the appendix mainly focus on boundedness of support, algebraic consistency, and stability of threshold updates under fixed targets, all relying on specific assumptions, without establishing a clear link between the defined task relevance, redundancy metrics, and final classification performance. These properties only show that certain mathematical constructs hold under given conditions, but are insufficient to prove that the selected thresholds can retain more discriminative information or improve audio-visual fusion performance. Therefore, the existing theoretical analysis is insufficient to support the claimed core mechanism advantages.
- W7. Figures are unclear, and key implementation details lack transparency. The core architecture diagram is overly crowded, with multiple modules, data flows, and threshold update paths intertwined, making it difficult to clearly understand the actual training and inference processes. In particular, the extraction path of task gradients, the training method of the auxiliary classification branch, and the threshold search process lack intuitive explanation, and relevant visualizations do not provide sufficient quantitative evidence. Therefore, the core mechanism of the paper is difficult to accurately understand and independently reproduce.
References
[1] Neftci E. O., Mostafa H., Zenke F. Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks. IEEE Signal Processing Magazine, 36(6): 51–63, 2019. https://doi.org/10.1109/MSP.2019.2931595
[2] Gygax J., Zenke F. Elucidating the Theoretical Underpinnings of Surrogate Gradient Learning in Spiking Neural Networks. Neural Computation, 37(5): 886–925, 2025. https://doi.org/10.1162/neco_a_01752
[3] Yu C., Liu L., Wang G., et al. Advancing Training Efficiency of Deep Spiking Neural Networks through Rate-based Backpropagation. NeurIPS, 2024. https://proceedings.neurips.cc/paper_files/paper/2024/hash/d1bdc488ec18f64177b2275a03984683-Abstract-Conference.html
[4] Jiang H., De Masi G., Xiong H., Gu B. NDOT: Neuronal Dynamics-based Online Training for Spiking Neural Networks. ICML, 2024. https://proceedings.mlr.press/v235/jiang24a.html
Task-Aware Spike Selection and Shared Threshold Optimization
Thank you for your constructive feedback. We expanded the method and theory in Sections 4.1–4.3 and Appendix A, clarified Figure 2, and added controlled results for threshold granularity (main-text Table 3), auxiliary heads (Appendix G), calibration (H), and response selection (K).
Weakness 1 & Question 1
ST-MAT deploys a single scalar θm,l per modality–stage pair, shared across time, tokens, and channels: Sm,l,b,t,n,d=H(Um,l,b,t,n,d−θm,l). It does not assign distinct spikes to identical membrane potentials through position-wise gates. Instead, training-time TRSSE computes Ai,d=∣Ui,d∂M/∂Ui,d∣ and characterizes effective task-relevant spatiotemporal support. SRB constructs semantic mass p+ and suppressible-redundancy mass p− from the task-aware evidence and selects an observed cutoff θ∗ maximizing Rsem(θ)αRred(θ)1−α, where Rsem=∑p+1[U≥θ] and Rred=∑p−1[U<θ]. SBTA updates the deployed scalar toward that target with bounded steps. Fine-grained evidence determines a shared conversion boundary, not per-position inference thresholds.
Table A10 (Appendix K) confirms the intended selectivity. In Stage 3's highest task-relevance quintile (F5), semantic retention increases from 41.85% to 57.63% for audio and 39.14% to 57.92% for video (+15.78%/+18.78%). For the highest suppressible-redundancy quintile, rejection increases from 83.98% to 86.79% and 85.17% to 87.89% (+2.81%/+2.72%). Corresponding high-relevance firing rates are 54.53%/55.16% with ST-MAT versus 38.67%/36.84% for fixed thresholds; high-redundancy firing rates fall to 13.10%/12.00% versus 15.81%/14.47%. These results demonstrate preferential preservation of discriminative responses and suppression of repetitive spikes. Crucially, neuron-wise threshold adaptation attains 76.73%, below ST-MAT's 78.47% on AVE (main-text Table 3), indicating that coherent population-level selection matters more than threshold granularity alone.
Weakness 1 & Question 2
In Figure 3(d), θ∗ denotes an intermediate target threshold.
Weakness 1 & Question 3
The calibration step uses activations obtained before the weight update, without recomputing a forward pass.