- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Classifier-Evidence Token Selection (CETS), a training-free approach to reducing redundant tokens in spiking transformers. The method reuses a frozen classifier to estimate token-wise uncertainty through Dirichlet evidence and combines temporal mean and variation for token ranking. Experiments on static and neuromorphic vision datasets show promising accuracy retention under moderate token pruning, although the effectiveness varies across datasets and backbone architectures.
Critical strengths
- The proposed method is simple and practical, requiring neither additional training nor auxiliary predictors. Reusing classifier information for token selection is an interesting direction.
- The paper provides useful analyses of different ranking signals, temporal aggregation strategies, and backbone architectures. The discussion of the method’s limitations is also appreciated.
Critical weaknesses
- The practical efficiency benefits remain unclear. As described in Appendix A.2, the current implementation requires an additional unpruned forward pass to obtain token scores. The reported operation reductions only concern the target attention module and exclude this overhead. Consequently, it is difficult to determine whether CETS provides actual end-to-end computational savings.
- The advantage over simpler feature-based criteria is not sufficiently established. Table 3 shows that feature norm provides comparable or even stronger ranking separation, while its correlation with classifier evidence reaches 0.914 on QKFormer. A direct comparison of classification accuracy and computational overhead against feature-norm-based selection would help clarify the added value of CETS.
- The effectiveness appears somewhat dependent on the backbone and dataset. The ranking signal becomes considerably less discriminative on MaxFormer, and the improvements from temporal variation are not consistent across datasets. These observations raise some questions about the general applicability of the proposed approach.
Questions for authors
- Could the authors provide end-to-end inference latency measurements that include the additional scoring pass? This would help establish the practical efficiency benefits.
- How does CETS compare with simple feature-norm-based token selection under identical retention ratios, particularly when accounting for computational overhead?
- Could the authors further explain why classifier-derived evidence becomes less effective on MaxFormer? Understanding this behavior may help identify when CETS is most useful.
Response to Reviewer DbUw
Thank you for recognizing the simplicity of reusing the frozen classifier. We focus below on practical inference cost, the direct FN comparison and the weaker MaxFormer results.
1. Complete inference latency, including score extraction
To measure the complete end-to-end cost, we implemented an equivalent single-pass version of CETS that computes evidence scores online without a separate dense recapture pass. In the original backbone, selection is applied to a late SSA computing
Q(KᵀV), which limits the remaining computation available to offset the scoring overhead. We therefore additionally evaluated frozen SpikingResformer-Ti, whose channel-compatible third-stage DSSAs allow KV selection to reduce both attention matrix multiplications and the intervening attention-neuron computation. Timing includes evidence scoring, temporal aggregation, sorting, token gathering, state reset, and complete inference; data loading and host-to-device transfer are excluded.Whole-model latency
Measurements use an RTX 4090, batch size 64, T=4, AMP, six warm-up iterations, and 40 interleaved repetitions. One ranking is shared across the three Stage-3 DSSAs. CETS reduces end-to-end latency by 1.18% and 2.62% at 60% and 40% retention, respectively.
2. Direct comparison with feature-norm selection
Values are full-test Acc@1 (%). A/B/C denote three checkpoints within each dataset. Methods share the checkpoint, selection site, operator and retention ratio within each row.
Averaged across checkpoints, CETS achieves 80.95%/80.52% versus FN's 80.96%/80.59% on CIFAR-100, and 81.20%/75.37% versus 80.83%/75.23% on DVS-CIFAR10. Individual DVS gains include 81.20% versus 80.00% at 80% retention for A and 76.90% versus 75.90% at 60% for C, although gains vary across checkpoints and retention ratios. The latency table accounts for CETS's additional cost.
These comparisons support classifier-derived evidence as a useful training-free selection signal, while showing that its accuracy benefits over FN are setting dependent and come with additional execution cost.
3. Why is classifier evidence less effective on MaxFormer?
Our diagnostics point to weaker subset-selection discrimination in the tested MaxFormer representation. At 80%/60% retention, retaining high-score tokens achieves 81.68%/80.45%, while retaining low-score tokens gives 81.51%/80.34%. These near-identical results indicate that simply reversing the ranking does not resolve the issue.
A plausible explanation is that the frozen classifier, trained on pooled features, produces less useful token-level ordering on this intermediate representation. The relationship between CETS and feature magnitude also changes markedly across backbones:
Spearman compares the final CETS score S=μ(U)+0.9σ(U) with FN within each image, then averages. Overlap is the intersection size divided by the number of selected tokens, not IoU.
These results indicate that classifier-derived scoring depends on the target representation. We will make this dependence explicit when discussing its applicability across backbones. We will also clarify that temporal variation is a dataset-dependent refinement.