acceptodds
Under review as a conference paper at ICLR 2027

Classifier Evidence Guided Token Selection for Spiking Transformers

Abstract

Spiking transformers achieve strong performance in neuromorphic vision, but multi-step token processing still introduces substantial redundancy and inference cost. Existing token reduction methods mainly rely on response-driven cues and do not explicitly exploit the temporal class evidence encoded by the frozen classifier. Since token utility is ultimately reflected in downstream prediction, we investigate the classifier as an additional source of ranking information. We observe that different tokens exhibit distinct uncertainty trajectories across spiking steps, suggesting that temporally aggregated classifier evidence can provide a complementary signal for token selection. Based on this observation, we propose CETS (Classifier-Evidence Token Selection), a training-free and plug-and-play framework that models token-wise class evidence with a Dirichlet distribution and aggregates its temporal uncertainty statistics into an inference-time ranking score. CETS requires no auxiliary predictor or parameter update and can be directly applied to token reduction. Experiments on static and neuromorphic benchmarks demonstrate favorable accuracy-efficiency trade-offs under controlled retention ratios. Further analyses show that the discriminative strength of classifier-derived uncertainty varies across representations, providing empirical insight into when classifier-side evidence is informative for token selection.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Spiking Transformers, Token Selection, Classifier Evidence, Temporal Uncertainty, Training-Free Inference

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_DbUwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Classifier-Evidence Token Selection (CETS), a training-free approach to reducing redundant tokens in spiking transformers. The method reuses a frozen classifier to estimate token-wise uncertainty through Dirichlet evidence and combines temporal mean and variation for token ranking. Experiments on static and neuromorphic vision datasets show promising accuracy retention under moderate token pruning, although the effectiveness varies across datasets and backbone architectures.

Critical strengths
  1. The proposed method is simple and practical, requiring neither additional training nor auxiliary predictors. Reusing classifier information for token selection is an interesting direction.
  2. The paper provides useful analyses of different ranking signals, temporal aggregation strategies, and backbone architectures. The discussion of the method’s limitations is also appreciated.
Critical weaknesses
  1. The practical efficiency benefits remain unclear. As described in Appendix A.2, the current implementation requires an additional unpruned forward pass to obtain token scores. The reported operation reductions only concern the target attention module and exclude this overhead. Consequently, it is difficult to determine whether CETS provides actual end-to-end computational savings.
  2. The advantage over simpler feature-based criteria is not sufficiently established. Table 3 shows that feature norm provides comparable or even stronger ranking separation, while its correlation with classifier evidence reaches 0.914 on QKFormer. A direct comparison of classification accuracy and computational overhead against feature-norm-based selection would help clarify the added value of CETS.
  3. The effectiveness appears somewhat dependent on the backbone and dataset. The ranking signal becomes considerably less discriminative on MaxFormer, and the improvements from temporal variation are not consistent across datasets. These observations raise some questions about the general applicability of the proposed approach.
Questions for authors
  1. Could the authors provide end-to-end inference latency measurements that include the additional scoring pass? This would help establish the practical efficiency benefits.
  2. How does CETS compare with simple feature-norm-based token selection under identical retention ratios, particularly when accounting for computational overhead?
  3. Could the authors further explain why classifier-derived evidence becomes less effective on MaxFormer? Understanding this behavior may help identify when CETS is most useful.
1 comment
AuthorsOpenReview ↗

Response to Reviewer DbUw

Thank you for recognizing the simplicity of reusing the frozen classifier. We focus below on practical inference cost, the direct FN comparison and the weaker MaxFormer results.

1. Complete inference latency, including score extraction

To measure the complete end-to-end cost, we implemented an equivalent single-pass version of CETS that computes evidence scores online without a separate dense recapture pass. In the original backbone, selection is applied to a late SSA computing Q(KᵀV), which limits the remaining computation available to offset the scoring overhead. We therefore additionally evaluated frozen SpikingResformer-Ti, whose channel-compatible third-stage DSSAs allow KV selection to reduce both attention matrix multiplications and the intervening attention-neuron computation. Timing includes evidence scoring, temporal aggregation, sorting, token gathering, state reset, and complete inference; data loading and host-to-device transfer are excluded.

Whole-model latency

RetentionDense (ms/batch)CETS (ms/batch)Speedup
60%150.482148.7121.012×
40%150.482146.5431.027×

Measurements use an RTX 4090, batch size 64, T=4, AMP, six warm-up iterations, and 40 interleaved repetitions. One ranking is shared across the three Stage-3 DSSAs. CETS reduces end-to-end latency by 1.18% and 2.62% at 60% and 40% retention, respectively.

2. Direct comparison with feature-norm selection

Dataset / checkpointCETS: 80% / 60%FN: 80% / 60%
CIFAR-100 / A80.93 / 80.6280.94 / 80.61
CIFAR-100 / B81.06 / 80.5580.94 / 80.57
CIFAR-100 / C80.86 / 80.4081.00 / 80.59
DVS-CIFAR10 / A81.20 / 74.1080.00 / 74.40
DVS-CIFAR10 / B80.80 / 75.1080.70 / 75.40
DVS-CIFAR10 / C81.60 / 76.9081.80 / 75.90

Values are full-test Acc@1 (%). A/B/C denote three checkpoints within each dataset. Methods share the checkpoint, selection site, operator and retention ratio within each row.

Averaged across checkpoints, CETS achieves 80.95%/80.52% versus FN's 80.96%/80.59% on CIFAR-100, and 81.20%/75.37% versus 80.83%/75.23% on DVS-CIFAR10. Individual DVS gains include 81.20% versus 80.00% at 80% retention for A and 76.90% versus 75.90% at 60% for C, although gains vary across checkpoints and retention ratios. The latency table accounts for CETS's additional cost.

These comparisons support classifier-derived evidence as a useful training-free selection signal, while showing that its accuracy benefits over FN are setting dependent and come with additional execution cost.

3. Why is classifier evidence less effective on MaxFormer?

Our diagnostics point to weaker subset-selection discrimination in the tested MaxFormer representation. At 80%/60% retention, retaining high-score tokens achieves 81.68%/80.45%, while retaining low-score tokens gives 81.51%/80.34%. These near-identical results indicate that simply reversing the ranking does not resolve the issue.

A plausible explanation is that the frozen classifier, trained on pooled features, produces less useful token-level ordering on this intermediate representation. The relationship between CETS and feature magnitude also changes markedly across backbones:

CIFAR-100 backboneMean within-image Spearman: CETS vs FNTop-20% / Top-40% overlap
QKFormer+0.91475.54% / 84.82%
MaxFormer−0.2677.43% / 26.61%

Spearman compares the final CETS score S=μ(U)+0.9σ(U) with FN within each image, then averages. Overlap is the intersection size divided by the number of selected tokens, not IoU.

These results indicate that classifier-derived scoring depends on the target representation. We will make this dependence explicit when discussing its applicability across backbones. We will also clarify that temporal variation is a dataset-dependent refinement.

Reviewer_HRy7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CETS, a training-free, plug-and-play token-selection framework for spiking transformers. The key idea is to reuse the frozen classification head as an additional ranking signal: token-wise class logits are converted to non-negative evidence via Softplus\mathrm{Softplus}, mapped to Dirichlet parameters (α=e+1\alpha = e + 1), and summarized into a per-step uncertainty U=C/∑αU = C/\sum\alpha. The temporal trajectory of this uncertainty across simulation steps is aggregated as Si=μi+0.9σiS_i = \mu_i + 0.9\sigma_i, and the top-KK spatial tokens under this score are retained (with a shared mask across all simulation steps) before the stage-3.1, i.e., spike-driven self-attention of QKFormer; a cross-backbone study with MaxFormer is also provided. Validation of CETS covers CIFAR-10/100, DVS-CIFAR-10 and DVS-Gesture, with additional analyses of ranking sources (trained vs. channel-perm. vs. norm-matched random heads), softmax-based uncertainty alternatives, backbone dependence of the evidence signal, and the contribution of the temporal-variation term. The method requires no auxiliary predictor and no parameter updates.

Critical strengths
  1. Thorough mechanistic analysis. The controls go well beyond what token-pruning papers typically provide: a norm-matched random-head control (which reverses the ranking), a channel-permutation control (which largely preserves it), comparisons against softmax-based uncertainty signals, a cross-backbone study (QKFormer vs. MaxFormer), seed-level robustness over three checkpoints, and a drop-vs-merge study. These analyses give the reader a clear picture of when the proposed signal works.
  2. Relatively clear positioning. The Discussion openly states that the criterion is not a direct measure of token importance and that two-pass operation counts do not establish end-to-end acceleration.
  3. Simple to deploy. Reusing the frozen classifier head requires no auxiliary selector, no retraining, and integrates directly with existing pretrained checkpoints; the temporal aggregation rule and its λ\lambda-sensitivity are simple and reproducible.
Critical weaknesses
  1. CETS's evidence signal may be a redundant and weakened variant of the almost-free feature norm, which perhaps undermines the paper's central claim. The High-minus-Low Acc@1 gaps (henceforth hl-acc gaps) are the ranking-utility metric defined by the authors, and their magnitude reflects the utility of the evidence signal. In Table 4, the classifier-evidence hl-acc gaps on QKFormer (80%/60% retention: +7.23/+12.12) are strictly smaller than those of the trivially computable temporal-RMS feature norm (+8.03/+12.52); moreover, the two rankings agree with Spearman ρ=0.914\rho=0.914 and a top-set overlap of 75~85%. These results show that on QKFormer the two are almost the same signal or, put differently, that the CETS evidence signal is a degraded approximation of the feature norm: they are highly similar and non-complementary. Going further, the Channel-perm. results in Table 3 (+7.53/+12.15) are essentially consistent with the default setting (unchanged up to randomness). Taken together, these two experiments indicate that what drives the ranking is the norm-like property of the tokens rather than class-semantic evidence, because the point of the Channel-perm. control is precisely to observe the utility of the ranking signal when the semantic structure has been destroyed. Does this mean the frozen classifier head effectively serves only as a norm calculator with slightly degraded performance? Given that the paper neither runs the only experiment that could demonstrate complementarity (a norm + CETS combination score vs. the norm alone) nor compares CETS head-to-head against feature-norm selection accuracy in the main tables, practitioners cannot find any reason in this paper to use this logits-dependent classifier machinery instead of the free norm computation.
  2. CETS's utility is inconsistent and sometimes negative, and the efficiency comparison is incomplete. In Table 1, at 40% retention on DVS-CIFAR-10, CETS falls below random selection (55.40 vs. 60.60±0.66); the Temporal-variance criterion also outperforms CETS at several retention settings; and in Table 5, on DVS-Gesture the σ\sigma term actually degrades performance (−0.34/−1.39). Meanwhile, all reported MACs describe only the forward pass of the pruned SSA, while the computation overhead of evidence computation, temporal aggregation of the evidence, and Top-K selection is not reflected at all. Such an efficiency comparison is unfair. Note that the core contribution of this paper is efficient inference, so this metric comparison must be complete enough. Moreover, ranking involves many indexing operations, whose computational logic differs from that of normal multiply-accumulate operations and whose required computational resources also differ; at a minimum, these deserve an analysis.
Questions for authors
  1. After channel-permuting the input of the trained classifier, the h-l Acc gaps barely change, whereas the norm-matched random head reverses the effect of the ranking. In view of this phenomenon, the authors need to answer: what does the trained head contribute beyond a norm-like computation? Could you report the Spearman correlation and Top-K overlap between CETS scores and feature-norm scores for all datasets and both backbones (currently only CIFAR-100 with QKFormer/MaxFormer)?
  2. Given the near-redundancy with the feature norm on QKFormer, what reason would a practitioner have to choose CETS over the simple temporal-RMS norm (which requires no classifier projection and no evidence computation)? Please provide a direct accuracy comparison of feature-norm selection against CETS in the main table setting.
  3. The efficiency analysis excludes the scoring pipeline by design (Appendix A.2 explicitly states that these counts "exclude the dense capture pass, computation outside the target SSA, and scoring and Top-K overhead"), yet the computational cost of token ranking has never been quantified. Could you provide end-to-end latency and energy measurements that include the dense capture pass and the complete ranking computation: the frozen-head projection over all N×TN\times T tokens, the Softplus/Dirichlet evidence, temporal aggregation, and Top-K selection, and express this ranking overhead as a fraction of the theoretical SSA savings reported in Table 6, so that the net accuracy-efficiency trade-off can be assessed in absolute terms?
  4. The uncertainty U=C/∑αU=C/\sum\alpha is the vacuity measure in evidential deep learning (Sensoy et al., 2018), yet the paper cites only Shafer (1976). More importantly, the u=K/Su=K/S in EDL has a calibrated meaning only when the class evidence genuinely follows a Dirichlet distribution; however, the frozen head here was trained on pooled features and has never been calibrated on per-token inputs, so the per-token "evidence" does not strictly carry the EDL statistical meaning. As the authors themselves state, it is merely a "convenient parameterization". Then, in this setting, what does UU mean statistically? Could you cite the EDL literature?
  5. The symbol UU denotes the LIF membrane potential in Eqs. (1)–(3), while from Sec. 4.2 onward it denotes the uncertainty score. This increases the reading cost from the outset; a replacement is suggested, if the Uncertainty semantics is to be kept, a calligraphic U could be used.
  6. Figure 4 shows a single token over three consecutive steps, which has no statistical meaning; moreover, as the input varies over time, it is only natural that other related metrics change accordingly. So what this figure is meant to convey is unclear, and no information gain is evident.
  7. Figure 2 contains an interesting finding: retaining high-uncertainty tokens consistently outperforms the control of retaining low-uncertainty tokens. For this somewhat counterintuitive finding, the paper offers no mechanistic explanation: why are the tokens that the classifier is least certain about precisely the important ones?
  8. Please define the drop operator completely: what is the attention-branch output at the unselected positions (zero, identity, or original features preserved via the residual path)? The current description ("scattered back to their original spatial locations to restore the dense layout") does not define what fills the dropped positions, and this affects the interpretation of the accuracy-efficiency trade-off.

3 comments
AuthorsOpenReview ↗

Response to Reviewer HRy7

Thank you for the detailed feedback.

Q1. What does the classifier contribute beyond magnitude?

Input-channel permutation largely preserves the high–low accuracy gap, whereas matched random heads reverse it. These controls show that the learned projection affects ranking behavior, although they do not establish precise class-semantic alignment as its source.

Coverage: all four datasets on QKFormer, plus MaxFormer on CIFAR-100.

SettingSpearmanTop-20%/40% overlap
CIFAR-100/QKFormer+0.91475.54%/84.82%
CIFAR-100/MaxFormer−0.2677.43%/26.61%
CIFAR-10/QKFormer+0.61849.16%/66.42%
Gesture/QKFormer+0.23828.85%/43.72%
DVS-CIFAR10/A+0.88570.37%/81.90%
DVS-CIFAR10/B+0.86266.82%/79.96%
DVS-CIFAR10/C+0.86769.94%/80.81%

Spearman compares final CETS scores S=μ(U)+0.9σ(U) with FN per image, then averages. Overlap is the intersection size divided by the number of selected tokens, not IoU.

Q2. Direct FN comparison and practical added value

Dataset/checkpointCETS:80%/60%FN:80%/60%FN+CETS:80%/60%
CIFAR-100/A80.93/80.6280.94/80.6181.00/80.51
CIFAR-100/B81.06/80.5580.94/80.5781.01/80.63
CIFAR-100/C80.86/80.4081.00/80.5980.89/80.42
DVS-CIFAR10/A81.20/74.1080.00/74.4080.80/73.40
DVS-CIFAR10/B80.80/75.1080.70/75.4081.00/75.60
DVS-CIFAR10/C81.60/76.9081.80/75.9081.90/75.90

Full-test Acc@1 (%). A/B/C are checkpoints per dataset. Methods share checkpoint, site, operator and retention ratio per row. Fusion sums within-image standardized scores with equal weights.

CETS closely matches FN on CIFAR-100 and provides setting-specific DVS gains. For your requested fusion control, DVS checkpoint B gives 81.00%/75.60% for FN+CETS, versus 80.70%/75.40% for FN and 80.80%/75.10% for CETS. At 80%/60% retention, CIFAR-10 CETS/FN is 96.21/96.10 versus 96.26/96.17. Gesture is 96.88/94.79 versus 98.26/97.92. We will include direct FN comparisons in the main results.

Q3. Inference latency and selection overhead

Equivalent single-pass execution removes dense recapture without changing the scoring rule or operator. The original backbone computes Q(KᵀV), linear in token count at fixed channel dimensions. Late selection limits savings available to offset scoring overhead.

We additionally evaluated frozen SpikingResformer-Ti with CETS at channel-compatible third-stage DSSAs. KV selection reduces both attention matmuls and intervening attention-neuron computation.

Whole model

RetentionDense ms/batchCETS ms/batchSpeedup
60%150.482148.7121.012×
40%150.482146.5431.027×

Target DSSA at 20% retention

Stage 3 DSSADense msCETS msSpeedup
Block 110.3907.8131.330×
Block 210.2867.7791.322×
Block 39.7539.1021.071×

All measurements use an RTX 4090 with batch size 64, T=4, and AMP, comparing dense and pruned execution under matched conditions and including online scoring, sorting, and gathering overhead. Here the full model inference shares one ranking across the three target DSSAs. These results indicate considerable speedups of the attention module and the whole model.

Q4. Statistical meaning and EDL reference

We will cite Sensoy et al. (NeurIPS 2018). Here α=1+softplus(logits), U=C/Σα parameterize inverse response strength for ranking. The head is not evidentially trained or calibrated on tokens. We claim neither calibrated uncertainty nor a validated Dirichlet posterior. Controls assess ranking utility.

Q5. Notation

We will use V for voltage and calligraphic U for uncertainty.

Q6. Figure 4

We will clarify that Figure 4 is illustrative. Controlled test-set analyses support quantitative claims.

Q7. Why retain high-score tokens?

Low evidence on an individual token does not establish dispensability in joint inference. Across three CIFAR-100 checkpoints, high-score retention achieves 80.95%/80.52%, versus 74.01%/68.83% for low-score retention. At 80%/60% retention, these results support high-score over low-score selection under the tested intervention. They do not establish superiority over FN or isolated causal importance. DVS at 40% is below Random (55.40% versus 60.60%), and Gesture favors mean-only temporal scoring. We will qualify these boundaries.

Q8. Drop operator

For retained indices MM:

x′=x+ScatterM(SSA(GatherM(x))),x′′=x′+MLP(x′).x'=x+\mathrm{Scatter}_M(\mathrm{SSA}(\mathrm{Gather}_M(x))),\quad x''=x'+\mathrm{MLP}(x').

Scatter zero-fills unselected updates. The residual preserves features, and the MLP uses the dense layout.

Reviewer_HRy7OpenReview ↗

Request for more analyses of Q6 and Q7

Thanks for the detailed rebuttal. In brief: Q1–Q5 and Q8 are essentially addressed to my satisfaction, i.e., the full Spearman/overlap coverage (Q1), the direct FN and fusion comparisons (Q2), the single-pass latency measurements (Q3), the Sensoy et al. citation and the explicit disclaimer on calibration (Q4), the notation fix (Q5), and the precise Scatter/Gather definition of the drop operator (Q8). Two questions remain insufficiently answered: Q6 (Figure 4 is only relabeled as illustrative, with no statistical support added) and Q7 (the rationale for why high-uncertainty tokens are important is asserted rather than analyzed). I ask that the revision add quantitative analyses for these two points. My rating of weak acceptance stands.

Thank you for your patience and for clarifying the remaining concerns.

Q6: We performed quantitative analyses on CIFAR-100 dataset. Despite identical static inputs across four simulation steps, token-wise uncertainty rankings vary over time. Mean within-image Spearman correlations between adjacent steps are 0.630, 0.646 and 0.671, with image-bootstrap 95% confidence intervals of [0.628, 0.633], [0.643, 0.648] and [0.668, 0.673], respectively. Adding the temporal-standard-deviation term to mean-only scoring replaces, on average, 4.02%/6.99% of the retained tokens at 80%/60% retention.

Q7: Low classifier evidence at an individual token does not necessarily imply that it is dispensable as attention context. We examined this distinction on the full CIFAR-100 by selecting only K/V contributions while preserving all query positions, output positions, residual features and MLP computation. At 80%/60% retention, high-score context selection achieves 80.79%/80.57% Acc@1, compared with 73.43%/66.27% for low-score selection. High-score subsets also better preserve the complete KᵀV aggregation factor, with relative Frobenius errors of 0.167/0.310, versus 0.509/0.723 for low-score subsets.

These results support attention-context preservation as an explanation for the ranking’s usefulness in this setting. The uncertainty denotes inverse aggregate evidence strength.

Reviewer_dFPAOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes CETS, a training-free token selection method for Spiking Transformers. It uses a frozen classifier to estimate token-wise Dirichlet uncertainty and combines temporal statistics to rank tokens. Experiments on static and neuromorphic datasets examine its effectiveness under different token retention ratios.

Critical strengths

The use of classifier-side evidence for token ranking is interesting and requires no additional training. The paper provides useful comparisons of uncertainty measures, classifier projections, and different backbones. The experiments also identify situations where the proposed ranking is less effective.

Critical weaknesses
  1. The ranking signal is strongly correlated with feature norm, and its advantage over this simpler criterion is not fully established.

  2. The reported computational savings exclude the dense capture pass and scoring overhead, leaving the end-to-end efficiency benefit unclear.

  3. Temporal variation provides only small gains on CIFAR-100 and can reduce accuracy on DVS-Gesture.

  4. The method is less effective under aggressive token reduction, particularly on DVS-CIFAR-10.

Questions for authors
  1. Table 4 shows a Spearman correlation of 0.914 between classifier evidence and feature norm on QKFormer; could the authors clarify what additional ranking information CETS provides beyond feature magnitude?

  2. Appendix A.2 excludes the dense capture pass and scoring overhead from the operation counts; could the authors clarify the practical efficiency implications of the two-pass procedure?

  3. Table 5 shows that the temporal standard-deviation term can reduce accuracy on DVS-Gesture; could the authors explain the choice of a fixed λ = 0.9 across datasets?

2 comments
AuthorsOpenReview ↗

Response to Reviewer dFPA

Thank you for highlighting the training-free classifier-evidence formulation and its diagnostic analyses. We address your concerns about ranking information, evaluation cost and temporal aggregation below.

Classifier-derived scoring versus feature magnitude

FN measures token magnitude, whereas CETS evaluates tokens through the frozen, task-trained classifier projection and aggregates inverse evidence strength over time. We assess this classifier-conditioned criterion through matched comparisons:

Dataset / checkpointCETS: 80% / 60%FN: 80% / 60%FN+CETS: 80% / 60%
CIFAR-100 / A80.93 / 80.6280.94 / 80.6181.00 / 80.51
CIFAR-100 / B81.06 / 80.5580.94 / 80.5781.01 / 80.63
CIFAR-100 / C80.86 / 80.4081.00 / 80.5980.89 / 80.42
DVS-CIFAR10 / A81.20 / 74.1080.00 / 74.4080.80 / 73.40
DVS-CIFAR10 / B80.80 / 75.1080.70 / 75.4081.00 / 75.60
DVS-CIFAR10 / C81.60 / 76.9081.80 / 75.9081.90 / 75.90

Values are full-test Acc@1 (%). A/B/C denote three pretrained checkpoints within each dataset. Within each row, checkpoint, selection site, operator and retention ratio are matched. Fusion sums the within-image standardized FN and CETS scores with equal weights.

CETS closely matches FN on CIFAR-100 and achieves setting-specific DVS gains: 81.20% vs. 80.00% at 80% retention for A, and 76.90% vs. 75.90% at 60% for C. Fusion improves DVS checkpoint B at both retention ratios, achieving 81.00%/75.60% compared with 80.80%/75.10% for CETS and 80.70%/75.40% for FN.

Separately, across three CIFAR-100 checkpoints, CETS achieves 80.95±0.10% / 80.52±0.11%, exceeding Random (80.29±0.05% / 78.87±0.08%) and low-score retention (74.01±0.62% / 68.83±0.78%). Values are mean±sample SD across checkpoints. Random first averages five mask realizations within each checkpoint. These controls establish useful ordering, while the FN table assesses relative benefit.

On CIFAR-100, mean within-image Spearman correlation with FN changes from +0.914 on QKFormer to −0.267 on MaxFormer. Matched random heads reverse QKFormer's high–low separation, whereas input-channel permutation largely preserves it. These controls support sensitivity to learned projection structure, without establishing precise class-semantic alignment as the main source.

These comparisons support classifier-derived evidence as a useful training-free selection signal whose benefits depend on the representation and token retention ratio.

Two-pass evaluation and complete inference cost

The additional dense pass belongs to the original controlled evaluation procedure. In single-pass execution, scores are computed from features already available at the target site, followed by token selection and the remaining network. Both FN and CETS use this implementation, with unchanged scoring and selection rules.

Whole-model mean latency (ms/batch):

Dataset / checkpointDense, unprunedSingle-pass FN: 80% / 60%Single-pass CETS: 80% / 60%
CIFAR-100 / B34.8936.50 / 36.7037.83 / 37.78
DVS-CIFAR10 / A23.7125.23 / 23.5628.00 / 26.56

Measurements use an RTX 4090, batch 64 with AMP for CIFAR-100 and batch 16 with FP32 for DVS. We use 10 warmup rounds and 50 repetitions per measurement group. Dense is unpruned. Timing includes state reset, scoring and selection where applicable, indexing and model execution, excluding loading and host-to-device transfer.

These timings show that removing dense recapture does not by itself produce whole-model GPU acceleration. The revision will separate target-SSA operation reductions from observed inference latency.

Fixed temporal coefficient and dataset-dependent behavior

The original implementation used a fixed λ=0.9, which we retained across datasets without dataset-specific tuning. We will include the λ-sensitivity analysis and clarify that λ=0.9 is a shared configuration, not a dataset-wise optimum, and that temporal variation is a dataset-dependent refinement whose benefits are not universal.

Aggressive token reduction

At 40% retention on DVS-CIFAR10 (checkpoint A), CETS achieves 55.40%, below Random at 60.60±0.66% (mean±sample SD over five mask realizations). This result limits the applicable pruning range of the current criterion. We will explicitly distinguish these regimes and qualify the supported retention ratios, rather than extrapolate moderate-reduction benefits to aggressive pruning.

I appreciate the authors' detailed response, which partially address my concerns. And I will maintain my original rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.