- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces USHE, a probabilistic spike encoding method designed to reduce encoding variance within a fixed number of simulation steps. The method combines stochastic rounding with sampling without replacement to reduce temporal variance while keeping the encoding unbiased. The authors provide theoretical results on variance optimality and its relationship to Bernoulli encoding. Experiments on CIFAR-10 and CIFAR-100 show improved clean accuracy compared with Bernoulli encoding, along with different trade-offs between accuracy and adversarial robustness.
Critical strengths
-
The motivation is clear and relevant. Encoding variance can be a significant issue for SNNs, especially when only a small number of time steps is available. Addressing this problem without increasing the simulation horizon is worthwhile.
-
The theoretical analysis is one of the stronger parts of the paper. The authors provide results on unbiasedness, variance optimality, and convergence to Bernoulli encoding, which help explain the behavior of the proposed method.
-
USHE consistently achieves better clean accuracy than standard Bernoulli encoding in the reported experiments. The comparison between different training and inference encoding strategies also provides some useful observations.
Critical weaknesses
-
I have some concerns about the methodological novelty. The main idea is based on stochastic rounding and sampling without replacement, both of which are well-established techniques for variance reduction. While their application to spike encoding is reasonable, it is not entirely clear how USHE differs from existing structured or variance-reduced encoding methods. More comparisons with related approaches would help clarify its contribution.
-
The adversarial robustness evaluation is not fully convincing. The attacks mainly rely on single-sample BPDA gradient estimation. Since USHE uses stochastic encoding, this may not be sufficient to properly evaluate its robustness. Stronger adaptive attacks using EOT with multiple samples would be important to rule out possible gradient masking effects. Also, the reported robustness improvements are not consistent across different architectures and attacks, making it difficult to draw a general conclusion.
-
The practical advantage over existing encoding strategies is somewhat limited. Although USHE improves clean accuracy compared with Bernoulli encoding, it still performs considerably worse than direct encoding. In addition, the theoretical variance optimality applies to input-rate reconstruction, but it does not necessarily translate into better classification accuracy or adversarial robustness. The connection between these theoretical results and the downstream improvements could be explained more clearly.
-
The experimental evaluation is relatively narrow. The results are limited to CIFAR-10/100, two network architectures, and short simulation horizons. More importantly, each configuration appears to use only one training seed. The reported standard deviations mainly reflect randomness during evaluation rather than variation across independent training runs. This makes it difficult to assess how stable and reproducible the improvements are.
-
The computational overhead of USHE is not sufficiently discussed. Although the method does not require additional simulation steps, the sampling procedure itself may introduce extra computation compared with standard Bernoulli encoding. Without actual latency or energy measurements, it is unclear whether the lower encoding variance leads to meaningful efficiency gains in practice.
Questions for authors
See weaknesses
4 comments
W3 - Connecting encoding theory to classification.
Let be the average input spike vector and the class scores of a fixed affine classifier, with weights W and biases b. For independently encoded input coordinates we have,
where y is the true class and N=∞ denotes Bernoulli. This expected cross-entropy ordering follows from convex ordering of the count distributions, not from variance alone. We also obtain a classification-error bound through true-class-versus-competitor score margins: USHE preserves their means and reduces their variances, improving the bound when the mean margins are positive. With these results we hint how encoding variability connects to downstream loss and error for a specified classifier class.
W2 — Adaptive attacks: the 200-image sensitivity sweep
We tested the attack changes you recommend: longer optimization, multiple stochastic gradient samples, and additional restarts. The complete PGD-100 sweep below uses the same 200 class-balanced images per CIFAR dataset, training seed 1001, T=4 and ε=8/255. E denotes EOT samples; R5 denotes five restarts. Other columns use one restart; α=0.01 unless indicated. Entries are accuracies (%).
| Setting / encoder | E20 | E50 | E100 | E20, R5 | E20, α=2/255 |
|---|---|---|---|---|---|
| C10/VGG / Bernoulli | 26.76 | 26.69 | 26.41 | 26.60 | 26.79 |
| C10/VGG / USHE₄ | 21.66 | 21.41 | 21.43 | 21.22 | 21.59 |
| C10/VGG / USHE₁₂ | 26.97 | 26.70 | 26.52 | 26.66 | 27.16 |
| C10/ResNet19 / Bernoulli | 32.14 | 32.15 | 31.70 | 32.21 | 32.15 |
| C10/ResNet19 / USHE₄ | 27.09 | 26.88 | 26.54 | 27.07 | 26.91 |
| C10/ResNet19 / USHE₁₂ | 32.74 | 31.90 | 32.05 | 32.49 | 32.52 |
| C100/VGG / Bernoulli | 11.27 | 11.18 | 11.20 | 11.20 | 11.31 |
| C100/VGG / USHE₄ | 10.11 | 10.03 | 10.05 | 10.21 | 10.25 |
| C100/VGG / USHE₁₂ | 12.30 | 12.30 | 12.30 | 12.24 | 12.46 |
| C100/ResNet19 / Bernoulli | 15.11 | 15.04 | 15.03 | 15.18 | 15.08 |
| C100/ResNet19 / USHE₄ | 15.25 | 15.48 | 15.10 | 15.23 | 15.56 |
| C100/ResNet19 / USHE₁₂ | 15.07 | 15.23 | 15.03 | 15.27 | 15.18 |
USHE₁₂ exceeds Bernoulli under all five CIFAR-100/VGG schedules, by 1.03–1.15 points. Raising EOT from 20 to 100 changes accuracy only modestly: the average reduction across all twelve configurations is 0.26 points (maximum 0.68); five restarts reduce it by 0.07 on average. The main reduction occurs when strengthening the original attack: CIFAR-10/ResNet/USHE₁₂ falls from 46.99% under PGD-7/EOT-1 to 32.74% under PGD-100/EOT-20 on these same images.
Further study: 2,000 images and three training seeds. We then extended PGD-100/EOT-20 to a fixed, class-balanced 2,000-image cohort per dataset, disjoint from the sensitivity cohort, across all 36 Bernoulli/USHE₄/USHE₁₂ checkpoints. This study uses ε=8/255, α=0.01 and one restart. Mean ± sample SD across three training seeds:
| Dataset / architecture | Bernoulli | USHE₄ | USHE₁₂ |
|---|---|---|---|
| CIFAR-10 / VGG-SNN | 26.70 ± 0.10 | 20.01 ± 0.27 | 25.36 ± 0.14 |
| CIFAR-10 / ResNet-19 | 31.38 ± 1.06 | 25.30 ± 0.64 | 30.87 ± 0.21 |
| CIFAR-100 / VGG-SNN | 11.97 ± 0.15 | 9.34 ± 0.42 | 11.23 ± 0.11 |
| CIFAR-100 / ResNet-19 | 16.71 ± 0.41 | 13.98 ± 0.56 | 16.54 ± 0.26 |
The extension checks whether the small-cohort ranking persists across more images and independently trained models. USHE₁₂ stays much closer to Bernoulli than USHE₄, while Bernoulli has the highest mean in these four settings. The favorable CIFAR-100/VGG ranking from the sensitivity sweep does not persist here. Together with the clean results in Comment 3, this supports a tunable clean–robustness trade-off; the advantage over Bernoulli depends on the attack and setting.
For CIFAR-100, we also evaluated DRC and cyclic burst on the same 2,000 images with training seed 1001. USHE₄ exceeds DRC by 1.67/3.65 points and cyclic burst by 0.28/0.07 points (VGG/ResNet), extending the original-attack comparisons in Comment 1.
Gradient estimation and scoring. The forward pass retains actual sampled spikes. EOT averages per-realization cross-entropy gradients through the expected-rate BPDA surrogate, reducing sampling variability while leaving the backward approximation explicit. Candidate selection uses 32 separate draws and final scoring 128 fresh draws. Each scoring draw makes one prediction; we average correctness indicators, so this evaluates single-encoding predictions rather than a 128-sample ensemble.
Alternative attacks. In addition, we tested on custom APGD-CE (100 steps/EOT-10) and gradient-free square-patch search useing matched 500-image cohorts (seed 1001). On CIFAR-100 VGG/ResNet, USHE₄ exceeds Bernoulli by 5.26/5.74 pp under the search and DRC by 1.04/3.62 pp under APGD and 8.99/5.94 pp under the search.
W4 — Independent training runs and broader evaluation
We added training seeds 1001 and 1002 to seed 1000. Bernoulli, USHE₄ and USHE₁₂ now have three checkpoints per CIFAR dataset/architecture setting (36 total), with clean and classical-attack evaluation on all 10,000 test images. Dedicated clean accuracy (%) and paired gains are:
| Dataset / architecture | Bernoulli | USHE₄ | Paired gain (pp) |
|---|---|---|---|
| CIFAR-10 / VGG-SNN | 79.32 ± 0.17 | 84.98 ± 0.08 | +5.67 ± 0.18 |
| CIFAR-10 / ResNet-19 | 79.18 ± 0.25 | 85.46 ± 0.15 | +6.28 ± 0.30 |
| CIFAR-100 / VGG-SNN | 51.60 ± 0.11 | 57.90 ± 0.04 | +6.30 ± 0.09 |
| CIFAR-100 / ResNet-19 | 51.33 ± 0.51 | 57.89 ± 0.11 | +6.56 ± 0.41 |
Both USHE populations improve clean accuracy for every seed in all four settings; USHE₁₂ gains 1.27–1.87 points on average. SD is computed across checkpoint means after averaging inference repetitions. Some attacked gains also reproduce: CIFAR-10/ResNet-19 RFGSM gives 69.58 ± 0.39% for USHE₄ versus 66.89 ± 0.34% for Bernoulli, a paired gain of 2.69 ± 0.06 points, positive in all three seeds.
Additional datasets. SVHN and Tiny ImageNet extend the clean-accuracy evidence beyond CIFAR and add a spiking ResNet-18. At the endpoint N=T:
| Dataset / architecture | T | Direct | Bernoulli | USHE | USHE − Bernoulli (pp) |
|---|---|---|---|---|---|
| Tiny ImageNet / ResNet-18 | 4 | — | 48.94 | 52.43 | +3.49 |
| Tiny ImageNet / ResNet-18 | 8 | — | 52.84 | 55.67 | +2.84 |
| SVHN / VGG-SNN | 4 | 96.17 | 86.99 | 92.55 | +5.56 |
| SVHN / VGG-SNN | 8 | 96.49 | 90.29 | 93.58 | +3.28 |
| SVHN / ResNet-19 | 4 | 96.22 | 88.59 | 94.16 | +5.57 |
| SVHN / ResNet-19 | 8 | 96.68 | 91.94 | 95.29 | +3.34 |
All results use one training seed (1000), with repeated inference evaluations. Differences are calculated from the unrounded means and expressed in percentage points. “—” indicates that a Direct result is not available in the supplied report.
These additional datasets use one training seed (1000) per configuration, with repeated single-encoding evaluations. On SVHN, USHE at T=4 exceeds Bernoulli at T=8 by 2.25/2.22 points (VGG/ResNet), using half as many simulated steps. It also approaches Direct's clean accuracy: on SVHN/ResNet-19 at T=8, USHE reaches 95.29% versus Direct's 96.68%, a gap of about 1.4 points.
The submitted CIFAR T=8 results retain clean gains of 3.40–6.01 points over Bernoulli (one training seed). Our theory covers arbitrary horizons; experiments cover T=4 and T=8.
W5 — Encoding operations and neuromorphic cost
USHE retains binary input computation and adds a small budget counter plus conditional sampling. For intensity p, we initialize K=floor(Np)+B, with B Bernoulli of parameter Np−floor(Np). Further, at a visible step t, we set m=N−t+1, draw J uniformly from {1,…,m}, emit s=1{J≤K}, and update K←K−s. This implements probability K/m without division in the per-step threshold. The value m is shared across synchronized input coordinates; the unused N−T positions are never simulated or stored.
| Resource | Bernoulli | USHE |
|---|---|---|
| Initialization | Input probability p | Form Np and stochastically round |
| Per-step sampling | Uniform draw, compare with p | Bounded-integer draw, compare with K |
| Count update | None | K←K−s |
| Extra count state per coordinate | None | ⌈log₂(N+1)⌉ bits |
| Visible steps / expected spikes | T / Tp | T / Tp |
The counter needs three bits for N=4 and four for N=12. Both encoders use O(Td) logical sampling/comparison operations for d inputs; USHE adds O(d log N) counter bits. Deterministic K=0 or K=m cases can skip sampling. Bounded-integer draws may require range conversion or rejection, so equal operation order does not mean equal random-number cost.
With fan-out f_j for input j, both encoders have expected first-layer event count TΣ_j f_jx_j. Binary spikes permit gated weight additions in that layer, as explained in Comment 1. This reduces the need for floating-point input–weight multiplication in an event-driven implementation.
Thank you for the detailed responses and the considerable effort put into the additional experiments. I appreciate that the revision has addressed several of my concerns, particularly regarding stronger attacks and statistical reliability. However, the new results also weaken the original robustness claims, and I still have reservations about the novelty and practical benefits of the method. I will therefore maintain my original rating and look forward to seeing these claims revised and the work further improved in a future version.
Thank you for recognizing the motivation, theoretical analysis and clean-accuracy gains. We have strengthened the comparisons and clarified the practical role of USHE. We address the encoding contribution and Direct comparison here, the adaptive-attack sweep in Comment 2, and training replication, broader datasets and implementation cost in Comment 3.
W1 — Contribution beyond stochastic rounding and sampling without replacement
These sampling primitives are established. Our contribution is a population-indexed family that preserves binary spikes, each timestep's Bernoulli marginal, expected input activity and the visible horizon T, while controlling temporal dependence through N. Only T timesteps are sampled, even when N>T. The endpoint N=T minimizes unbiased rate-reconstruction variance; increasing N connects it to independent Bernoulli encoding.
The analysis characterizes this entire family: variance optimality among unbiased exchangeable length-N populations observed for T steps (Theorem 3.2), global count-variance optimality at N=T without requiring exchangeability (Corollary 3.3), convex-order guarantees and quantitative convergence to Bernoulli. This identifies both the achievable variance reduction and the effect of the tuning parameter.
Matched structured-encoder comparisons. We added DRC, which deterministically rounds the spike count (generally biasing the rate) and places spikes uniformly, and cyclic burst, which uses USHE₄'s unbiased count distribution but places spikes contiguously with a random cyclic shift. These controls test count quantization and temporal placement under the same CIFAR-100 training recipes.
USHE₄ improves full-test PGD-7 accuracy over DRC by 2.97 ± 0.53 points on VGG and 4.79 ± 0.63 on ResNet across three training seeds. The adaptive comparisons in Comment 2 also support this advantage and assess the effect of alternative temporal placement. These results support USHE's practical benefit over deterministic count rounding, while the cyclic control shows comparable strong-PGD performance with another arrangement of the same counts. USHE additionally provides an explicit population parameter connecting the low-variance endpoint to Bernoulli.
W3 — Practical advantages over Direct encoding.
USHE provides a way to balance clean accuracy and adversarial performance while retaining binary input spikes. Direct encoding achieves strong clean accuracy, but stochastic encoders retain substantially higher attacked accuracy in the paper’s evaluations. The following CIFAR-100/ResNet-19 results at T=4 illustrate how the USHE population parameter adjusts this balance. All accuracies are percentages, evaluated on the full test set.
Bernoulli and USHE entries are means ± sample SD across three independently trained models. Direct is the original single-checkpoint reference; its training variability has not been measured.
USHE₄ improves clean accuracy over Bernoulli by 6.56 percentage points, while retaining substantially higher attacked accuracy than Direct under these evaluations. Increasing the population to N=12 recovers Bernoulli-level PGD-7 accuracy while improving both clean and RFGSM accuracy over Bernoulli. In particular, USHE₁₂’s paired RFGSM gain is 0.83 ± 0.05 percentage points, positive in all three training seeds. Its mean PGD-7 accuracy is also slightly higher, by 0.09 points. These results demonstrate useful operating points: the smaller population favors clean accuracy, while the larger population moves toward Bernoulli’s PGD-7 performance without giving up the clean-accuracy improvement. The stronger adaptive evaluations are reported separately in Comment 2. We also note that on certain datasets the difference between direct and USHE encodings becomes even less emphasized (see SVHN table bellow).
USHE also retains a computational advantage of binary input encoding. For a synaptic weight w and spike s∈{0,1}, the product ws is either zero or w, so an event-driven first layer can use gated weight additions in place of floating-point input–weight multiplication. USHE therefore combines an adjustable clean–robustness balance with the binary input arithmetic shared by Bernoulli; its additional sampling cost is accounted for in Comment 3.