- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Lapis replaces query–key dot products with an L1 distance between TTFS latency codes and a Laplacian affinity, relates the exponential kernel to leak-only LIF dynamics, and approximates row normalization by power-of-two scaling. It trains a quantized model and converts its codebook losslessly into TTFS spike timing. Experiments cover CIFAR-10/100 and ImageNet-1K, including 6-bit deployment results and an operation-level arithmetic energy estimate. The paper is technically coherent at the operator level, but its incremental novelty and practical energy benefit are not yet convincingly isolated.
Critical strengths
- Coherent operator construction and carefully scoped equivalence. The affine relation between nonnegative integer codes and first-spike latencies preserves the L1 query–key distance; a leaky recurrence reproduces the exponential affinity in the stated model, and the power-of-two row scaling has a bounded multiplicative approximation (Secs. 4.2–4.5). The claimed losslessness is relative to the quantized codebook, not arbitrary full-precision ANN activations.
- Nontrivial scalability and accuracy evidence. Lapis-S reports 96.56% CIFAR-10 accuracy, 0.53 points below the controlled dot-product variant (Table 3). The deployed Lapis-L reports 83.25% ImageNet-1K accuracy with 6-bit body weights. The supplied Appendix A makes the operation-count assumptions inspectable rather than presenting energy measurements without a model.
Critical weaknesses
- The contribution of the new relation relative to prior L1 Laplacian attention and existing TTFS Transformers is not sufficiently disentangled. EcoTransformer already formulates L1 Laplacian query–key relations, while TTFSFormer addresses TTFS Transformer conversion (both are discussed in Related Work). Equation (18) shows that the latency L1 distance is numerically identical to the L1 distance between quantized activation codes. The genuinely new value must therefore come from a validated spike-timing/leakage realization and/or efficiency under matched TTFS conditions. Furthermore, Appendix C states that dot-product/Gaussian ablations use signed LSQ Q/K representations while Hamming/Laplacian use nonnegative codes; the accuracy differences are not kernel-only effects.
- The largest efficiency claims depend on an unvalidated execution and accounting model. Appendix A charges dense L1 pairwise distance calculations but excludes the repeated leakage updates as explicit arithmetic; it also omits routing, memory, state maintenance and temporal buffering. Yet the proposed realization requires per-pair D_ij decay and the same attention weights are then applied to time-resolved value spikes (Secs. 4.3–4.4). The paper does not show how all needed Q/K first-spike times and V spikes are synchronized or stored in a realizable platform, nor does it provide matched-TTFS hardware latency/energy. Thus, the 15.2x attention-path and 93.3x whole-model energy figures should be understood strictly as conditional arithmetic proxies, not measured deployment savings.
- The proposed TTFS encoding assigns the same terminal latency to all silent neurons. Consequently, query–key pairs with identical silent patterns can receive high or even maximal attention affinity despite containing little active information. The paper does not sufficiently examine whether this behavior affects attention quality in highly sparse regimes.
Questions for authors
- Can L1 Laplacian, Gaussian, Hamming and dot-product comparisons be repeated using identical Q/K code ranges, precision, training and normalization? What benefit remains specifically attributable to the relation metric?
- How are first-spike Q/K codes collected, the per-pair leak states evolved, and the V spike trains buffered or replayed before applying attention? Please provide a scheduling diagram and explicit latency/state/memory or digital-operation costs for a defined hardware target.
- Under the same TTFS execution model and weight precision, how much arithmetic energy and accuracy does Lapis save versus a matched TTFS dot-product or other TTFS attention operator? Please separate TTFS sparsity, scoring and W6 quantization effects.
- How does Lapis behave when query and key representations contain many silent neurons? Can the authors analyze whether high affinity between jointly inactive representations affects attention quality and evaluate the sensitivity of performance to different levels of spike sparsity?
4 comments
Follow-up on our response
Thank you again for your review. As the discussion period is drawing to a close, we would be grateful to know whether our response has addressed your concerns. We are happy to clarify any remaining points.
I sincerely appreciate the additional experiments conducted to address my concerns. The additional results suggest that most of the reported energy savings come from TTFS execution and quantization, while the incremental benefit of Lapis over matched TTFS attention is relatively modest. Moreover, the relationship between the proposed leakage-based realization and the simulated hardware implementation remains somewhat unclear. Although the rebuttal strengthens the technical justification of the method, I am not yet fully convinced that the incremental contribution is sufficiently significant. Therefore, I will maintain my rating of 2 (Weak Rejection).
Response to Reviewer e2v5: the Significance of the Contribution
Thank you for the follow-up and for acknowledging that the rebuttal strengthens the technical justification of Lapis. We would like to respectfully share our view on the significance of the contribution.
An improvement without a trade-off. Against matched TTFS attention, Lapis is more accurate, uses less energy and adds no latency. These margins are measured on a fully digital core that charges in full the stage leakage would perform physically, the setting least favorable to Lapis. An improvement on every axis under these conditions reflects a consistent advantage of the method.
A new direction rather than a refinement. We would also suggest that these margins do not fully reflect what Lapis contributes. Prior spiking attention approximates the dot product with spikes. Lapis instead uses what the neuron already computes. The interval between two spikes is a distance, and leakage over it is an exponential, so the neuron's own dynamics carry out the relation. Lapis validates this route on attention up to ViT-L on ImageNet, and we believe establishing a new route, even at an early stage, can matter more than a further step along a well-explored one.
We would be grateful if you would take this perspective into account in your final assessment.
Response to Reviewer e2v5
We thank the reviewer for the detailed feedback and for recognizing the coherent construction of Lapis and its scalability.
W1 & Q1: Relation to prior work and matched comparison. As Eq. 18 shows, Lapis computes the same values as quantized ℓ1 attention, and it differs from prior work in how they are computed. EcoTransformer evaluates the exponential on dense activations, and TTFSFormer keeps the dot product and softmax and builds dedicated neuron networks to compute them. In Lapis, every step runs on a native spiking operation, as membrane leakage over the delays between query and key spikes yields the exponential weight and normalization reduces to a bit shift.
For Q1, Hamming and Laplacian already share nonnegative codes in Table 3, so their gap comes from the relation itself. We re-trained dot product, Gaussian and Laplacian with both signed and nonnegative 4-bit codes, using the same training and exact row normalization.
Under both codes, Laplacian stays above Gaussian, so this gap comes from the kernel itself. Laplacian barely changes with the code sign, whereas the dot product drops without signed codes. Since a first-spike latency can only encode a nonnegative code, the distance suits spikes, and Laplacian ranks first among nonnegative codes. This comparison is added as Table 4.
W2 & Q2: Execution schedule and cost. By design, the leakage needs no update schedule. Each pair's membrane leaks in parallel over the delays between query and key spikes, so the affinity is ready at the end of the coding window. We additionally charge the readout of each affinity as one 8 KB SRAM access, 10 pJ at 45 nm (Horowitz, 2014), which changes the reductions only to 14.8× for the attention path and 88.9× for the whole 6-bit model.
As a defined hardware target, we simulate a bit-true attention core in 22 nm fixed-point logic at 500 MHz, with 16 query lanes per head, and the new Fig. 3 in Appendix A.5 shows its schedule. During the coding window, each query, key and value spike stores its time step in on-chip SRAM as a 4-bit code. Keys and values are then read once per tile of queries. Each affinity is accumulated into the row sum and the weighted value sum as soon as it is computed, and each row is shifted once by kih in Eq. 12 at the end, so no per-pair state is stored.
For Lapis-S, the attention takes 0.41 ms and 50.6 µJ per image including SRAM access and data movement, the Q/K/V buffer needs 110.8 KiB per layer, and the lane registers hold 3.2 KiB.
W2 & Q3: Matched TTFS comparison and effect separation. The matched TTFS operator is dot-product attention on the same nonnegative codes, whose accuracy by Eq. 18 equals that of TTFS dot-product attention. Against it, Lapis is 0.29 points more accurate on CIFAR-10 and uses 8.6% less energy on the simulated 22 nm core above. For Lapis-L on ImageNet under the 45 nm model of Sec. 5.3, where the affinity comes from membrane leakage, TTFS execution reduces attention-path energy by 14.5×, replacing the dot product with Lapis saves a further 4%, and 6-bit weights save another 3.9×. TTFS execution thus provides the primary gain, as Sec. 5.3 states, and its conversion is lossless. With 6-bit weights, scoring takes a larger share of the remaining energy, and Lapis uses 15.6% less attention-path energy than TTFS dot-product attention, without any multiplication between dynamic operands.
W3 & Q4: Silent neurons and sparsity. A silent neuron encodes zero activation (Eq. 5), so two silent channels agree exactly as two zeros do in the equivalent quantized network. A maximal affinity without any agreeing active channel therefore requires both tokens to be silent in every channel of a head. No such token appears on CIFAR-10, and only 0.3% of key tokens on ImageNet are fully silent. To test whether their affinity affects attention, we mask all fully silent pairs. Predictions on ImageNet stay unchanged, and even when we raise sparsity far beyond the operating point, masking never improves accuracy by more than 0.09 points. For sensitivity, we end the coding window of queries and keys early in Lapis-S on CIFAR-10, which silences the spikes carrying the smallest values. Reducing the share of firing neurons from 60% to 41% and to 25% lowers accuracy by only 0.19 and 1.0 points. Appendix D reports the full analysis.
We hope these results address the reviewer's concerns and are happy to clarify any remaining point.
Reference. Horowitz M. 1.1 computing's energy problem (and what we can do about it)[C]//2014 IEEE international solid-state circuits conference digest of technical papers (ISSCC). Ieee, 2014: 10-14.