acceptodds
Under review as a conference paper at ICLR 2027

Lapis: Laplacian Spiking Attention via First-Spike Timing and Membrane Leakage

Abstract

Self-attention has become central to spiking vision transformers, yet its query-key scoring is still largely inherited from dense networks. Existing spiking variants either simplify dot product scoring or replace it with discrete operators, but spike timing, the native variable of a spiking network, does not directly define how tokens are related. We propose Lapis, a spiking attention mechanism that scores each token pair by the L1 distance between its query and key first-spike latency vectors under time-to-first-spike coding, and maps this distance to an affinity through a Laplacian kernel. The kernel's exponential decay matches the impulse response of a leaky integrate-and-fire membrane, so the accumulated latency difference determines the decay of a membrane trace, while row normalization reduces to a bit shift under power-of-two rounding. Scoring therefore needs only subtraction, absolute value, and accumulation, and removes all multiplication between query and key channels. Under a matched backbone and training schedule, Lapis reaches 96.56% top-1 accuracy on CIFAR-10, within 0.53 points of dot-product scoring. On ImageNet-1K, it reduces the estimated arithmetic energy of the attention path by 15.2x relative to dense dot-product attention. The deployed 6-bit model attains 83.25% top-1 accuracy at an estimated arithmetic energy of 3.04mJ per image.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Spiking Neural Networks, Spiking Attention, Time-to-First-Spike, Energy Efficiency

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_e2v5OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Lapis replaces query–key dot products with an L1 distance between TTFS latency codes and a Laplacian affinity, relates the exponential kernel to leak-only LIF dynamics, and approximates row normalization by power-of-two scaling. It trains a quantized model and converts its codebook losslessly into TTFS spike timing. Experiments cover CIFAR-10/100 and ImageNet-1K, including 6-bit deployment results and an operation-level arithmetic energy estimate. The paper is technically coherent at the operator level, but its incremental novelty and practical energy benefit are not yet convincingly isolated.

Critical strengths
  1. Coherent operator construction and carefully scoped equivalence. The affine relation between nonnegative integer codes and first-spike latencies preserves the L1 query–key distance; a leaky recurrence reproduces the exponential affinity in the stated model, and the power-of-two row scaling has a bounded multiplicative approximation (Secs. 4.2–4.5). The claimed losslessness is relative to the quantized codebook, not arbitrary full-precision ANN activations.
  2. Nontrivial scalability and accuracy evidence. Lapis-S reports 96.56% CIFAR-10 accuracy, 0.53 points below the controlled dot-product variant (Table 3). The deployed Lapis-L reports 83.25% ImageNet-1K accuracy with 6-bit body weights. The supplied Appendix A makes the operation-count assumptions inspectable rather than presenting energy measurements without a model.
Critical weaknesses
  1. The contribution of the new relation relative to prior L1 Laplacian attention and existing TTFS Transformers is not sufficiently disentangled. EcoTransformer already formulates L1 Laplacian query–key relations, while TTFSFormer addresses TTFS Transformer conversion (both are discussed in Related Work). Equation (18) shows that the latency L1 distance is numerically identical to the L1 distance between quantized activation codes. The genuinely new value must therefore come from a validated spike-timing/leakage realization and/or efficiency under matched TTFS conditions. Furthermore, Appendix C states that dot-product/Gaussian ablations use signed LSQ Q/K representations while Hamming/Laplacian use nonnegative codes; the accuracy differences are not kernel-only effects.
  2. The largest efficiency claims depend on an unvalidated execution and accounting model. Appendix A charges dense L1 pairwise distance calculations but excludes the repeated leakage updates as explicit arithmetic; it also omits routing, memory, state maintenance and temporal buffering. Yet the proposed realization requires per-pair D_ij decay and the same attention weights are then applied to time-resolved value spikes (Secs. 4.3–4.4). The paper does not show how all needed Q/K first-spike times and V spikes are synchronized or stored in a realizable platform, nor does it provide matched-TTFS hardware latency/energy. Thus, the 15.2x attention-path and 93.3x whole-model energy figures should be understood strictly as conditional arithmetic proxies, not measured deployment savings.
  3. The proposed TTFS encoding assigns the same terminal latency to all silent neurons. Consequently, query–key pairs with identical silent patterns can receive high or even maximal attention affinity despite containing little active information. The paper does not sufficiently examine whether this behavior affects attention quality in highly sparse regimes.
Questions for authors
  1. Can L1 Laplacian, Gaussian, Hamming and dot-product comparisons be repeated using identical Q/K code ranges, precision, training and normalization? What benefit remains specifically attributable to the relation metric?
  2. How are first-spike Q/K codes collected, the per-pair leak states evolved, and the V spike trains buffered or replayed before applying attention? Please provide a scheduling diagram and explicit latency/state/memory or digital-operation costs for a defined hardware target.
  3. Under the same TTFS execution model and weight precision, how much arithmetic energy and accuracy does Lapis save versus a matched TTFS dot-product or other TTFS attention operator? Please separate TTFS sparsity, scoring and W6 quantization effects.
  4. How does Lapis behave when query and key representations contain many silent neurons? Can the authors analyze whether high affinity between jointly inactive representations affects attention quality and evaluate the sensitivity of performance to different levels of spike sparsity?
4 comments
AuthorsOpenReview ↗

Response to Reviewer e2v5

We thank the reviewer for the detailed feedback and for recognizing the coherent construction of Lapis and its scalability.

W1 & Q1: Relation to prior work and matched comparison. As Eq. 18 shows, Lapis computes the same values as quantized ℓ1\ell_1 attention, and it differs from prior work in how they are computed. EcoTransformer evaluates the exponential on dense activations, and TTFSFormer keeps the dot product and softmax and builds dedicated neuron networks to compute them. In Lapis, every step runs on a native spiking operation, as membrane leakage over the delays between query and key spikes yields the exponential weight and normalization reduces to a bit shift.

For Q1, Hamming and Laplacian already share nonnegative codes in Table 3, so their gap comes from the relation itself. We re-trained dot product, Gaussian and Laplacian with both signed and nonnegative 4-bit codes, using the same training and exact row normalization.

CodesLaplacianGaussianDot product
Nonnegative96.5690.2596.27
Signed96.6591.4497.09

Under both codes, Laplacian stays above Gaussian, so this gap comes from the kernel itself. Laplacian barely changes with the code sign, whereas the dot product drops without signed codes. Since a first-spike latency can only encode a nonnegative code, the distance suits spikes, and Laplacian ranks first among nonnegative codes. This comparison is added as Table 4.

W2 & Q2: Execution schedule and cost. By design, the leakage needs no update schedule. Each pair's membrane leaks in parallel over the delays between query and key spikes, so the affinity is ready at the end of the coding window. We additionally charge the readout of each affinity as one 8 KB SRAM access, 10 pJ at 45 nm (Horowitz, 2014), which changes the reductions only to 14.8× for the attention path and 88.9× for the whole 6-bit model.

As a defined hardware target, we simulate a bit-true attention core in 22 nm fixed-point logic at 500 MHz, with 16 query lanes per head, and the new Fig. 3 in Appendix A.5 shows its schedule. During the coding window, each query, key and value spike stores its time step in on-chip SRAM as a 4-bit code. Keys and values are then read once per tile of queries. Each affinity is accumulated into the row sum and the weighted value sum as soon as it is computed, and each row is shifted once by kihk_i^h in Eq. 12 at the end, so no per-pair state is stored.

For Lapis-S, the attention takes 0.41 ms and 50.6 µJ per image including SRAM access and data movement, the Q/K/V buffer needs 110.8 KiB per layer, and the lane registers hold 3.2 KiB.

W2 & Q3: Matched TTFS comparison and effect separation. The matched TTFS operator is dot-product attention on the same nonnegative codes, whose accuracy by Eq. 18 equals that of TTFS dot-product attention. Against it, Lapis is 0.29 points more accurate on CIFAR-10 and uses 8.6% less energy on the simulated 22 nm core above. For Lapis-L on ImageNet under the 45 nm model of Sec. 5.3, where the affinity comes from membrane leakage, TTFS execution reduces attention-path energy by 14.5×, replacing the dot product with Lapis saves a further 4%, and 6-bit weights save another 3.9×. TTFS execution thus provides the primary gain, as Sec. 5.3 states, and its conversion is lossless. With 6-bit weights, scoring takes a larger share of the remaining energy, and Lapis uses 15.6% less attention-path energy than TTFS dot-product attention, without any multiplication between dynamic operands.

W3 & Q4: Silent neurons and sparsity. A silent neuron encodes zero activation (Eq. 5), so two silent channels agree exactly as two zeros do in the equivalent quantized network. A maximal affinity without any agreeing active channel therefore requires both tokens to be silent in every channel of a head. No such token appears on CIFAR-10, and only 0.3% of key tokens on ImageNet are fully silent. To test whether their affinity affects attention, we mask all fully silent pairs. Predictions on ImageNet stay unchanged, and even when we raise sparsity far beyond the operating point, masking never improves accuracy by more than 0.09 points. For sensitivity, we end the coding window of queries and keys early in Lapis-S on CIFAR-10, which silences the spikes carrying the smallest values. Reducing the share of firing neurons from 60% to 41% and to 25% lowers accuracy by only 0.19 and 1.0 points. Appendix D reports the full analysis.

We hope these results address the reviewer's concerns and are happy to clarify any remaining point.

Reference. Horowitz M. 1.1 computing's energy problem (and what we can do about it)[C]//2014 IEEE international solid-state circuits conference digest of technical papers (ISSCC). Ieee, 2014: 10-14.

AuthorsOpenReview ↗

Follow-up on our response

Thank you again for your review. As the discussion period is drawing to a close, we would be grateful to know whether our response has addressed your concerns. We are happy to clarify any remaining points.

I sincerely appreciate the additional experiments conducted to address my concerns. The additional results suggest that most of the reported energy savings come from TTFS execution and quantization, while the incremental benefit of Lapis over matched TTFS attention is relatively modest. Moreover, the relationship between the proposed leakage-based realization and the simulated hardware implementation remains somewhat unclear. Although the rebuttal strengthens the technical justification of the method, I am not yet fully convinced that the incremental contribution is sufficiently significant. Therefore, I will maintain my rating of 2 (Weak Rejection).

AuthorsOpenReview ↗

Response to Reviewer e2v5: the Significance of the Contribution

Thank you for the follow-up and for acknowledging that the rebuttal strengthens the technical justification of Lapis. We would like to respectfully share our view on the significance of the contribution.

An improvement without a trade-off. Against matched TTFS attention, Lapis is more accurate, uses less energy and adds no latency. These margins are measured on a fully digital core that charges in full the stage leakage would perform physically, the setting least favorable to Lapis. An improvement on every axis under these conditions reflects a consistent advantage of the method.

A new direction rather than a refinement. We would also suggest that these margins do not fully reflect what Lapis contributes. Prior spiking attention approximates the dot product with spikes. Lapis instead uses what the neuron already computes. The interval between two spikes is a distance, and leakage over it is an exponential, so the neuron's own dynamics carry out the relation. Lapis validates this route on attention up to ViT-L on ImageNet, and we believe establishing a new route, even at an early stage, can matter more than a further step along a well-explored one.

We would be grateful if you would take this perspective into account in your final assessment.

Reviewer_48VWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper replaces the query–key dot product in spiking vision transformers with an l1 distance between first-spike latency vectors under time-to-first-spike (TTFS) coding, mapped to an affinity A_{ij} by a Laplacian kernel with a per-head learnable \tau. The exponential is identified with the subthreshold decay of a LIF membrane, so the affinity for a pair is exactly the membrane state after D_{ij} decay steps (Prop. 1). Experimental results show the promissing outcomes of the method.

Critical strengths
  1. The operator is well-posed, cheap to state, and immediately reusable. The score needs only subtraction, absolute value and accumulation; the power-of-two normalization has a genuinely tight bound (a factor \sqrt{2}, and it preserves all pairwise ratios exactly, which is the property that actually matters); and the QNN->TTFS conversion is exact on the codebook, so the training path needs no surrogate gradients. This is a cleaner construction than most "multiplication-free attention" proposals, because the approximation (power-of-two norm) is isolated, bounded and separately ablatable rather than folded into the operator. For anyone working on spiking transformers, this is a drop-in operator with a clear cost model.

  2. Table 3 is the right experiment, and its result is informative. Holding backbone, teacher, schedule and value aggregation fixed while swapping in four relation functions isolates the kernel choice better than most ablations in this area. The 5.12-point gap over the Gaussian kernel (91.44) and the 8.6-point gap over Hamming (87.92) are large enough to say something real: graded temporal proximity, not merely binary latency matching, is what carries the accuracy, and a symmetric \ell_2 kernel is markedly worse than \ell_1 in this representation. Combined with 96.56 vs 97.09 for dot product, the "within 0.53 points" claim is at least measured under controlled conditions.

Critical weaknesses
  1. The efficiency claim is conditional on a cost model that assigns zero cost to the operation the method is named after. The affinity is βDij\beta^{D_{ij}} with Dij∈{0,…,CT}D_{ij}\in\{0,\dots,CT\}; for Lapis-L, C=64C=64 (D=1024D=1024, H=16H=16) and T=20T=20, so DijD_{ij} ranges over up to 1280. A.1 and A.4 then state that "membrane leakage is treated as intrinsic neuron dynamics rather than as repeated digital multiplications", i.e. the decay is not charged at all.

  2. The evidence does not isolate the proposed relation function, because every comparison that carries the argument is confounded.

2.1 C.1 states that dot product and Gaussian use signed LSQ query/key codes while Hamming and Laplacian use nonnegative codes. Signed vs nonnegative is a change in the representable range of exactly the quantity the ℓ1\ell_1 distance is computed on, and it changes what the network can express independently of the kernel. The 5.12-point Gaussian deficit and the 0.53-point dot-product gap therefore each mix two interventions and cannot be attributed to the kernel. This is the single experiment the paper most needs to be clean.

2.2 Lapis-S is trained with logit and feature distillation from a BEiT-v2-Large teacher adapted for 20 epochs, with four student blocks aligned to teacher blocks 5/11/17/23 through a 384->1024 projection. The Spikformer/QKFormer numbers in Table 2 come from their own papers without such a teacher. "Highest accuracy on both benchmarks, exceeding QKFormer by 0.38 and 0.26 points" is therefore not attributable to the relation function, and no ablation removes KD to quantify its contribution — a 300M-parameter ImageNet-pretrained teacher is exactly the kind of thing that supplies 0.38 points.

Questions for authors

See the weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 48VW

We thank the reviewer for the detailed feedback and for recognizing the clean construction of Lapis.

W1: Cost of membrane leakage. The decay carries no arithmetic cost by design, because the affinity is read from physical membrane decay rather than from repeated digital updates. Otters (Yan et al., 2026) uses the same principle in optoelectronic synapses. Prop. 1 runs DijhD^h_{ij} leak steps, but since βhaβhb=βha+b\beta_h^{a}\beta_h^{b}=\beta_h^{a+b}, these steps need not run in sequence. Each channel leaks only between its query and key spikes, at most T=20T=20 steps, and all channels discharge one membrane in parallel. The affinity is therefore ready at the end of the coding window rather than after up to 1280 steps.

We additionally charge the readout of each affinity, which a fully digital realization can implement as one lookup in a per-head table, since AijhA^h_{ij} takes at most CT+1CT+1 values by Prop. 2(i). Charging each readout conservatively as one 8 KB SRAM access, 10 pJ at 45 nm (Horowitz, 2014), adds 0.15 mJ to Lapis-L, so the reduction changes only from 15.2× to 14.8× for the attention path and from 93.3× to 88.9× for the whole 6-bit model. Sec. 4.3 now describes the parallel form, and Appendix A.4 includes the readout charge.

W2.1: Kernel comparison under matched codes. We thank the reviewer for highlighting this control. We re-trained dot product, Gaussian and Laplacian under one protocol with both signed and nonnegative 4-bit codes, keeping the backbone, distillation recipe, schedule and exact row normalization fixed.

CodesLaplacianGaussianDot product
Nonnegative96.5690.2596.27
Signed96.6591.4497.09

Under both codes, Laplacian stays above Gaussian, so the Gaussian gap comes from the kernel itself. The dot-product gap instead depends on the code sign, as the dot product drops without signed codes while Laplacian barely changes. Since a first-spike latency can only encode a nonnegative code, the distance suits spikes, and Laplacian ranks first among nonnegative codes. This comparison is added as Table 4.

W2.2: Effect of distillation. To attribute the gain to the relation function rather than to distillation, we placed the attention of Spikformer, QKFormer and Spike-driven Transformer into the same ViT-S backbone and re-trained all variants with the same teacher, distillation recipe and schedule. Following QKFormer, its variant uses Q–K token attention in the first half of the layers and SSA in the second half.

AttentionCIFAR-10 (%)
SSA from Spikformer, binary90.53
SSA from Spikformer, 4-bit92.46
Q–K attention from QKFormer87.89
SDSA from Spike-driven Transformer86.55
Lapis96.56

With the teacher held fixed, the differences between rows come from the attention alone, and Lapis leads all other relations. These numbers are below the published ones because the original models use their own architectures and train from scratch with multiple time steps, whereas here all relations are fine-tuned from the same ViT-S backbone under one recipe. These rows are added to Table 3, and Table 2 now notes the distillation.

We hope these results address the reviewer's concerns and are happy to clarify any remaining point.

References.

  • Yan Z. et al. Otters: An energy-efficient spiking transformer via optical time-to-first-spike encoding[C]//International Conference on Learning Representations. 2026: 90203-90225.
  • Horowitz M. 1.1 computing's energy problem (and what we can do about it)[C]//2014 IEEE international solid-state circuits conference digest of technical papers (ISSCC). Ieee, 2014: 10-14.
AuthorsOpenReview ↗

Follow-up on our response

Thank you again for your review. As the discussion period is drawing to a close, we would be grateful to know whether our response has addressed your concerns. We are happy to clarify any remaining points.

Reviewer_DSN1OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Lapis, an attention operator that replaces query–key dot products with L1 distances between first-spike latency vectors. These distances are mapped to affinities through a Laplacian kernel, with a proposed realization based on membrane leakage. Power-of-two normalization replaces general division with shift-based scaling. The model is trained as a quantized network and converted to an equivalent TTFS representation. Experiments cover CIFAR-10, CIFAR-100, and ImageNet-1K, including relation-function ablations and operation-level energy estimates. Lapis achieves 96.56% accuracy on CIFAR-10 and 83.25% on ImageNet-1K with 6-bit body weights.

Critical strengths
  1. The paper presents a coherent alternative to dot-product attention that avoids query–key channel multiplications while retaining competitive accuracy. Under the same CIFAR-10 backbone and training schedule, Lapis achieves 96.56% compared with 97.09% for dot-product attention. This supports the practical potential of the proposed relation function.

  2. The quantized-to-TTFS equivalence and the error bound for power-of-two normalization are clearly formulated. These analyses help separate conversion error from approximation error, while the ImageNet experiments demonstrate that the operator can be integrated into large pretrained Transformers.

Critical weaknesses
  1. The energy estimates exclude membrane leakage from explicit arithmetic costs, although the proposed realization involves a distance-dependent number of decay updates. Appendix A also charges subtraction, absolute value, and accumulation together at the cost of one accumulation. Without a concrete implementation accounting for these operations, state storage, and readout, the practical energy advantage remains uncertain.

  2. Avoiding general query–key multiplication is not unique to Lapis, since binary and spike-gated attention can already use logical operations or gated accumulation. The matched comparisons do not include these alternatives or Hadamard-based attention. Comparisons under the same backbone, precision, and training conditions are needed to establish the accuracy–cost advantage.

  3. The contribution of spike timing is not clearly separated from that of quantized L1 attention. Equation 18 shows that the first-spike distance is exactly the distance between the underlying quantized codes. This equivalence supports conversion correctness, but leaves unclear what the temporal implementation offers beyond an alternative representation of the same computation.

  4. The kernel ablation changes both the relation function and the query/key representation. Appendix C uses signed codes for dot-product and Gaussian attention, but nonnegative codes for Hamming and Laplacian attention. Consequently, the accuracy differences cannot be attributed solely to the kernels. Comparisons with matched quantization ranges would provide stronger evidence.

Questions for authors
  1. Could you clarify how the leakage-based affinity is implemented during inference? Is it evaluated through explicit exponential computation, repeated digital updates, a lookup table, or physical membrane decay? How would including its implementation costs affect the reported energy savings and latency?

  2. What supports charging subtraction, absolute value, and accumulation together at 0.9 pJ? Please clarify the required operand and accumulator precision, and whether the energy advantage persists under a more detailed accounting.

  3. Do you have matched results against binary or spike-gated attention and low-bit dot-product attention using the same backbone, training resources, and energy model? Such results would materially affect my assessment of the efficiency contribution.

  4. Does the kernel comparison remain consistent when all variants use the same signed or nonnegative query/key representation? This would help determine how much of the reported difference comes from the relation function rather than quantization.

4 comments
AuthorsOpenReview ↗

Response to Reviewer DSN1 (1/2)

We thank the reviewer for the detailed feedback and for recognizing the competitive accuracy and clear TTFS equivalence of Lapis.

W1 & Q1: Leakage implementation and cost. By design, the affinity is read from physical membrane decay rather than from an explicit exponential or repeated digital updates. Otters (Yan et al., 2026) uses the same principle in optoelectronic synapses. Prop. 1 runs DijhD^h_{ij} leak steps, but since βhaβhb=βha+b\beta_h^a\beta_h^b=\beta_h^{a+b}, these steps need not run in sequence. Each channel leaks only between its query and key spikes, and all channels discharge one membrane in parallel, which holds AijhA^h_{ij} at the end of the coding window. The decay therefore needs no arithmetic, and its latency is the coding window (T=20T=20) regardless of DijhD^h_{ij}.

We additionally charge the readout of each affinity, which a fully digital realization can implement as one table lookup, since AijhA^h_{ij} takes at most CT+1CT+1 values by Prop. 2(i). Charging each readout conservatively as one 8 KB SRAM access, 10 pJ at 45 nm (Horowitz, 2014), adds 0.15 mJ to Lapis-L, so the reduction changes only from 15.2× to 14.8× for the attention path and from 93.3× to 88.9× for the whole 6-bit model. We also evaluated a fully digital attention core in a bit-true simulation with 22 nm costs. It keeps no per-pair state, and including SRAM access and data movement, the attention of Lapis-S takes 0.41 ms and 50.6 µJ per image, 8.6% less energy than the same core with dot-product scoring. Sec. 4.3 now describes the parallel form, Appendix A.4 includes the readout charge, and the new Appendix A.5 describes the core.

W1 & Q2: Operation cost and precision. The 0.9 pJ price is that of an FP32 addition (Horowitz, 2014). It is conservative for these three operations, whose operands are small integers. Latencies take 5 bits (T=20T=20), the signed difference fits in 6 bits, and the accumulator needs 11 bits since Dijh≤CTD^h_{ij}\le CT by Prop. 2(i). A finer accounting charges subtraction, absolute value and accumulation separately at 0.1 pJ each, the 32-bit integer addition cost already used for the 6-bit layers in Appendix A.1. This gives 0.3 pJ per channel and lowers the relation energy of Lapis-L from 0.86 mJ to 0.29 mJ. The reductions reported in the paper therefore remain lower bounds, even with the lookup charge of W1. Appendix A.3 now states these operand widths.

References.

  • Yan Z. et al. Otters: An energy-efficient spiking transformer via optical time-to-first-spike encoding[C]//International Conference on Learning Representations. 2026: 90203-90225.
  • Horowitz M. 1.1 computing's energy problem (and what we can do about it)[C]//2014 IEEE international solid-state circuits conference digest of technical papers (ISSCC). Ieee, 2014: 10-14.
AuthorsOpenReview ↗

Response to Reviewer DSN1 (2/2)

W2 & Q3: Matched comparison with other attention. Every step of Lapis runs on a native spiking operation. Membrane leakage over the delays between query and key spikes directly yields the exponential weight, and normalization reduces to a bit shift. Binary, gated and Hadamard relations avoid multiplication by simplifying the interaction in each channel and dropping the exponential. To measure this trade-off, we placed SSA from Spikformer with binary and 4-bit inputs, Hadamard-based SDSA from Spike-driven Transformer, and a 4-bit dot product into Lapis-S. All variants are re-trained under one protocol with the same backbone, distillation recipe and schedule.

Relation in Lapis-SACC (%)Energy (mJ, 45 nm)
4-bit dot product97.093.50
SSA, 4-bit92.462.83
SSA, binary90.532.80
SDSA86.552.77
Lapis96.563.05

SSA and SDSA save a little energy by skipping the exponential and the normalization, at a large cost in accuracy. The dot product keeps accuracy but multiplies two dynamic operands in every channel and costs more energy. Lapis keeps every step of attention without multiplication while staying close to dot-product accuracy. These numbers are below the published ones because the original models use their own architectures and train from scratch with multiple time steps, whereas here all relations are fine-tuned from the same ViT-S backbone under one recipe. These rows are added to Table 3.

W3: Role of spike timing. The equivalence in Eq. 18 is intentional, since it lets the trained quantized model run as a TTFS network. Running in time changes how attention is computed and what it costs. In the quantized network, the relation needs subtraction, absolute value, accumulation and an exponential. In the TTFS network, the leakage described in W1 & Q1 produces the same affinity from the spike delays alone. Each activation is also a single spike, which yields the primary energy reduction of Sec. 5.3, 22.0× before weight quantization. This is why Lapis pairs the L1 distance with a Laplacian kernel, since spike timing and leakage compute this relation natively, whereas a dot product would have to decode both latencies and multiply them.

W4 & Q4: Kernel comparison under matched codes. We thank the reviewer for highlighting this control. We re-trained dot product, Gaussian and Laplacian under one protocol with both signed and nonnegative 4-bit codes, keeping the backbone, distillation recipe, schedule and exact row normalization fixed.

CodesLaplacianGaussianDot product
Nonnegative96.5690.2596.27
Signed96.6591.4497.09

The ranking against Gaussian holds under both codes, so this gap comes from the kernel itself. Laplacian barely changes with the code sign, whereas the dot product drops without signed codes. Since a first-spike latency can only encode a nonnegative code, the distance suits spikes, and Laplacian ranks first among nonnegative codes. This comparison is added as Table 4.

We hope these results address the reviewer's concerns, and we are happy to clarify any remaining point.

Reviewer_DSN1OpenReview ↗

Review

Thank you for the additional experiments. The matched-code comparisons and the new SSA, SDSA, and 4-bit dot-product baselines address my concerns about the kernel ablation and the choice of baselines. My remaining concern is the accounting for affinity computation. Appendix A.4 describes a lookup-table implementation, while Appendix A.5 describes a digital core that reads affinities from membrane states and treats leakage as intrinsic neuron dynamics. It is therefore unclear how the simulated core computes each affinity and which costs are included in the reported 8.6% energy saving. Please clarify the exact implementation used in the simulation and report energy and latency for Lapis and the matched dot-product baseline, including the cost of producing and reading each affinity. This would help establish whether the TTFS implementation offers a practical efficiency gain beyond the arithmetic estimates. I maintain my original rating.

AuthorsOpenReview ↗

Response to Reviewer DSN1: Digital core implementation, energy, and latency

Thank you for the follow-up, and for confirming that the new experiments addressed the earlier concerns. We appreciate your pointing out that Appendix A.5 was unclear, and we have revised it. Its earlier mention of membrane states referred to the physical realization of Sec. 4.3. The simulated core is fully digital, and nothing in it is treated as intrinsic.

Implementation. By Prop. 2(i) the affinity depends only on the integer distance, so the core reads it from a table. It scales the distance to a base-2 exponent with a per-head constant, then uses one shared 16-entry table and a shift. All of these steps are charged. The dot-product baseline runs on the same core and differs only in scoring, where each channel multiplies the two codes.

Energy and latency per image, attention of Lapis-S on CIFAR-10.

LapisDot product, nonnegative codesDot product, signed codes
Affinity computation and readout (µJ)16.9418.0124.81
Code buffer SRAM (µJ)10.7210.7210.72
Row sum, weighted value sum, row scaling (µJ)20.0423.7224.08
Data movement (µJ)2.892.902.97
Total (µJ)50.5955.3662.58
Latency (ms)0.4050.4110.408

Lapis is the cheapest at producing and reading affinities, and uses 8.6% and 19.2% less energy in total at the same latency. It also yields fewer nonzero affinities at 8-bit precision, which lowers the third row. The table read is 0.11 µJ of the first row, as the table holds only 16 entries. Latency is the cycle count of the schedule at 500 MHz. The table above appears as Table 5 in the revised Appendix A.5.

We hope this resolves the remaining concern, and we would be grateful if you would consider it in your final assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.