acceptodds
Under review as a conference paper at ICLR 2027

Spec-Spike: Spiking Drafters for Vision-Language-Action Speculative Decoding

Abstract

Spiking neural networks (SNNs), the native substrate of neuromorphic hardware, compute with sparse spike events and need far less synaptic arithmetic than dense networks. Speculative decoding of vision-language-action (VLA) policies relies on dense drafters whose action-token proposals a 7B-parameter verifier checks in one pass, and a spiking drafter should make these proposals far cheaper. Making only the drafter's core spiking, however, cuts the core's cost over 15x but the drafter's only about 3x, because the dense projections that feed the core and read out its logits recur at every draft position and take 87% of its analytical cost. We therefore design Spec-Spike around these interfaces in two coupled steps. First, it restructures each interface by how often its input changes: context is cached per observation, token projections are tabulated, and the hidden-state and context projections are factored to low rank. Second, because this factorization costs a spiking drafter 14% of its accepted prefix, against 5% for a dense one, and more than halves its spike activity, Spec-Spike recovers the factors on its own prediction histories. On LIBERO-Long, Spec-Spike reaches mean task success not detectably different from a parameter-matched Spec-VLA-style dense drafter with the same interface optimizations (49.93% vs. 48.21%) at 11.35x lower analytical drafter cost, and a second branch, still 6.35x cheaper in drafter arithmetic than one dense branch, accepts longer prefixes than the single-branch dense drafter with fewer verifier passes. The released Spec-VLA drafter, run with official Spec-VLA and KERV code, spends 750-1,540x more drafter arithmetic.

Primary area:
applications to robotics, autonomy, planning
Keywords:
spiking neural networks, speculative decoding, vision-language-action models, energy-efficient inference, neuromorphic computing, robot manipulation.

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ze7HOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

Spec-Spike combines a spiking action drafter with cached and low-rank interfaces for speculative decoding under a frozen OpenVLA verifier. The framework recovers compressed projections using the original drafter’s prediction histories and expands candidate branches to explore the trade-off between proposal cost and task performance.

Critical strengths
  1. The paper identifies dense interfaces as an important bottleneck in spiking drafters and evaluates optimizations beyond the spiking core.

  2. The matched ANN/SNN evaluation covers multiple LIBERO suites and three training seeds with paired closed-loop comparisons.

  3. The paper transparently reports training exposure, negative outcomes, adaptive development, and the distinction between analytical cost and measured latency.

Critical weaknesses
  1. All evaluation initial states were used to generate drafter-training trajectories, so the decision-excluded masks do not establish generalization to training-unseen states and limit comparisons with externally trained systems.

  2. Parameter matching does not isolate spiking dynamics from architecture, integer activations, or the SNN-only activity penalty; activation-quantized and compute-matched ANN controls are missing.

  3. The cost model excludes memory traffic and neuron-state updates, while Table 6 shows neither lower GPU latency nor lower total modeled arithmetic cost than the matched ANN; an end-to-end energy benefit remains unverified.

  4. Three training seeds do not establish task-success non-inferiority, and significant declines occur against the ANN on LIBERO-Goal and against verifier-only decoding on LIBERO-Long.

  5. Although prior history-aligned distillation and low-rank methods are cited, the additional benefit of own-history recovery is not established through matched closed-loop comparisons with alternative recovery strategies, and its implementation requires more precise definitions.

Questions for authors
  1. How does Spec-Spike perform when evaluation initial states are excluded from drafter training, calibration, recovery, and all development decisions?

  2. Can you compare against activation-quantized and compute-matched ANN drafters, including a control with matched architecture and activity regularization, to isolate the contribution of spiking dynamics?

  3. What are the measured end-to-end latency and energy when verifier execution, memory traffic, neuron-state updates, CPU–GPU communication where applicable, and lookup-table construction are included?

  4. What additional independent-seed results and prespecified non-inferiority margin would support preserved task success, and how does this conclusion change across acceptance radii?

  5. Can you compare recovery strategies in closed loop under matched training budgets and specify the I-LIF recurrence, initial hidden state, and SmoothL1 reduction required to reproduce the recovery procedure?

Reviewer_g9vCOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SPEC-SPIKE, which uses a spiking neural network as the drafter for VLA speculative decoding, and reduces proposal cost through context caching, token projection lookup tables, low-rank interface compression, and recovery training based on its own prediction history. On the LIBERO benchmark, its analytical drafter cost is about 1/11 that of a parameter-matched ANN, while no stable difference in task success rate is detected.

Critical strengths
  1. The paper's discussion and analysis of the computational bottleneck are relatively clear, and this can be used as a focus to explain how the spike-based use of VLA should be further reduced in computation.

  2. The paper's explanation of the design and how the related computations are performed is very detailed, and the experimental transparency is relatively high.

  3. Judging from the computational results, the Spec-VLA baseline that the paper compares against and reproduces performs well, with the basic effect being close, while achieving better simulated energy consumption reduction.

Critical weaknesses
  1. Compared with ANN computation, the simulated numerical values currently used are not fully credible. Because it still uses 45nm, fp32 simulated operation values, distinguishing the entire AC and MAC operations with 4.6 and 0.9 pJ, but the current experiments are based on bf16 GPU implementations, more computations are excluded.

  2. According to the paper's explanation, the verifier prefill computation occupies 8.8 J of energy consumption. Compared with the drafter optimized to a final 3.8 mJ and then 0.34 mJ, the difference is negligible, which substantially affects the practical system significance. Moreover, the current SNN end-to-end tests show no advantage. The 750-1540x computation gap in the paper essentially has similar problems.

  3. From cross-seed reproduction, the computed intervals vary widely, and the overall stability is weak.

  4. Fig. 6 is quite obviously AI-drawn, with distorted vertical lines.

Questions for authors

see weakness

Reviewer_nrp9OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This work proposes Spec-Spike, a spiking neural network (SNN) drafter for speculative decoding in vision-language-action (VLA) models. The authors find that replacing the drafter's core with an SNN only reduces about 3× overall cost because of dense interface projections. To address this limitation, Spec-Spike restructures each interface by how often its input changes and recovers the factors on its own prediction histories. Experiments on LIBERO show that Spec-Spike achieves comparable task success to ANN drafter while greatly reducing drafter cost.

Critical strengths
  1. This work investigates an interesting problem of reducing the computational overhead of speculative decoding for VLA models. The analysis of the drafter's computational cost provides useful insights into how dense interface operations can limit the benefits of an SNN core.

  2. The experiments cover three LIBERO suites and include comparisons with a ANN drafter. The results suggest that substantial reductions in the drafter's cost can be achieved without degradation in task success relative to the ANN baseline.

Critical weaknesses
  1. The motivation and energy benefits of using an SNN drafter are insufficiently established. The proposed approach retains the large ANN verifier, which remains the dominant source of computational cost. The paper should better justify why optimizing this relatively inexpensive component is worthwhile when the dominant verifier remains dense.

  2. The related-work discussion is insufficient. Although SpikeVLA [1] is briefly mentioned, the paper does not mention recent SNN-VLA approaches [2, 3]. It also overlooks relevant work on spiking Decision Transformers [4] and provides limited engagement with the broader literature on SNN-based reinforcement learning.

Reference

[1] Song R, Nie D, Teng S, Ding B, Zhang X, Li D, Zhang C, Li Y, Wu H, Chen L. SpikeVLA: Vision-Language-Action Models with Spiking Neural Networks.

[2] Wang J, Zhang D, Wang S, Zhang M, Yang Y, Li H. SpikingVLA: Asynchronous Spiking Vision-Language-Action Models.

[3] Wang S, Zhang M, Liu M, Dai W, Zhang D, Zhang J, Shan Y, Zhou Z, Yang Y. Spike-driven Vision-Language-Action Model.

[4] Huang W, Gu Q, Ye N. Decision spikeformer: Spike-driven transformer for decision making.

Questions for authors

Could you provide a comparison with recent SNN-VLA approaches [1, 2, 3], covering task performance and energy consumption?

How does Spec-Spike compare with approaches that use SNNs throughout the VLA pipeline rather than only in the drafter?

Given the recent progress in SNN-VLA and spiking sequence-decision models [1–4], what is the principal advantage of using an SNN specifically as a speculative drafter, compared with directly developing an end-to-end SNN-based VLA policy?

Under what hardware or deployment conditions would the proposed hybrid architecture be deployed and preferable?

Reference

[1] Song R, Nie D, Teng S, Ding B, Zhang X, Li D, Zhang C, Li Y, Wu H, Chen L. SpikeVLA: Vision-Language-Action Models with Spiking Neural Networks.

[2] Wang J, Zhang D, Wang S, Zhang M, Yang Y, Li H. SpikingVLA: Asynchronous Spiking Vision-Language-Action Models.

[3] Wang S, Zhang M, Liu M, Dai W, Zhang D, Zhang J, Shan Y, Zhou Z, Yang Y. Spike-driven Vision-Language-Action Model.

[4] Huang W, Gu Q, Ye N. Decision spikeformer: Spike-driven transformer for decision making.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.