acceptodds
Under review as a conference paper at ICLR 2027

Energy-Based Single-Layer Spiking Graph Neural Networks

Abstract

Spiking graph neural networks have shown strong potential for efficient graph representation learning due to their discrete communication mechanism. However, most existing methods still inherit the hierarchically stacked message-passing paradigm of conventional graph neural networks and rely on multi-step output averaging for prediction, making them vulnerable to over-smoothing, spike information degradation, and high inference latency. To address these issues, we propose an energy-based single-layer spiking graph representation learning model that formulates node representation learning as a finite-horizon spiking dynamical process on graphs. Specifically, we decouple the neuronal dynamics into three components: a constant external current for mitigating over-smoothing, a bidirectional graph attention current for structural interaction, and a Hopfield current for refining node states via prior feature memory. Furthermore, we establish a rigorous theoretical framework based on a global energy functional, mathematically proving that the network's temporal evolution intrinsically executes an energy minimization process, thereby guaranteeing system stability and convergence. Extensive node classification experiments on eight benchmark datasets demonstrate that the proposed method achieves competitive performance while exhibiting clear advantages in dynamical stability, anti-over-smoothing capability, and inference efficiency.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph Neural Network, Spike Neural Network, Hopfield Network

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_XQdcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a single-layer spiking graph neural network that replaces conventional spatial layer stacking with temporal evolution of spiking neuron dynamics. The input current of each LIF neuron is decoupled into an external driving current and an internal interaction current, where the latter consists of a graph-attention current and a Hopfield associative-memory current. The authors construct a global energy functional with membrane-recovery, internal-aggregation, and external-driving terms, and provide theoretical analyses showing monotonic energy decrease under soft-threshold continuous dynamics and hard-threshold discrete state flips. Experiments on eight node-classification benchmarks show strong performance among spiking GNNs, competitive results against continuous GNNs, resistance to over-smoothing under long temporal evolution, and favorable theoretical computational cost. Overall, the paper presents a biologically motivated and theoretically grounded alternative to hierarchical spiking GNNs.

Critical strengths
  1. The temporal-evolution paradigm is a clean and biologically plausible alternative to spatial stacking in spiking GNNs.

  2. The decoupling of external driving and internal graph-attention/Hopfield currents is well motivated and leads to an interpretable energy-based formulation.

  3. The paper provides convergence/stability analyses for both soft-threshold and hard-threshold firing, which is stronger than many empirical spiking GNN papers.

Critical weaknesses
  1. The energy functional is used for theoretical analysis, but it is not clear whether it participates in the training objective or only serves as an analytical tool. This should be stated explicitly.

  2. The ablation study focuses on current components but does not sufficiently analyze key hyperparameters.

Questions for authors
  1. How often does the condition σ2(HIext)>0\sigma_{2}(H I_{\mathrm{ext}}) > 0 hold empirically for trained models on the eight datasets?

  2. Does the energy-descent guarantee still hold under synchronous hard-threshold updates used in Algorithm 1, or only under asynchronous updates?

  3. What are the training-time memory and runtime costs compared with SpikeGCL, SpikeNet, and at least one ODE-based GNN?

  4. How does the method perform on heterophilous graphs and on tasks beyond node classification?

2 comments
AuthorsOpenReview ↗

rebuttal(1/2)

We thank the reviewer for the positive assessment of our temporal evolution architecture, current decomposition, and energy analysis. In response to your questions, we have added relevant explanations and experimental results and respond point by point below.

Q1. Is the energy function involved in the training objective?

The parameters in the energy function are continuously updated during training. In fact, the energy function is not used as an additional training loss, but it can describe the state of the network's evolution. The energy function unifies membrane recovery, graph attention, Hopfield memory, and sustained external drive within the same dynamical framework and is used to analyze its stability. We will explain this in the paper.

Q2. How sensitive is the method to key hyperparameters?

We added sensitivity experiments on PubMed and CS for the number of memories MM, the time constant τ\tau, and the step size dtdt. The default setting is M=256,τ=2,dt=1M=256,\tau=2,dt=1. Only one parameter was changed at a time, and each configuration was trained with 3 seeds for 200 epochs. The table below reports validation accuracy (%), with the sweep columns reporting means in the order given in their headings.

DatasetDefault settingM=64/128/512M=64/128/512τ=1/4\tau=1/4dt=0.5/2/3dt=0.5/2/3
PubMed92.47 ± 0.4292.40 / 92.13 / 92.4092.40 / 92.3392.53 / 92.67 / 92.27
CS94.16 ± 0.4094.03 / 94.07 / 94.1194.33 / 94.0394.13 / 93.94 / 93.65

Although different hyperparameter choices lead to small performance variations, none of the individual hyperparameters causes large fluctuations in model performance within the tested range. These results demonstrate that our model remains relatively stable over reasonable ranges of hyperparameter values.

Q3. Does the condition σ2(HIext)>0\sigma_2(HI_{\mathrm{ext}})>0 hold in the trained models?

The minimum second singular values of the trained models on seven datasets are shown below.

DatasetMinimum σ2(HIext)\sigma_2(HI_{\mathrm{ext}})
Cora78.359
CiteSeer138.676
PubMed311.443
Computers235.275
Photo128.458
CS389.590
Physics580.105
ogbn-arxiv872.762

All values are clearly greater than zero, indicating that the centered external currents of the evaluated models retain at least two linearly independent directions of variation across nodes, supporting the nondegeneracy condition required by Theorem 1.

Q4. Do synchronous hard-threshold updates have an energy descent guarantee?

The current proof of hard-threshold energy descent applies to asynchronous updates. For the synchronous updates in Algorithm 1, we have not yet established a corresponding rigorous descent guarantee. We will clarify this scope in the revised manuscript and further investigate sufficient conditions for descent and theoretical analysis of synchronous updates in future work.

AuthorsOpenReview ↗

rebuttal (2/2)

Q5. How do training time and GPU memory usage compare with SpikeGCL, SpikeNet, and ODE models?

Figure 6 already reports training cost comparisons with SpikeGCL and SpikeNet. Following your suggestion, we retained the original experimental configuration and added the ODE model GREAD. The results below were obtained on the Computers dataset using an RTX 3090 with T=5T=5, and the timing statistics use 5 random seeds.

MethodTime per epoch (ms)Peak GPU memory during training (MiB)
Our method83.98 ± 0.53773.45
SpikeGCL102.99 ± 2.80721.81
SpikeNet219.02 ± 9.331045.92
GREAD78.57 ± 0.323902.64

Under these configurations, our time per epoch is reduced by approximately 18.5% and 61.7% compared with SpikeGCL and SpikeNet, respectively. GREAD is slightly faster per epoch, but our peak GPU memory usage during the procedure is approximately 80.2% lower. In addition, at T=16T=16 in Figure 6, the peak GPU memory usage of our method and SpikeGCL is 848.30 and 1665.74 MiB, respectively, demonstrating the memory advantage with longer temporal unfolding.

Q6. How does the method perform on heterophilous graphs and tasks beyond node classification?

Heterophilous graphs. Our local interaction mechanism is more naturally suited to homophilous graphs because it tends to strengthen related states between adjacent nodes. On strongly heterophilous graphs, one-hop neighbors may contain information inconsistent with the central node's label, so relying solely on local neighborhood interactions may limit the model's performance.

However, the external current design in our framework is highly flexible and can explicitly incorporate multi-hop structural information.

Iext(K)=Z0+∑k=1KA^kZ0Wk,I_{\mathrm{ext}}^{(K)} = Z_0+\sum_{k=1}^{K}\widehat A^{k}Z_0W_k,

Here, Z0∈RN×DZ_0\in\mathbb R^{N\times D} denotes the encoded node features, A^\widehat A is the graph propagation matrix, WkW_k is the learnable mapping for each hop, and KK is the maximum number of aggregation hops. K=0K=0 corresponds to the original external current, while K=1,2K=1,2 additionally incorporate one-hop and two-hop information, respectively. We conducted experiments on Chameleon and Squirrel using the 10 publicly available fixed splits provided by Geom-GCN [1]. The test accuracies are shown below.

MethodChameleon (%)Squirrel (%)
SpikeNet60.68 ± 1.4340.49 ± 1.03
SpikeGCL75.29 ± 1.4259.61 ± 1.22
MSG42.02 ± 2.9527.48 ± 1.31
Ours(K=0K=0)54.82 ± 1.9339.00 ± 1.07
Ours(K=1K=1)66.58 ± 1.2649.84 ± 1.20
Ours(K=2K=2)67.43 ± 1.6250.71 ± 1.56

Compared withK=0 , both K=1K=1 and K=2K=2 lead to clear accuracy improvements on Chameleon and Squirrel. These results demonstrate that incorporating multi-hop structural information into the external current can improve performance on heterophilic graphs. Although our method does not yet achieve the best performance under this setting, the results validate the extensibility of our external-current design to heterophilic scenarios. A more systematic design of heterophily-aware external currents will be investigated in future work.

Graph-level tasks. Our method can be naturally extended to graph-level tasks. Without modifying the underlying dynamics, we only need to replace the node-level readout layer with a graph-level readout operation.

We conducted additional experiments on two graph-level regression datasets, ESOL and FreeSolv. Each dataset was split into training, validation, and test sets using an 8:1:1 ratio. The evaluation metric is the test RMSE, and all results are reported as the mean ± standard deviation over five random seeds.

MethodESOL RMSE ↓FreeSolv RMSE ↓
GCN0.801 ± 0.0391.257 ± 0.051
GAT0.668 ± 0.0151.085 ± 0.069
SpikeGCL0.908 ± 0.0541.616 ± 0.080
SpikeGraphormer0.695 ± 0.0491.004 ± 0.161
Ours0.678 ± 0.0250.902 ± 0.037

Our method achieves the best performance on FreeSolv and competitive performance on ESOL, demonstrating that the proposed dynamics can also be applied to graph-level prediction tasks.

It should be noted that graph-level tasks are fundamentally different from node-classification tasks in terms of the energy formulation. For node classification, the energy function is constructed over the entire graph dataset. In contrast, for graph-level tasks, an energy function is constructed independently for each graph sample.

[1] Geom-GCN: Geometric Graph Convolutional Networks

Reviewer_9aiLOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

To address the issues of over-smoothing, spike information degradation, and the reliance on multi-step rate coding in existing spiking graph neural networks, this paper proposes an energy-based single-layer spiking graph neural network method. By formulating graph message-passing as a finite-horizon spiking dynamical process and decoupling it into a constant external current, a bidirectional graph attention current, and a Hopfield current, this method accomplishes node state evolution and classification through the minimization of a global energy functional. Comparative experiments were conducted on multiple public benchmark datasets, validating the effectiveness of the proposed method.

Critical strengths

1.The model successfully transforms the complexity of graph representation learning from "hierarchical spatial stacking" to "continuous temporal evolution." This single-layer dynamical design effectively circumvents the pervasive issues of over-smoothing and spike vanishing induced by traditional multi-layer GNNs, offering a highly promising and efficient new paradigm for spiking graph representation learning.

2.The authors not only decouple the neuronal dynamics, but also prove, through a series of theorems, the mathematical guarantee of resistance to over-smoothing brought by introducing the external driving current, as well as conduct a theoretical analysis on the monotonic energy decrease of the system under both soft-threshold and hard-threshold firing mechanisms, thereby providing certain theoretical support for the reliability of the proposed method.

3.The authors evaluate node classification on eight public datasets and compare with existing spiking GNNs. The reported performance generally exceeds that of the baselines, and ablation studies assess the contribution of each current component. These experiments provide empirical support for the effectiveness of the proposed method, though the evaluation is limited to small- and medium-scale datasets.

Critical weaknesses

1.Although the method transforms hierarchical spatial stacking into temporal evolution, its substantial contributions at the theoretical level require further clarification. Compared to existing related works on energy-based models, the specific formulation of the energy functional and the positioning of the theoretical contributions are not sufficiently clear. The paper does not adequately discuss the mathematical relationship between the Hopfield current term PTPP^TPand the QKTQK^T term in self-attention mechanisms, leaving the argumentation for its novelty somewhat weak.

2.The research motivation in Section 1 provides insufficient support for practical demands or theoretical gaps, Emphasizing biological plausibility cannot mask the lack of motivation for handling graph topology evolution. The paper fails to clearly articulate the irreplaceable advantages of introducing spiking nonlinearity over traditional aggregation when capturing complex topological structures. Furthermore, the efficiency argument that "the number of layers equals the number of forward propagations" entails conceptual confusion and requires further clarification.

3.The scalability of the global Hopfield memory matrix in Eq. (4) of Section 3.1 is not clearly described; furthermore, the initialization strategy, potential orthogonality constraints, and storage capacity limits of the memory matrix PP within the Hopfield current remain undiscussed. For high feature dimensions or large-scale networks, optimizing the global associative memory matrix will incur a substantial computational burden, necessitating a specific supplementary complexity analysis of the proposed model.

Questions for authors

See Critical Weaknesses

5 comments

We thank the reviewer for the detailed questions about the contribution of the energy construction, the temporal evolution architecture, and the meaning of the memory component. We clarify each point below.

Q1. What is the contribution of the energy construction? How does P⊤PP^\top P differ from QK⊤QK^\top?

Existing work has applied energy functions to graph representation learning, describing structural relationships by summing difference terms between node pairs ∑ijρ(∥yi−yj∥2)\sum_{ij}\rho(\|y_i-y_j\|^2) and deriving propagation or adaptive attention from energy optimization [1]. Other work combines the matching energy between nodes and memory patterns with a graph Laplacian term [2]. Our work differs fundamentally from these methods in the interacting entities described by the energy and the resulting dynamics. Our graph attention potential describes the degree of matching between a node and its neighboring nodes. Since each node participates in the energy of both its own neighborhood and the neighborhoods of other nodes, the gradient of the global potential naturally produces forward and reverse attention currents. We further combine this attention potential with membrane recovery, sustained external drive, and memory feedback to form a unified global energy.

The difference between$P^\top P$ and QK⊤QK^\top. The two differ in their scope of operation. P⊤P∈RD×DP^\top P\in\mathbb R^{D\times D} provides shared feature memory feedback within an individual node, while QK⊤∈RN×NQK^\top\in\mathbb R^{N\times N} in node self-attention describes similarities between node pairs. In our work, the graph attention current handles interactions across nodes, and the Hopfield current provides feature-space feedback. The two do not duplicate the same operation.

Q2. Why use temporal evolution and spiking nonlinearity, and what does single layer mean?

We thank the reviewer for raising this question. The main motivation for applying spiking neural networks to graph learning is to reduce the computational and communication costs of repeated neighborhood propagation. Many existing studies combining spiking neural networks with graphs explore precisely this direction. Our contribution is to address the problem of layer stacking in existing spiking graph neural networks. As shown in Figure 1, many methods still follow the multilayer structure of conventional GNNs and perform computation over multiple time steps at each layer, compounding the costs of spatial depth and temporal unfolding. We directly use the graph topology as the connections between neurons and aggregate information through the temporal evolution of the same graph layer, thereby reducing the reliance on stacking spatial layers. In addition, we need to clarify that we did not state in the paper that the number of layers is equivalent to the number of complete forward passes. Single layer refers to the unified graph dynamics layer and the reuse of its parameters over time.

Q3. Initialization, capacity, and scalability of the Hopfield memory matrix

We thank the reviewer for their attention to this issue. Our memory matrix P∈RM×DP\in\mathbb R^{M\times D} is shared across all nodes, and its size is independent of the number of nodes N. Therefore, its parameter count is MDMD and does not increase with the number of nodes NN. In the implementation, PP uses the default uniform initialization of a linear layer without bias, in the range [−1/D,1/D][-1/\sqrt D,1/\sqrt D], and is then learned jointly through the supervised classification loss. Our energy construction does not require the memory vectors to be mutually orthogonal. We conducted a sensitivity analysis of M, using MM=64, 128, 256, and 512, with Pubmed and Computers as examples. The results are shown below.

DatasetM=256M=256M=64M=64M=128M=128M=512M=512
PubMed92.4792.4092.1392.40
CS94.1694.0394.0794.11

Validation performance varies very little across these settings. Therefore, the choice of MM in practice does not need to depend on the number of nodes.

[1] Graph neural networks inspired by classical iterative algorithms.

[2] Graph Hopfield Networks: Energy-Based Node Classification with Associative Memory.

Reviewer_9aiLOpenReview ↗

Response to Authors

Thank you for your response and clarifications, which have addressed most of my concerns. However, the analysis of model complexity remains unresolved, so I have decided to keep my current score unchanged.

AuthorsOpenReview ↗

Model Complexity and Parameter Analysis

Thank you for your follow-up. We apologize for overlooking this question in our previous response. Below, we analyze the computational complexity and parameter count of the attention and Hopfield currents. Let NN, EE, DD, MM, and KK denote the numbers of nodes, edges, hidden dimensions, memory patterns, and time steps, respectively.

Attention current. Node-state projection and feature mapping after aggregation require O(ND2)O(ND^2) operations. Similarity computation, neighborhood normalization, and bidirectional message aggregation operate only on existing edges, requiring O(ED)O(ED) operations. The learnable matrix W∈RD×DW\in\mathbb{R}^{D\times D} is shared by both current directions, contributing D2D^2 parameters, independently of NN.

Hopfield current. The memory matrix P∈RM×DP\in\mathbb{R}^{M\times D} is shared across all nodes and time steps. Mapping node states into the memory space and back into the feature space requires O(NMD)O(NMD) operations per step. These mappings share PP and its transpose, contributing MDMD matrix parameters, independently of NN.

The complexity of the two currents is summarized below:

ModulePer-step computational complexityMatrix parameter count
Attention currentO(ND2+ED)O(ND^2+ED)D2D^2
Hopfield currentO(NMD)O(NMD)MDMD

Over KK time steps, their total computational complexity is O(K(ND2+ED+NMD))O\left(K(ND^2+ED+NMD)\right). Their combined matrix parameter count is D2+MDD^2+MD, independent of the graph size and number of time steps. This parameter sharing does not eliminate the dependence of computation and intermediate storage on graph size.

Thank you again for your feedback. We would be happy to provide further clarification if needed.

Reviewer_25aHOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper formulates graph representation learning as the temporal evolution of a single-layer spiking network. Nodes repeatedly exchange information through their evolving spike states while remaining conditioned on their original features. Predictions use only the final state, avoiding the temporal averaging used in many existing spiking graph models. The proposed method constructs a global energy function to explain how these interactions guide representation updates and establish stability properties under stated assumptions. Experiments on eight node classification benchmarks report competitive accuracy against the evaluated baselines. Further analyses report stable accuracy and positive Dirichlet energy over extended evolution, alongside lower training overhead and estimated inference energy than the evaluated spiking baselines.

Critical strengths
  1. The formulation explicitly connects the interaction currents to a global energy function. Including both incoming and outgoing attention contributions makes the relationship between graph interactions and energy gradients concrete.

  2. The experiments report competitive results across eight datasets under both threshold mappings. Empirical experiments provide useful evidence that the evaluated configurations retain predictive performance as propagation continues.

Critical weaknesses
  1. Energy-based graph propagation has been explored [1][2]. The paper provides limited discussion of its methodological differences from these approaches. Neither Eq. (1) nor Algorithm 1 includes membrane reset, raising the question of what the proposed spiking dynamics contribute beyond thresholded recurrent message passing.

  2. The ablation study provides limited support for the emphasized Hopfield mechanism. Removing it reduces accuracy by only 0.2–1.0 percentage points, while removing attention performs almost identically to removing all internal currents.

  3. Theorem 1 shows that membrane potentials do not become identical across all nodes. However, different membrane potentials can still produce the same binary outputs after thresholding. The theorem therefore does not guarantee that the representations used for prediction preserve differences between nodes.

  4. Figure 6 reports training time rather than inference latency. Appendix E estimates inference energy, but does not explain how the costs of real-valued attention and repeated membrane updates are calculated. This makes the claimed inference savings difficult to assess.

References:

[1] Graph neural networks inspired by classical iterative algorithms.

[2] Graph Hopfield Networks: Energy-Based Node Classification with Associative Memory.

Questions for authors
  1. Compare continuous feature injection with using node features only to initialize membrane potentials. Report paired accuracy differences across runs for the Hopfield ablation to show whether its gains are consistent.

  2. Explain how the theoretical inference costs are calculated and how all operations during temporal evolution are accounted for. Report measured inference latency using the configurations evaluated in Table 1.

6 comments
AuthorsOpenReview ↗

rebuttal (1/2)

We thank the reviewer for the careful reading and constructive questions. We respond point by point below.

Q1. Differences from TWIRLS and GHN and the meaning of no reset

We thank the reviewer for raising this question. The key difference between our work and TWIRLS [1] and GHN [2] lies in the interacting entities described by the energy and the resulting dynamics. The energy in TWIRLS penalizes deviations of node representations from input features, as well as differences in representations between adjacent nodes, thereby yielding reweighted graph diffusion. GHN combines associative memory retrieval with graph Laplacian regularization. Its LogSumExp potential describes the matching between nodes and memory patterns, while the Laplacian term describes the structural relationships between nodes. In contrast, we construct a LogSumExp potential directly from learnable similarities between nodes within graph neighborhoods, and its gradient naturally produces forward and reverse attention currents. This attention potential, together with the membrane recovery potential, external drive potential, and Hopfield potential, forms the global energy, unifying graph structural interactions, sustained feature drive, and memory feedback within the temporal evolution of the same layer. We will explain this further in the Related Work section.

We adopt reset-free spiking dynamics precisely to unify spiking graph interactions with analyzable energy evolution. Without reset, the membrane potential retains its accumulated state after firing and continues to be regulated by the leak term and net input current. In fact, whether to use reset can be adjusted according to the specific modeling and computational objectives. For example, PSN [3] and SPSN [4] both adopt reset-free spiking neuron designs.

Q2. How consistent are the gains from Hopfield feedback? Why are the results from removing attention close to those from removing all internal currents?

We have added the per-seed results of the Hopfield current ablation experiments to the Figure 10 in Appendix H. Overall, the gains are approximately 0.2–1.0 percentage points. When graph attention is retained, Hopfield feedback provides positive but generally small gains.

In terms of the mechanism, the attention current and Hopfield current serve different functions. The former exchanges neighborhood information, while the latter provides learnable feature memory feedback to the current node state. Therefore, after attention is removed, retaining only the Hopfield current leaves no information exchange between nodes, which explains why its results are close to those obtained by removing all internal currents. When attention is retained, Hopfield can participate in the continuous correction of node states, further improving the final representations.

Q3. Does non-collapse of membrane potentials guarantee non-collapse of the final spikes?

Whether differences in membrane potentials translate into differences in spikes depends on whether they cross the threshold. When different nodes lie on opposite sides of the threshold in at least one channel, their spike representations must differ. To directly examine the spike representations used for prediction, we compute the Dirichlet energy of the spike matrix ST∈{0,1}N×DS_T\in\{0,1\}^{N\times D} at T=100:

D(ST)=12N∑i,jAij∣si(T)−sj(T)∣22.\mathcal D(S_T) =\frac{1}{2N}\sum_{i,j}A_{ij}|s_i(T)-s_j(T)|_2^2.

Here, AA is the adjacency matrix. The squared Euclidean distance equals the Hamming distance between the two spike encodings. The results are shown below.

DatasetDirichlet energy of the final spikes at T=100T=100
Cora4.99 ± 1.32
CiteSeer12.54 ± 1.35
PubMed16.89 ± 1.03
Computers95.71 ± 5.32
Photo81.68 ± 6.91
CS26.82 ± 0.38
Physics19.09 ± 1.52

All spike Dirichlet energies remain strictly positive, indicating that the node differences preserved by the model are indeed reflected in the spike representations.

[1] Graph neural networks inspired by classical iterative algorithms.

[2] Graph Hopfield Networks: Energy-Based Node Classification with Associative Memory.

[3] Parallel Spiking Neurons with High Efficiency and Ability to Learn Long-term Dependencies.

[4] Accelerating SNN Training with Stochastic Parallelizable Spiking Neurons.

AuthorsOpenReview ↗

rebuttal (2/2)

Q4. How does continuous injection differ from using the input only for initialization?

We retrained the following three variants on seven datasets, using five random seeds for each setting.

  • Continuous (continuous injection) uses U(0)=0U(0)=0, with a constant external current injected at every update.
  • First-step (injection only at the first step) uses U(0)=0U(0)=0, with the external current injected only in the first Euler update.
  • Initial-state (setting only the initial state) uses U(0)=IextU(0)=I_{\mathrm{ext}}, with no external current injected thereafter.

The table below reports test accuracy (%) as mean ± sample standard deviation.

DatasetContinuousFirst-stepInitial-state
Cora86.06 ± 0.4031.90 ± 0.0086.32 ± 0.53
CiteSeer78.76 ± 0.8820.86 ± 3.1073.94 ± 1.46
PubMed90.28 ± 0.3940.82 ± 0.2767.94 ± 16.09
Computers89.73 ± 0.2771.75 ± 4.0272.98 ± 9.28
Photo93.57 ± 0.3025.35 ± 0.0039.90 ± 8.63
CS93.70 ± 0.1956.33 ± 16.0185.74 ± 4.77
Physics96.19 ± 0.0966.39 ± 4.5495.26 ± 0.50

Compared with Initial-state, continuous injection achieves higher mean accuracy on six of the seven datasets, with gains of 22.34 and 7.96 percentage points on PubMed and CS, respectively. On Cora, Initial-state is higher by 0.26 percentage points. Under the tested configurations, these results show that sustained external drive can provide benefits beyond setting the initial membrane potential.

Q5. Does the efficiency evaluation include full inference latency and real-valued operations?

We thank the reviewer for their attention to actual computational costs. Our efficiency evaluation covers both actual runtime costs and theoretical energy consumption. Figure 6 already reports training time per epoch and peak GPU memory usage. Appendix E reports theoretical inference energy consumption based on MAC/AC operation counts.

Measured forward-pass latency. In addition to the training costs reported in Figure 6, we measured forward-pass time on the Computers dataset using the same RTX 3090. The table below reports results in milliseconds as mean ± population standard deviation.

TTOur methodSpikeGCLSpikeNet
13.07 ± 0.033.94 ± 0.01261.99 ± 2.29
411.02 ± 0.0214.93 ± 1.801017.19 ± 3.03
821.88 ± 0.0729.47 ± 2.242757.77 ± 82.88
1232.87 ± 0.1342.12 ± 2.083684.68 ± 419.78
1643.68 ± 0.3356.71 ± 2.904185.12 ± 41.60

Our method recorded lower latency in these measurements.

Arithmetic energy accounting. We adopt the operation-level energy estimation framework widely used in spiking neural networks, counting multiply–accumulate operations (MACs) and spike-driven accumulation operations (ACs) separately.

Earith=EMACNMAC+EACNAC,E_{\mathrm{arith}} =E_{\mathrm{MAC}}N_{\mathrm{MAC}} +E_{\mathrm{AC}}N_{\mathrm{AC}},

Here, NMACN_{\mathrm{MAC}} and NACN_{\mathrm{AC}} are the respective operation counts over the full inference process, with the latter counted based on actual spike activity. For example, one membrane potential decay requires one MAC operation. This is consistent with the estimation approach used in works such as Spikformer [1].

Since the parameters are fixed during inference, A=W⊤WA=W^\top W and B=P⊤PB=P^\top P in the attention current and Hopfield current can be precomputed once and shared across all nodes and all time steps. For example,

W⊤∑jαijWsj=∑jαijAsj.W^\top\sum_j\alpha_{ij}Ws_j =\sum_j\alpha_{ij}As_j.

AsjAs_j and BsjBs_j can be implemented by summing the matrix columns activated by binary spikes, which is also an advantage of our single-layer neural network.

[1] Spikformer: When Spiking Neural Network Meets Transformer.

Reviewer_25aHOpenReview ↗

Responses to the authors

Thank you for your responses. Although the authors provide additional experiments and explanations, my concerns still remain as follows:

  1. For Q1, the authors clarify the difference between the proposed method and two existing methods (TWIRLS and GHN). However, my concern about the method’s novelty still remains, because Energy Transformer [1] already combines a neighborhood LogSumExp energy with two directional attention updates and node-local Hopfield feedback.
  2. For Q2, Figure 10 only reports the mean decrease in accuracy after removing the Hopfield current. The authors should provide the paired test accuracy difference for each random seed and report the number of seeds with a negative difference after removing the Hopfield current.
  3. For Q3, I appreciate that the authors provide an additional experiment for Dirichlet energy. However, it is worth noting that my concern is related to the theoretical perspective. The paper should clearly distinguish this empirical evidence from the theoretical guarantee in Theorem 1, which applies only to membrane potentials.

Due to the above concerns, I will keep my score unchanged.

Reference:

[1] Energy Transformer

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.