acceptodds
Under review as a conference paper at ICLR 2027

A Spiking Framework for Efficient Graph Representation Learning

Abstract

Integrating spiking neural networks (SNNs) into graph neural networks (GNNs) offers a promising approach to energy-efficient graph representation learning. However, existing integration methods can still exhibit performance gaps compared with strong real-valued GNNs, while adapting SNNs to these diverse GNN architectures remains challenging. To bridge this gap, we propose a spiking GNN framework applicable to various message-passing neural networks (MPNNs) and graph Transformers. Specifically, for MPNNs, we integrate spiking neurons between layers and reposition the linear transformation modules to the start of each layer. This structure enables the computation of linear transformation to become sparse addition operations. Moreover, our method exhibits a linear decay rate of Dirichlet energy, effectively alleviating the over-smoothing problem. For graph Transformers, we design a novel spike-based self-attention mechanism and use layer normalization to replace the softmax function, which can better align with the discrete characteristics of SNNs and greatly reduce computational overhead. We extend our method to six MPNN models and three graph Transformer models. Without sacrificing model performance, the energy consumption of spike-based models is approximately 10% of that of non-spiking counterparts.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph Neural Network, Spike Neural Network

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_fnCzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a general framework for integrating spiking neural networks (SNNs) into message-passing neural networks (MPNNs) and graph Transformers. For MPNNs, the authors reposition linear transformations and introduce spiking neurons to enable sparse accumulation operations. They also provide a theoretical analysis suggesting that spike thresholding can mitigate over-smoothing. For graph Transformers, the paper introduces spike-based attention using binary queries and keys, replaces softmax with layer normalization, and supports different attention structures.

Experiments across nine graph architectures and six datasets demonstrate competitive predictive performance and substantially lower estimated computational energy, with spiking models consuming approximately 6–19% of their non-spiking counterparts' estimated energy.

Critical strengths

Broad architectural applicability. The framework is evaluated on six MPNNs and three graph Transformers, demonstrating that the proposed spiking integration strategy can accommodate diverse graph learning architectures.

Encouraging empirical performance. Table 1 reports competitive results across six datasets, with improvements in most spiking versus non-spiking comparisons. The attention ablation and over-smoothing visualizations provide additional evidence supporting the proposed design.

Critical weaknesses

Energy efficiency is not validated through actual hardware measurements. The reported energy reductions rely on theoretical MAC/AC operation counts (Section 4.2 and Figure 4), without fully accounting for memory access, spike communication, normalization, and temporal execution overhead. Therefore, the claimed practical energy-efficiency gains remain uncertain.

The theoretical analysis does not establish a general guarantee against over-smoothing. Theorem 1 characterizes the local sensitivity of thresholded spike outputs under distributional assumptions, rather than the propagation of node representations across network depth. Furthermore, the proofs in Appendix C establish asymptotic upper bounds, which alone do not guarantee the claimed separation between spiking and continuous activations. Thus, the theoretical result provides useful intuition but does not rigorously establish that the proposed architecture prevents over-smoothing.

Questions for authors

Can the authors clarify how memory access, spike communication, layer normalization, and temporal execution costs affect the estimated energy savings? Are measured latency or energy results available?

How does the local Dirichlet-energy analysis in Theorem 1 extend to the evolution of node representations across multiple graph layers? Can the authors clarify the assumptions needed for the claimed over-smoothing mitigation?

Could the authors provide an ablation separating the effects of spiking neurons, linear-layer repositioning, and layer normalization, to better identify their individual contributions?

Reviewer_vfgQOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a framework for constructing spiking counterparts of message-passing neural networks and graph Transformers. For message passing, it places spiking neurons between layers and moves linear transformations before aggregation, allowing these transformations to process binary spike inputs through accumulations. For graph Transformers, it introduces binary queries and keys, retains continuous values and replaces softmax with layer normalization after aggregation. The framework covers global attention, adjacency-masked attention and attention with structural bias. Experiments evaluate six message-passing architectures and three graph Transformers on six graph regression and classification datasets. The spiking variants outperform their non-spiking counterparts in most evaluated settings while achieving lower estimated energy consumption.

Critical strengths
  1. Experiments on nine baseline models demonstrate the framework’s applicability.

  2. The firing-rate measurements and temporal-step analysis help explain the computational trade-off of the proposed framework.

Critical weaknesses
  1. The proposed modifications to MPNNs and graph Transformers closely resemble designs used in existing SNNs, including spike-driven linear transformations and softmax-free spiking attention. The paper needs to clarify what unifying design principle makes these adaptations a methodological framework and what it contributes beyond the existing components.

  2. The efficiency claims rely on operation-based energy estimates for an ESOL batch. Although binary inputs enable sparse accumulations in linear transformations, the accounting for aggregation, normalization and multiple time steps is not sufficiently explicit. A clearer MAC/AC breakdown, together with runtime and memory measurements, would help establish how the estimated savings translate into practical efficiency.

  3. The claim that spike-based GCN mitigates over-smoothing is not sufficiently supported by controlled comparisons. Prior work [1] highlights how simple choices such as normalization and residual connections can strengthen conventional GCNs. arxiv.org The paper does not specify these settings for the GCN used in the over-smoothing analysis or isolate which components of the spike-based model produce the observed effect. It therefore remains uncertain whether the improvement comes from spiking dynamics or other architectural changes.

  4. For masked attention in Eq. (14), clarifying whether dense attention or mask matrices are materialized would help assess the implementation. Reporting runtime and peak-memory scaling on larger graphs would further substantiate the scalability claim.

[1] Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification

Questions for authors

Please see the Weaknesses.

Reviewer_z1KTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes a unified spiking neural network framework for graph representation learning to preserve the predictive performance of conventional graph neural networks while reducing computational energy consumption through spike-driven operations. Specifically, it uses LIF neurons and binary spike representations to reduce linear transformation computation and then introduces a theoretical analysis of its effect on graph Dirichlet energy. Experiments on six datasets and nine architectures show competitive performance and low energy consumption.

Critical strengths
  1. This paper addresses the important challenge of improving the computational efficiency of graph representation learning while retaining the architectural advantages of established GNNs and graph Transformers.
  2. Conducting linear transformation based on spike representations can avoid dense multiply-accumulate operations with sparse accumulation. Besides, the authors further extend the standard attention computing to adjacency-masked attention and attention with structural bias..
  3. The analysis of thresholded representations through Dirichlet energy provides a useful theoretical perspective. The visualizations in Figure 5 offer additional empirical evidence that spiking representations can preserve differences between node features.
  4. The proposed framework achieves competitive performance on six MPNN architectures and three graph Transformer architectures.
Critical weaknesses
  1. Several modules have presented in the published literature including spiking graph convolutions, spike-based attention, LayerNorm. The author should identify the fundamentally technical contributions and explain the relationship with existing spiking (graph) Transformers and spiking GNNs.
  2. The claimed linear complexity of attention may not hold in dense structural biases or masked attention learning. Besides, runtime and memory costs are not sufficiently evaluated.
  3. Replacing softmax with LayerNorm may affect attention weights and the model's ability to capture important graph relationships.
  4. The evidence for statistical significance is limited, particularly for results with substantial variance. Besides, more ablations are needed to demonstrate the performance gains w.r.t. spiking dynamics, architectural modifications or additional normalization.
  5. The comparison with existing spiking baselines is not sufficiently comprehensive to establish a clear advantage over the strongest relevant approaches.
Questions for authors
  1. Can the theoretical analysis be extended to multiple layers and more realistic feature distributions?
  2. Does the claimed linear attention complexity still hold when structural biases and attention masks are included? Can authors provide actual runtime, memory usage or other complexity analysis results?
  3. How does replacing softmax with LayerNorm affect attention quality and predictive performance?
  4. Are the performance improvements statistically significant? Which design choices contribute most to the gains?
  5. How does the proposed framework compare with existing spiking graph learning methods, and how well does it scale to larger graphs?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.