acceptodds
Withdrawn from ICLR 2027

High-Degree Node Energy Smearing for Graph Neural Networks quantization

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Graph Neural Networks (GNNs) achieve state-of-the-art accuracy on irregularly structured data, yet efficient deployment on resource-constrained platforms remains challenging. This challenge is growing in importance as GNNs are increasingly incorporated into real-world products. In recent years, graphs underlying real-world applications have increased dramatically in size, making it necessary to operate under strict energy, latency, and memory constraints. Quantization has emerged as a promising approach to address these challenges. However, its accuracy degradation in message-passing GNNs is often severe. We show that this degradation is fundamentally topological: activation energy grows with node degree, causing hub nodes to dominate the dynamic range and reducing quantization resolution. Existing methods treat quantization error as a calibration or smoothness problem and fail to address this deterministic range explosion. We propose High-Degree Node Energy Smearing (HD-NES), a topology-aware method that applies lightweight, reversible rotation transformations to diffuse the feature energy of hub nodes. By reducing activation variance while preserving information via reversibility, HD-NES mitigates hub-induced outliers and enables accurate low-bit quantization. HD-NES integrates into existing GNN quantization pipelines without architectural changes and consistently outperforms prior approaches.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph Neural Networks, Quantization

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_CWqAOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a quantization method called High-Degree Node Energy Smearing, which suppresses the numerical magnitudes of hub nodes via rotation transformations, thereby reducing the variance in activations for quantization and improving quantization accuracy. Experiments show that the method achieves good results across various tasks.

Critical strengths
  1. The paper is well written and easy to follow.
  2. The motivation is clear and well grounded: the existence of hub nodes and tail nodes indeed makes GNN quantization difficult.
Critical weaknesses
  1. The novelty is limited. First, the outlier phenomenon in GNNs has been extensively discussed in prior work. Second, rotation-matrix-based solutions have been widely adopted for outlier suppression in LLM quantization. What is the fundamental difference between the rotation method proposed in this paper and existing rotation-based methods?
  2. The experiments are insufficient. The paper does not conduct direct comparisons with current state-of-the-art methods, such as DQ and A2QA^2Q, which makes it hard to demonstrate the effectiveness of the proposed method.
Questions for authors
  1. What is the fundamental difference between the rotation method proposed in this paper and rotation methods applied in LLM quantization (e.g., SpinQuant: LLM quantization with learned rotations)?
  2. Please supplement the experiments with direct comparisons against previous state-of-the-art methods, such as DQ and A2QA^2Q.
Reviewer_fGM7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper attributes the accuracy loss of low-bit GNN quantization to graph topology: under sum-style aggregation, high-degree nodes dominate the per-tensor quantization scale and low-degree nodes are rounded to zero. HD-NES mean-centres the activation matrix, splits it into contiguous blocks of B rows, and applies a randomized Hadamard rotation along the node axis (GCN/GIN) or the edge-message axis (GAT) before quantization. Experiments compare against a uniform symmetric QAT baseline at W8A8 and W4A4 on five datasets and stack HD-NES on DegreeQuant and SMP, and show superior performance.

Critical strengths
  1. The rotation axis is shown to matter through a controlled contrast. On ogbn-arxiv at W4A4 (Figs 3-4), rotating along the node axis improves GIN and rotating along the edge-message axis improves GATv2, while a QuaRot-style rotation along the feature axis with the same block sizes helps neither model. This isolates the choice of axis as the active ingredient, and it is the paper's genuinely new observation relative to rotation-based LLM quantization.
  2. Consistent and inexpensive improvements over the paper's own baseline. At W4A4, HD-NES improves on the uniform QAT baseline in almost every cell of Table 1, usually by far more than the reported 5-seed standard deviation, with the largest gains on ogbl-citation2 and the attention models. It also adds modest gains on top of DegreeQuant and SMP (Tables 2-3), and the latency overhead measured on one Reddit GCN layer is well below one millisecond (Table 4). The method needs no change to the architecture or the training recipe.
Critical weaknesses
  1. The Table 1 margins cannot be attributed to the rotation, and the claim of outperforming prior approaches has no head-to-head support. The sole Table 1 comparator uses a max-based scale and already loses a large part of its accuracy at 8 bits on several datasets, most severely on ogbl-citation2 (Table 1), yet somewhere describes W8A8 as "reasonable fidelity". No clipped, LSQ, per-channel or per-group baseline is run on the Table 1 datasets, there is no identity-rotation control with the same centering and blocking, and the quantizer granularity is never stated. Prior GNN quantizers appear only as "X" versus "HD-NES + X" on two small datasets (Tables 2-3), and MixQ and A2Q are not run at all. The reader therefore cannot tell whether the large gains measure the rotation or the weakness of the baseline.
  2. The topological thesis is never tested and is partly contradicted by the paper's own data. Blocks are contiguous index slices that use no degree or adjacency information, and choosing B from degree centrality is left to future work. No degree-bucketed error analysis or node-ordering ablation is reported. GAT, which Sec. 3.4 describes as degree-independent, is the architecture that degrades most at W8A8 on ogbl-citation2 (Table 1). The block-size curve is essentially flat from B = 32 up to the largest block tested (Fig. 3), which is hard to reconcile with the claim that blocks are "local enough to preserve graph structure".
Questions for authors
  1. What scale granularity and range estimator are used for the Table 1 QAT baseline and for HD-NES, and do the Table 1 margins survive an identity-rotation control with the same blocking and centering plus one clipped or per-group uniform baseline?
  2. Which split was used to select B and the HD-NES layer subset, and does Table 1 report test or validation accuracy? The peaks of Figs. 3-4 coincide with the corresponding Table 1 entries.
  3. Why does GAT, which your analysis treats as degree-independent, degrade most at W8A8 on ogbl-citation2, and are the link-prediction decoder and edge scores quantized?
  4. The peaks in Fig. 2(b) and Fig. 5 fall at unit fractions, which is what near-uniform attention over a low-degree node's inputs produces. How do you rule out that the multimodal structure is a degree artefact rather than a property of attention mechanics?
  5. Please report HD-NES alone against DegreeQuant, SMP and A2Q alone on the Table 1 datasets at matched backbone and bit-width, and state whether the baseline rows in Tables 2-3 were re-run under the same seeds and folds as the "+HD-NES" rows.
  6. In the W4A4 pipeline, which tensor does neighbour aggregation consume and at what integer width, and which INT4 kernels produced the INT4 rows of Table 4?
  7. Please report degree-bucketed error with and without HD-NES, and results with blocks formed from random, degree-sorted and native node order. If ordering does not matter, the "topology-aware" framing should be dropped in favour of node/edge-axis rotation.
  8. Please confirm the Pubmed GIN entries in Table 1 and correct the Table 2 delta.
Reviewer_pgnyOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

GNN quantization has a well-known problem of dealing with hub node features: hub nodes can exhibit very large outlier magnitudes after sum-based aggregation, reducing quantization precision for most nodes. This paper proposes HD-NES, which distributes and restores the hub node feature magnitude by using reversible rotation operations with Hadamard Transformation. The proposed method shows accuracy benefits compared to a naive QAT baseline.

Critical strengths
  1. The paper has a detailed analysis of the well-known problem for GNN quantization, and where degradation comes from.
Critical weaknesses
  1. The problem and the method that the paper suggests lacks new findings or advancement for the challenges of GNN quantization, and the method itself has insufficient evidence of advantages over prior works.

    • The problem and findings in analysis elaborated at section 3 seems to be identical to those in DegreeQuant [1], with limited findings.
    • The paper provides limited evidence of improved quantization, either alone or in combination with prior methods.
      • The proposed method were only compared with the naive QAT baseline, and whether the experiment were done in transductive setting or inductive setting for node classification is unclear.
      • Although memory requirements from large graphs motivate the method, its benefits on larger graphs, such as Reddit and ogbn-products, remain unclear without evaluation.
      • Using more rotation matrices appears to introduce a trade-off between memory usage and quantization quality. The paper does not quantify the memory overhead of storing rotation matrices relative to model weights or peak memory usage during quantized GNN inference.
  2. The overall manuscript is not self-contained enough and therefore feels rather incomplete.

    • The method description does not clearly explain where rotation and restoration are applied within a GNN layer or how the transformation matrix is constructed. Also, it lacks essential background of how GNN operates (e.g., the message-passing mechanism). These hinders the understanding of readers without background knowledge.
    • Output equivalence is not clearly demonstrated through figures or equations, leaving unclear whether the added operations preserve the layer’s exact output or some other information.
    • Moreover, the final contribution stated, and significant amount of experiments are found only at the appendix. Every component necessary for presenting the method should be in the main manuscript. The experimental section 5.5 relies too heavily on references to the appendix.

[1]. Tailor, Shyam A., Javier Fernandez-Marques, and Nicholas D. Lane. "Degree-quant: Quantization-aware training for graph neural networks." arXiv preprint arXiv:2008.05000 (2020).

Questions for authors
  1. Since solving outlier issues with rotation matrices is already common in the field of LLM quantization, it is encouraged to introduce the new challenges when adapting the same remedy for GNNs, or provide convincing evidence of why using rotation matrices is better for GNN quantization.

    • Are there new challenges? If so, how does the method address that?
    • How does the proposed method differ from rotation-based methods in LLM quantization, besides the targeted model architecture?
    • Why should we use rotation matrices for quantized GNNs, instead of prior works, in terms of quality, speed, and memory costs?
  2. Please clarify where rotation and restoration occur within a GNN layer and whether they preserve its output. If the operation does not preserve output equivalence, the paper can specify what information is preserved.

    • In methods[1,2] that propose similar approaches, they use figures and equations to demonstrate output equivalence. This paper should provide similar verification. How does the method integrate into a GNN layer?
    • How is the output modified, or preserved in the modified GNN layer?
  3. As adding rotations lead to adding computation and memory, empirical results showing that the cost is negligible may be provided. There are some ambiguities to be clarified, regarding such a point.

    • Table 4 reports inference cost for one graph with block size 64, while accuracy experiments use dataset-specific block sizes. How does inference cost vary with graph size and block size?
    • If one does minibatch inference for a very large graph, will the additional computation cost scale with batch size? If so, how much?
    • How much memory is needed to store the rotation matrices relative to model weights, and how does this affect peak memory during quantized GNN inference?
  4. Evaluation on larger datasets, with comparison with baselines, is recommended.

    • How are the results compared with prior GNN quantization works, not a naive QAT baseline?
    • How are the results in a much larger graph, such as Reddit and ogbn-products?

[1] Ashkboos, Saleh, et al. "Quarot: Outlier-free 4-bit inference in rotated llms." Advances in Neural Information Processing Systems 37 (2024).

[2] Liu, Zechun, et al. "Spinquant: Llm quantization with learned rotations." International Conference on Learning Representations. 2025.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.