acceptodds
Withdrawn from ICLR 2027

Spectral Graph Representation Learning with Cross-Layer Spectral Alignment for Multi-Label Node Classification

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Current graph neural networks for multi-label node classification predominantly rely on entangled node-level representations for direct node classification. This limits their ability to capture global structural dependencies. Motivated by the foundational theories of classical manifold learning and spectral dimensionality reduction, we propose a framework that shifts the paradigm from local node classification to explicit graph-level representation learning. We introduce a dedicated low-rank spectral manifold graph representation as the final output, effectively decoupling the classification space from noisy node embeddings. To optimally deduce this representation, we introduce a novel cross-layer spectral alignment mechanism to preserve multi-scale structural cues, alongside a closed-form manifold induction process that enforces strict geometric orthogonality. Evaluated on four biological benchmark datasets, our method reduces Hamming Loss by 5.0% and increases Micro-F1 by 1.7%, demonstrating that explicit spectral manifold representation with cross-layer alignment yields sharper decision boundaries.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph Neural Networks, Multi-Label Node Classification, Spectral Graph Theory, Manifold Learning, Over-smoothing, Representation Learning, Generalized Power Iteration

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_EZn4OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper attempts to address the over-smoothing problem, or low-frequency bias, in traditional GNN models. It proposes to combine spectral representation learning, cross-layer alignment, and multi-layer GNNs to mitigate this issue. Instead of directly relying on conventional node embeddings, the authors claim to learn, for each node, a low-dimensional, globally optimized spectral manifold representation.

Critical strengths
  1. The paper claims to test on a lot of different datasets and claims good results.
Critical weaknesses

Weakness:

  1. The introduction section is poorly motivated and overclaims the research gap. It does not clearly explain why the MLNC problem requires learning high-frequency signals. It also does not provide enough theoretical or empirical evidence that this is the dominant problem in MLNC. The discussion also largely ignores modern heterophily-aware and frequency-adaptive GNNs, which makes the claimed research gap appear incomplete and insufficiently justified.
  2. The methodology claims to bring a “paradigm shift,” but it is actually a repackaging of well-known ideas, such as learning node representations in the frequency domain using eigenvectors and reformulating the learning process to prevent representation collapse or basis coupling.
  3. Once the optimization problem becomes Equation 7, the optimal FF can be obtained from the eigenvectors of MM. Therefore, the need for SVD is not well explained.
  4. The paper calls the GPI update an “exact closed-form solution,” but it is actually iterative. The matrix GG depends on the matrix FF, and the SVD provides an updated FF, which is still an iterative step. So the algorithm flow should be double-checked before submission.
  5. The paper does not properly explain how high-frequency signals are actually learned or preserved. A lot of prior work has suggested that learning low- and high-frequency components separately, without mixing them, helps resist over-smoothing.
  6. Alignment is not the same as high-frequency preservation. A model can enforce alignment with the previous layer, but that does not guarantee that it learns distinctive or high-frequency signals. Figure 4(d) appears unusually clean and idealized. The figure seems more like an illustrative portrayal of an ideal graph. A real-world graph, where both the structure and features are often noisy, would not usually produce such a clean diagonalization.
  7. A flatter singular-value spectrum only tells us that ZZ uses more independent representation dimensions. It does not necessarily mean that ZZ contains more high-frequency graph signals. The paper should directly analyze the representation energy across Laplacian frequencies.
Questions for authors
  1. Can the authors clarify why the GPI procedure is necessary after equation 7, because the problem is the standard Rayleigh-Ritz problem whose solution is the eigenvectors corresponding to the smallest eigenvalues of M. Since G depends on F, how can equation 9 be considered a closed-form solution??
Reviewer_R6CLOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

SSR-MLNC targets multi-label node classification, aiming to preserve global graph structure and resist over-smoothing while retaining label information. It combines an orthogonal node embedding with label fitting, an adaptive graph Laplacian, and projection-based alignment to a shallow-layer spectral subspace. The proposed training procedure alternates GCN updates with SVD/GPI updates of the embedding, accompanied by optimization claims and spectral analyses. Experiments report improvements across eight datasets and four metrics, together with ablations and robustness studies intended to support the spectral-alignment mechanism.

Critical strengths
  1. Relevant motivation for preserving global graph structure.
  2. Basis-invariant projector alignment and orthogonality-preserving SVD updates.
Critical weaknesses
  1. The "exact closed-form" claim. F = UVᵀ is one GPI [1] step rather than an exact solution: for M = diag(1,2) it gives objective 1.2 against a minimum of 1, and has a fixed point at the maximum. Revising the claim and comparing with a converged eigensolver (e.g., LOBPCG [2]) would help.
  2. No splits or decision thresholds are given, and Y enters the objective over all nodes, so it is unclear whether held-out labels are used. Specifying the label masking, splits and decision rule is essential.
  3. An orthonormal F with m = 16 has an eRank of exactly 16, yet Fig. 4 reports 28.7 and 32 eigenvalues, and Supp. Fig. 4 calls the same curves Laplacian spectra. Clarifying what is measured would help.
  4. Fig. 5(a) (0.758 on PCG) conflicts with the text and Table 3 (0.708), and four datasets in Tables 1–4 are never described. Reconciling these and releasing per-seed results would help.
  5. Alg. 1 trains the GCN only on L_OLSP, which makes the "w/o OLSP" ablation hard to interpret; describing the gradient path and adding a frozen-encoder control would clarify it.

[1] Nie, Zhang and Li. A Generalized Power Iteration Method for Solving Quadratic Problem on the Stiefel Manifold. Science China Information Sciences 60:112101, 2017.

[2] Knyazev. Toward the Optimal Preconditioned Eigensolver: Locally Optimal Block Preconditioned Conjugate Gradient Method. SIAM J. Sci. Comput. 23(2), 2001.

Questions for authors

See weaknesses.

Reviewer_otDCOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a "spectral manifold graph representation" framework for multi-label node classification: two GCN layers produce node embeddings Z, on which a kNN adaptive graph and its Laplacian L are built; a trace minimization problem with a hard orthogonality constraint is solved to obtain a low-rank representation, and per-node predictions are then made from F together with a linear classifier P. A cross-layer spectral alignment loss subsequently aligns the spectral subspace projection matrices of the kNN graph at each layer with the projection matrix of F. The authors describe this as a paradigm shift from node-level classification to graph-level representation, and support it with experimental results.

Critical strengths
  • The motivation is reasonable. Decoupling the classification space from over-smoothed node embeddings is a sensible starting point in the context of deep graph networks, and the authors correctly note that low-pass filtering attenuates discriminative high-frequency signals.
  • Exploiting the orthogonality constraint allows the least-squares readout to be written in closed form, which is of reference value for related work.
Critical weaknesses
  • The claimed "graph-level representation" is in fact a per-node low-rank embedding, which substantially reduces the information it carries. The main conceptual contribution, namely the shift from node classification to graph-level representation learning, is therefore not realized by the method. It is not shown what new graph-level predictive capability this representation provides.
  • The experimental section is self-contradictory. For example: (a) the paper states "four biological MLNC benchmarks: PPI, PCG, HumLoc, EukLoc", yet Tables 1-4 contain 8 datasets, adding Scene, Corel16k, NUS-WIDE and VOC2007; these four are multi-label image datasets, neither biological nor node-classification graphs, and Section 4.1 gives no construction details; (b) the paper states "10 representative baselines" but also "we compare SSR-MLNC against 11 baselines across 8 diverse MLNC datasets"; (c) the paper states "removing the OLSP module drops the Macro-F1 from 0.708 to 0.665", whereas the three bars in Fig. 5(a) on the same page are 0.758, 0.721 and 0.742.
Questions for authors
  • Do the experimental results remain incompletely prepared and consolidated?
  • The "exact closed-form solution" appears to be a single step of a generalized power iteration rather than an analytical solution of the constrained problem. Could the authors clarify this?
  • Each row corresponds to a node, and the prediction is still a per-node label. What exactly is the methodological object that the authors call the "graph-level representation"? What testable predictive capability does it add relative to spectral node embeddings?
  • Please explain how the image datasets are converted into node-classification graphs, and reconcile the number of baselines as well as the discrepancies between the ablation text and the figures.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.