acceptodds
Under review as a conference paper at ICLR 2027

Band-Masked Spectral Autoencoding for Self-Supervised Graph Representation Learning

Abstract

Self-supervised graph representation learning is currently dominated by contrastive methods relying on engineered augmentations and masked autoencoders that reconstruct spatially masked features or edges. We introduce Band-Masked Spectral Autoencoding (BMSAE), a self-supervised framework that instead masks and reconstructs structured components of graph signals in the spectral domain. BMSAE decomposes node features into spectral bands using a fixed Meyer-wavelet Parseval frame, masks a subset of these bands, and predicts them from the remaining ones. Crucially, Parseval tightness yields an exact synthesis relation. This allows masked-band prediction error to bound reconstruction error in the original feature space, thereby controlling the risk of Lipschitz predictors evaluated on the reconstruction. Because the spectral transform is graph-local, applying the node-level objective to a disjoint union is equivalent to applying it independently to each graph, while segment pooling yields graph-level representations. Experiments show that BMSAE is competitive with or improves on strong contrastive and masked-autoencoder baselines on graph-level benchmarks. For node classification, its strongest results occur on heterophilic graphs, where BMSAE outperforms competing methods on most of the evaluated benchmarks.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
self-supervised graph representation learning, spectral graph signal processing, graph wavelets, Parseval frames, masked autoencoder, generalization bounds, graph classification, node classification

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_qPxsOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Band-Masked Spectral Autoencoding (BMSAE), a self-supervised graph representation learning framework that masks and reconstructs graph spectral bands rather than node features or edges. Specifically, BMSAE decomposes node features into multiple spectral bands using a fixed Meyer-wavelet Parseval frame, randomly masks selected bands, and predicts them from the remaining bands through an encoder-decoder architecture. The paper also establishes a reconstruction error bound based on the Parseval property. Experiments demonstrate competitive performance on graph classification and heterophilic node classification benchmarks.

Critical strengths
  • The proposed spectral-band masked reconstruction provides an interesting alternative to conventional spatial-domain masking (e.g., GraphMAE) and spectral contrastive learning.
  • The paper establishes a reconstruction bound showing that masked-band prediction errors upper-bound the reconstruction error in the original feature space.
  • BMSAE achieves the best reported mean performance on five of six graph classification datasets and four of five heterophilic node classification benchmarks.
Critical weaknesses
  • Table 3 compares BMSAE against frozen spectral features, but this does not disentangle the contributions of masked-band reconstruction and the nonlinear encoder. An architecture-matched baseline using the same spectral inputs but a different self-supervised objective would better establish the effectiveness of the proposed masking strategy.
  • The main experiments use uniform single-band masking, and four of six graph classification datasets select V=2, reducing training to reciprocal prediction between two broad frequency bands. The paper lacks a systematic investigation of how masking multiple bands, varying masking ratios, or selecting different frequency bands affects representation quality.
  • The node classification experiments focus exclusively on five heterophilic benchmarks. The paper does not report these results or provide a detailed analysis of when and why spectral-band masking is beneficial.
Questions for authors

Please refer to weaknesses above.

Reviewer_WZEqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper dicussed graph recovery in two conditions: (1) mask part of nodes (2) mask signal from a certain frequency

Critical strengths

(1) Novel and well-motivated self-supervised objective. The paper introduces a spectral-domain masked autoencoding framework that predicts withheld graph-frequency bands instead of reconstructing spatially masked node features or edges. This provides an interesting alternative to existing contrastive and masked graph representation learning approaches.

(2) The use of a Parseval tight frame provides an elegant connection between spectral-band prediction error and reconstruction error in the original feature space. The analysis also considers deviations from ideal tightness arising from the practical Chebyshev approximation.

Critical weaknesses

(1) The primary bound is derived directly from the non-expansive synthesis property of Parseval tight frames. While this provides a valid and useful guarantee, it does not necessarily constitute a significant theoretical innovation. In particular, it fails to establish a direct link between spectral-band prediction and the acquisition of higher-quality downstream representations. (2) In Table 3, the authors include a control experiment using "frozen" spectral features—concatenating the spectral bands and training a linear probe on them. BMSAE does indeed outperform this baseline. However, this does not sufficiently disentangle the respective contributions of spectral filtering, the encoder’s non-linear mapping, and the masked reconstruction objective. The authors themselves acknowledge this limitation on page 9.

Questions for authors

Could the authors provide an additional ablation comparing BMSAE against an encoder trained on the same spectral features without band masking, or with full-band reconstruction? This would help isolate the contribution of the proposed masked-band prediction objective from spectral preprocessing and nonlinear representation learning.

Reviewer_sAiXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper focuses on graph representation learning and studies how to use a masked spectral auto-encoding method for the task. The paper introduces a new framework, BMSAE, which masks and reconstructs spectral graph signals rather than spatial features or edges. It utilizes a fixed Meyer-wavelet Parseval frame to decompose features into spectral bands and learns graph representation by predicting the masked ones using the others. The paper gives a theoretical analysis to establish the relation between small masked-band prediction error and reconstruction error in node-feature space.

Critical strengths

Based on the fixed Meyer-wavelet Parseval frame, the paper proposes a new masked spectral graph representation framework, BMSAE. The paper gives a solid reconstruction guarantee for the used pretext task.

Critical weaknesses

For the graph-level classification task, as shown in Table 1, the performance gain of the proposed BMSAE is limited. For the four datasets (COLLAB, PROTEINS, D&D, NCI1), the WL baseline already gets strong results.

Questions for authors

Why do we need to introduce the fixed Meyer-wavelet Parseval frame if the performance gain is not so significant? Is there any other spectral method that could be utilized?

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response to Reviewers and New Ablations

Thank you for taking the time to review our paper and for your feedback.

Reviewer sAiX

We agree that WL is a strong graph-classification baseline. In Table 1, BMSAE improves over WL by 1.4 points on COLLAB, 1.8 on PROTEINS, 2.2 on D&D, and 2.3 on NCI1. Against the strongest reported comparator on each dataset, the differences are +0.6, +1.8, +1.0, and −2.0 points, respectively. We view these as modest gains on three datasets and a clear loss to GraphCL on NCI1.

We chose Meyer filters because they give a fixed set of smooth, overlapping frequency bands that can be evaluated with Chebyshev polynomials. The ideal Parseval property gives exact synthesis and a unit-constant bound from masked-band prediction error to feature reconstruction error. The argument also applies to other Parseval spectral banks. For example, overlapping cosine filters can be normalized so that their squared responses sum to one across the Laplacian spectrum. Whether that choice gives better downstream performance is an empirical question.

The new controls in Appendix C.2 hold the Meyer bank fixed and vary the training objective. BMSAE has a higher mean than both unmasked controls in all eight distinct graph configurations, supporting the contribution of target withholding with the chosen bank.

Reviewer WZEq

We agree that the bound follows from the non-expansiveness of Parseval synthesis. Its role is to connect the pretext loss to feature reconstruction fidelity. We evaluate downstream representation quality empirically with frozen probes.

To address the attribution gap in Table 3, we added both requested controls in Appendix C.2 and linked them from §6.4. The target-visible control uses the same sampled target and single-band reconstruction loss as BMSAE, with every band visible to the encoder. The full-band control also exposes every band and averages reconstruction losses across all bands on each update. Both reuse the selected filter bank, encoder and decoder architectures, training budget, and frozen-probe protocol.

BMSAE has a higher mean than both controls in all eight distinct graph configurations. Across ten node configurations, it has a higher mean than target-visible in nine, with one tie at displayed precision, and a higher mean than full-band in eight. On Roman-empire Linear, the gains are +2.7 accuracy points over target-visible and +1.9 over full-band. The two full-band exceptions are on Minesweeper, where BMSAE trails by 0.4 and 0.2 ROC-AUC points. These results support a contribution from target withholding in most of the tested configurations.

Reviewer qPxs

Contribution of masking. Appendix C.2 adds two architecture-matched controls using the same spectral filter bank and frozen-probe protocol. One exposes all bands and reconstructs the same sampled target as BMSAE. The other exposes all bands and reconstructs every band on each update. BMSAE has a higher mean than both controls in all eight distinct graph configurations. Across ten node configurations, it has a higher mean than target-visible in nine, with one tie at displayed precision, and a higher mean than full-band in eight. Full-band leads on both Minesweeper configurations.

Masking ratio and band choice. We agree that the four V=2V=2 graph configurations reduce to reciprocal low-/high-band prediction. All four have higher mean accuracy than both controls, so this simple setting can benefit from withholding the target. Eight of the ten node configurations use V>2V>2, and the reported comparison still uses uniform single-band masking. The effect of withholding multiple bands or favoring particular frequencies remains open.

Scope of the node results. Our quantitative node-classification evidence covers five heterophilic benchmarks. The benefit varies within that panel. BMSAE is 6.7 accuracy points ahead of GraphMAE2 on Roman-empire and 1.5 points behind it on Amazon-ratings. As noted in §6.3, the homophilic cases we examined showed smaller gains. The current results leave open which graph and feature properties predict when spectral-band masking will help.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.