- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how to align feature, structure and label in graph learning. Specifically, the proposed method, GraphALFA, uses contrastive learning framework, where the two input views augments attribute and structure, followed by a learned gate for fusion. The model is optimized by two objectives, i.e., feature-lable and feature-structure alignments.
Critical strengths
None.
Critical weaknesses
- The authors combine several standard operations into one model. For example, the adopted attribute view, structure view and intra-/inter-class contrastive learning have been standard components in graph learning. Also, the used technique in structure view construction, (e.g., node2vec, LINE) are very old-fashioned.
- Potential high complexity. In feature-structure alignment loss computing, the authors firstly computed band-wise ratios of spectral energy change, then averaged them across layers for the final g_emp. That means we need to do spectrum-related operations every layer and every spectral band, which could incur high computing complexity.
Questions for authors
- In feature-structure alignment loss computing, it also requires label distribution over structure to be same as that of feature. So, could I view the two losses basically do the same thing that both align feature and label?
- This paper claimed the feature-structure-label alignment. But if the given graph natually does not satisfy this alignment, can we still learn such features to do the alignment?
- Lack the comparison with graph contrastive learning methods.
- In standard graph contrastive learning, the input views need to be changeable every epoch, for example randomly mask features or delete edges. GraphALFA tooks unchanged input views.
3 comments
Q2: We note that feature-structure-label alignment is not an assumption about the data, but an optimization objective for addressing existing misalignment. Through task-supervised feature augmentation, the framework aims to construct representations that better exploit complementary attribute and structural information while accounting for the downstream GNN's propagation behavior. This alignment concerns the learned augmented features and their task relevance, rather than requiring agreement between graph connectivity and class labels or enforcing homophily. Our empirical results on heterophilic datasets support this capability under imperfect alignment. Gains over the original features are substantially larger on heterophilic than on homophilic graphs (Table 1; average +5.5 vs. +0.7 points with GCN and +7.4 vs. +2.9 with GAT). The controlled cSBM experiments (Figure 2) demonstrate improvements across varying feature and structural signal conditions, where gains for GCN and GAT are largest in heterophilic settings. Furthermore, the robustness analysis (Table 12) shows that GraphAlFA remains effective under feature and structural perturbations, with adaptive fusion adjusting the relative contributions of the two views. Appendix D.3.7 provides a complementary measure of how the augmented features relate to class labels, by showing that the adjusted feature homophily rises from 0.057 to 0.290 on Arxiv-Year and from 0.010 to 0.173 on Cora. Accuracy also improves on nodes whose neighborhoods are mostly dissimilar (Table 13: Arxiv-Year, local homophily ≤ 0.25: 39.6 → 45.4; Cora, (0.25, 0.5]: 46.2 → 55.3).
Q3: We agree that including graph contrastive learning methods would strengthen the empirical assessment. We have conducted additional experiments with representative methods (GRACE [1], GraphACL [2], and PolyGCL [3]). Specifically, for each contrastive learning baseline, we train its encoder using the recommended settings, then use the learned node embeddings with the same backbones. We have incorporated these additional baseline comparisons in table 1 of the manuscript. GraphAlFA achieves the best overall performance, with average improvements of 2.7 to 4.1 percentage points across all datasets and backbones. These results further demonstrate the effectiveness of our propagation-aware feature augmentation.
[1] Zhu et al. Deep Graph Contrastive Representation Learning (2020)
[2] Xiao, T., Zhu, H., Chen, Z., & Wang, S. Simple and Asymmetric Graph Contrastive Learning without Augmentations (NeurIPS 2023)
[3] Chen, J., Lei, R., & Wei, Z. PolyGCL: GRAPH CONTRASTIVE LEARNING via Learnable Spectral Polynomial Filters (ICLR 2024)
Q4: We note that while stochastic views are often constructed through masking or perturbations in self-supervised graph contrastive learning to form positive pairs, stochastic perturbations are not required [1]. GraphAlFA follows a different paradigm, where positive pairs are defined by shared task labels, following a supervised contrastive learning setup [2, 3]. This objective promotes class-discriminative representations without requiring independently perturbed contrastive views. Accordingly, the attribute-derived and structure-derived views capture complementary information, rather than stochastic contrastive views, and regenerating them at each epoch is not required. Although input views are precomputed (Alg. 1, lines 1–7; App. B.1.1), their learnable projections and adaptive fusion (Sec. 3.2) are updated under task supervision (Sec. 3.3; Alg. 1, lines 22–32), providing evolving augmented features. This allows GraphAlFA to learn task-relevant representations while preserving the observed graph topology. Stochastic view perturbations could be incorporated as an extension. We clarify this methodological distinction in Sections 3.1 and 3.3.
[1] Xiao, T., Zhu, H., Chen, Z., & Wang, S. Simple and Asymmetric Graph Contrastive Learning without Augmentations (NeurIPS 2023)
[2] Wang et al. ClusterSCL: Cluster-Aware Supervised Contrastive Learning on Graphs (WWW 2022)
[3] Khosla et al. Supervised Contrastive Learning (NeurIPS 2020)
Thank you again for your thoughtful and constructive feedback. We would like to check whether our initial response has adequately addressed your concerns, or if there are any remaining questions or points you would like us to clarify further. We would be happy to make additional revisions or provide further details if needed.
Thank you for the detailed, constructive feedback!
W1: While the individual feature construction and contrastive learning components build upon established techniques, GraphAlFA's main contribution is the propagation-aware alignment mechanism, which conditions feature augmentation on the downstream GNN's observed spectral propagation behavior. This enables the augmentation to adapt to the graph, task supervision, and backbone. The broader feature-structure-label alignment formulation provides a unified framework for this mechanism. Importantly, we show that combining the standard components alone is not sufficient (Table 2), specifically, concatenating the standard attribute- and structure-derived views without the proposed alignment generally underperforms GCN on four of five datasets (“Concat” in Table 2), whereas the full framework consistently improves. Furthermore, removing either alignment objective degrades performance (Tables 2 and 8), supporting the contribution of the proposed alignment mechanism beyond combining standard components. We note that GraphAlFA is designed to be flexible with respect to the choice of structure-derived representation, rather than introducing a new structural embedding method. We employ established techniques as efficient, widely applicable implementations. We empirically evaluate various structural embedding (structure-view) configurations (Table 7) and demonstrate the framework's generalizability and effectiveness. LINE is selected as the default due to its low preprocessing cost (Table 14), contributing to the framework's overall computational efficiency (Table 3). More recent structural or positional encoders can be plugged in directly.
W2: We note that GraphAlFA computes the per-layer band-energy ratios only when it refreshes g_emp, every U epochs (Algorithm 1, lines 11-21). Between updates, the spectral alignment loss reuses the most recent profile. For each energy estimate, stochastic Lanczos spectral slicing uses a shared low-dimensional Krylov basis across bands and avoids a full Laplacian eigendecomposition. As analyzed in Appendix B.1.4, the dominant spectral computation scales linearly with graph size. Empirically, on Arxiv-Year with GCN, Table 9 reports that refreshing g_emp adds 14.80 ms per epoch (7.64% of training time) at U=10 and 4.86 ms per epoch (3.09%) at U=20, averaged over training. Table 3 shows that GraphAlFA achieves the lowest or near-lowest overall augmentation runtime among the compared methods, supporting the computational efficiency of our framework.
Q1: While both alignment objectives incorporate task supervision, they impose distinct constraints on the learned representations rather than performing the same feature-label alignment (defined in Sections 3.3.1 and 3.3.2). The augmented feature-label alignment loss shapes the geometry of node representations by promoting intra-class compactness and inter-class separation. In contrast, the augmented feature-structure alignment loss matches the relative spectral energy profiles of the augmented features and train-masked labels across graph Laplacian frequency bands, with each band's contribution weighted by the downstream GNN's empirically observed propagation behavior. Their complementary roles are empirically supported by our ablation studies (Tables 2 and 8), which support the benefit of combining these alignments, e.g., as shown in Table 2, on Chameleon with GCN, the full model achieves 48.69%, compared with 44.16% with only SupCon and 43.60% with only spectral alignment. We clarify this distinction of the structure- and propagation-conditioned nature of the spectral alignment in Section 3.3.