- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes CARAT (Curvature- and Attention-Aligned Rewiring with Adaptive Topology), a graph-rewiring method for attention-based GNNs. Its central motivation is the rewiring-utilization gap: adding a structurally desirable shortcut edge does not mean that a GAT-style model will actually use it, because the new edge must compete with existing neighbors through attention normalization. CARAT addresses three questions: where to add shortcuts, which shortcuts are useful, and how strongly they should be used. The method has three main parts:
- Curvature-guided candidate generation. It computes exact Balanced Forman Curvature (BFC) on original edges, interprets strongly negative curvature as evidence of a bottleneck, and scores candidate two-hop shortcuts based on the curvature of the corresponding two-hop paths. Only the top-k candidates per source node are retained, bounding the number of new directed edges by roughly k|V|.
- Directional compatibility. For each candidate shortcut i -> j, CARAT learns separate query/key projections and computes a directional compatibility score . The motivation is that useful communication need not correspond to raw feature similarity, especially on heterophilous graphs.
- Soft task-adaptive gating. The topology score and compatibility score are combined, and a monotone sigmoid gate determines how strongly each shortcut is exposed to GATv2. The authors additionally use bridge and sparsity regularizers to encourage useful bottleneck shortcuts while discouraging excessive activation.
Critical strengths
- The central motivation (i.e., the rewiring-utilization gap) is easy to understand
Critical weaknesses
W1. The novelty of the proposed “rewiring–utilization gap” appears somewhat overstated relative to prior task-adaptive and relational rewiring work. The distinction between adding a structurally desirable edge and actually using it downstream is intuitive for attention-based aggregation and has close precedents. In particular, FoSR (Karhadkar et al., ICLR 2023), which is already cited, explicitly distinguishes original and rewired edges through relation-specific learned parameters so that the model can suppress unhelpful rewired connections; however, the related-work discussion describes FoSR only in terms of spectral edge selection. Probabilistic Task-Adaptive Graph Rewiring (Qian et al., ICML DAE Workshop 2023) likewise learns to add task-relevant edges while omitting less beneficial ones, but is not discussed. Greener GRASS (ICLR 2025) further already combines rewiring and graph attention. The paper should more clearly distinguish this narrower contribution from the broader, established idea that structurally proposed edges should be selectively utilized according to the downstream task.
- FoSR: First-order Spectral Rewiring for Addressing Oversquashing in GNNs. ICLR 2023.
- Probabilistic Task-Adaptive Graph Rewiring. ICML DAE 2023.
- Greener GRASS: Enhancing GNNs with Encoding, Rewiring, and Attention. ICLR 2025.
W2. The proposed soft gate does not necessarily control actual shortcut utilization. In Section 4.3, the learned gate $g_{ij}$ is passed to GATv2 as a structural edge attribute rather than directly multiplying the message or attention coefficient. Since GATv2 processes edge attributes through unconstrained learned transformations, increasing $g_{ij}$ does not necessarily increase attention, and a gate approaching zero does not guarantee that the corresponding shortcut is suppressed. Thus, the monotonicity of the gate with respect to the proposed shortcut score does not imply monotonicity of actual attention or message contribution.
W3. The learned directional compatibility score is not explicitly aligned with the downstream attention mechanism. Section 4.2 introduces a separately learned compatibility score $c_{ij}=q_i^\top k_j$ to estimate whether a candidate shortcut will be useful to GATv2. However, this score and GATv2's attention coefficients are computed using different learned parameters, without an explicit constraint ensuring their agreement. Consequently, a shortcut assigned high compatibility may still receive little attention, leaving the proposed rewiring–utilization gap unresolved. Moreover, since GATv2 already learns directional attention scores, the necessity of an additional compatibility network is unclear.
W4. The ablation experiments do not sufficiently isolate the contributions of the proposed components. In Table 1, the Topology-only variant differs from CARAT in three aspects simultaneously: directional compatibility, soft gating, and hard consolidation. Therefore, their performance difference cannot establish which component is responsible for the improvement. Similarly, Uniform-2Hop and CARAT differ in both shortcut selection and adaptive utilization, making it difficult to isolate the contribution of curvature-guided selection. This is particularly important because Uniform-2Hop already captures most of CARAT's performance gains on the Peptides benchmarks.
W5. The experimental comparisons do not sufficiently establish the practical competitiveness of CARAT, particularly on the LRGB benchmarks. Although the paper intentionally focuses on controlled GATv2 comparisons, CARAT's reported performance on Peptides-func (0.5412 AP) and Peptides-struct (0.3671 MAE) is below even the original GCN baselines reported by Dwivedi et al. (NeurIPS 2022), which achieve 0.5930 AP and 0.3496 MAE, respectively. Subsequent work by Tönshoff et al. (TMLR 2024) reports substantially stronger results from appropriately tuned conventional GNNs. While cross-paper comparisons are not directly controlled, these gaps raise questions about the strength of the underlying backbone and the practical significance of the reported improvements.
- Dwivedi et al. (NeurIPS 2022): Long range graph benchmark
- Tönshoff et al. (TMLR 2024): Where Did the Gap Go? Reassessing the Long-Range Graph Benchmark
W6. The manuscript contains several presentation and implementation-consistency issues. Appendix E refers to "AFP-enabled" and "non-AFP" variants without clearly defining AFP, while the model-name correspondence table retains implementation-specific names such as "fecasa v3 without afp," suggesting that the final terminology has not been consistently applied. More importantly, Table 1 specifies that the Topology-only variant does not use soft gating but does perform hard consolidation, whereas Appendix C defines the consolidation criterion using the learned gate $g_{ij}$. It is unclear how this criterion is implemented for Topology-only.
Questions for authors
See weaknesses above.