- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
In this work, the authors propose AW-RoPE, which learns a rotation angle on each edge from the features of its two endpoints. Queries and keys are then propagated along walks in the graph and are rotated at every edge they cross, and walks of all lengths are summed with decaying weights. An exact variant computes the full sum, and a cheaper sparse variant stops at a fixed walk length. The paper proves approximation bounds for the truncation. The method is evaluated on synthetic graph tasks and on standard graph benchmarks, using AW-RoPE inside a linear-attention (Performer) graph GPS and also in a message-passing GNN (GIN).
Critical strengths
- The core idea is elegant : WIRE rotates each node's query and key by an angle from its position, AW-RoPE adds in neighbours' queries and keys, rotating each by a learned angle for every edge it travels along, so different paths between the same two nodes can give different total rotations.
- The method acts on Q/K before the attention kernel, so it works with linear attention and also as a standalone propagation layer in GIN.
- The empirical results show advantage of the method over the baseline WIRE.
Critical weaknesses
- The rotations do not seem to drive the gains. Turning the edge rotations off while keeping the multi-hop mixing gives the same results (Table I.4), and rotation free version is quite strong (Fig H1).
- It is hard to understand the significance of the results as the baseline and WIRE do not have the standard deviations. Its also unclear why on PascalVOC/COCO, AW also uses a different kernel (FAVOR+) . Moreover, the GIN numbers are far below standard results (e.g. MNIST 75.6% vs 96.5%) (see [1]).
- The writing is extremely hard to understand and it is not clear why different tables use different kernels (for ex in H1, Favor+ was used in the ablation whereas the main paper mixes ReLU and Favor+).
- The results seem incremental given the speed and the memory overhead.
- The paper argues that node-based RoPE cannot tell apart different paths between two nodes, because the rotation between them depends only on the two endpoints. But WIRE's attention between two nodes decays on average with their effective resistance (Reid et al., 2026, Thm. 3), which is lower when the nodes are joined by more and shorter paths. The synthetic tasks used to show the limitation depend on node features, which no positional encoding sees, so they show the value of feature-dependent mixing rather than a gap in positional information. Finally, AW only sees paths between nodes at most L hops apart; for farther pairs it gives no path information, whereas WIRE still gives a relative position.
[1] Recipe for a General, Powerful, Scalable Graph Transformer Rampášek et al. Neurips 2022
[2] Rotary Position Encodings for Graphs Reid et al. ICML 2026
Post rebuttal : The authors ran additional evaluations to answer the above concerns so I improved the score to a 2.
Questions for authors
- Why were the published MalNet-Tiny and Pascal VOC numbers replaced?
- What WIRE configuration produced 0.073 in Table I.4? The numbers seem lower than what was reported by the original authors.
- Why is accuracy used for GIN for Pattern/Cluster? And how many seeds are the results in Table 4 run?
- The paper introduces two techniques and the ablation is only carried out for two datasets where it is showing that Mixing is quite strong so an ablation over more datasets can help us understand the significance of the method.
- Since Sparse AW costs O(L(N+M)d), how does it scale on dense graphs, where the number of edges M grows quadratically with the number of nodes N?
Not questions per se but the paper could benefit from some presentation improvements.
- Use standard GNN terms instead of physics ones: "propagation" for "transport", "edge angle" for "displacement" etc.
- State the update rule (a decaying sum of neighbours' rotated queries and keys) plainly in the introduction.
- Give the ablation variants in Table 2 descriptive names (e.g. instead of "Dynamic nonlocal Flat").
- Please give proper numbering to figures and tables.
2 comments
Thank you for the detailed reply and re-running the evaluations. They answer my questions. I will revise my score to a 2 but I am still a little concerned by the novelty of the method over existing techniques and the results are not as competitive given the additional cost.
Response to 2VNJ
Thank you for the thorough review; your comments shaped the new experiments and the revision.
Why were the MalNet-Tiny and PascalVOC-SP numbers replaced?
The revision reports both datasets from this matched cohort, with baseline SDs. Per-seed values and protocols are in our reply to 1xPx.
What WIRE configuration produced 0.073 in Table I.4? It is a local matched run, not a published mean. The setup is four width-32, single-head dense-softmax layers with the common input/spectral scaffold, 250 epochs, lr 10−4, batch 16, seeds 0/1/2. Test nRMSE is 0.0773692, 0.0775564 and 0.0643272, giving 0.0730843 ± 0.0075844. Backbone initialization is shared within each seed. "WIRE + mixing" applies depth-8, decay-0.8 phase-free propagation before WIRE rotation. Protocol.
Why accuracy for GIN on PATTERN/CLUSTER, and how many seeds? You are right that the benchmark uses class-balanced accuracy. We re-evaluated the saved checkpoints on the full splits with that metric:
The revision uses balanced accuracy. Table 4 uses seeds 0/1 for both methods: 12 datasets, 48 runs. Per-seed results.
GIN MNIST baseline. We agree the original baseline was too weak: width 32, and it did not use node coordinates. We built a stronger GIN: width 110, learned ε, BatchNorm, pooled heads, intensity plus coordinates, 105,434 parameters. We trained NoPE, zero angle and AW on it for 250 epochs, seeds 0/1. Test accuracy:
The stronger backbone reaches the ~96.5% level you cite, and AW does not add to it on MNIST. The revision replaces the MNIST row with this comparison.
Ablation over more datasets. Beyond the two submitted datasets, zero-angle and gradient-field controls are now matched on PascalVOC-SP, Peptides-func, corrected MalNet-Tiny, MolHIV, Peptides-struct and MNIST. Learned edge angles help on Peptides-struct (I.4), PascalVOC-SP against the gradient field, and MalNet-Tiny; phase-free propagation is as strong or stronger elsewhere. The paired table is in our reply to 1xPx.
Constructed tasks and WIRE. You are right that WIRE's Theorem 3 links its expected Q/K similarity to effective resistance, which encodes path multiplicity and length. Our distinction is narrower: node-wise phase differences telescope along any route, while learned edge angles need not. The constructed tasks isolate exactly this, using opposite-label pairs with identical topology but different feature arrangements. With a full attention block or a global gradient field, simpler controls also solve them. The revision presents them as a demonstration of feature-conditioned propagation under stated assumptions. WIRE v5; constructed-task protocols.
Sparse depth and distant pairs. At one layer, Sparse AW propagates along walks of length ≤ L. Global attention then compares the propagated summaries for any pair, and stacking layers extends the reach.
Cost and dense graphs. Sparse AW costs O(L(N+M)d). On dense graphs, M=Θ(N2), so it becomes O(LN2d): the order of dense attention, times the depth L. Measured costs:
The revision reports these costs next to accuracy.
Presentation. We use "propagation" and "edge angle" throughout, describe the update plainly (rotate neighboring Q/K features by learned edge angles, aggregate, sum the propagated states with decaying weights), rename "Dynamic nonlocal Flat" to "global gradient-projected edge field", and renumber all figures and tables.