- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Using quiver representations, the paper characterizes standard multi-head attention as diagonal transport on its native head space. It introduces Topological Attention (Top-A), which uses to add edge-dependent off-diagonal routing across value heads before neighborhood aggregation while preserving the original same-head attention paths. For fixed attention and transport operators, the paper establishes a necessary and sufficient condition under which the routed aggregation can be reproduced by a shared post-aggregation linear map. Experiments with GATv2 and Graph Transformer (GT) backbones show improvements on STaR, MovieLens-100K, and CLRS.
Critical strengths
1. A simple and coherent design that facilitates controlled architectural comparisons. Top-A preserves the original same-head attention paths and exactly recovers the corresponding baseline function at zero initialization, while allowing the routing generator to receive gradients. This design facilitates controlled comparisons and direct integration with different attention backbones.
2. The main empirical gains are supported. The CLRS experiments cover all 30 tasks and 20 paired seeds for both attention backbones. The paper also reports task-specific regressions and the absence of systematic gains on heterophilic node classification, providing a balanced account of the proposed inductive bias and its empirical scope.
Critical weaknesses
1. The core mechanism is not sufficiently isolated to attribute the gains specifically to edge-dependent cross-head routing. The main experiments primarily compare each vanilla backbone with the full Top-A model, without key controls such as globally shared routing, relation-type-conditioned routing, or alternative value transformations of comparable capacity. Matching initialization controls the starting point but does not disentangle the contributions of additional capacity, multiplicative transformations, and the edge-dependent cross-head structure.
2. The distinct contribution and advantages over the closest existing mechanisms remain insufficiently established. Copresheaf attention [1] already applies learned edge-dependent matrices to value representations before aggregation, while Compositional Attention [2] allows context-dependent composition of search and value-retrieval mechanisms; the latter is not discussed in this paper. However, the main experiments demonstrating gains compare only against vanilla GATv2 and GT.
[1] Copresheaf Topological Neural Networks: A Generalized Deep Learning Framework, NeurIPS 2025
[2] Compositional Attention: Disentangling Search and Retrieval, ICLR 2022
Questions for authors
See Weaknesses.
5 comments
Thank you for your response
Thank you for your detailed response and the clarifications. I have carefully read the rebuttal and the other reviewers' comments. After consideration, I have decided to maintain my original score, as it remains unclear whether Top-A can be integrated with more advanced and powerful GNNs and further enhance their performance; therefore, it is not yet possible to determine Top-A’s general applicability and usefulness in this field.
Dear Reviewer vLNh,
Thank you for reconsidering our work and for your additional feedback.
We would like to clarify that Top-A is designed as a general augmentation of multi-head attention mechanisms, rather than of arbitrary GNN architectures. Its construction is independent of the specific attention-scoring function, as it introduces edge-conditioned cross-head transport before aggregation while preserving the underlying attention mechanism.
To demonstrate this flexibility, we evaluated Top-A on two distinct and widely used attention-based GNN families, GATv2 and Graph Transformers, observing improvements across multiple tasks. We therefore believe that the architectural applicability of Top-A extends beyond a particular backbone, although we agree that evaluating additional, more advanced architectures would further strengthen the empirical evidence.
As the rebuttal period is approaching its end, we would particularly appreciate your guidance on which architectures or additional comparisons you would consider most informative for establishing the broader applicability of Top-A. We would be happy to address this concern as concretely as possible.
Thank you again for your time and consideration.
Best regards,
The Authors
Dear Reviewer vLNh,
Thank you again for your careful review and valuable suggestions.
We hope that our responses have helped clarify the contribution of Top-A, the scope of our experimental controls, and its relationship to existing attention mechanisms.
We would appreciate knowing whether our responses have addressed your main concerns, and we welcome any further questions or suggestions.
Best regards,
The Authors