- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper develops applications of discrete Morse theory to graph transformers, providing both mathematical and experimental results. The paper includes a nontrivial amount of mathematical flaws as mentioned in the weakness comments below. Due to those issues, I cannot recommend acceptance of the paper.
Critical strengths
The idea of applying discrete Morse theory to graph transformer seems interesting (although discrete Morse theory has already been applied in the context of machine learning and topological data analysis; even so, I have never seen an application of discrete Morse theory to graph transformers).
Critical weaknesses
The paper is not mathematically sound to a non-negligible degree. For example, the gradient-path definition is inconsistent with the orientation of the gradient graph, making the cancellation algorithm incorrect. Also, the attribute projection is orientation-dependent, and essential graph-isomorphism invariance is not established.
At the same time, apart from these mathematical issues, practical applicability in machine learning is not fully convincing and the proposed method seems premature to be published at a top conference like ICLR.
Questions for authors
Can you fix the mathematical issues as mentioned in the weakness?
4 comments
Part 2/2
At the same time, apart from these mathematical issues, practical applicability in machine learning is not fully convincing and the proposed method seems premature to be published at a top conference like ICLR.
Our experiments demonstrate the practical applicability of MorseFormer across several established benchmarks and evaluation protocols. Under the HOPSE/TopoTune protocol, cellular MorseFormer++ has the highest mean on all five benchmarks (MUTAG, PROTEINS, NCI1, NCI109, and ZINC) among graph, simplicial, and cellular baselines; on NCI1 and NCI109 the improvements over the strongest cellular baselines are accompanied by statistical evidence ( and ; Appendix D). Under the TopoFormer protocol, it achieves the highest mean accuracy on COX2 and PROTEINS. On molecular benchmarks with edge features, it obtains MAE on ZINC, outperforming GPS, CIN, CIN++, and the Cellular Transformer.
Beyond predictive performance, our training-free Morse descriptors distinguish BREC pairs, compared with for SpectRe. The scalability experiments further demonstrate a reduction in token count and a reduction in attention time on larger graphs. These results demonstrate both predictive effectiveness and computational benefits across distinct graph-learning settings (Sections 4.1–4.3).
We thank the reviewer for encouraging broader experimental validation. We have added ModelNet40 classification on rank-3 alpha complexes containing tetrahedra to Section 4.1 of the revised manuscript (new table marked in blue):
| Model | Rank | Representation size | OA (%) ↑ | GPU (GB) ↓ | s/epoch ↓ |
|---|---|---|---|---|---|
| SCCNN | 2 | 8,072 cells | 2.6 | 66 | |
| SCCN | 3 | 8,072 cells | 0.9 | 72 | |
| Cellular Transformer | 3 | 8,538 cells | — | OOM | — |
| MorseFormer | 3 | 202 cells | 3.3 | 76 | |
| MorseFormer++ | 3 | 1,224 cells | 4.3 | 109 |
The Cellular Transformer exceeds the V100-32GB memory budget. The 1,224 representation elements of MorseFormer++ consist of 200 critical cells and 1,024 graph nodes processed in separate branches; higher-order attention operates only on the critical cells.
MorseFormer++ improves over SCCNN by 4.9 percentage points and rank-3 SCCN by 8.6 points, while global attention over its higher-order representation requires only 200 critical-cell tokens. MorseFormer reaches without a graph backbone, the second-highest accuracy among the successfully evaluated models in the table and 1.2 percentage points above SCCNN. The additional scaling experiments, summarized in Section 4.3 and detailed in Appendix C, demonstrate that Morse attention remains feasible on substantially larger rank-3 complexes.
We once again thank the reviewer for contributing to our work and for helping us better position the importance of it for the TDL community. We truly believe (as showed in the experiments) that combining discrete Morse theory with transformers could lead us to powerful topological neural networks and we hope that this work could also serve by inspiration for future TNNs to also have the possibility to go to ranks/dimensions higher than 2 without combinatorial explosion/time complexity issues. We expect that our rank-3 experiments are the first of many more to come along this path.
[1] C. Bodnar et al. Weisfeiler and Lehman Go Cellular: CW Networks. NeurIPS, 2021.
[2] S. Ebli, C. Hacker, and K. Maggs. Morse Theoretic Signal Compression and Reconstruction on Chain Complexes. Journal of Applied and Computational Topology, 2024.
Thank you for the reply. Considering the response, I still think my evaluation is appropriate for this paper.
Thank for your response. Could you please clarify why and what other points remain for this assessment?
Thank you for your service once again.
We thank the reviewer for their time serving our community and helping us improve our work.
Part 1/2
We thank the reviewer for identifying this inconsistency. The direction of the Hasse arcs was incorrectly stated in Section 2.1: the correct convention is τ→σ, from a cell to its facets. Importantly, both Figure 2 and its original caption already follow this convention, with the explicitly stated gradient path 2→23→3→13→1, alternating between upward matched arcs and downward unmatched arcs. The implementation also follows this convention, so the inconsistency is confined to the written Hasse-arc definition and does not affect the matching algorithm. We have corrected the arrow direction in Section 2.1 of the revised paper PDF (marked in blue).
Our attribute construction follows established approaches in topological deep learning and algebraic discrete Morse theory. Higher-rank features are initialized using unsigned incidences ∣Bk∣, consistently with the higher-order feature initialization of Bodnar et al. (2021) [1]. The subsequent projection ϕkXk follows the Morse deformation-retraction framework of Ebli et al. (2024, Theorem 2.9) [2], who explicitly discuss Morse-based signal compression for pooling in cellular neural networks (Section 6.3.2).
As in Ebli et al., the retraction is defined with respect to oriented cellular bases. Under a change of orientation, it transforms as ϕk′=SkcϕkSk. The corresponding Morse complexes are chain-isomorphic, preserving homology and Hodge spectra. We agree, however, that this algebraic property does not imply invariance of the projected attributes: Xk is initialized using unsigned incidences, and SkcϕkSkXk need not equal ϕkXk. Similarly, SignNet handles eigenvector-sign ambiguity but does not by itself establish invariance under arbitrary cell reorientations.
Regarding graph isomorphism, the lifting maps respects graph isomorphisms at the combinatorial level, and every valid acyclic matching preserves the homology of the lifted complex. Our topological descriptors are computed from the resulting Morse complex and are invariant under graph isomorphisms whenever the corresponding Morse reductions are related by a cellular isomorphism that preserves the filtrations and orientations up to a change of basis. Thus, the topological readout is invariant under such corresponding reductions. While different valid matchings may produce different descriptors, the homotopy type and homology of the lifted complex are always preserved, which is the fundamental topological guarantee of our method.
We have corrected the Hasse-arc convention in Section 2.1 and clarified the unsigned attribute initialization and its connection to Morse-based signal compression in Section 3.2 (revisions marked in blue).