acceptodds
Under review as a conference paper at ICLR 2027

MorseFormer: Scalable Topology-Aware Graph Transformers via Discrete Morse Theory

Abstract

Topological Deep Learning extends graph neural networks to higher-order domains, but its dominant message-passing schemes are bounded by cellular WL, and attention-based extensions operate on the full cell complex, where sequence length grows with the lifting and global attention quickly becomes infeasible. Independently, persistent homology and Hodge Laplacian descriptors add expressivity to graph learners but are expensive and thus rarely used inside the training loop. We introduce **MorseFormer**, a topology-aware graph transformer built on Discrete Morse Theory. After a (possibly learnable) lifting, an acyclic matching collapses the cell complex onto its Morse complex — a CW complex on the critical cells, homotopy-equivalent to the original. The Morse retraction transports cell attributes basin-wise to the critical cells, and the resulting stream becomes the token sequence of a dual-branch transformer: a graph branch over the original graph and a Morse branch over critical cells with cross-rank global attention. Because the Morse complex is smaller, persistent homology and Hodge eigendecompositions become tractable as a fixed-size topological readout inside training. On graph classification and expressivity benchmarks MorseFormer is competitive with or improves on state-of-the-art topological baselines. Under static liftings, the entire topological machinery is one-shot preprocessing; training-time cost reduces to a transformer over the Morse complex.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
topological deep learning, graph transformers, discrete Morse theory, cell complexes, graph classification

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_AvSROpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper develops applications of discrete Morse theory to graph transformers, providing both mathematical and experimental results. The paper includes a nontrivial amount of mathematical flaws as mentioned in the weakness comments below. Due to those issues, I cannot recommend acceptance of the paper.

Critical strengths

The idea of applying discrete Morse theory to graph transformer seems interesting (although discrete Morse theory has already been applied in the context of machine learning and topological data analysis; even so, I have never seen an application of discrete Morse theory to graph transformers).

Critical weaknesses

The paper is not mathematically sound to a non-negligible degree. For example, the gradient-path definition is inconsistent with the orientation of the gradient graph, making the cancellation algorithm incorrect. Also, the attribute projection is orientation-dependent, and essential graph-isomorphism invariance is not established.

At the same time, apart from these mathematical issues, practical applicability in machine learning is not fully convincing and the proposed method seems premature to be published at a top conference like ICLR.

Questions for authors

Can you fix the mathematical issues as mentioned in the weakness?

4 comments

We thank the reviewer for their time serving our community and helping us improve our work.

Part 1/2

The gradient-path definition is inconsistent with the orientation of the gradient graph, making the cancellation algorithm incorrect.

We thank the reviewer for identifying this inconsistency. The direction of the Hasse arcs was incorrectly stated in Section 2.1: the correct convention is τ→σ\tau \to \sigma, from a cell to its facets. Importantly, both Figure 2 and its original caption already follow this convention, with the explicitly stated gradient path 2→23→3→13→12 \to 23 \to 3 \to 13 \to 1, alternating between upward matched arcs and downward unmatched arcs. The implementation also follows this convention, so the inconsistency is confined to the written Hasse-arc definition and does not affect the matching algorithm. We have corrected the arrow direction in Section 2.1 of the revised paper PDF (marked in blue).

Also, the attribute projection is orientation-dependent, and essential graph-isomorphism invariance is not established.

Our attribute construction follows established approaches in topological deep learning and algebraic discrete Morse theory. Higher-rank features are initialized using unsigned incidences ∣Bk∣|B_k|, consistently with the higher-order feature initialization of Bodnar et al. (2021) [1]. The subsequent projection ϕkXk\phi_k X_k follows the Morse deformation-retraction framework of Ebli et al. (2024, Theorem 2.9) [2], who explicitly discuss Morse-based signal compression for pooling in cellular neural networks (Section 6.3.2).

As in Ebli et al., the retraction is defined with respect to oriented cellular bases. Under a change of orientation, it transforms as ϕk′=SkcϕkSk\phi_k'=S_k^c\phi_kS_k. The corresponding Morse complexes are chain-isomorphic, preserving homology and Hodge spectra. We agree, however, that this algebraic property does not imply invariance of the projected attributes: XkX_k is initialized using unsigned incidences, and SkcϕkSkXkS_k^c\phi_kS_kX_k need not equal ϕkXk\phi_kX_k. Similarly, SignNet handles eigenvector-sign ambiguity but does not by itself establish invariance under arbitrary cell reorientations.

Regarding graph isomorphism, the lifting maps respects graph isomorphisms at the combinatorial level, and every valid acyclic matching preserves the homology of the lifted complex. Our topological descriptors are computed from the resulting Morse complex and are invariant under graph isomorphisms whenever the corresponding Morse reductions are related by a cellular isomorphism that preserves the filtrations and orientations up to a change of basis. Thus, the topological readout is invariant under such corresponding reductions. While different valid matchings may produce different descriptors, the homotopy type and homology of the lifted complex are always preserved, which is the fundamental topological guarantee of our method.

We have corrected the Hasse-arc convention in Section 2.1 and clarified the unsigned attribute initialization and its connection to Morse-based signal compression in Section 3.2 (revisions marked in blue).

Part 2/2

At the same time, apart from these mathematical issues, practical applicability in machine learning is not fully convincing and the proposed method seems premature to be published at a top conference like ICLR.

Our experiments demonstrate the practical applicability of MorseFormer across several established benchmarks and evaluation protocols. Under the HOPSE/TopoTune protocol, cellular MorseFormer++ has the highest mean on all five benchmarks (MUTAG, PROTEINS, NCI1, NCI109, and ZINC) among graph, simplicial, and cellular baselines; on NCI1 and NCI109 the improvements over the strongest cellular baselines are accompanied by statistical evidence (p=0.025p=0.025 and p=0.003p=0.003; Appendix D). Under the TopoFormer protocol, it achieves the highest mean accuracy on COX2 and PROTEINS. On molecular benchmarks with edge features, it obtains 0.0640.064 MAE on ZINC, outperforming GPS, CIN, CIN++, and the Cellular Transformer.

Beyond predictive performance, our training-free Morse descriptors distinguish 314/400314/400 BREC pairs, compared with 216216 for SpectRe. The scalability experiments further demonstrate a 17.5×17.5\times reduction in token count and a 250×250\times reduction in attention time on larger graphs. These results demonstrate both predictive effectiveness and computational benefits across distinct graph-learning settings (Sections 4.1–4.3).

We thank the reviewer for encouraging broader experimental validation. We have added ModelNet40 classification on rank-3 alpha complexes containing tetrahedra to Section 4.1 of the revised manuscript (new table marked in blue):

ModelRankRepresentation sizeOA (%) ↑GPU (GB) ↓s/epoch ↓
SCCNN28,072 cells85.3±0.885.3\pm0.82.666
SCCN38,072 cells81.6±1.081.6\pm1.00.972
Cellular Transformer38,538 cells—OOM—
MorseFormer3202 cells86.5±0.786.5\pm0.73.376
MorseFormer++31,224 cells90.2±0.390.2\pm0.34.3109

The Cellular Transformer exceeds the V100-32GB memory budget. The 1,224 representation elements of MorseFormer++ consist of 200 critical cells and 1,024 graph nodes processed in separate branches; higher-order attention operates only on the critical cells.

MorseFormer++ improves over SCCNN by 4.9 percentage points and rank-3 SCCN by 8.6 points, while global attention over its higher-order representation requires only 200 critical-cell tokens. MorseFormer reaches 86.5±0.7%86.5\pm0.7\% without a graph backbone, the second-highest accuracy among the successfully evaluated models in the table and 1.2 percentage points above SCCNN. The additional scaling experiments, summarized in Section 4.3 and detailed in Appendix C, demonstrate that Morse attention remains feasible on substantially larger rank-3 complexes.

We once again thank the reviewer for contributing to our work and for helping us better position the importance of it for the TDL community. We truly believe (as showed in the experiments) that combining discrete Morse theory with transformers could lead us to powerful topological neural networks and we hope that this work could also serve by inspiration for future TNNs to also have the possibility to go to ranks/dimensions higher than 2 without combinatorial explosion/time complexity issues. We expect that our rank-3 experiments are the first of many more to come along this path.


[1] C. Bodnar et al. Weisfeiler and Lehman Go Cellular: CW Networks. NeurIPS, 2021.

[2] S. Ebli, C. Hacker, and K. Maggs. Morse Theoretic Signal Compression and Reconstruction on Chain Complexes. Journal of Applied and Computational Topology, 2024.

Thank you for the reply. Considering the response, I still think my evaluation is appropriate for this paper.

Thank for your response. Could you please clarify why and what other points remain for this assessment?

Thank you for your service once again.

Reviewer_gmisOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces MorseFormer, an efficient attention-based architecture for topological graph learning. The method first lifts a graph into a higher-order cell complex and then uses discrete Morse theory to reduce the number of cells while preserving its homotopy type. The features of the removed cells are aggregated into the remaining critical cells, which are then processed using a GPS-style architecture. Since the resulting complex is considerably smaller, attention can be computed more efficiently. The authors evaluate their models on several standard graph benchmarks, demonstrating competitive predictive performance. They also evaluate their method on BREC, a synthetic expressivity benchmark, showing strong discriminative power. Finally, they demonstrate substantial reductions in attention computation on large synthetic graphs.

Critical strengths
  1. The proposed approach is simple and elegant, providing a principled way to reduce the theoretical computational cost of higher-order graph learning while preserving topological information.

  2. The method achieves competitive results across the evaluated graph benchmarks, outperforming several existing approaches on some datasets.

  3. The experiments on large synthetic graphs demonstrate substantial reductions in the number of cells and the computational cost of attention in this setting.

Critical weaknesses
  1. Limited evidence of practical computational advantages. One of the main motivations of the paper is that lifting graphs into cell complexes increases the computational cost of attention. However, for many real-world graphs, including those used in several of the paper's benchmarks, lifting introduces a relatively modest number of additional higher-order cells, making it unclear how significant this bottleneck is in practice. I also suspect that some of the computational overhead of existing topological neural networks could be significantly reduced through more efficient implementations, without requiring an additional reduction step. While the paper demonstrates impressive attention speedups on large synthetic graphs, there is limited evidence of substantial end-to-end efficiency improvements on real-world benchmarks. For example, on ZINC, MorseFormer++ takes 9.1 seconds per epoch, compared with 11.0 seconds for SMCN and 7.2 seconds for GPS. Even compared with SMCN, which involves relatively expensive computations, the speedup is fairly modest. A broader efficiency comparison against existing more efficient topological neural networks on more real-world benchmarks would therefore be important to establish the practical advantages of the proposed approach.

  2. Potential loss of important structural information. While discrete Morse reduction preserves the homology of the lifted complex, it may discard other potentially important structural information, such as the number of higher-order cells, which can reflect the number of cycles, cliques, or other meaningful substructures in the original graph. Although the method aggregates attributes from removed cells into the remaining critical cells, it is unclear to what extent this compensates for the loss of structural information or how this loss affects downstream performance.

  3. Limited evidence of a clear practical advantage in predictive performance. While common in this field, many of the graph classification benchmarks used in the paper are small, noisy and saturated, making it difficult to assess the practical significance of the results. The improvements are also somewhat difficult to interpret: the method outperforms several baselines on NCI1 and NCI109, but these datasets are noisy, and it does not consistently outperform existing approaches on other baselines. Since the paper's primary motivation is computational efficiency, a more direct comparison against topological neural networks with comparable real time computational complexity, demonstrating a clearer and more consistent advantage in the accuracy-efficiency trade-off, would strengthen the paper considerably.

Questions for authors
  1. Can the authors demonstrate substantial end-to-end computational improvements over existing higher-order graph learning methods on several real-world benchmarks, beyond the attention-only speedups reported on synthetic graphs?

  2. Can the authors demonstrate improvements in predictive performance over strong baselines when controlling for computational cost, for example by comparing models with matched training or inference budgets?

5 comments

We thank the reviewer for the valuable contribution to our community and for spending time in evaluating and strenghtening our work.

Part 1/2

Limited evidence of practical computational advantages. [...] While the paper demonstrates impressive attention speedups on large synthetic graphs, there is limited evidence of substantial end-to-end efficiency improvements on real-world benchmarks.

Can the authors demonstrate substantial end-to-end computational improvements over existing higher-order graph learning methods on several real-world benchmarks, beyond the attention-only speedups reported on synthetic graphs?

Our current evaluation splits between end-to-end computational costs from the savings obtained directly through Morse reduction (although they are both correlated, but since a transformer is a heavier architecture, a message passing TNN can be faster while operating in much more cells due to sparse message passing vs global attention). On ZINC, MorseFormer++ requires 9.1 seconds per epoch, compared with 7.2 for GPS and 11.0 for SMCN, while MorseFormer without the graph backbone requires only 3.1 seconds. Under static liftings, matching and retraction are computed once (so it can be a offline preprocessing): the Morse stage takes roughly 7–10 ms on the smaller molecular graphs (Appendix C). On larger complexes, the reduction decreases 38,076 cells to 2,174 critical cells, reducing attention time from 285 to 1.1 ms (Section 4.3).

We thank the reviewer for requesting practical measurements on larger higher-order structures. We have added ModelNet40 classification experiments in Section 4.1, including per-epoch training costs on V100 GPUs (new Table 4 in the revised PDF):

ModelRankRepresentation sizeOA (%) ↑GPU (GB) ↓s/epoch ↓
SCCNN28,072 cells85.3±0.885.3\pm0.82.666
SCCN38,072 cells81.6±1.081.6\pm1.00.972
Cellular Transformer38,538 cells—OOM—
MorseFormer3202 cells86.5±0.786.5\pm0.73.376
MorseFormer++31,224 cells90.2±0.390.2\pm0.34.3109

The Cellular Transformer [1] computes global attention in all original lifted cells and thus exceeds the V100-32GB memory budget. MorseFormer++ consists in MorseFormer combined with a GPS, so the 1,224 cells in fact means 200 cells for the MorseFormer and 1024 nodes that the GPS operates.

MorseFormer++ achieves 90.2±0.3%90.2\pm0.3\% accuracy, improving over SCCNN by 4.9 points and SCCN by 8.6 points. MorseFormer reaches 86.5±0.7%86.5\pm0.7\% without GPS, ranking second among the successfully evaluated models and outperforming SCCNN by 1.2 points. The table reports full-classifier per-epoch training costs

We have also added a scaling experiment on ModelNet40 alpha complexes to Appendix C of the revised manuscript, with the main result summarized in Section 4.3 (marked in blue):

PointsCritical cellsGPU memoryTraining step
1,0241750.52 GB32 ms
4,0968002.84 GB90 ms
8,1921,7429.69 GB234 ms

These measurements isolate the Morse branch on a V100-32GB, with lifting and reduction cached. As showed, Morse attention remains feasible at 8,192 points.

At 1,024 points, the Morse branch requires 32 ms per training step, compared with 74 ms for SCCN and 161 ms for SAN in the corresponding profiling experiment. Thus, the new evaluation reports full-classifier training costs separately from controlled higher-order encoder measurements.

Potential loss of important structural information. While discrete Morse reduction preserves the homology of the lifted complex, it may discard other potentially important structural information, such as the number of higher-order cells [...]

Indeed, homology or homotopy preservation does not imply preservation of every structural property of the original complex. However, the retraction can aggregate attributes from original cells into the surviving critical cells through its nonzero coefficients, using fixed or learned aggregation (Section 3.2). Although this does not guarantee preservation of all cell counts or task-relevant information, we can see by the experiments that MorseFormer keeps preserving information about the initial features and connectivity, moreover, this is exactly why our more powerful model is MorseFormer++, specially because we incorporate a graph transformer and merge it with the MorseFormer, this way the graph transformer keeps encoding all the necessary information about the original features and combinatorial structure, while the higher-order topological information is learned by MorseFormer in a scalable reduced way. Either way, even using only MorseFormer we can rely on powerful aggregations and our ablation confirms that this choice matters: DeepSets improves over fixed projection from 0.070 to 0.064 MAE on ZINC and from 72.68% to 74.11% accuracy on PROTEINS (Appendix D). Summarizing, MorseFormer relies on the aggregations for retaining the original information and MorseFormer++ retains the original graph through its GPS backbone, allowing the model to combine graph-level and Morse-reduced representations.

We have added ModelNet40 component experiments to Appendix D of the revised manuscript (new table marked in blue), separating standalone MorseFormer (86.5±0.7%86.5\pm0.7\%), MorseFormer++ without the auxiliary topological readout (89.8±0.6%89.8\pm0.6\%), and MorseFormer++ (90.2±0.3%90.2\pm0.3\%). Together with the existing ZINC and PROTEINS attribute-transport experiments, these results examine the contribution of the Morse representation and its readout.

A separate experiment, added to Appendix C and summarized in Section 4.3, demonstrates a further benefit of Morse reduction:

ConfigurationCritical cellsOA (%)GPU memorys/epoch
Rank 21,78389.3±0.489.3\pm0.425.3 GB193
Rank 320090.2±0.390.2\pm0.34.3 GB109

Adding tetrahedra reduces the critical-cell sequence almost ninefold and the number of possible attention-score entries by approximately 80 times, with memory falling from 25.3 to 4.3 GB and epoch time from 193 to 109 s. The rank-3 configuration reaches 90.2±0.3%90.2\pm0.3\% at 109 s/epoch. Thus, increasing the maximum rank can make Morse attention cheaper without requiring attention over the unreduced complex.

Limited evidence of a clear practical advantage in predictive performance. [...] The improvements are also somewhat difficult to interpret [...]

Can the authors demonstrate improvements in predictive performance over strong baselines when controlling for computational cost, for example by comparing models with matched training or inference budgets?

Our predictive evaluation already covers multiple established protocols. Under HOPSE/TopoTune, cellular MorseFormer++ achieves the highest reported mean performance on all five benchmarks. Under the TopoFormer protocol, it leads on COX2 and PROTEINS. On ZINC with edge features, MorseFormer++ obtains 0.064 MAE, improving over GPS, CIN, CIN++, and the Cellular Transformer. These comparisons establish competitive predictive performance across different settings.

To clarify the practical trade-off, we now report ModelNet40 classification accuracy alongside per-epoch classifier training time and peak GPU memory in Section 4.1. MorseFormer++ reaches 90.2±0.3%90.2\pm0.3\%, improving over SCCNN by 4.9 points and SCCN by 8.6 points. Standalone MorseFormer achieves 86.5±0.7%86.5\pm0.7\% without the graph backbone, the second-highest accuracy among the successfully evaluated models in the table. The new table allows accuracy and training cost to be evaluated together, while Appendix C separately measures the savings in the Morse encoder. These results support the feasibility of global attention after Morse reduction.

We once again thank the reviewer for taking the time to help us improve our work. We believe that one of the main challenges for topological deep learning is making higher-order information accessible to more powerful architectures without making their computational cost prohibitive. This is precisely what motivated MorseFormer, and we hope our new experiments help demonstrate why this work can be a start for powerful scalable TDL using computationally expensive architectures such as Transformers while being fast and preserving the higher-order topology.


[1] R. Ballester et al. Attending to Topological Spaces: The Cellular Transformer. arXiv:2405.14094, 2024.

I thank the authors for their rebuttal and the additional experiments. My main concerns remain. On ZINC, MorseFormer++ takes 9.1 seconds per epoch compared with 11.0 seconds for SMCN, a relatively expensive baseline. I consider this a modest speedup, and the evidence for larger end-to-end savings on other real-world datasets remains limited.
 The new ModelNet40 results partly address my scalability concern by showing that Morse reduction makes global attention feasible on the chosen alpha complex, which contains thousands of cells. It remains unclear whether simply using a coarser or sparser lift could match the accuracy of constructing a large complex and then reducing it, at comparable or lower computational cost.

I also agree with reviewer w3cg that the differences between MorseFormer and MorseFormer++ and their competitors are, in most cases, within the range of run-to-run fluctuations, and that the set of competitors is limited and omits state-of-the-art baselines. I therefore retain my original score.

Reviewer_w3cgOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper is motivated by an idea to improve the computational performance of topological deep learning (TDL) methods by lifting a graph to a cell complex. To ensure that lifting preserves the essential higher-order topological information, such lifting is performed using the notions of Discrete Morse theory (DMT). Particularly, DMT allows for splitting cells into non-critical (and as such potentially collapsable) and critical (that are to be preserved). Then, armed with the resulting Morse retraction, the authors then propose a Transformer-based model (MorseFormer) operating on the resulting critical cells. The utility of MorseFormer is shown in respect to graph classification.

Critical strengths

Addressing computational costs of TDL is an important and timely topic. Bringing topological methods to enhance computational efficiency, while preserving the key structural properties of the data is certainly a promising direction.

Critical weaknesses

While I was very excited to read the paper, I found that its promise does not reflect its content. I have many major concerns, and one of the primary ones is that it's very sloppy, both in terms of mathematics and presentation, with many missing, incomplete citations, definitions, results etc. One such example is "EMP (?)" on page 8. It looks as the paper has been conceived in an extreme hurry. Unfortunately, this sloppiness propagates into other more essential aspects beyond presentation. For example, the essential terms of DMT such as the critical vertex and the critical edge are mentioned cursorily and are not defined. Another example is Betti curves that are also not defined, and there are other instances.

More importantly, it appears that the method largely reduces to carrying topological information only on H1 (one-dimensional topological features) or even trivial case of H0 (connected components) with the cycle lift. This implies that 1) gains if any may be due to topological readout ztopoz_{topo}, GPS or other components; 2) it is unclear how the approach will perform on the data with richer topological structures, both in terms of accuracy and computational costs.

The ablation study is limited and does not delineate contributions of each component on the datasets with richer topology, i.e. the ones where it would make sense to apply such reduction.

The list of the competing models is very limited and omits many recent SOTAs, especially, for the considered task of graph classification. I have strong doubts that graph classification is even a good illustrative task for this type of models. The method must be evaluated on the task that inherently needs information on the higher order structures and datasets that have both richer topology and are substantially larger so that TDL is prohibitive.

Now, to the question above, not surprisingly, the performance of MorseFormer++ is very lukewarm. Indeed, mean accuracy of MorseFormer++ is either very close to the next best model (and we don't even have SOTAs here) or worse than the listed competitor. Moreover, a closer examination of standard deviations reveal that MorseFormer has substantially higher standard deviation than that of competitors, e.g. for MUTAG 13.42 for MorseFormer vs. 3.56 for GCN+PSEs which is 3.77 times higher. Similar dynamics is observed for other experiments. Now, MorseFormer++ has lower standard deviation than MorseFormer but its performance is statistically indistinguishable from the competing methods, that is, it is within the random fluctuations (except of NCI1 and NCI09 which have very small numbers of nodes). Remarkably, note how much lower standard deviations for competing methods in Table 2; only MorseFormer++ and TopoFormer have high standard deviations and are indistinguishable statistically from other SOTAs. Hence, there is no convincing evidence in the paper that MorseFormer++ bring any gains in performance.

Finally, methodologically, the paper never defines a discrete Morse function and uses directly the acyclic matching, i.e., the gradient vector field of some undefined discrete Morse function. However, the gradient is not intrinsic to the complex, and the critical cells and their basins depend on the discrete Morse function, so different Morse functions can result in the same number of critical cells but at the entirely different locations. The choice of discrete Morse function, or the associated score function, is hence essential and it is even more important to discuss and evaluate sensitivity of the model performance to different discrete Morse functions.

Questions for authors

See the critical weaknesses above

16 comments

We sincerely thank the reviewer for taking the time to carefully evaluate our work, especially at a time when scientific research and peer review face so many challenges. We truly value this contribution to our community. We also want to apologize if the presentation of our original submission gave the impression of rushed or careless work. Having also served as reviewers during this cycle, we understand how frustrating it can be to encounter papers whose presentation makes them unnecessarily difficult to evaluate. We know how harmful such practices can be to the scientific community, and this is something we take very seriously. We have invested considerable effort into this research, and it is difficult to hear that some presentation issues may have obscured its contributions. We hope that the revisions motivated by your comments make our work clearer, more rigorous, and easier to evaluate and understand, while better communicating why we believe discrete Morse theory can open an important direction for scalable higher-dimensional topological deep learning.

Part 1/3

I have many major concerns, and one of the primary ones is that it's very sloppy, both in terms of mathematics and presentation, with many missing, incomplete citations, definitions, results etc. One such example is "EMP (?)" on page 8. [...] For example, the essential terms of DMT such as the critical vertex and the critical edge are mentioned cursorily and are not defined. Another example is Betti curves that are also not defined [...]

Thank you for bringing our attention to this. Critical cells were defined in Section 2.2 as cells not belonging to the acyclic matching (we left this marked in green in the PDF for confirmation); critical vertices and edges are their dimension-0 and dimension-1 cases. Betti numbers were defined in Section 2.1, and Betti curves in the topological-readout appendix (Appendix E.1) as Betti numbers along a filtration. Nevertheless, we agree that these notions needed to be presented clearer. We now identify critical vertices and edges explicitly in Section 2.2, recall the formal discrete Morse function in Appendix D, and correct the missing EMP citation to Chen et al. (2024) [1] in Section 4.1. The revisions are marked in blue. So altough these information were already present in the paper, for us this feedback is highly valuable since it pointed to us where we could improve the general clarity of the paper.

More importantly, it appears that the method largely reduces to carrying topological information only on H1 (one-dimensional topological features) or even trivial case of H0 (connected components) with the cycle lift. [...] it is unclear how the approach will perform on the data with richer topological structures, both in terms of accuracy and computational costs.

The framework is not restricted to H0H_0 or H1H_1: ϕk\phi_k, B~k\tilde B_k, and Δ~k\tilde\Delta_k are defined for each rank kk, with rank-specific encoders and attention across critical cells (Sections 2.2–3.4). The maximum rank depends on the input lifting, not on a restriction of MorseFormer. However, this is precisely one of the contributions we consider most important, and we acknowledge that our original experiments did not adequately demonstrate it. We thank the reviewer for encouraging us to evaluate this capability, which motivated our new experiments on rank/dim greater than 2 .

We have therefore added ModelNet40 classification on rank-3 alpha complexes with vertices, edges, triangles, and tetrahedra (new Table 4, Section 4.1, marked in blue):

ModelRankRepresentation sizeOA (%) ↑GPU (GB) ↓s/epoch ↓
SCCNN28,072 cells85.3±0.885.3\pm0.82.666
SCCN38,072 cells81.6±1.081.6\pm1.00.972
Cellular Transformer38,538 cells—OOM—
MorseFormer3202 cells86.5±0.786.5\pm0.73.376
MorseFormer++31,224 cells90.2±0.390.2\pm0.34.3109

MorseFormer++ reaches 90.2±0.3%90.2\pm0.3\%, 4.9 points above SCCNN and 8.6 above SCCN; standalone MorseFormer reaches 86.5±0.7%86.5\pm0.7\% without GPS. The Morse branch uses 200 critical-cell attention tokens, while the 1,224 elements reported for MorseFormer++ also include 1,024 graph nodes processed separately. In this scenario, the full-complex Cellular Transformer [2] exceeds the V100-32GB memory budget. This is an evaluation on genuinely rank-3 structures, more specifically we are dealing with nodes, edges, triangles and tetrahedra (3-dimensional simplicial complexes).


[1] Y. Chen et al. EMP: Effective Multidimensional Persistence for Graph Representation Learning. Proceedings of the Learning on Graphs Conference, 2024.

[2] R. Ballester et al. Attending to Topological Spaces: The Cellular Transformer. arXiv:2405.14094, 2024.

Part 2/3

Finally, methodologically, the paper never defines a discrete Morse function and uses directly the acyclic matching, i.e., the gradient vector field of some undefined discrete Morse function. [...] The choice of discrete Morse function, or the associated score function, is hence essential and it is even more important to discuss and evaluate sensitivity of the model performance to different discrete Morse functions.

We agree that a scalar-valued discrete Morse function should have been recalled explicitly, and have added its formal definition to Appendix D (marked in blue). Such a function assigns values to cells so that each cell has at most one coface with a value no greater than its own and at most one face with a value no smaller than its own. However, a separate scalar score is not required to construct a valid Morse reduction. For a finite regular cell complex, an acyclic matching of incident cells on its face poset defines a discrete gradient vector field realizable by a discrete Morse function [3]. This is why our algorithm works directly with matchings: it constructs an acyclic gradient, instead of first assigning scalar scores and deriving the matching from them. In fact, the manuscript consistently follows this matching-based formulation, which was formally introduced in Section 2.2 and used throughout the subsequent construction. Nevertheless, we recognize that we did not explain its connection to Forman's original formulation clearly enough, which may have given the impression that an essential mathematical definition was missing. We thank the reviewer for pointing this out, as it allowed us to make this connection explicit in the revised manuscript.

We also agree with the reviewer that the matching is not intrinsic to the complex. Different valid matchings can produce different critical cells, basins, and transported attributes, even when the number of critical cells is the same. What discrete Morse reduction guarantees for each valid matching is preservation of the homotopy type and homology of the lifted complex; it does not guarantee identical learned representations across matchings. We now make this distinction clear.

To test how much this choice affects learning, we have added a matching-sensitivity experiment in Appendix D (marked in blue). We use acyclic matchings obtained from random cell orderings and a degree-based ordering, training three seeds per matching:

DatasetMatchingsStd. across matchingsStd. across seeds
MUTAG62.47.8
PROTEINS61.32.0
NCI120.21.4

Within these tested configurations, variability across matching choices is smaller than variability across training seeds. We do not claim matching independence: in particular, NCI1 includes only two matchings, and the experiment does not exhaust the possible discrete Morse functions. We thank the reviewer for identifying a dependence that deserved both a mathematical clarification and an empirical check.


[3] R. Forman. Morse Theory for Cell Complexes. Advances in Mathematics, 1998.

Part 3/3

The ablation study is limited and does not delineate contributions of each component on the datasets with richer topology, i.e. the ones where it would make sense to apply such reduction.

Appendix D already compared attribute-transport mechanisms with the remaining architecture fixed: DeepSets improves over fixed projection from 0.070 to 0.064 MAE on ZINC and from 72.68% to 74.11% accuracy on PROTEINS. The original manuscript also reports GPS alone under the HOPSE/TopoTune and TopoFormer protocols. In Table 1, MorseFormer++ improves over GPS by 7.2, 7.6, and 5.7 points on PROTEINS, NCI1, and NCI109, respectively.

We have added a ModelNet40 component-ablation table to Appendix D (marked in blue): standalone MorseFormer obtains 86.5±0.7%86.5\pm0.7\%, MorseFormer++ without the auxiliary topological readout 89.8±0.6%89.8\pm0.6\%, and MorseFormer++ 90.2±0.3%90.2\pm0.3\%. These measurements help separate the Morse branch and topological readout.

The list of the competing models is very limited and omits many recent SOTAs [...] The method must be evaluated on the task that inherently needs information on the higher order structures and datasets that have both richer topology and are substantially larger so that TDL is prohibitive.

We used established protocols to keep published comparisons meaningful: HOPSE/TopoTune includes graph, simplicial, and cellular approaches (including GCCN and HOPSE); TopoFormer includes eight published methods (including EMP and DASP); and our molecular comparisons include Subgraphormer, CIN++, SMCN, and the Cellular Transformer. In response to this criticism, we added rank-3 ModelNet40 experiments (Part 1) and further scaling results in Appendix C. Morse attention remains feasible on 8,192-point alpha complexes (1,742 critical cells; 9.69 GB; 234 ms per training step at micro-batch 16, with lifting/reduction cached).

We also compared the reported rank-2 and rank-3 configurations (Appendix C):

ConfigurationCritical cellsOA (%)GPU memorys/epoch
Rank 21,78389.3±0.489.3\pm0.425.3 GB193
Rank 320090.2±0.390.2\pm0.34.3 GB109

Tetrahedral cofaces allow additional Morse cancellations, producing almost nine times fewer attention tokens and approximately 80 times fewer possible pairwise attention scores.

Moreover, a closer examination of standard deviations reveal that MorseFormer has substantially higher standard deviation than that of competitors, e.g. for MUTAG 13.42 for MorseFormer vs. 3.56 for GCN+PSEs [...] Hence, there is no convincing evidence in the paper that MorseFormer++ bring any gains in performance.

We agree that standalone MorseFormer has high variability on MUTAG (84.21±13.4284.21\pm13.42), a dataset of only 188 graphs. MorseFormer++ reduces this variability (84.59±5.1984.59\pm5.19), but its MUTAG and PROTEINS improvements remain uncertain given the reported deviations. On NCI1 and NCI109, cellular MorseFormer++ improves over the strongest reported cellular baselines by 4.5 and 3.1 points. Exploratory unpaired Welch comparisons from five-seed summary statistics give p≈0.025p\approx0.025 and p≈0.003p\approx0.003 (Appendix D). The new ModelNet40 results show 90.2±0.3%90.2\pm0.3\% for MorseFormer++, compared with 85.3±0.8%85.3\pm0.8\% for SCCNN and 81.6±1.0%81.6\pm1.0\% for SCCN.

Finally, we would like to emphasize what we believe is the main contribution of MorseFormer to the TDL community. Just as graph Transformers have not replaced message-passing GNNs, we do not expect topological Transformers to replace message-passing TNNs. In fact, we believe both approaches have important roles to play. Our goal with MorseFormer is to make global attention a scalable alternative for higher-order topological learning, particularly in dimensions where operating on the full lifted complex becomes prohibitively expensive. Discrete Morse theory allows us to reduce these complexes before applying attention while preserving their homotopy type, making it possible to use Transformers on higher-dimensional structures with far fewer tokens. We believe the new rank-3 experiments help demonstrate why this direction is important and what it could offer to future TDL architectures. We sincerely thank the reviewer for pushing us to better demonstrate this contribution, as these experiments have become an important part of our work.

Comments on the paper

1 comment

Dear Reviewers and AC,

We thank you for helping us improve our paper. We updated the manuscript (changes in blue) and anonymous repo.

During the rebuttal, we added 18 datasets, new ablations, multi-seed experiments, efficiency studies and mathematical analyses. Tables 1–5 and 13 now contain 69 baseline comparison rows, including 24 distinct link-prediction baselines. This was substantial work in a short period.

Richer topology and ablations (w3cg, gmis, AvSR). We added ModelNet40 classification on rank-3 alpha complexes containing tetrahedra (Section 4.1, Table 4). MorseFormer++ reaches 90.2±0.3% accuracy, versus 85.3±0.8% for SCCNN and 81.6±1.0% for SCCN, with a statistically significant gain over SCCNN. Cellular Transformer runs out of memory on a 32 GB GPU, while our higher-order attention operates on about 200 critical cells. Standalone MorseFormer reaches 86.5±0.7% without a graph backbone. Tables 10–11 examine attribute transport and individual model components.

Stronger baselines and link prediction (w3cg, gmis, AvSR). We evaluated 15 link-prediction datasets and included the top five entries on the official OGB leaderboard (Table 5; Appendix G). On ogbl-collab, MorseLP achieves 72.70±0.31 Hits@50, surpassing HyperFusion, the highest-ranked public leaderboard entry, at 71.29±0.18 (Table 13). The gains over all five leading leaderboard entries are statistically significant (see our Nov. 18 reply for assumptions). MorseLP also has the highest mean on 9/15 datasets, with gains of 13.8, 14.1 and 14.5 points over the strongest listed baselines on Physics, Computers and Photo. NCI1 and NCI109 also show significant gains over HOPSE-G (Appendix D.3). Learning on critical cells raises ogbl-collab Hits@50 from 64.53 (heuristics) and 68.19 (fixed descriptors) to 69.51, with gains in all ten seeds (Table 18).

Generality across architectures (w3cg, gmis). Even without lifting, MorseFormer++ improves on GPS across all four classification datasets in Table 1. With GRIT, MorseFormer++ obtains 0.057±0.002 ZINC MAE, compared with 0.059±0.002 for GRIT and 0.060±0.004 for SMCN (Table 3; Appendix E). Morse reduction and attribute transport are backbone-independent, so advances in graph Transformers are complementary to our framework.

Practical efficiency (gmis, w3cg). On 20,000 ogbl-collab pair complexes, reduction shortens attention from 983 to 6.8 tokens on average, making it 40 times faster and reducing peak attention memory by a factor of 70 (Section 4.3; Appendix G.3). We compute the same extended-persistence images as TopoLink up to 14.8 times faster (Appendix G.1). Incorporating this computation and structural features into TopoLink reduces total wall-clock time from 121 to 74 minutes on CS and 6 to 2 minutes on Ecoli (Table 15). We report full-classifier costs and encoder scaling to 8,192 points (Tables 4, 9).

Richer liftings and large-scale graph coarsening (gmis, w3cg). Simply using fewer cells can change the topology of the lifted complex. Morse reduction preserves its homotopy type. Interestingly, adding tetrahedra on ModelNet40 reduces critical cells from 1,783 to 200 and memory from 25.3 to 4.3 GB (Table 8). We further demonstrate the potential of discrete Morse theory for graph coarsening (Appendix H). On ogbn-arxiv, we outperform the strongest reported baseline by 6.4 percentage points while retaining fewer vertices (15% vs. 20%). On ogbn-products (2.45M nodes and 61.9M edges), we retain 32% of the vertices in 201 seconds, achieving 72.33±0.11% accuracy.

Mathematical clarifications (AvSR, w3cg). We corrected the Hasse-arc convention, clarified acyclic matchings and attribute transport, and made the scope of our topological guarantees explicit (Sections 2–3). We studied matching sensitivity (Table 12) and related PrunIT to filtered Morse matchings (Proposition 6).

Our main contribution. One of the major challenges in TDL is making learning on higher-order topological domains scalable, particularly when using powerful architectures such as Transformers. Previous works, including the Cellular Transformer of Ballester et al., acknowledge this limitation. With discrete Morse theory, we can choose richer topological domains without necessarily increasing the cost of learning on them, while preserving their homotopy type. The new experiments also demonstrate its generality across architectures, topological domains, tasks and scales. We believe this is a significant contribution to TDL and can serve as a foundation for more powerful and scalable TNNs.

We firmly believe we have addressed the concerns raised by all three reviewers, and would greatly appreciate it if you could reconsider your assessments and increase your scores in light of the new evidence. We remain happy to discuss any remaining questions.

Thank you again for your valuable service to our community.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.