acceptodds
Withdrawn from ICLR 2027

Batch Before You Lift: Scalable Topological Deep Learning on Large Graphs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

11 views

Abstract

Topological Deep Learning extends graph-based learning to higher-order domains, such as hypergraphs, cellular, and simplicial complexes. These domains are typically constructed from patterns in an input graph through a process of graph lifting. Current methods operate only after this lifting phase, requiring the entire lifted domain to be materialized prior to training. On large and dense datasets like Reddit (k nodes and M edges), this global materialization becomes a severe computational bottleneck, often rendering training infeasible. To address this limitation, we introduce Cluster-TNN, a domain-agnostic framework that avoids this bottleneck by lifting locally instead. After partitioning the input graph during preprocessing, at runtime Cluster-TNN dynamically samples groups of node clusters, reconstructs their induced subgraphs to form mini-batches, and applies the chosen lifting within each mini-batch. By retaining all edges among the sampled nodes, Cluster-TNN preserves higher-order structures spanning sampled clusters, producing topological mini-batches that existing Topological Neural Networks can process directly. Across evaluated domains, lifting procedures, and architectures, Cluster-TNN reduces peak GPU memory by an average of % while maintaining competitive predictive performance. Notably, such a reduction enables, to our knowledge, the first training of multiple different higher-order Topological Neural Networks on large datasets such as Reddit and OGBN Products. These results establish Cluster-TNN workflow as a general strategy for scaling Topological Deep Learning beyond the limits of global domain construction.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Topological Deep Learning, Topological Neural Networks, Graph Neural Networks, Higher-Order Networks, Scalable Graph Learning, Hypergraphs, Cell Complexes, Simplicial Complexes

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_MZNqOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper tackles the problem of applying Topological Deep Learning (TDL) architectures to large graphs, for which lifting procedures can exacerbate computational difficulties by inducing the need to materialise a combinatorially large number of additional higher-order structures.

The authors advocate for an approach whereby lifting is applied only after graph minibatching, posterior to merging graph clusters in output from a graph partitioning procedure. This allows to only materialise the higher order structures for each minibatch allowing applications to graphs which, after lifting, would induce a too large higher-order domain that would potentially be hard to fit in memory and deal with. At the same time, by lifting only after clusters are merged, this approach allows to retain higher-order structures supported across partitions.

The authors experiment with various graph datasets and TDL architectures, exploring aspects such as variations in runtime, memory footprint and performance. They then run ablation studies on the impact of design choices, including inference and partitioning scheme.

Critical strengths
  • (S1) The paper motivation is relatively clear and the authors present it in an understandable way.
  • (S2) The authors work on an extensive set of datasets and architectures, proposing a valuable stratification of the former ones.
Critical weaknesses
  • (W1) Critically, the reviewer finds the methodological contribution rather thin: essentially, the proposed method is a simple application of ClusterGCN (Chiang et al, 2019) whereby minibatches are lifted to capture the contained higher-order structures by downstream topological deep learning architectures. Importantly, the authors are not very explicit abut this. In fact, partitioning and stochastically combining partitions at training time is not their contribution, rather ClusterGCN's. While they do cite and actually mention the contributions by ClusterGCN in the related works, they are not upfront enough on the fact that, e.g., only card (4) in Figure 1 (a) is their contribution, the rest is ClusterGCN. As another example, the method presented in section 3.1 is essentially ClusterGCN except for the minibatch-wise lifting procedure.

  • (W2) Results presented in Section 3.3 (Theoretical Guarantees) are mostly reasonably trivial. With due respect, the reviewer deems it rather trivial that "A structure can be recovered only if its supporting cluster fit within one minibatch", and that, in the limit, if one runs enough epochs the probability to observe every observable structure approaches one. It is basically obvious that increasing q also increases the number of observable structures and their recovery probability. The authors report various statistics for different values of q, but it is not clear what is the take home message. These results depend on many other factors, including the clustering approach and the size of the partitions. Generally speaking, this section whole section does not provide any interesting insight. ... More interesting results could investigate, instead, the inter-relation between the partitioning procedure and the lifting function or non-random cluster merging approaches that leverage the graph structure to improve the coverage probability. There is not much in the paper and these results in particular that is really specific to higher-order topological domains beyond graphs.

  • (W3) The experimental section is not well structured and clear. The authors propose a series of questions, but then proceed to discuss results in a way that is sometimes confusing.

    • Often, seemingly relevant tables and figures are relegated to appendix with much clarity on this choice.
    • Separating the computational gains from performance in different sections (4.2, 4.3) does not really help: computational gains are only relevant for reasonably interesting performance (and "viceversa"). Figure 3 puts the two together, but it is quite hard to parse.
    • It is not clear what corresponds to GCN + Cluster-TNN in Table 1: is it just ClusterGCN?
    • There is essentially no mention and discussion on the utilised lifting procedures by the various considered approaches. It is understandable that the method is to be shown agnostic to the lifting procedure, yet there is not even one mention to the various lifting procedures, and whether they can explain differences in the performance. This is a work on Topological Deep Learning, after all, the reviewer deems it striking that the authors do not discuss these components at all.
  • (W4) The presented results are either somewhat expected, rather underwhelming, or not discussed much in their nuances

    • Section 4.2: Isn't it expected that the method allows for lower GPU footprint? It would be more interesting to compare them to other approaches for scalable training?
    • Except for OGBN-Products, it does not seem that higher-order methods are significantly better than the basic GCN approach, see, e.g. Coauthor Physics and Reddit. What is the point then?
    • The method seems not to be compared to other approaches for scalable TDL.
    • There is a lot of variability on results across various TDL methods, why is that and how can we interpret these, and in relation to the proposed contributions? This is not clear or discussed much in the main paper.
Questions for authors

Does the GCN baselines use a comparable inference ensembling scheme ensuring, e.g., comparable results in Table 1?

3 comments

We thank the reviewer for the thorough review, for recognizing the clear motivation (S1) and the breadth of datasets and architectures (S2), and for the concrete suggestions on what a more informative analysis would study. We respectfully disagree, however, with two of its conclusions: that the contribution is a thin application of Cluster-GCN (W1), and that the structural analysis is trivial and not specific to higher-order domains (W2).

The batching is Cluster-GCN's, and the paper builds on it explicitly (Section 3.1). What we add is lifting within each mini-batch, which lets the mini-batch control both the construction of the higher-order domain and the execution of the model. As a result, an entire family of existing TNNs, across all three higher-order domains, trains unchanged on Reddit and OGBN Products, where full-domain execution exceeds the available resources for several of them. Below, we answer each point and the question in turn.

W1. Methodological Novelty and Relationship to Cluster-GCN

We agree that partitioning and stochastic cluster grouping are Cluster-GCN's contributions. The paper credits Cluster-GCN as the methodological foundation (Section 2) and introduces our batching as "following the multi-cluster batching scheme of Cluster-GCN" (Section 3.1), but this should be clear from the start. In the revised version, we state this in the introduction. However, we see this simplicity as a strength: full lifting is what has kept TDL benchmarks small, and batching before lifting lets any TNNs and liftings run unchanged on graphs where it fails. What changes for TDL is the role of cross-cluster edges: for a GNN they carry messages, but for a TNN they do both: decide whether higher-order structures exist at all and thereby allow them to carry the messages. In fact, our ablation reflects this: on Cora Full, removing them costs GCN 0.08 pp but TopoTune 10.01 pp (Table 2). The core contribution is the ability to scale an entire family of architectures: higher-order TNNs from all three domains now train on large benchmarks like Reddit and OGBN Products (Table 1). It also includes the recovery analysis and the measurement of full-domain construction costs.

W2. Depth of Theoretical Guarantees and Alternative Partitioning Strategies

We respectfully disagree that the analysis is trivial or not specific to higher-order domains. Its starting condition, that a structure is recovered exactly when its supporting clusters share a mini-batch, is an assumption on the lifting procedure. It holds trivially for graphs (the identity lifting) and holds for the neighbourhood and clique-complex liftings, but it can fail for cycle-basis lifting, where restricting the graph can change the selected basis (Appendix C.1). On Cora Full, the analytical prediction matches the measured recovery of hyperedges, triangles and simple cycles, while cycle-basis recovery falls below it for exactly this reason (Figure 7, Appendix C.13.2).

The lifting also determines how many clusters a structure needs: a graph edge spans at most two, whereas a triangle can span three, a retained cycle up to nine, and a neighbourhood hyperedge has no fixed bound (Section 3.3, Appendix C.13.1). This separates lifting families: on Coauthor Physics, complete neighbourhood hyperedges reach only 62.8% recovery even at q=1,000q = 1{,}000, whereas both 2-cell families exceed 99.8% (Appendix C.13.1). Therefore, observability does not imply exposure: on Questions at q=20q = 20, every cellular and simplicial 2-cell is observable, yet only 22.0% and 31.9% of their supports are encountered within 200 epochs (Appendix C.13.1).

The reviewer is right that these quantities depend on the partition. For predictive performance, a new sweep over KK and qq (see our reply to Reviewer CeGJ, Q4) shows that the higher-order models vary by at most 3.9 points on Questions and 2.3 points on Amazon Ratings, while GCN on Questions is the most sensitive. We thank the reviewer for the suggestion to study the interplay between partitioning and lifting, which we find convincing. Our ablation points in this direction: on Cora Full, METIS improves GCN by 0.98 pp but EDHNN, UniGNN and SCCNN by 5.3–6.5 pp. Structure-aware cluster merging is a natural extension that we plan to investigate.

W3. Discussion of Lifting Procedures

We thank the reviewer for the comments. GCN + Cluster-TNN in Table 1 is GCN run through the same workflow with the identity lifting, that is Cluster-GCN mini-batching combined with our ensemble inference.

We acknowledge that the paper does not analyse whether the choice of lifting explains the differences in performance between models. Our focus is feasibility, and the pipeline is agnostic to the lifting and applies to any lifting that can be evaluated on an induced subgraph (Section 3.1). We therefore use TopoBench's default lifting for each domain (Appendix A.2.1 and B.2.1): closed one-hop neighbourhood hypergraphs (EDHNN, UniGNN), a cycle basis with cycles of length at most nine (CWN, TopoTune), and a clique complex up to dimension two (SCN, SCCNN). Comparing liftings within a domain at this scale is what the pipeline now makes possible, and we see it as a natural extension.

W4. Performance Expectations, Nuances, and Comparisons with Scalable Baselines

The lower GPU footprint is expected in direction, and our measurements quantify it (83.2% mean reduction over 21 comparisons) and show that the binding limit can often be CPU-side construction. For example, for TopoTune on Reddit, mean peak CPU memory is 480.1 GB during full-graph preprocessing, compared with a mean run-wide peak of 11.81 GB under Cluster-TNN (Section 4.2). For other scalable TDL approaches, please see our reply to Reviewer Rzz4 (W2.A): FastHGNN and SaNN operate on an already constructed domain, so a matched comparison is only possible where the full domain fits.

On the comparison with GCN, higher-order models do not uniformly outperform GCN. Our claim is that these comparisons can now be made at this scale, not that TNNs always win, as we emphasise in the revised manuscript. On OGBN Products, however, EDHNN, SCN and SCCNN outperform GCN (86.51%, 87.03% and 88.63% vs. 85.81%), and on the three controlled datasets the best model is a TNN under both settings (Table 1). These differences motivate comparing higher-order representations across datasets, a task enabled at scale by the common Cluster-TNN workflow.

The variability comes largely from the models rather than from the batching. It is already present under full-graph training with the same liftings: on Cora Full, full-graph accuracy ranges from 46.40% (TopoTune) to 70.81% (SCCNN), compared with 56.45–70.32% under Cluster-TNN (Table 1). Models that share a domain also differ in architecture, for example in the neighbourhood operators they use on the same complex (Table 5: CWN uses the rank-1 upper adjacency, TopoTune also a two-hop rank-0 upper adjacency), and they differ in how much they rely on cross-cluster structure: on Cora Full, removing cross-cluster edges costs TopoTune 10.01 pp but CWN only 0.66 pp, although both use the same cycle-basis lifting (Table 2).

Q1. Clarifying Inference Ensembling in GCN Baselines

Yes, all Cluster-TNN results in Table 1, including the GCN baseline, use the same inference protocol, and all full-graph results, including GCN, use full-graph inference, as the Table 1 caption states. GCN is therefore evaluated under the same conditions as the higher-order models in both settings.

The ensemble benefit is also not specific to TNNs. Table 2 reports the gain of ensemble over single-pass mini-batch inference for every model, and for GCN it is +2.51 pp on Cora Full, +0.05 pp on Amazon Ratings, and +0.65 pp on Questions. More generally, ensemble inference "yields higher mean than a single-pass in all 21 comparisons" (Section 4.4), with gains from 0.04 to 4.65 pp (Table 2), and these 21 comparisons include GCN on all three datasets.

Reviewer_MZNqOpenReview ↗

Response to rebuttal

I would like to genuinely thank the authors for their responses.

My main concerns however still remain, mostly in relation to the depth of the contribution. The additional methodological substance on top of ClusterGCN is thin and I believe the current additional analyses are not interesting enough to compensate that. The current theoretical claims seem to be become less trivial only for a specific lifting procedure, thus limiting its impact. Experimentally, the authors acknowledged that more analyses and studies are possible by lifting after batching, and I believe that some should minimally be addressed already in this paper.

I believe the paper still requires additional work to be ready for publication and I maintain my current score.

Reviewer_Rzz4OpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2(+1)
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Cluster-TNN, a framework for scaling Topological Neural Networks by applying graph mini-batching before higher-order lifting. The authors use Cluster-GCN to construct induced subgraphs, which are independently lifted into hypergraphs, cellular complexes, or simplicial complexes and processed by existing TNN architectures.

Critical strengths

The main strength of the paper is that it addresses a relevant computational limitation of Topological Deep Learning. Constructing complete higher-order domains can become prohibitively expensive on large graphs, particularly for clique-complex lifting.

I appreciate that the authors implement their approach within TopoBench, making it compatible with several existing TNN architectures and lifting procedures. The paper also provides detailed computational measurements, including preprocessing costs, memory consumption, and runtime. Demonstrating that previously infeasible configurations can be trained is a useful practical contribution.

Critical weaknesses

1. The difference of batch then lift when compared to lift then batch is not well described. For instance, when we are thinking in static liftings such as k-hop, clique lifting and cycle lifting, we can always set their hyperparameters to reflect the size of the locality we are trying to describe, eg size of the cycle, how many hops (k), etc... So this seems much more a engineering perspective than a new method that contributes to TDL community and the gains obtained from using cluster instead of other approaches is not well ablated or explained.

2. The comparison with recent GNN and TNN papers is hard to understand, papers like FastHGNN [1] and SaNN [2] already investigate scalability in higher-order networks, but I could not find direct experimental comparisons with these approaches. I recognize that they address different computational bottlenecks, but this makes it difficult to understand the practical advantages of Cluster-TNN over existing alternatives. Another point is that recent learnable lifting methods such as DCM [3] and Difflift [4] are not discussed, despite showing that learning higher-order structures can improve downstream predictive performance compared to static liftings. he scalability of higher-order learning.

The theory also seems somewhat limited to me. For instance, in cellular complexes, different higher-order cells can share boundaries in the original domain but be processed in different mini-batches. Even if these cells are eventually recovered, it is not clear whether the interactions between them are also recovered. I am not sure if I am missing something in the theoretical analysis, but I could not find a clear justification connecting this structural recovery to the actual higher-order message passing performed by the TNN.

References

[1] Lu et al. FastHGNN: A New Sampling Technique for Learning with Hypergraph Neural Networks. ACM Transactions on Knowledge Discovery from Data, 2024.

[2] Gurugubelli et al. SaNN: Simple Yet Powerful Simplicial-aware Neural Networks. ICLR, 2024.

[3] Battiloro et al. From Latent Graph to Latent Topology Inference: Differentiable Cell Complex Module. ICLR, 2024.

[4] Franco et al. Differentiable Lifting for Topological Neural Networks. ICLR, 2026.

Questions for authors
  1. What exactly is the methodological advantage of batch-before-lift compared to existing sampling procedures? Could the authors better justify why clustering is necessary in this problem and why and if is the optimal approach?

  2. Why are recent higher-order scalability methods not included in the experiments? Also why are learnable lifting approaches not discussed?

  3. Could the authors better clarify what the structural recovery theory imply for the actual TNN computations? For instance, when higher-order cells share boundaries in the original cellular complex but are recovered in different mini-batches, how are their interactions preserved during message passing?

5 comments

We thank the reviewer for the review and for recognizing the practical contribution, the TopoBench integration, and the detailed computational measurements. In summary, batching before lifting addresses a different bottleneck from restricting lifting hyperparameters or subsampling an already constructed domain: it avoids building the full lifted domain at all. Furthermore, our work targets the memory cost of constructing and processing the lifted domain on large graphs, which is orthogonal to how the lifting is chosen or learned, and could in principle be combined with learnable liftings (W2.B). We respectfully disagree that tuning lifting hyperparameters achieves the same effect (W1). We answer each point and question in detail below.

W1. Methodological Clarity and Ablation Studies

Restricting lifting locality and restricting the graph presented to the lifting address different quantities, and the liftings we evaluate are already minimal: k=1k = 1 neighbourhood hypergraphs and clique complexes up to dimension two, i.e., triangles only (Appendix A.2.1). Clique lifting in particular leaves nothing further to restrict: even at dimension two, Reddit contains about 8.36 billion triangles, and its full simplicial feature matrices alone are estimated at 20.3 TB (Table 4). Reducing the dimension further would remove the higher-order structures under study. For cycle-basis lifting, the length cap filters cycles only after a complete cycle basis has been computed, so it does not reduce construction cost (Appendix B.7). Batch-before-lift instead limits the input to each construction call while keeping the lifting parameters fixed, so the two strategies are complementary rather than interchangeable.

Regarding the ablation of clustering, Section 4.4 evaluates the choices it involves (Table 2), and Q1 below explains why we use clustering. Retaining cross-cluster edges has a higher mean than removing them in 19 of 21 comparisons. For TDL, cross-cluster edges decide whether higher-order structures exist at all: on Cora Full, removing them costs TopoTune 10.01 pp but GCN only 0.08 pp.

W2.A. Extended Baselines

FastHGNN and SaNN are discussed in Section 2. Both operate on an already constructed domain: FastHGNN subsamples a preconstructed hypergraph and SaNN precomputes aggregations over the full simplicial complex, whereas our aim is to avoid constructing the full domain. The reviewer is right that we do not compare with them experimentally. Such a comparison is only possible where the full domain fits, for example, FastHGNN on Reddit's neighbourhood hypergraph, whose full construction takes 137.7 s and 20.77 GB of CPU memory (Table 5). SaNN on Reddit would first require the full clique complex, which our full-graph preprocessing could not build (Table 5) since its feature matrices alone are estimated at 20.3 TB (Table 4).

W2.B. Learnable Methods

We thank the reviewer for pointing to DCM and DiffLift. They are relevant recent approaches, but they address a different problem: learnable liftings learn from the data which higher-order structures to build, while our approach changes where a given lifting is computed. They also operate at a different scale. The DCM authors identify message passing and the search for induced cycles as its main bottlenecks and note that it "may benefit from further improvements to effectively scale on very large datasets" ([3], Section 6). Its largest benchmark, Coauthor Physics, has about 248K edges, whereas Reddit and OGBN Products have 57M and 62M. The DiffLift authors likewise list scaling to larger graphs as future work ([4], Section 6). Since computing the lifting is the step that batch-before-lift restricts to induced subgraphs, the two approaches are complementary, although we have not shown that arbitrary learnable liftings can be combined with ours. We discuss both methods in the related work of the revised version.

W2.C. Structural Recovery Theory

The reviewer is correct in understanding that recovering individual cells over training does not imply that all their interactions are seen in one forward pass. For example, take two triangles sharing an edge, with the shared edge's vertices in one cluster and each remaining vertex in a different cluster: with q=2q = 2, either triangle can appear, but never both together.

The appendix connects recovery to message passing in a narrower sense. Under the compatibility condition of Appendix C.9, a message along an incidence between two cells is available exactly when the higher-ranked cell is recovered. Appendix C.10 quantifies how often such messages are exposed, and Appendix C.11 bounds the gap between full-domain and expected mini-batch aggregation for a single layer. As Appendix C.11 states, this analysis does not cover same-rank messages, such as those between two 2-cells sharing an edge, which are available only when both cells appear in the same mini-batch. As with Cluster-GCN on graphs, the mini-batch computation therefore approximates the full-domain computation rather than reproducing it, and predictive adequacy is evaluated empirically (Tables 1–2). We made this explicit in the main text.

Q1. Justifying the Methodological Advantage of Clustering

Existing sampling procedures for higher-order models operate on an already constructed domain, so the full lifted domain must be built and stored before any sampling takes place. Batch-before-lift samples the input graph instead, so a lifted domain only ever exists for one mini-batch. On Reddit, where the full clique complex cannot be built (Table 5), SCN and SCCNN have mean run-wide peak GPU memory of 6.84 and 7.45 GB, respectively (Table 9).

Clustering is not necessary: batch-before-lift requires a sampler that returns induced subgraphs, and we do not claim that Cluster-GCN's is optimal. We adopt Cluster-GCN’s multi-cluster batching because METIS groups locally connected nodes, helping keep the nodes supporting triangles, short cycles and neighbourhoods together. Recombining clusters and retaining their connecting edges also allows structures spanning partition boundaries to appear in a mini-batch.

The structural analysis provides evidence for this choice. On Cora Full and Amazon Ratings (K=32K = 32), 83–99% of the reference 2-cells already fall within single METIS clusters. With q=4q = 4, the supporting nodes of more than 99% of these reference structures appear together in a mini-batch at least once within 200 epochs (Appendix C.13.1). The partition ablation points the same way: METIS achieves a higher mean score than random partitioning in 15 of 21 comparisons (Table 2). We thank the reviewer for this question: comparing alternative induced-subgraph samplers is a natural next step that we plan to investigate in the future.

Q2. Addressing Missing Scalability Baselines and Learnable Lifting Literature

The scalability methods the reviewer mentions reduce model-execution cost on a domain that has already been constructed (Section 2), whereas our method targets the regime in which the full domain cannot be constructed. They therefore address a complementary stage of the pipeline, and a matched comparison is only possible where the full domain fits (see W2.A). We did not discuss learnable liftings because they address which structures are built rather than where the lifting is computed. As noted under W2.B, we added a discussion of DCM and DiffLift to the related work.

Q3. Clarifying Structural Recovery and Message Passing Interactions Across Mini-Batches

Within a single training step, these interactions are not computed: messages pass only between cells present in the same lifted mini-batch, so two cells that share a boundary but appear in different mini-batches do not interact in that step (W2.C gives an example). The recovery analysis nevertheless extends directly to such pairs. Both cells, and hence their interaction, are present exactly when all clusters supporting their union fall into one mini-batch, so the pair's co-occurrence probability follows the same expression as the recovery of a single structure (Section 3.3), with the union's cluster span in place of a single structure's. Independent cluster recombinations give such pairs repeated chances to co-occur during training, whenever their union spans at most qq clusters, and ensemble inference averages predictions over several recombinations. This holds for liftings that satisfy the compatibility condition, such as the clique and neighbourhood liftings. Moreover, for the cycle-basis cellular lifting, a mini-batch can also contain cells that differ from the full-graph ones (Appendix C). As noted under W2.C, the mini-batch computation therefore approximates the full-domain computation.

References

  • [1] Lu et al. FastHGNN: A New Sampling Technique for Learning with Hypergraph Neural Networks. ACM Transactions on Knowledge Discovery from Data, 2024.
  • [2] Gurugubelli et al. SaNN: Simple Yet Powerful Simplicial-aware Neural Networks. ICLR, 2024.
  • [3] Battiloro et al. From Latent Graph to Latent Topology Inference: Differentiable Cell Complex Module. ICLR, 2024.
  • [4] Franco et al. Differentiable Lifting for Topological Neural Networks. ICLR, 2026.
Reviewer_CeGJOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper is clearly trivial where the authors just attempt to batch the graph before lifting vs. traditional lifting then batching. While the paper provides many experiments, many of them are trivial and based on a simple fact: sampling causes lossy structure, so they either have to repeatedly sample more during inference or training longer to observe all samples. The authors didn’t consider the real-world scenarios for node-level and link-level tasks like online inference that may pose a major challenge for lifting after batching when the graph is large-scale. For this reason, I believe the paper should be a workshop-level paper, where the novelty and significance are extremely limited.

Critical strengths

None. While the paper attempts to solve the scalability problem of Topological Deep Learning, the paper failed to consider the practical aspect of the method. The proposed solution is unpractical for any large-scale node-/link-level problems (see Weaknesses below). Furthermore, by materializing the complexes after batching is more like an engineering trick rather than a scientific contribution.

Critical weaknesses
  1. The method doesn't work on node classification nor link prediction. For example, in inductive node classification, we often have unseen nodes appearing in the graph during evaluation, thus leading to different graph partitions for validation/test graphs. For this reason, every new node requires online lifting, causing high latency. For link prediction, the problem is even worse. In link prediction without temporal split, a link in graph often chosen randomly and hidden from message passing graph. Therefore, for a set of links, the sampled nodes may not faithfully maintain the original graph structure, thus degrading the topological complex's original structure. In other words, the framework has problems as the sampling rate increases since the more we sample, the more edges hidden. Therefore, the lifted graphs can't be cached offline, and thus have a high overhead in online stage. Even when the experiments do provide node-level classification, I doubt the applicability of the framework in practice since every node needs either lifted complex cached or online lifting.
  2. On line 80, the authors should be more explicit about what existing mini-batch strategies are.
  3. On line 85, the statement regarding limitations of batching directly on different lifted representations requires different procedures is unsupported. I do not think the statement is true either. Specifically, the only difference between these topologic domains is the lifting function, while the batching procedure is identical since all of them share the same interface (see TopoX library implementation).
  4. Again, the framework claims to have a unified interface across topological domains is an overstatement. The shared interface between topological domains has been implemented in libraries like TopoX.
  5. The assumption for the method to work is trivial: the paper assumes that a complex is recoverable if all supporting clusters are included in the mini-batch. Since the sampling algorithm is induced subgraph sampling, the underlying sub-graph is thus the same graph that is used to perform topological lifting in prior literature.
  6. The ensemble necessity is trivial since the complex isn’t fully recoverable in 1 sample for large graphs. For this reason, the claim regarding scalability is thus further challenged since now we need to sample more due to ensemble during inference (please see Q3).
  7. Reproducibility Statement and AI Use Statement should be right after the main text and before references.
Questions for authors
  1. Can you additionally plot ogbn-products in Figure 3?
  2. Is the reported memory usage Figure 4 ensemble or single-model?
  3. Can you benchmark the inference time latency due to online lifting and compare it with inference time of methods whose lifting is at pre-processing?
  4. Can you report the predictive performance vs cluster sizes vs number of clusters?
8 comments

We thank the reviewer for their feedback, but respectfully and strongly disagree with the characterization of our work as “clearly trivial.” We find that several of the reviewer’s criticisms overlook important technical distinctions and experimental evidence presented in the manuscript, which we clarify in detail below. In particular, we disagree with the statement that the method does not work for node classification, as well as with the claim that shared software interfaces imply identical batching procedures across topological domains.

Large-scale TDL faces two coupled challenges: constructing the higher-order domain and executing neural networks over its elements and incidence relations. Sampling an already constructed domain can reduce execution costs, but leaves the initial construction requirement in place. Building on Cluster-GCN, Cluster-TNN moves lifting after graph mini-batching, allowing the mini-batch to control both construction and model execution. This enables the evaluated TNNs across all three higher-order domains to run on Reddit and OGBN Products without changes to their architectures or lifting procedures. We measure the resulting structural exposure, memory requirements, runtime and predictive performance.

Beyond the transductive experiments in the main text, we have now evaluated all seven models on Reddit under an inductive protocol and conducted the requested predictive-performance sweep over KK and qq. We present these results below, together with responses to the specific methodological claims and questions.

W1. Inductive node classification, link prediction and online lifting

We would like to clarify that Cluster-GCN, which we build on, is itself evaluated inductively. Its Table 12 marks all four tasks as inductive, including Reddit and Amazon2M (the graph released as OGBN Products), and its Section 6.2 states: "the testing nodes are not visible during the training process. Thus, we construct an adjacency matrix containing only training nodes and another one containing all nodes. Graph partitioning are applied to the former one". The code of [1] re-partitions the full graph for validation and test. Our new inductive experiments below follow the same protocol.

The experiments in the main text use transductive node classification. To address the reviewer’s concern about inductive applicability, we conducted additional experiments with all seven models on Reddit’s official split, excluding validation and test nodes from the training graph.

Inductive results on Reddit (official split, 3 seeds) are shown below. We reuse the transductive Reddit hyperparameters without retuning, so these results demonstrate feasibility rather than tuned performance. Validation and testing use a separate partition of the full graph, with the 10-recombination ensemble at test time. We adapted the shared data pipeline to handle separate training and evaluation graphs, while the model architectures and lifting procedures remain unchanged.

ModelTest accuracy (%)
GCN92.30±0.0192.30 \pm 0.01
EDHNN91.82±0.6691.82 \pm 0.66
UniGNN91.32±0.1091.32 \pm 0.10
CWN89.84±1.6789.84 \pm 1.67
TopoTune89.83±0.1689.83 \pm 0.16
SCN90.89±0.1590.89 \pm 0.15
SCCNN91.53±0.2091.53 \pm 0.20

All seven models train and evaluate under this protocol and stay within 1.5 pp of their transductive Reddit accuracies in Table 1 (which use a random 50/25/25 split).

The runtime cost of repeated lifting is reported in the manuscript (Section 4.2, Appendix B.5.1), and the Conclusion names developing strategies for caching of lifted mini-batches as future work.

Furthermore, the need to incorporate newly arriving nodes also affects methods based on offline full-graph lifting. Even a single newly connected node can introduce higher-order structures and change incidence relations, requiring the precomputed representation to be updated or rebuilt. Without an incremental update procedure that handles the chosen domain, lifting, and required operators, the offline pipeline must repeat full-domain preprocessing. Its cost is substantial: on Reddit, cellular preprocessing for CWN takes approximately 4,685s and 480 GB of peak CPU memory, while clique-complex preprocessing runs out of memory (Appendix B.3, Table 5).

Cluster-TNN instead constructs lifted mini-batches on demand, allowing new nodes to be processed without rebuilding or storing the complete lifted domain. Accommodating new nodes therefore reinforces the motivation for our approach: Cluster-TNN avoids the full-domain construction requirement by design, without requiring the development of a separate, domain- and lifting-specific incremental update algorithm.

Link prediction is outside the paper's scope: all experiments and claims concern node classification, and we state this explicitly in the revised version.

W2. Existing mini-batching strategies

Section 2 already discusses relevant domain- and architecture-specific procedures:

  • FastHGNN samples nodes and hyperedges layer by layer for hypergraph convolution [2].
  • Hypergraph Neural Networks through the Lens of Message Passing samples hyperedges and then nodes within them [3, Section 4.4].
  • TopoMamba uses node-to-cell incidence for batching and explicitly distinguishes its procedure from those used for SCN and SCCNN [4, Section 4.2 and Section 5.2].

These provide concrete support for the discussion. We added a direct pointer to Section 2 in the introduction.

W3-W4. Shared interfaces and batching across topological domains

Libraries such as TopoX provide a common interface across topological domains. However, the statement in Section 1 (W3) concerns sampling algorithms rather than software interfaces. Sampling within a lifted domain must respect domain-specific constraints, such as simplicial face closure and cellular boundary relations [5, 6], and existing samplers are accordingly domain- or architecture-specific [2–4]. As noted in Appendix B.7, the hypergraph samplers “do not directly specify how to sample simplicial or cellular domains while preserving the face or boundary relations used by those models”. We acknowledge that the original wording could also be interpreted as referring to software interfaces. We clarified that our discussion concerns sampling within a large lifted domain, rather than collating independently constructed examples.

Regarding W4, the manuscript explicitly presents these interfaces as existing infrastructure that Cluster-TNN reuses. The contribution in Section 1 is titled “Reusable open-source integration” and states that we implement Cluster-TNN in TopoBench, “reusing existing lifting and models across different higher-order domains without requiring architecture-specific modifications.” Section 2 credits TopoX’s data structures and neural operators and TopoBench’s common benchmarking framework. Section 3.1 likewise specifies execution “through the same interface used for full-domain execution.” The manuscript therefore does not claim to have developed these interfaces. Cluster-TNN is domain-agnostic because it relies on them for lifting and model execution, and because it performs batching on the input graph, which all liftings share (Section 1).

Our methodological contribution reverses the usual order: we batch the input graph first and lift each mini-batch, instead of lifting the full graph and then batching the lifted domain. The full higher-order domain is therefore never built and each training step constructs and processes one lifted mini-batch. On Reddit, TopoTune's mean peak CPU memory is 480.1 GB during full-graph preprocessing, compared with a mean run-wide peak of 11.81 GB under Cluster-TNN (Section 4.2). Because batching acts on the input graph, which all liftings share, the same procedure serves hypergraph, cellular and simplicial models. Shared interfaces do not provide this: TopoNetX's restriction routines, for instance, take an existing complex as input. Shared interfaces support this implementation, but do not themselves remove the requirement to construct the full lifted domain. To the best of our knowledge, Cluster-TNN is the first domain- and model-agnostic approach to scaling graph-supported TDL.

W5. Recovery analysis and the construction bottleneck

We respectfully disagree that the assumption is trivial. While it holds for the neighbourhood and clique-complex liftings, it can fail for cycle-basis lifting, where restricting the graph can change the selected basis (Appendix C).

Moreover, the recovery condition makes the approximation explicit and provides the basis for analysing structural exposure over a finite training budget. This distinction is already illustrated in Section 3.3 (Figure 2): on Coauthor Physics at q=100q = 100, every reference 2-cell is recoverable within a mini-batch, yet only 59.3% of cellular and 58.9% of simplicial supports are encountered within 200 epochs. The gap is more pronounced on Questions at q=20q = 20, where all 2-cells are recoverable, but only 22.0% and 31.9% of cellular and simplicial supports, respectively, are encountered by epoch 200 (Appendix C.13.1). Thus, recoverability does not guarantee that training encounters these structures within a finite training budget.

Reusing the same lifting on induced subgraphs is deliberate. Forming these subgraphs before lifting avoids constructing the full lifted domain while retaining existing lifting procedures and models. On Reddit, TopoTune’s mean peak CPU memory is 480.1 GB during full-graph preprocessing, compared with a mean run-wide peak of 11.81 GB under Cluster-TNN (Section 4.2).

W6. Ensemble inference

Ensembling is optional, not necessary, as already specified in Section 3.2, which defines single-pass, ensemble, and full-graph inference, and specifies ten recombinations for the reported ensemble results. The rationale and execution are also explicit. Section 3.3 describes averaging predictions “from the same trained model”, while “retaining the memory benefits of processing one lifted mini-batch at a time”. Table 2 compares the inference modes, and Section 4.4 explicitly reports that ensemble inference achieves a higher mean test metric than single-pass inference in all 21 comparisons. Without ensembling, single-pass inference is lower by 0.04–4.65 pp (Table 2). Thus, the additional passes introduce a runtime trade-off without requiring multiple models or lifted mini-batches to be held in memory simultaneously.

Q1. OGBN Products in Figure 3

Figure 3 reports runtime and memory ratios relative to full-graph execution, together with differences in predictive performance (Section 4.2). These comparisons require completed full-graph baselines, which are unavailable for the affected OGBN Products configurations, as stated in the experimental setup: "For the intermediate-density and large-scale regimes, full-domain execution of at least one higher-order TNN exceeds the available resources" (Section 4.1). The evaluated cellular and simplicial input feature matrices alone require an estimated 39.414 GB and 274.156 GB, respectively, excluding connectivity representations, temporary allocations, and training state (Table 4). Appendix B.3 details the full-domain resource constraints. We therefore report OGBN Products separately through predictive results in Table 1 and runtime and memory measurements in Tables 6 and 9.

Section 4.1 characterizes the dataset regimes in terms of graph size, connectivity, and lifting-induced expansion, placing OGBN Products in the large-scale regime, where full-domain execution of at least one higher-order TNN exceeds the available resources. Accordingly, matched full-graph comparisons are restricted to the controlled regime.

These resource constraints motivate batch-before-lift. To the best of our knowledge, no prior work has trained TNNs from all three higher-order domains on a graph of this size. Cluster-TNN trains all six higher-order models we evaluate on it without changing their architectures or liftings, with mean peak GPU memory between 0.05 and 11.94 GB (Table 9).

Q2. Memory measurement in Figure 4

Figure 4 reports peak GPU memory usage during training, measured in training-only runs (Section 4.2), so it reflects neither single-pass nor ensemble inference. During inference, ensemble and single-pass evaluation have comparable peak GPU memory requirements, as the ensemble reuses the same trained model and processes one lifted mini-batch at a time. In our implementation, predictions are transferred to the CPU for aggregation, avoiding additional GPU memory overhead from storing ensemble predictions. We clarified the measurement scope in the Figure 4 caption.

Q3. Inference latency and existing runtime measurements

Table 6 provides the corresponding breakdown into preprocessing, training, and final evaluation. These comparisons use the same TNN architectures under full-graph execution, where lifting occurs during preprocessing, and Cluster-TNN, where mini-batches are lifted during execution (Appendix B.5.1). Hyperparameters are selected independently for full-graph and partitioned execution.

The additional evaluation cost is explicitly quantified: final evaluation (i.e. inference) is longer under Cluster-TNN, with a median ratio of 11.75×11.75\times for ten-recombination inference compared with one full-graph pass (Appendix B.5.1). The paper also specifies that validation batches are reconstructed and lifted online and explains where these costs are included (Appendix B.5.1). Thus, the reported measurements already account for the computational trade-off introduced by repeated lifting and evaluation. They measure final-evaluation time, including checkpoint validation and test inference, rather than isolated lifting time or per-request serving latency, which we did not measure. Our demonstrated benefit is enabling execution within memory constraints, with the associated runtime costs reported explicitly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.