acceptodds
Under review as a conference paper at ICLR 2027

HIGHER-ORDER GRAPH REPRESENTATION LEARNING NEEDS A ROSETTA STONE FOR PORTABILITY

14 views

Abstract

The last decade has seen a rapid proliferation of non-Euclidean neural network architectures to address so-called higher-order network data exhibiting multi-way interactions. These architectures run the gamut, differing in their choice of higher-order representations (e.g., hypergraphs and simplicial complexes), their mathematical vernacular (algebraic, topological, geometric), and their intended task (e.g., link prediction, node classification, or segmentation). To help navigate this diverse landscape, several detailed surveys on higher-order representations have recently appeared, attempting to categorize and taxonomize these architectures. Yet multiple blindspots remain, obstructing both the utility of higher-order network representation learning and the development of its associated theoretical guarantees. In this position paper, we argue that portability between higher-order architectures is one such critical, largely under-explored area, vital to the future of graph deep learning. Through the lens of this meta-analysis, we present a “Rosetta stone" for higher-order graph representation learning, highlighting the recent progress, connecting the mathematical dots among multiple fragmented frameworks, and offering a synergistic perspective on the open problems and future directions of non-Euclidean abstract representation learning.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
higher-order neural networks, transfer learning, hypergraphs, simplicial complex

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_bHA8OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper discusses four types of higher-order graph representations and their integration into deep learning through higher-order neural network architectures. It then conducts a meta-analysis of ten survey papers, highlighting gaps in the literature, particularly the limited attention given to portability and transfer learning across different higher-order representations.

Critical strengths
  • The paper is overall well written and well organised.
  • Portability and transfer learning across higher-order representations are relevant and potentially valuable research directions.
  • The paper provides a useful illustrative example showing how the ability to distinguish higher-order structures depends on the representation, propagation rule, and input features.
Critical weaknesses
  1. Overstatement of the contribution. The authors state that they aim to "fill these knowledge gaps and connect the mathematical dots among multiple fragmented representation frameworks". However, the paper primarily reviews existing representations and identifies open research directions, rather than establishing new connections between these mathematical frameworks. Therefore, I find this claim stronger than what the paper delivers.
  2. Clarification of expressivity claims. The authors suggest that general analytical results on the expressivity of higher-order neural networks remain relatively rare (Lines 380–383). However, several theoretical results already exist, including those of Bodnar et al. (2021) on the expressive power of simplicial and cellular neural networks (Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networks [1] and Weisfeiler and Lehman Go Cellular: CW Networks [2]). I am not necessarily disputing the authors' claim, but I would appreciate a clarification of what they mean by "general and analytic statements" in this context, ideally with an example and a reference to a result that fits this description.
  3. Distinction between higher-order representations and their algebraic constructions. I am not convinced that cochain complexes should be presented on the same footing as hypergraphs, simplicial complexes, and cellular complexes in Section 2.1. While the latter describe underlying combinatorial or topological structures, chain and cochain complexes are algebraic constructions that can be associated with such structures, for instance through the corresponding boundary and coboundary operators. In this sense, they appear more naturally as algebraic representations of the underlying structures rather than as an analogous category of higher-order networks. I think the taxonomy would benefit from making this distinction more explicit, particularly given the paper's emphasis on separating mathematical abstractions from their algebraic instantiations.
  4. Ambiguity of the central position. The paper moves between portability, principled representation selection, and automated model adaptation without sufficiently clarifying their conceptual relationship or identifying which constitutes its primary thesis. While portability and transfer learning are presented as central questions in the abstract and introduction, the conclusion advocates principled representation selection (Lines 454–456) and ultimately automated model selection and adaptation (Lines 484–485). Although these objectives are related, the connections between them are not sufficiently developed. As a result, the paper sometimes appears to pursue several directions without establishing a sufficiently clear overarching argument.
  5. Limited development of the proposed perspective. Although the paper identifies relevant open problems, it could provide a more concrete conceptual account of portability and explain what distinguishes it from established transfer-learning settings. In particular, the relationship between Open Problems 2–4 and the central objective of portability could be developed more explicitly. This would help establish a more coherent research agenda and clarify how the proposed directions contribute to the main position.

References

[1] Bodnar, C., Frasca, F., Wang, Y., Otter, N., Montufar, G. F., Lió, P., & Bronstein, M. (2021). Weisfeiler and Lehman go topological: Message passing simplicial networks. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research, 139, 1026–1037.

[2] Bodnar, C., Frasca, F., Otter, N., Wang, Y. G., Liò, P., Montúfar, G., & Bronstein, M. M. (2021). Weisfeiler and Lehman go cellular: CW networks. Advances in Neural Information Processing Systems, 35, 2625-2640.

Questions for authors
  1. Could the authors clarify the central objective of the paper and the relationship between portability, principled representation selection, and automated model adaptation? This is related to weaknesses 4-5 and is essential to my assessment.
  2. Could the authors clarify what they mean by "general and analytic statements" about expressivity, ideally by providing an example of such a result and an appropriate reference?
5 comments
AuthorsOpenReview ↗

Response to Reviewer bHA8 - Part 1

We sincerely thank you for your thoughtful comments. We greatly appreciate that you find our portability and transfer learning across higher-order representations are potentially valuable research directions. We are here to provide detailed replies to your comments and hope we can resolve your major concerns. We have revised our manuscript accordingly and uploaded a highlighted PDF.

Q: Overstatement of the contribution. The authors state that they aim to "fill these knowledge gaps and connect the mathematical dots among multiple fragmented representation frameworks". However, the paper primarily reviews existing representations and identifies open research directions, rather than establishing new connections between these mathematical frameworks. Therefore, I find this claim stronger than what the paper delivers.

A: We have revised the sentence the referee quotes so that it matches what the paper delivers: the introduction now says that we aim to identify these knowledge gaps and begin connecting the mathematical dots among the fragmented frameworks, and the abstract says that we identify mathematical connections among them.

To show concretely what connecting the dots means, the new Appendix D works through one example: propagation over the triangles of a graph, encoded in three ways, as a simplicial complex whose triangles are filled, as a hypergraph whose hyperedges are the triangles, and as an order-3 adjacency tensor. As we explain there, three short identities relate the encodings. The hypergraph incidence matrix is half the product of the unsigned simplicial boundary matrices. Summing the tensor over one index gives the hypergraph's triangle-weighted adjacency matrix. And the tensor's quadratic propagation equals one hypergraph message-passing layer with a squaring activation, minus terms that depend only on that adjacency matrix. So for linear, unsigned propagation through the triangles, the three encodings agree, and they come apart through the nonlinearity, which is what our Gram-mate experiment also shows.

The example is illustrative and does not cover all cases. Connecting all of the mathematical dots is the research program we advocate, and no single paper can complete it. Our contribution is to pose that program explicitly. We separate three choices the literature often conflates (the abstraction, its algebraic instantiation, and the learning rule), use the meta-analysis to locate where the frameworks fail to connect, namely the transfer of trained models, and define in Section 4 what a successful connection must achieve and which open problems stand in its way. The triangle example shows that once the problem is posed this way, connections become concrete and checkable.

Q: Clarification of expressivity claims. The authors suggest that general analytical results on the expressivity of higher-order neural networks remain relatively rare (Lines 380–383). However, several theoretical results already exist, including those of Bodnar et al. (2021) on the expressive power of simplicial and cellular neural networks (Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networks [1] and Weisfeiler and Lehman Go Cellular: CW Networks [2]). I am not necessarily disputing the authors' claim, but I would appreciate a clarification of what they mean by "general and analytic statements" in this context, ideally with an example and a reference to a result that fits this description.

A: Thanks for mentioning these references, we are indeed aware of both of them and agree they do show that there is at least some prior work in this area. We previously did not cite them explicitly only because they already implicitly present in our meta-analysis, and surveys by Besta et al. and Papamarkou et al. have a rating of “substantive” for “expressiveness” precisely because these two surveys cite and discuss both of these references.

Because these constitute some of the only work we found on this topic, we still concluded that results on the expressivity of higher-order neural networks remain relatively rare.

We have cited these works explicitly in the updated manuscript to highlight that the literature is not completely silent on these topics, even if results are rare.

AuthorsOpenReview ↗

Response to Reviewer bHA8 - Part 2

Q: Distinction between higher-order representations and their algebraic constructions. I am not convinced that cochain complexes should be presented on the same footing as hypergraphs, simplicial complexes, and cellular complexes in Section 2.1. While the latter describe underlying combinatorial or topological structures, chain and cochain complexes are algebraic constructions that can be associated with such structures, for instance through the corresponding boundary and coboundary operators. In this sense, they appear more naturally as algebraic representations of the underlying structures rather than as an analogous category of higher-order networks. I think the taxonomy would benefit from making this distinction more explicit, particularly given the paper's emphasis on separating mathematical abstractions from their algebraic instantiations.

A: We thank the referee for this careful point and agree. A cochain complex is an algebraic construction on an underlying simplicial or cell complex, defined through its coboundary operators, and so it belongs as much to the algebraic instantiations of Section 2.2 as to the abstractions of Section 2.1. We list it in Section 2.1 because one line of work, built on discrete exterior calculus, takes cochains (discrete differential forms) as its primary modeling object, and our meta-analysis tracks this as a separate vernacular. As suggested, Section 2.1 now states this distinction explicitly.

Q: Ambiguity of the central position. The paper moves between portability, principled representation selection, and automated model adaptation without sufficiently clarifying their conceptual relationship or identifying which constitutes its primary thesis. While portability and transfer learning are presented as central questions in the abstract and introduction, the conclusion advocates principled representation selection (Lines 454–456) and ultimately automated model selection and adaptation (Lines 484–485). Although these objectives are related, the connections between them are not sufficiently developed. As a result, the paper sometimes appears to pursue several directions without establishing a sufficiently clear overarching argument.

A: Our central thesis is that higher-order graph learning cannot choose among its many representations in a principled way until trained models can be ported between them, and that this capability is missing from the field today. The paper is built to establish both halves of this claim. Section 2 shows that representation choices are consequential: the same data admit several abstractions and algebraic encodings, and our Gram-mate experiment shows that a single encoding can discard or expose structure depending on the propagation rule. Section 3 shows that the field has no way to move between these choices. All ten organizing works discuss translating data between representations, but none addresses transferring a trained model, and our broader search of primary work found none either. Section 4 explains why this gap matters and what stands in the way of closing it. Principled representation selection and automated adaptation, discussed in the conclusion, follow from the thesis, since both require comparing or switching representations without retraining from scratch. This conclusion is not obvious: each representation's literature looks complete from within, and the gap becomes visible only when they are read side by side, as our meta-analysis does. The introduction and conclusion now state this thesis in one sentence.

AuthorsOpenReview ↗

Response to Reviewer bHA8 - Part 3

Q: Limited development of the proposed perspective. Although the paper identifies relevant open problems, it could provide a more concrete conceptual account of portability and explain what distinguishes it from established transfer-learning settings. In particular, the relationship between Open Problems 2–4 and the central objective of portability could be developed more explicitly. This would help establish a more coherent research agenda and clarify how the proposed directions contribute to the main position.

A: We thank the referee for pressing on this. By portability we mean reusing a model trained on one higher-order representation on another representation of the same data, without retraining from scratch. Established transfer-learning settings, such as fine-tuning or domain adaptation, keep the representation and change the task or the data distribution, so the source model applies to the target as it is. Portability changes the representation, and with it the operator the model is built on. For example, a hypergraph network propagates through incidences and a simplicial network through boundary operators, so the two share no parameters. Porting a model therefore requires correspondences between representations.

The open problems are organized around these correspondences. Open Problem 1 asks whether representations that disagree about the data can share a latent space through which models are ported. Open Problems 2-4 ask whether random walks, notions of dimension, and spectra supply the correspondences that could ground that space. Open Problem 5 asks what porting through that space costs in expressivity. As requested, we've updated Section 4 to clarify these relationships explicitly.

Q: Could the authors clarify the central objective of the paper and the relationship between portability, principled representation selection, and automated model adaptation? This is related to weaknesses 4-5 and is essential to my assessment.

A: Our central objective is the thesis stated in our response to Weakness 4: higher-order graph learning cannot choose among its representations in a principled way until trained models can be ported between them, and this capability is missing today. In our view, the three notions the referee highlights (portability, principled representation selection, and automated model adaptation) form a chain in which each depends on the one before. That is, portability is a prerequisite for principled representation selection, which in turn is a prerequisite for automated model adaptation.

Portability is needed for principled selection because choosing between, for example, a hypergraph and a simplicial complex for a task requires comparing them with what the model has learned held fixed. Today, changing the representation means training a new model, so any comparison confounds the representation with the training run (Section 4). Principled selection is in turn needed for automated adaptation, because an agent that changes representations as new evidence arrives makes this same choice repeatedly and without human judgment (Section 5). It can do so reliably only if each choice rests on evidence, not intuition.

Portability is therefore the foundation of the chain. Without it, representation choices cannot rest on evidence, whether they are made by researchers or by agents, which is why portability is the focus of our paper.

Q: Could the authors clarify what they mean by "general and analytic statements" about expressivity, ideally by providing an example of such a result and an appropriate reference?

A: We had originally attempted to communicate what we mean by “general and analytical statements” about expressivity (and give examples) in definition elsewhere in the text: “Expressivity is an account of which structures a model can and cannot tell apart, like isomorphism based expressivity tests (e.g. the Weisfeiler-Leman hierarchy) and motif count based expressiveness.”

In our updated manuscript, we have included additional text to make this even clearer, and included additional references (Bodnar et al. ICML 2021, and Bodnar et et al. NeurIPS 2021) on papers that do contribute such results.

Reviewer_Y28hOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper argues that portability across higher-order graph representations deserves more attention. It distinguishes mathematical abstractions, algebraic encodings, and learning operators, reviews ten surveys, and outlines open problems for transferring models across representations.

Critical strengths

The distinction between representations, encodings, and propagation rules provides a useful framework for understanding higher-order learning. The Gram-mate example clearly illustrates why an information-preserving encoding does not guarantee that a model exploits that information. Cross-representation portability is a relevant research direction, with useful connections to expressivity and computational cost.

Critical weaknesses

The proposed portability agenda remains insufficiently concrete. The paper distinguishes translation, pretraining, and transfer, but does not specify a concrete transfer setting with clear success criteria. The numerical example concerns distinguishability within one encoding, rather than portability. This leaves the reader without a sufficiently actionable account of how to test the central proposal.

The literature analysis does not fully support the breadth of the gap claimed. Table 1 evaluates ten surveys, but limited coverage in surveys does not establish that primary research on portability is absent. The paper provides inclusion criteria but little detail on the search process or evidence behind individual coverage ratings. This weakens the claim that portability is a largely unaddressed research area.

Questions for authors
  • Could you specify one concrete source-to-target transfer task, including what is reused, what may be adapted, and how success would be measured against training from scratch?

  • How were the ten surveys selected, and could you provide supporting references for the coverage ratings and discuss primary research on cross-representation transfer?

  • Could you release the script for Figure 3, including the random seed and exact feature-mean grid?

7 comments
AuthorsOpenReview ↗

Response to Reviewer Y28h - Part 1

We sincerely thank you for your in-depth and thoughtful comments. We are here to provide detailed replies to your comments and hope we can resolve your major concerns. We have revised our manuscript accordingly and uploaded a highlighted PDF.

Q: The proposed portability agenda remains insufficiently concrete. The paper distinguishes translation, pretraining, and transfer, but does not specify a concrete transfer setting with clear success criteria. The numerical example concerns distinguishability within one encoding, rather than portability. This leaves the reader without a sufficiently actionable account of how to test the central proposal.

A: Section 4 now specifies a concrete transfer setting with explicit success criteria, which we describe in full below, in our response to the referee's question asking for a concrete source-to-target transfer task.

Q: The literature analysis does not fully support the breadth of the gap claimed. Table 1 evaluates ten surveys, but limited coverage in surveys does not establish that primary research on portability is absent. The paper provides inclusion criteria but little detail on the search process or evidence behind individual coverage ratings. This weakens the claim that portability is a largely unaddressed research area.

A: We agree more can be said about how we found the surveys and how we arrived at each rating. Of course, no literature review can be definitively shown to be exhaustive, but we have now made our search process transparent and every judgment easily checkable, via a detailed Appendix C that addresses the following:

Search process: Appendix C describes the search we carried out in August 2026: starting from works already known to us, manual Google Scholar searches (terms including "hypergraph", "higher-order network", "simplicial complex", and "transfer learning"), and LLM-assisted searches (Claude Opus with Exa web search, and Gemini) to find works the manual searches missed. Every candidate was screened against the three inclusion criteria in Section 3, and the appendix names the surveys we excluded and why.

Primary research: Our searches were not limited to surveys: they also covered primary research, from which we downselected to the works in Table 1. We found no primary work on transferring trained models across higher-order representations, so the gap is not an artifact of looking only at surveys.

Evidence for every rating: For each of the ten works, Appendix C states the reason for every substantive and partial rating and pinpoints where the work treats the topic, down to specific pages and sections, and it lists each work's absent topics. Every page and section number is hyperlinked to the exact version of the paper we read, so any rating can be checked in one click. This also makes the meta-analysis a practical guide in its own right: a researcher can see at a glance where each survey discusses each topic.

Comparison with LLM-generated ratings: Although not requested, we also compared our ratings with ratings produced by Claude Opus from the same inclusion criteria and topic definitions (Appendix C, Table 2). The two agree on 72 of the 100 cells, and in 26 of the 28 disagreements the LLM rated coverage higher. The substantive ratings for expressivity, pretraining, and transfer are identical, so the gaps we identify are unchanged.

We believe these changes greatly improve the utility and auditability of the meta-analysis; we thank the referee for raising this point.

AuthorsOpenReview ↗

Response to Reviewer Y28h - Part 2

Q: Could you specify one concrete source-to-target transfer task, including what is reused, what may be adapted, and how success would be measured against training from scratch?

A: We agree the paper should specify a concrete test, and Section 4 now does. To answer each of the referee's requested points:

Task. Higher-order link prediction on the data of Benson et al. (2018): predicting which three nodes, already linked in pairs, will later interact as a group, for example three researchers who have each co-authored with the other two later writing a paper together. Section 4 cites the common intuition that simplicial complexes are the natural representation for this task, and this test can check it.

Source model. A model trained on the data represented as a hypergraph. It has an encoder, which computes a vector for each node, and a predictor, which makes the prediction from those vectors.

Target. The same data represented as a simplicial complex, which also includes every subset of each group, for example every pair within a group of three.

What is reused and what is adapted. The predictor is reused unchanged. Only a new encoder is trained for the simplicial complex, without labels, so that it gives each node the same vector the hypergraph encoder gave it.

How success is measured. The ported model is given a small number of labeled examples on the simplicial complex and compared with two models trained from scratch there. It succeeds if it comes close to a model trained on all of the labeled data, and beats a model trained on the same small number of examples. The second comparison ensures that any gain comes from what was carried over, not from the new examples.

Q: How were the ten surveys selected, and could you provide supporting references for the coverage ratings and discuss primary research on cross-representation transfer?

A: Our response to Weakness 2 above covers the search, the supporting references for each rating (Appendix C), and primary research on cross-representation transfer. On selection specifically, each of the ten works meets the three inclusion criteria of Section 3. It explicitly defines the higher-order structures it considers, it primarily aims to survey, categorize, or generalize the field, and it focuses on neural networks rather than solely on the mathematical object. Surveys of higher-order networks that do not focus on learning were excluded under the third criterion, and Appendix C now names them.

Q: Could you release the script for Figure 3, including the random seed and exact feature-mean grid?

A: We agree Figure 3 should be reproducible. The experiment behind it is short, so we have specified it completely in the new Appendix B: pseudo-code, the feature-mean grid (29 equally spaced values from -1 to 2.5), the number of draws per grid point, the threshold, and the random number generator and seed (NumPy's default generator, seed 1). With these, Figure 3 can be reproduced exactly. As we note there, the results do not hinge on the seed: rerunning with ten other seeds changes no plotted value by more than 0.003 in median separation or 0.007 in the fraction of separating draws.

I thank the authors for the response and the revisions.

There are still some important concerns:

  • There are no transfer results. The ZINC experiment is the only experiment that tests portability, and its results are still missing (“[Results coming]”, empty Table 3). In addition, the HNN/SNN/CNN variants are reweightings of the Laplacian on the same molecular graph. The evaluation is zero-shot and mixes in transfer across molecule sizes, so it does not follow the (ε,k) definition in Section 4.
  • The Section 4 protocol cannot test the motivating claim. The target encoder is trained to reproduce the source (hypergraph) embeddings. By construction it cannot exploit the extra structure of the simplicial complex, so the protocol cannot show whether simplicial complexes are better suited to link prediction. It measures distillation, not which representation is better suited to the task.
  • The claim about primary literature is not in the paper. The response says the search also covered primary research. The manuscript contains no account of this, so the claim that “portability is absent” still rests on surveys only.
  • The LLM is not an independent second coder (raised by Reviewer tdbK). Agreement with the LLM coder is moderate (κ = 0.56). The LLM received the authors’ own criteria, so it is not an independent coder, and in most disagreements it rates coverage higher than the authors do.

The revisions improve clarity and reproducibility. However, the central proposal is still untested

Reviewer_tdbKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This is a position paper. The authors argue that higher-order network deep learning has split across hypergraphs, simplicial complexes, cell complexes, and differential forms, and that the under-discussed problem is portability: whether a model trained on one algebraic instantiation can be used on another, instead of retraining whenever the representation changes.

The paper separates the abstract representation, the algebraic instantiation, and the learning rule. A co-authorship example shows that the same collaborations can be encoded in four structures, and that a simplicial complex invents edges absent from the data. A Gram-mate experiment shows that linear propagation never separates the pair, while nonlinear activations sometimes do. A meta-analysis scores ten surveys and finds that transfer of a trained model across higher-order representations is absent. The paper then lists five open problems. The authors state that this is a research agenda, not a validated transfer procedure.

Critical strengths
  1. One example connects all three decisions. Clique expansion turns multiway interactions into pairwise summaries. The same incidence matrix cannot be separated with identity activation, but it can be separated with tanh. So a richer complex does not automatically give you more expressive power.

  2. Portability is a real problem: if you change the representation, you have to retrain, your tools do not carry over, and common beliefs, like “simplicial complexes are the default for link prediction”, are tangled up with one particular training run. Open Problem 5 says a predictor cannot recover structure that the encoder has already thrown away.

Critical weaknesses
  1. The title promises a Rosetta stone, but the paper does not really deliver one. The coverage table is basically a survey of surveys, and the five open problems are just a research to-do list. Another possible explanation is that these representations are combinatorially inequivalent, so forcing them into one shared latent space would average away differences that a task actually needs. The paper never says what empirical result would prove that portability is the wrong goal. Without a core claim that can be tested, the paper reads like an annotated reading list.

  2. The claim that transfer is the current blind spot depends on the authors’ own three-level coding scheme, and the experiments never train or transfer a model. The Gram-mate study only changes the activation function. Open Problem 5 just restates Section 2.3.

  3. Change the title so it matches what the paper actually delivers. “Rosetta stone” suggests that a correspondence between these representations already exists, but the paper does not establish one.

Questions for authors
  1. Can an independent second coder reproduce Table 1? For every cell marked absent or partial, please point to the exact passage that supports that label.

  2. Is there any attempt to transfer between representations on a real dataset—even a failed attempt?

  3. What does “portability” add beyond the “translation” that Table 1 marks substantive for all ten surveys, plus ordinary transfer learning? Why is a JEPA-style shared latent space the right instrument rather than direct operator alignment?

4 comments
AuthorsOpenReview ↗

Response to Reviewer tdbK - Part 1

We thank you for your time and valuable feedback! Below we present our answers to questions. We have revised our manuscript accordingly and uploaded a highlighted PDF.

Q: The title promises a Rosetta stone, but the paper does not really deliver one. The coverage table is basically a survey of surveys, and the five open problems are just a research to-do list. Another possible explanation is that these representations are combinatorially inequivalent, so forcing them into one shared latent space would average away differences that a task actually needs. The paper never says what empirical result would prove that portability is the wrong goal. Without a core claim that can be tested, the paper reads like an annotated reading list.

A: We believe that there is a misunderstanding here, and we apologize for not making it transparent. Our title states that higher-order graph representation learning needs a Rosetta stone, not that we deliver one, and the paper argues for and explains that need. The submitted abstract, however, did wrongly say that we "present" a Rosetta stone, and we have revised it to match the title. Thanks for challenging on this.

Regarding the referee's suggestion that our thesis is not falsifiable, we would like respectfully to push back on this. Our thesis has two parts, and both can be tested. The first, that trained models cannot currently be ported between higher-order representations, is a claim about the existing literature. Our search found no such papers (Appendix C), although we admit that there is a non-zero probability of their existence. The second, that portability is the right goal, can be tested using the definition that Section 4 now provides. That is, a model trained on one representation is portable to another if a transfer procedure, using a limited number of labeled target samples, comes within a tolerance of a model trained from scratch on all of the target data and outperforms a model trained from scratch on those same samples. Hence, portability may be a wrong goal for a task if no procedure meets this standard. It is also the wrong goal if procedures meet it only where the two representations carry the same information, for example when the hypergraph is already a simplicial complex. That would mean a shared latent space succeeds only when there is nothing to reconcile, which is the alternative the referee raises, i.e. the representations are inequivalent, and aligning them averages away differences a task needs.

Our open problems are not an unordered to-do list but are the steps needed to decide between portability and this alternative. Open Problem 1 asks whether representations that disagree about the data can share a latent space at all, and a negative answer would settle the question against portability. Open Problems 2-4 ask whether specific quantities, namely random walks, notions of dimension, and spectra, correspond across representations. Where they do, a shared space can be built on them; where they do not, they locate the differences that alignment would average away. Open Problem 5 asks whether porting through a shared space loses expressivity, which is the referee's concern posed as a research question.

Q: The claim that transfer is the current blind spot depends on the authors’ own three-level coding scheme, and the experiments never train or transfer a model. The Gram-mate study only changes the activation function. Open Problem 5 just restates Section 2.3.

A: While the coding scheme is indeed ours, we’ve revised the paper so that every rating can now be checked. Appendix C gives the reasons and linked pages behind each substantive and partial rating, and a second coding by Claude Opus from the same criteria also rates transfer absent in all ten works.

The Gram-mate study changes only the activation by design. With the representation and input features held fixed, it isolates the propagation rule as what decides whether higher-order structure survives. Under the identity activation the two hypergraphs produce identical outputs for every input, so no later layer, however it is trained, can tell them apart. Under tanh their outputs differ on every sampled input, and under ReLU on some. This matters for portability because it shows that the representation alone does not determine what a model can use, so translating the data is not the same as porting a trained model.

Open Problem 5 builds on Section 2.3 rather than restating it. Section 2.3 shows that an encoder can discard structure within one representation; Open Problem 5 asks whether a model ported to a finer target representation inherits that loss, and whether joint training across representations can recover it.

AuthorsOpenReview ↗

Response to Reviewer tdbK - Part 2

Q: Change the title so it matches what the paper actually delivers. “Rosetta stone” suggests that a correspondence between these representations already exists, but the paper does not establish one.

A: As in our response to Weakness 1, the title states that the field needs a Rosetta stone, and the paper argues for that need. We do not claim that such a correspondence already exists. In our previous version the abstract indeed erroneously stated that “we present a Rosetta stone” (we are sorry about it) and we corrected it now. So the title and abstract now match. If the referee still feels that the title should be changed, we would also be happy to do it. Some of our alternative titles are, for example, "In Search of a Rosetta Stone for Portability in Higher-Order Graph Representation Learning", "Toward Portability in Higher-Order Graph Representation Learning", or "Missing Links in Higher-Order Neural Networks: A Meta-Analysis and Call for Transfer Learning".

Q: Can an independent second coder reproduce Table 1? For every cell marked absent or partial, please point to the exact passage that supports that label.

A: We agree that rating the coverage of existing literature inevitably involves some subjectivity. However, our revisions now make every rating in Table 1 transparent and auditable. The new Appendix C gives, for each of the ten works, the reason for every substantive and partial rating and the exact pages and sections that support it, each hyperlinked to the version of the paper we read, so a second coder can check any cell in one click. Absent topics are listed for each work. Where a cell is borderline, for example a topic a work mentions but does not develop, the entry says so and gives the page. As a further check on reproducibility, we ran an experiment in which we had Claude Opus complete the same Table 1, using the same criteria and topic definitions in the paper. We found that its ratings agree with ours on 72 of the 100 cells, with identical substantive ratings for expressivity, pretraining, and transfer, the gaps our conclusions rest on. We thank the referee for this suggestion, which has made Table 1 much easier to verify and has strengthened the impact of the meta-analysis.

Q: Is there any attempt to transfer between representations on a real dataset—even a failed attempt?

A: Yes. Our revision now adds a second experiment, complementing the Gram-mate study, that trains models on a real dataset and transfers them across higher-order structures. On the ZINC AMONS dataset of about 140,000 small organic molecules, we train models to predict molecular potential energy and then evaluate the trained weights after the structure is changed, comparing each transferred model with one trained on the target from scratch. All models are built on the recent Inner Product Laplacian framework, whose choice of inner products on vertices and edges reproduces the structure of graph, hypergraph, simplicial, and chain-complex neural networks. Because the four variants share parameter shapes, a model trained on one can be applied to another directly.

Of course, this experiment does not show everything our framework calls for. All four variants live on the same molecular graph, so it does not test transfer between a hypergraph and a simplicial complex built as separate objects, and it tests transfer through a common operator family rather than through the shared latent space of Open Problem 1, and both remain important open questions. However, we believe the experiment is an original contribution in this space and adds value along the lines the referee suggests, and we are not aware of any prior experiment that transfers trained models across higher-order structures in this way. Even a failed transfer would be informative, since it would point to the inequivalence the referee raises in Weakness 1.

We have updated new experimental results in the revised version. The revised PDF marks the results as forthcoming, and we will post them here as soon as they are in.

AuthorsOpenReview ↗

Response to Reviewer tdbK - Part 3

Q: What does “portability” add beyond the “translation” that Table 1 marks substantive for all ten surveys, plus ordinary transfer learning? Why is a JEPA-style shared latent space the right instrument rather than direct operator alignment?

A: Thanks for this interesting question!

Translation can be viewed as a data-level operation before training. For example, a hypergraph may be converted into a simplicial complex or a clique expansion, and a new model is then trained from scratch on the result, so little if anything that the original model has learned carries over. (Also, a small correction that Table 1 rates translation substantive in six of the ten works and partial in the other four.)

In contrast, transfer learning may change the task or the data distribution but keeps the input domain (i.e., the space of objects the model takes as input) and often the model architecture, which is why parameters can often be reused. In the higher-order setting this breaks down. For example, a hypergraph network propagates through vertex-hyperedge incidences, while a simplicial network propagates through boundary operators and Hodge Laplacians, so their parameter spaces do not in general match and we cannot fine-tune directly. The closest existing area is heterogeneous transfer learning, but it largely focuses on disparate feature spaces rather than on higher-order representations. Furthermore, as discussed in Open Problem 1, higher-order representations can disagree about the ground truth itself. A simplicial complex asserts relations that the hypergraph denies, such as the tie {a,c} in Figure 1, which no paper in the data has.

By portability, we mean the ability to carry what a model has learned, not just the data, from one higher-order representation to another without retraining from scratch, and to state what such a move preserves or discards. Translation, pretraining, and cross-representation transfer are the three levels of this notion in Table 1. The surveys cover the first in depth, partly cover the second, and do not address the third. Section 4 now makes this definition precise and states how to test it.

In regards to why a JEPA-style shared latent space rather than direct operator alignment. We view operator alignment as a valuable and complementary tool rather than a competing one, and Open Problem 4 already proposes spectral correspondences as targets for latent alignment. We advocate a shared latent space as the primary instrument for two reasons. First, the operators of different representations have no common index set (hyperedges in one, edges and triangles in the other), and where the representations disagree, as with {a,c}, the correspondence is undefined. Second, aligning operators does not determine what the model computes. The two Gram-mate hypergraphs in Section 2.3 have identical SS^T, so any alignment based on these Gram operators treats them as the same, yet with a tanh activation their outputs differ on every sampled input. The new Appendix D shows the same pattern for triangles. The simplicial, hypergraph, and tensor operators agree at the linear level and come apart through the nonlinearity. As our Gram-mate experiment shows, what a model learns is determined jointly by the representation, the propagation rule, and the input features. Operator alignment, hence, matches only the first of these factors, whereas a shared latent space aligns their combined effect.

AuthorsOpenReview ↗

Dear Reviewer tdbK, Have we addressed all your question?

Dear Reviewer tdbK,

Thanks a lot for your constructive feedback! We have tried to address all your questions. Since the deadline of the discussion period is around the corner, can you please let us know whether you have any other questions?

The Authors.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.