acceptodds
Under review as a conference paper at ICLR 2027

Graph Manifold: Rethinking Graph Learning Through Riemannian Geometry

Abstract

Graph Neural Networks (GNNs) have achieved remarkable success in graph representation learning by modelling graph topology and the relationships among nodes embedded within it. However, existing graph learning typically regards graphs as discrete structures and optimises them based on local neighbourhood relationships, ignoring the intrinsic geometric properties and global evolution patterns. To address this, we rethink graph learning from the perspective of Riemannian geometry and introduce a novel **Graph Manifold**, which provides a continuous geometric space for graph representation and optimisation. Specifically, we reconstruct graph topology geometrically based on the Cholesky manifold, representing graphs as structured objects on a manifold space and enabling holistic geometric graph optimisation. Furthermore, we define a distributional structure on the Graph Manifold by modelling directional distributions in the tangent space, allowing the characterisation of intrinsic graph variations. Moreover, to extend the applicability of Graph Manifold to non-Euclidean computational spaces, we formulate Graph Manifold within a gyrovector algebraic framework and rigorously prove the intrinsic consistency of its geometric operations from a mathematical perspective. Finally, we apply Graph Manifold to the graph topology-oriented prompting task for pre-trained GNNs and conduct extensive experiments on four graph datasets under four pre-training strategies. Experimental results demonstrate that the proposed Graph Manifold consistently improves performance across multiple node classification tasks, validating its effectiveness as a novel Riemannian geometry for graph learning.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph Learning, Riemannian Geometry, Graph Distribution

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_n3WCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes Graph Manifold, a continuous representation of an undirected graph's adjacency structure based on a lower-triangular parameterization and Log-Cholesky geometry. It defines tangent-space Gaussian distributions to represent local topology variations, maps samples back to graph structures through the Riemannian exponential map, and derives gyrovector operations for the resulting space. The method is evaluated as a topology-oriented prompting and fine-tuning approach for pretrained GNNs on node classification.

Critical strengths

1.This paper's motivation for using Riemannian manifolds to model graph structures is reasonable, and it is supported by many related works. 2.The framework is presented coherently, from a lower-triangular graph parameterization to tangent-space distributions and sampling on the manifold. 3.The experimental results demonstrate the powerful performance of the proposed method.

Critical weaknesses

1.The writing of this paper is extremely poor. Firstly, the intro section is confusing. The distribution of the graph structure is regarded as the continuous representation of learning explicit graph structures. However, the purpose of this paper is graph structure learning, to find a better graph structure for downstream tasks. Additionally, Section 5 is very abrupt. The author throws out a large number of geometric manifold operators, but there is no any textual explanation to clarify their meanings and why to write this. If it were supplementary calculation explanations, they should have been placed in the appendix. Moreover, this seems more like a definition rather than a theorem. 2.The definition of the proposed space needs clarification. The paper states that off-diagonal entries of the lower-triangular factor lie in [0, 1], but the Cholesky manifold used for the geometric construction has unconstrained off-diagonal entries. The stated geodesic and exponential maps update those entries linearly, so they do not in general preserve the [0, 1] constraint. Please specify whether the constraint is actually enforced during optimization and sampling. 3.The empirical benefit of the manifold component is mixed: GraphManifold does not consistently outperform GraphEuclidean, whereas the distributional variant accounts for much of the gain. This weakens the evidence that Riemannian geometry itself is responsible for the improvements. 4.The author fails to provide a discussion on the learning modeling of graphs based on hyperbolic geometry and sphere geometry as the foundation of Riemannian geometry.

Questions for authors

See Weakness.

Reviewer_y3zJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Graph Manifold, a Riemannian framework for representing and optimizing graph topology. It maps graph adjacency matrices to a Cholesky-based manifold and models structural uncertainty through tangent-space Gaussian distributions. The paper also develops a gyrovector formulation with theoretical analysis. Experiments on four datasets and four pre-training strategies show improvements in few-shot node classification.

Critical strengths

S1. Interesting research perspective. The paper explores graph topology optimization from a Riemannian geometric perspective. Representing entire graph structures as manifold points provides an interesting alternative to conventional edge-level optimization.

S2. Extensive mathematical formulation. The paper provides formal definitions of manifold operations, including tangent spaces, Riemannian metrics, geodesics, and gyrovector operations. The mathematical analysis offers a structured foundation for the proposed framework.

S3. Competitive experimental performance. The proposed method achieves the best performance in 13 of 16 settings across four datasets and four pre-training strategies. The ablation experiments also demonstrate the benefits of incorporating distribution modeling.

Critical weaknesses

W1. Insufficient justification for the proposed manifold geometry.

The paper maps adjacency matrices to a Cholesky manifold through a lower-triangular representation. However, the motivation for choosing this specific geometry is not sufficiently justified. The mapping in Eq. (3) essentially reconstructs a symmetric adjacency matrix from its lower-triangular entries. It is unclear what intrinsic graph properties this geometry preserves beyond the original adjacency representation. The paper also lacks comparisons with alternative manifold geometries to demonstrate why the proposed choice is more suitable for graph learning.

W2. Limited evidence for the benefits of Riemannian geometry.

Table 2 shows that the manifold-only variant does not consistently outperform its Euclidean counterpart. For example, under GraphCL, GraphManifold achieves 59.96% on Cora, compared with 61.56% for GraphEuclidean. Similar performance drops appear under SimGRACE and LP-GraphPrompt. Most improvements come from adding distribution modeling rather than manifold geometry alone. These results weaken the central claim that the proposed Riemannian representation provides a clear advantage over Euclidean graph optimization.

W3. Insufficient validation of intrinsic graph distribution modeling.

Section 4.3 introduces Gaussian distributions in the tangent space to characterize structural uncertainty. However, the paper does not explain why Gaussian assumptions are appropriate for graph topology variations. The experiments mainly evaluate classification accuracy and do not directly assess the quality of the learned graph distributions or uncertainty estimates. It remains unclear whether the proposed distributions capture meaningful structural uncertainty or simply act as an additional regularization mechanism.

W4. Limited practical applicability and scalability analysis.

The proposed manifold represents an entire graph using an (n \times n) matrix, where (n) is the number of nodes. This raises concerns about memory consumption and computational scalability on large graphs. The paper does not report training time, memory usage, or scalability comparisons with existing graph prompting methods. Furthermore, the framework focuses on undirected graphs with a fixed number of nodes. Its applicability to directed graphs, dynamic graphs, and graphs with varying sizes remains unclear.

W5. Limited experimental scope and statistical evidence.

The experiments are restricted to four datasets and 5-shot node classification. Although the method is presented as a general framework for graph learning, its effectiveness on link prediction, graph classification, and other graph learning tasks is not evaluated. Furthermore, several improvements in Table 1 are small relative to the reported standard deviations. For example, under GraphCL, GraphManifold improves over GraphTOP by only 0.69 percentage points on Cora. Statistical significance tests are needed to establish whether these improvements are reliable.

Questions for authors

Q1. What intrinsic graph properties are preserved by the proposed Cholesky manifold? Why is this geometry preferable to other manifold representations?

Q2. Table 2 shows that GraphManifold sometimes underperforms GraphEuclidean. Can the authors explain when and why Riemannian geometry provides an advantage?

Q3. How is the Gaussian assumption for graph topology variations justified? Can the authors provide direct evidence that the learned distributions capture meaningful structural uncertainty?

Q4. What are the computational complexity, training time, and peak memory requirements of the proposed method compared with GraphTOP and other baselines?

Q5. Can the authors provide statistical significance tests for the improvements in Table 1? Are the reported gains consistent across different random seeds and data splits?

Reviewer_eAUmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Graph Manifold, a continuous geometric representation of undirected graph structures. A graph is encoded by a lower-triangular matrix LL with strictly positive diagonal, i.e., a point of the Cholesky manifold, whose strictly lower part stores edge weights in [0,1][0,1] and whose diagonal stores self-loop weights, and it is mapped to a symmetric adjacency matrix through Φ(L)=L+L⊤−diag(L)\Phi(L) = L + L^\top - \mathrm{diag}(L). The Log-Cholesky geometry is then transferred to the image An\mathcal{A}^n of this map, which yields closed forms for its tangent space, metric, geodesics, and exponential and logarithmic maps. Furthermore, the authors define an intrinsic graph distribution by defining a Gaussian on the tangent space through the exponential map, and introduce gyro addition, inverse, and scalar multiplication relative to a reference graph, proving that An\mathcal{A}^n can be seen as a gyrovector space. The framework is applied to topology-oriented prompting of pre-trained GNNs, where structural variations are sampled on 2-hop subgraphs during fine-tuning and predictions are averaged over samples at inference. Experiments on node classification over Cora, PubMed, Amazon, and Minesweeper, under four pre-training strategies, report the best average accuracy in most settings, and an ablation separates the contribution of the manifold from that of the distribution modeling.

Critical strengths

S1. The paper is well-motivated, mathematically and practically. The question raised in the introduction, i.e., whether one can build a continuous space of graph structures and characterize the distribution of topologies within it, is relevant for graph structure learning and for topology-oriented prompting, where the structure itself is the object being optimized. Treating the whole structure as a single point, with a metric, geodesics, and closed-form exponential and logarithmic maps, provides a clean language for structural updates and interpolation, and placing the uncertainty in the tangent space before mapping it back through the exponential map gives a principled way to sample structures and, subsequently, use them for some downstream task. The method can in principle be attached to any pre-trained GNN that takes a weighted adjacency matrix, and the experiments cover both contrastive and link-prediction pre-training, together with an ablation designed to separate the role of the geometry from that of the distribution modeling.

S2. Theory seems sound and is definitely interesting. I checked the closed forms of Propositions 1 to 5 and Lemmas 1 to 3 against the Log-Cholesky geometry of Lin (2019) and Chen et al.\textit{et al.} (2026), and they are correct, as is Theorem 1. Connecting graph structures with Riemannian and gyrovector machinery is also an interesting perspective, since the gyro operations give closed-form addition, inversion, and scaling of graphs relative to a reference graph, which could be useful, e.g., to compose structural prompts.

Critical weaknesses

W1. The text is really hard to read. It lacks specificity at some moments and consistency at others, it repeats excerpts throughout the main body (e.g., l.191-192 and 198-199, or the two consecutive sentences on graph structure learning in l.124-128), and the preliminaries are too short and not self-contained, i.e., readers have to resort to the appendix multiple times when reading and reasoning about the following sections. At the very least, Section 3 should give the minimum information needed to understand the Cholesky manifold and gyrovector spaces, and why they are needed here. For instance, An\mathcal{A}^n appears in Section 3.2 before being defined in Section 4.1, and the parallel transport used in Lemma 1 is never defined, its expression only appearing without derivation in Eq. (109). Regarding specificity, it is not clear what a "complete graph topology" (l.234-235) is, nor what the "global evolution patterns" of the abstract refer to. Regarding consistency, Section 4.1 introduces G=(V,E,A)G = (V, E, A) with a standard adjacency matrix, whose diagonal is zero, but the entries of LL are then constrained to [0,1][0,1] with a strictly positive diagonal (l.217-220), so weighted graphs with self-loops should be defined from the start. Some citations are also misplaced. The adjacency matrix of an undirected graph and its symmetry are standard and need neither a citation nor a dedicated sentence (l.199-201), the role of Boumal (2020) in l.244 is unclear, and Riemannian Adam is due to Bécigneul and Ganea (2019), not to Kingma and Ba (2014). There are also typos, e.g., "positived iagonal" (l.218) and an unbalanced parenthesis (l.223). Finally, the proof of Lemma 1 contains wrong intermediate steps, since Eq. (112) adds a diagonal term outside the exponential and Eq. (113) assigns the diagonal D(K)−D(LE)D(K) - D(L_E) to [K]−[LE][K] - [L_E], which is strictly lower triangular. Both errors disappear without justification in Eqs. (114) and (115), though the final expression is nevertheless correct.

W2. The Cholesky manifold and the map Φ\Phi restrict the graph topology to have self-loops, i.e., every point of An\mathcal{A}^n has a strictly positive diagonal, which might be an issue especially for heterophilic datasets such as Amazon and Minesweeper. That weakens the claims that Φ\Phi recovers the complete topology of the undirected graph (l.229-230) and that An\mathcal{A}^n is the space of graph topologies over nn nodes (l.234-235), and with them the method's flexibility and generality when compared to fixed graph-structure ones. Similarly, the constraint that the lower-triangular entries lie in [0,1][0,1] is not preserved by the operations of Sections 4 and 5. Geodesics and the exponential map, as well as the Gaussian sampling, move these entries along straight lines without bounds, and the gyro inverse of Lemma 2 has off-diagonal part 2[LE]−[L]2[L_E] - [L], which is negative wherever LL has an edge that the reference graph lacks. As a consequence, An\mathcal{A}^n is closed under these operations, and Theorem 1 holds, only if the constraint is dropped, in which case An\mathcal{A}^n contains matrices with negative edge weights. Furthermore, Propositions 1 to 5 follow from the Log-Cholesky geometry of Lin (2019) through a linear change of variables, and the gyro addition of Lemma 1 is the abelian group operation that Lin (2019) already defines on the Cholesky space, recentered at EE. Finally, the claim of being the first investigation of graph learning from the perspective of Riemannian geometry (l.93-94 and l.110) overlooks Riemannian GNNs, e.g., Chami et al. (2019) and Bachmann et al. (2020), as well as continuous spaces of graph structures such as the graphon space of Lovász and Szegedy (2006) – though this last one is not geometrically motivated.

W3. It is very difficult to understand how the proposed method is even trained. The main text never describes the procedure used in the experiments, and Appendix G.3 only states that 2-hop subgraphs are extracted, that a "posterior" with learnable mean and variance is placed on the tangent space, that several samples are mapped back through the exponential map during training, and that predictions are averaged over samples at inference. Do we need to compute tangent vectors or even construct the tangent space? How are the parameters μA\mu_{\mathcal{A}} and ΣA\Sigma_{\mathcal{A}} estimated? Appendix G.3 calls this distribution a posterior, but no prior, likelihood, or divergence term is ever written, so the training loss remains unknown. Many other choices are left unspecified, such as the parameterization of μA\mu_{\mathcal{A}} and ΣA\Sigma_{\mathcal{A}} (shared or per subgraph, full or diagonal), the number of samples, which entries of LL are learned (observed edges only or all node pairs of the subgraph), how LL is initialized from the observed graph, how the [0,1][0,1] constraint is enforced, and what GraphEuclidean is exactly. None of the gyro operations of Section 5 appears in this pipeline either, which leaves the need for that extension unexplained. What is the time complexity of the method compared with the baselines? If LL is dense, its number of entries grows quadratically with the size of each subgraph, which can be large around high-degree nodes, and the Monte Carlo averaging multiplies the inference cost by the number of samples, so wall-clock time and memory against the baselines should be reported.

W4. The reported improvement is also less significant than the authors claim it to be. Given the reported standard deviations over the five seeds, the margin over the strongest baseline in Table 1 is smaller than the standard deviation of GraphManifold⋆\text{GraphManifold}^\star in every setting except SimGRACE on Cora, and the margin is negative in three settings. In the ablation of Table 2, the full model is indeed better given some pre-training methods and for homophilic datasets (SimGRACE/Cora and LP-GPPT/PubMed), but there is no statistically significant improvement in the other scenarios. The manifold alone never differs significantly from GraphEuclidean and is even worse on Cora under all four pre-training strategies, which does not support the "clear advantages" claimed in the main body. Moreover, GraphEuclidean already outperforms the strongest baseline of Table 1 in several settings, so the gains over the baselines seem to come from the prompting pipeline and the sampling rather than from the Riemannian structure. The two controls that would settle this are missing, namely a Euclidean counterpart of the distribution component and a version without the Monte Carlo averaging at inference, which acts as a test-time ensemble that the baselines do not use.

Questions for authors

Q1. Is there any component of GraphManifold⋆\text{GraphManifold}^\star that would not be reproduced by a Gaussian perturbation of the edge weights and of the logarithms of the self-loop weights? Could the authors add to Table 2 a GraphEuclidean variant with the same distribution component (same parameterization, number of samples, and Monte Carlo averaging), as well as GraphManifold⋆\text{GraphManifold}^\star without the averaging at inference? These controls would show whether the gains come from the geometry, from the stochastic perturbations, or from test-time ensembling.

Q2. How does the method differ from existing work, especially from Lin (2019)? As noted in W2, Section 4.2 transports the Log-Cholesky geometry through the linear map Φ\Phi, and Lemma 1 recovers the group operation of Lin (2019). Moreover, building gyro operations from exponential and logarithmic maps and parallel transport is the recipe of Nguyen (2022), and the intrinsic graph distribution, i.e., a Gaussian on the tangent space pushed forward through the exponential map, follows the construction of Skopek et al. (2019). On the application side, Bayesian GCN (Zhang et al., 2019) already infers a posterior over graph structures and averages predictions over sampled graphs. Could the authors state which results in Sections 4 and 5 they regard as new, and what the Graph Manifold offers over these approaches?

References

  • Bécigneul, Gary and Octavian-Eugen Ganea. “Riemannian Adaptive Optimization Methods.” International Conference on Learning Representations (ICLR), 2019.

  • Chami et al. “Hyperbolic Graph Convolutional Neural Networks.” Advances in Neural Information Processing Systems (NeurIPS), 2019.

  • Bachmann et al. "Constant Curvature Graph Convolutional Networks". In Proceedings of the 37th International Conference on Machine Learning (ICML), 2020, Vol. 119. JMLR.org, Article 46, 486–496.

  • Lovász, L., & Szegedy, B. "Limits of dense graph sequences". Journal of Combinatorial Theory, 2006, Series B, 96(6), 933-957.

Comments on the paper

1 comment

Dear Reviewers,

Please review the authors' rebuttal and participate in the discussion to address any remaining questions before the period closes on Nov 18 AoE.

Thank you!

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.