acceptodds
Under review as a conference paper at ICLR 2027

Learning Propagation Geometry from Message-Passing Feedback

Abstract

Learning local geometry enables graph neural networks (GNNs) to adapt how they compare and integrate neighborhood information. However, estimating geometry from aggregated representations can overlook variation among individual messages and dependencies across feature dimensions. We propose GeoF, a recurrent framework that jointly evolves node features and propagation geometry through message-passing feedback. Each node maintains a local symmetric positive-definite geometry, initialized from a structure-aware prototype atlas and parameterized in block log-triangular coordinates. At each step, the geometry determines neighborhood weights, while triangular frame transport maps transformed source messages into the target node’s local coordinates before aggregation. Weighted second-order statistics of residuals between aligned messages and the transformed target state capture directional variation and within-block dependencies, yielding a geometric update target. A shared controller learns complementary corrections through task supervision. A bounded log-triangular update combines these corrections, the target, and the previous geometric state while preserving positive definiteness. The geometry governs subsequent propagation, closing the feedback loop. With parameters shared across recurrent steps, task-specific readouts support node classification, link prediction, and graph classification. Experiments on benchmark datasets show that GeoF consistently outperforms state-of-the-art GNN baselines.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Graph Neural Networks, Learnable Propagation Space, Riemannian Geometry

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ygaoOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Graph neural networks typically compare neighbouring messages under a shared Euclidean structure and exploit only first-order information, which cancels under averaging. This paper proposes GeoF, a recurrent framework in which each node's local propagation metric is modeled as a state that persists and evolves across recurrent steps, driven by the second moments of per-message residuals computed after frame alignment, thereby closing a feedback loop. GeoF consistently outperforms general, manifold-based, and adaptive GNN baselines on node classification, link prediction, and graph classification.

Critical strengths
  1. The core idea is well targeted, and the necessity of the second order is proved. The failure mode identified is concrete and verifiable: a mean-based metric estimator assigns identical geometry to two neighbourhoods with identical means but different joint variation. The proposed remedy — per-message residuals under the current geometry, accumulated into a weighted second-moment matrix — is a direct answer, and Proposition 2 formalises the hierarchy: first-order updates cannot distinguish these configurations, the second-order target can, while 2(iv) volunteers that the target remains weaker than an injective aggregator.

  2. The controls indicate that the gain comes from the mechanism rather than from extra information or capacity. Table 3 feeds the same second-order statistics to GCN/GIN/ARGNN as extra input features with parameters matched; the augmentation sometimes hurts, and GeoF still beats the best augmented variant. Table 4's mean-only → diagonal → full ladder instantiates the moment hierarchy of Proposition 2 directly.

Critical weaknesses
  1. Effect sizes are untested and capacity is uncontrolled. Several per-setting margins are at or below the baselines' own reported standard deviations, yet the paper reports only an aggregate Friedman/Holm analysis — no per-setting paired test and no confidence interval for the differences. Tables 1–2 also carry no parameter-count column, while GeoF adds per-node geometric coordinates, a controller and prototypes on top of the base model.

  2. The central claim lacks an ablation that isolates it, and incrementality is not argued. Contributions (1) and (2) rest on the persistent state and the closed loop, yet no ablation localises them: w/o GE removes persistence, adaptation and feedback simultaneously, and w/o SF replaces the second-order target with the current geometry rather than removing it. Meanwhile, per-node SPD metrics have precedent in ARGNN, the triangular/Cholesky parameterisation comes from Lin (2019), and SPDGNN — which uses essentially the same SPD-via-Cholesky coordinates and is itself a baseline — is never discussed in Section 2.

Questions for authors
  1. (Cholesky convention). The paper is internally consistent: R is always lower triangular with R R^T = C + ηI, and ẑ is built from R. Under the g = L^T L readout, however, the metric actually implemented is R^T R = R^{-1}(C+ηI)R, which shares the spectrum of the regularised residual second moment but not its principal axes. Building the target from R^{-1} would normalise away the directions of high residual energy (whitening), whereas the current convention amplifies them; the two differ only when the residuals carry cross-dimensional coupling. Is this convention a deliberate choice, or simply inherited from the standard Cholesky factorisation? If the latter, the reading of Section 3.3 as "the target is the residual second moment" would need to be qualified, which affects my assessment of correctness.

  2. (Per-setting significance and parameter counts). Please report paired tests for each dataset–task setting (paired t / Wilcoxon over the 10 splits or 10 folds) together with 95% confidence intervals of the differences — especially for the settings whose margins are at or below the reported standard deviations — and add a parameter-count column to Tables 1–2. These two items directly determine the outcome of CW1 and are prerequisites for maintaining my score of 3.

  3. (Effective number of prototypes). The paper reports that accuracy is nearly flat over a wide range of K, which is compatible both with genuine insensitivity to atlas capacity and with a collapse in which only one prototype is effectively used. Please report the distribution of the effective number of prototypes per node, exp(H(α_i)) with H the Shannon entropy of α_i, together with the fraction of nodes whose α_i is dominated by a single prototype (e.g. max_k α_ik > 0.9). If the effective count is close to 1, the "structure-conditioned routing" component is not functioning as described and its design would need to be re-interpreted.

6 comments
AuthorsOpenReview ↗

Response to Reviewer ygao (1/4)

We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your comments in the following:

W1: Effect sizes are untested and capacity is uncontrolled. Several per-setting margins are at or below the baselines' own reported standard deviations, yet the paper reports only an aggregate Friedman/Holm analysis — no per-setting paired test and no confidence interval for the differences. Tables 1–2 also carry no parameter-count column, while GeoF adds per-node geometric coordinates, a controller and prototypes on top of the base model.

A1: Thanks for your comments. We have replaced the aggregate Friedman and Holm analysis in Appendix K.2 with per-setting paired tests and parameter counts for all 18 settings, as shown in the table below. For each setting, GeoF is compared with the strongest baseline on the same splits or folds, and we report the mean gain, the 95% confidence interval of the paired difference, the p-values ptp_t of the paired t-test and pWp_{\mathrm{W}} of the Wilcoxon signed-rank test, and the number of parameters. Node classification and link prediction use 10 random splits, and graph classification uses 10-fold cross-validation.

TaskDatasetBest baselineGain95% CIptp_tpWp_{\mathrm{W}}#Params (GeoF)#Params (Baseline)
NCCiteSeerARGNN+2.1[+0.94, +3.09]2.2×10−32.2\times10^{-3}<0.002572K648K
NCPubMedAMPs+0.7[+0.46, +0.96]1.3×10−41.3\times10^{-4}<0.002162K97K
NCCSAMPs+1.3[+0.97, +1.55]4.5×10−64.5\times10^{-6}<0.002194K209K
NCPhysicsARGNN+0.9[+0.78, +0.93]7.6×10−107.6\times10^{-10}<0.0021.18M1.25M
NCPhotoARGNN+1.1[+0.65, +1.47]2.3×10−42.3\times10^{-4}<0.002170K269K
NCComputersARGNN+0.8[+0.24, +1.28]9.0×10−39.0\times10^{-3}0.020197K272K
LPCiteSeerARGNN+0.8[+0.13, +1.07]0.0180.020589K681K
LPPubMedARGNN+0.9[+0.52, +1.17]2.2×10−42.2\times10^{-4}<0.002179K271K
LPCSBEC-GNN+0.5[+0.35, +0.68]7.3×10−57.3\times10^{-5}<0.002986K2.28M
LPPhysicsG²Former+0.5[+0.15, +0.94]0.0120.0131.19M2.22M
LPPhotoG²Former+0.6[+0.17, +0.72]5.5×10−35.5\times10^{-3}0.014210K258K
LPComputersWaveGC+0.5[+0.16, +0.92]0.0110.101213K560K
GCPROTEINSWaveGC+0.4[−1.14, +2.04]0.540.23109K438K
GCMutag.SPARROW+1.2[−0.39, +2.93]0.120.20111K179K
GCFRANK.WaveGC+0.7[−1.07, +2.41]0.410.23109K437K
GCNCI1G²Former+0.9[−0.40, +2.20]0.150.15114K110K
GCBBBPWaveGC+0.8[−0.49, +2.00]0.210.38107K438K
GCogbg-molhivWaveGC+0.9[−0.73, +2.40]0.260.27145K476K
AuthorsOpenReview ↗

Response to Reviewer ygao (2/4)

On node classification and link prediction, the confidence intervals of the paired differences exclude zero in all 12 settings, and the paired t-test gives p<0.05p<0.05 in all 12 settings. The Wilcoxon signed-rank test agrees in 11 of them; the remaining case is link prediction on Computers (pW=0.101p_{\mathrm{W}}=0.101), where the confidence interval still excludes zero and the paired t-test is significant (pt=0.011p_t=0.011). This holds even where the margin is close to the standard deviations reported for the baselines, because pairing on the same splits removes the variation shared by both methods, and GeoF improves on the strongest baseline consistently across splits rather than through a few favorable runs. On graph classification, GeoF improves on the strongest baseline on all six datasets, while the fold-level intervals include zero since the variance across folds on these datasets exceeds the gains. Appendix K.2 now describes these gains as consistent across datasets rather than individually significant.

Regarding capacity, GeoF uses fewer parameters than the strongest baseline in 16 of the 18 settings, often by a large margin on link prediction and graph classification. The two exceptions are node classification on PubMed and graph classification on NCI1, where the strongest baselines, AMPs and G²Former, are compact models. Capacity alone does not explain the gains in these cases either: in Section 4.6, when GCN, GIN, and ARGNN are given the same second-order statistics with parameter counts matched to GeoF within 0.5%, GeoF still outperforms them in every setting. The node-wise geometric coordinates are states computed from the shared prototype atlas and structural signatures and updated during propagation, so they are not learnable parameters and do not grow with the number of nodes. The learnable components added by GeoF are the shared controller and a small set of prototypes, and all recurrent parameters are shared across steps. The improvements therefore come from how GeoF learns propagation geometry rather than from a larger model.

W2: The central claim lacks an ablation that isolates it, and incrementality is not argued. Contributions (1) and (2) rest on the persistent state and the closed loop, yet no ablation localises them: w/o GE removes persistence, adaptation and feedback simultaneously, and w/o SF replaces the second-order target with the current geometry rather than removing it. Meanwhile, per-node SPD metrics have precedent in ARGNN, the triangular/Cholesky parameterisation comes from Lin (2019), and SPDGNN — which uses essentially the same SPD-via-Cholesky coordinates and is itself a baseline — is never discussed in Section 2.

A2: Thanks for your comments. To isolate the first two contributions, we added two variants that each change a single property while keeping the rest of the model unchanged, and report them together with the existing One-shot variant in Table 5 of Section 4.6. No persistence sets λ=1\lambda=1 in the geometry update (Eq. 12), so each step rebuilds the geometry from the current target and correction and discards the previous state, while feedback and adaptation remain. Fixed-frame computes the residual statistics under the initial geometry while propagation still uses the current geometry, so the feedback no longer follows the geometry it updates, while persistence remains. One-shot updates the geometry only once after the first step.

VariantPhotoCSMutagNCI1
One-shot94.6±0.594.8±0.383.1±1.879.8±2.7
No persistence95.0±0.595.2±0.384.1±1.482.3±2.1
Fixed-frame95.4±0.395.6±0.384.1±1.782.6±1.8
Full96.0±0.495.8±0.384.9±1.783.0±1.5

All three variants lower accuracy on all four datasets. Removing persistence causes a larger drop than fixing the feedback frame on average, and updating the geometry only once causes the largest drop on every dataset, especially on graph classification. Carrying the geometry across steps, updating it repeatedly, and computing feedback under the current geometry each contribute to performance, which localizes the persistent state and the closed loop beyond the joint effect measured by w/o GE. Regarding w/o SF, replacing the target with the current geometry reduces the update to the previous geometry plus the correction, so the second-order target is removed entirely while the correction is kept. We have clarified this in Section 4.3.

AuthorsOpenReview ↗

Response to Reviewer ygao (3/4)

Regarding incrementality, we have revised Section 2 to discuss SPDGNN, ARGNN, and the log-Cholesky parameterization of Lin [1]. GeoF uses this parameterization as a tool to keep the geometry positive definite during updates. SPDGNN embeds representations on the SPD manifold through Cholesky coordinates, while in GeoF the SPD geometry is a node-wise state that defines the frames used for weighting and transporting messages. ARGNN estimates node-wise metrics from features and neighborhood means at each layer, while GeoF carries the geometry as a persistent state and updates it from second-order statistics of residuals aligned under the current geometry. The No persistence variant is the closest to per-layer re-estimation, and its consistent drop shows that persistence brings improvements beyond node-wise metrics alone.

Q1: (Cholesky convention). The paper is internally consistent: RR is always lower triangular with RR⊤=C+IRR^\top = C + I, and z^\hat z is built from RR. Under the g=L⊤Lg = L^\top L readout, however, the metric actually implemented is R⊤R=R−1(C+I)RR^\top R = R^{-1}(C+I)R, which shares the spectrum of the regularised residual second moment but not its principal axes. Building the target from R−1R^{-1} would normalise away the directions of high residual energy (whitening), whereas the current convention amplifies them; the two differ only when the residuals carry cross-dimensional coupling. Is this convention a deliberate choice, or simply inherited from the standard Cholesky factorisation? If the latter, the reading of Section 3.3 as "the target is the residual second moment" would need to be qualified, which affects my assessment of correctness.

A3: Thanks for your comments. The convention is defined at the level of the frame rather than the metric. In GeoF, both the neighborhood weights and the message transport are computed through the frames themselves, via hi=Liξih_i=L_i\xi_i and Li−1LjL_i^{-1}L_j, so the object that acts on messages is LL and not g=L⊤Lg=L^\top L. The target factor RR satisfies RR⊤=C+ηIRR^\top=C+\eta I and maps the local unit ball onto the residual second-moment ellipsoid, so the target frame is aligned with the principal axes of the residual second moment and stretches high-energy residual directions. The induced metric R⊤RR^\top R shares the spectrum of C+ηIC+\eta I but not its principal axes, as you point out. We have revised Section 3.3 and Appendix F to state that the target aligns the frame with the regularized residual second moment and that the induced metric shares its spectrum, rather than stating that the metric equals the residual second moment.

To examine whether this choice matters, we compare three conventions on Photo, CS, Mutag, and NCI1 with the rest of the model unchanged. Whitened builds the target from R−1R^{-1} and normalizes away high-energy directions. Aligned builds the target so that the metric itself satisfies L⊤L=C+ηIL^\top L=C+\eta I and therefore shares the principal axes of the residual second moment. Ours is the convention used in the paper.

ConventionPhotoCSMutagNCI1
Whitened93.9±0.594.3±0.580.1±2.377.5±1.6
Aligned95.7±0.495.5±0.384.3±1.782.0±1.3
Ours96.0±0.495.8±0.384.9±1.783.0±1.5

Whitening consistently degrades performance on all four datasets, with the largest drops on graph classification, which shows that emphasizing rather than normalizing high-energy residual directions is essential to GeoF. Aligning the metric axes with the residual second moment performs close to our convention and slightly below it, consistent with your remark that the two differ only under cross-dimensional coupling. Together, these results indicate that the direction of the effect, amplification versus whitening, drives the benefit of second-order feedback, and that defining the target at the frame level, where it acts on messages, is a sound choice. We have added this comparison to Table 5 in Section 4.6.

Q2: (Per-setting significance and parameter counts). Please report paired tests for each dataset–task setting (paired t / Wilcoxon over the 10 splits or 10 folds) together with 95% confidence intervals of the differences — especially for the settings whose margins are at or below the reported standard deviations — and add a parameter-count column to Tables 1–2. These two items directly determine the outcome of CW1 and are prerequisites for maintaining my score of 3.

A4: Thanks for your comments. Please refer to our response to W1 (A1), which reports the per-setting paired tests, confidence intervals, and parameter counts against the strongest baseline for all 18 dataset–task settings.

Reviewer_3CJFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors proposed GeoF, utilizing message-passing feedback to learn a node-wise geometry in the feature representation space for node classification, graph classification, and link prediction. This node feature geometry is supervised directly by the task. The claimed contributions are that this node feature geometry is an information source for local geometry learning and that GeoF outperforms some baselines on 12 datasets.

Critical strengths
  1. GeoF is a fundamental method for node feature representation learning, so it has a wide range of downstream applications. There are 12 datasets, including one large-scale graph classification benchmark, that are tested in the paper, although only 4 datasets are considered in the ablation and analysis studies.
Critical weaknesses
  1. The presentation of the proposed methods uses uncommon terms. While some terms are illustrated with references in the literature so they can be mapped to common terms, some important terms were never defined or referenced. For example, "a common coordinate reference" in L.176 actually refers to the "common space of feature representation," according to the GCN and GAT references. Those terms frame the proposed methods but are undefined, making the design of GeoF difficult to follow, such as "feedback", "propagation geometry", "shared environment", and "TriSolve" in Sections 3.1 and 3.2. Meanwhile, this problem also happens in experimental results, such as "Evolution Depth" and "recurrent depth".

  2. The novelty of GeoF is not clear. Given those clearly and rigorously defined parts, GeoF is relevant to graph re-wiring methods according to Eq. 4, which shows how GeoF updates graph edges. The authors overlooked existing and relevant methods and the differences from them. Despite the section on related works in the paper illustrating GNN adaptation and geometric GNNs, the authors failed to position GeoF by showing the differences from the related works.

  3. The contribution of GeoF is not demonstrated soundly. Experiments lack computational complexity analysis, interpretation of GeoF performance, and dynamic analysis of the proposed geometric state evolution of GeoF. Hyperparameter settings are not present.

Questions for authors

None

5 comments
AuthorsOpenReview ↗

Response to Reviewer 3CJF

We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your comments in the following:

W1: The presentation of the proposed methods uses uncommon terms. While some terms are illustrated with references in the literature so they can be mapped to common terms, some important terms were never defined or referenced. For example, "a common coordinate reference" in L.176 actually refers to the "common space of feature representation," according to the GCN and GAT references. Those terms frame the proposed methods but are undefined, making the design of GeoF difficult to follow, such as "feedback", "propagation geometry", "shared environment", and "TriSolve" in Sections 3.1 and 3.2. Meanwhile, this problem also happens in experimental results, such as "Evolution Depth" and "recurrent depth".

A1: Thanks for your comments. We have revised our manuscript so that the key terms are defined at their first use and terms that previously referred to the same quantity are unified. The revised terms are summarized in the table below.

TermDefinition in the revised manuscriptLocation
Propagation geometryThe local metrics and coordinate mappings that govern how each node compares and transforms neighboring messages.Sec. 1
Message-passing feedbackThe signal computed from incoming messages under the current geometry, which provides information for subsequent geometric updates.Sec. 1
Shared spaceThe common feature space into which each frame maps the local coordinates of a node, and in which messages and representations of different nodes are compared and aggregated. It replaces "a common coordinate reference" and all variants of "environment".Sec. 3.1, Sec. 3.2
Triangular solveThe transported message is written explicitly as (Li,b)−1Lj,b(ϕ(ξj))b(L_{i,b})^{-1}L_{j,b}(\phi(\xi_j))_b and computed by forward substitution. It replaces "TriSolve".Sec. 3.2
Recurrent stepsThe number LL of recurrent updates of features and geometry. It replaces "Evolution Depth" and "recurrent depth".Sec. 3, Sec. 4.4

W2: The novelty of GeoF is not clear. Given those clearly and rigorously defined parts, GeoF is relevant to graph re-wiring methods according to Eq. 4, which shows how GeoF updates graph edges. The authors overlooked existing and relevant methods and the differences from them. Despite the section on related works in the paper illustrating GNN adaptation and geometric GNNs, the authors failed to position GeoF by showing the differences from the related works.

A2: Thanks for your comments. We would like to clarify the role of Eq. 4. The weights ωijl\omega_{ij}^l are computed by a softmax over the existing neighborhood N~(i)\widetilde{\mathcal{N}}(i), so they reweight the messages from the observed neighbors of node ii without adding or removing any edge, and the graph topology remains fixed throughout propagation. In this respect, Eq. 4 plays a role similar to the attention weights in GAT rather than to graph rewiring. The difference from attention is that the weights are determined by the current node-wise geometry, and the same geometry also aligns the messages before aggregation and is updated from them at the next step. We have made this explicit after Eq. 4 in Section 3.2 of our manuscript.

To clarify the positioning of GeoF, we summarize how representative lines of work adapt message passing in the table below, including graph rewiring and existing geometric GNNs.

Line of workWhat is adaptedGraph topologyHow the adapted object is obtained
Graph rewiring, e.g., SDRF, FoSR, DIGLEdgesModifiedFrom curvature, spectral, or diffusion properties of the graph
Graph structure learning, e.g., IDGL, Pro-GNNAdjacency matrixModifiedJointly with node representations or for robustness
Attention, e.g., GATEdge weightsFixedFrom the features at each layer
Fixed-manifold GNNs, e.g., HGCN, SPDGNNRepresentation spaceFixedPrescribed by the architecture
Adaptive geometric GNNs, e.g., ACE-HGNN, ARGNNCurvature or node-wise metricFixedLearned curvature, or metrics estimated from features and neighborhood means
Sheaf diffusion, e.g., NSDEdge-wise restriction mapsFixedFrom the features at each layer
GeoFNode-wise geometryFixedPersistent state updated from second-order statistics of aligned message residuals and task-guided corrections

As shown in the above table, graph rewiring and structure learning change which nodes exchange messages, whereas GeoF keeps the topology fixed and adapts how messages are compared and transformed along the existing edges. Compared with geometric GNNs, GeoF differs in two respects. First, the geometry is a persistent state that is carried across recurrent steps, rather than being prescribed or re-estimated from the current features. Second, the geometry is updated from the second-order statistics of message residuals aligned under the current geometry. These statistics capture directional variation that neighborhood means conceal, and the updated geometry in turn governs the next propagation step, forming a closed loop between propagation and geometry. We have revised Section 2 of our manuscript to include graph rewiring methods with their references and to state how GeoF differs from rewiring and geometric GNNs.

W3: The contribution of GeoF is not demonstrated soundly. Experiments lack computational complexity analysis, interpretation of GeoF performance, and dynamic analysis of the proposed geometric state evolution of GeoF. Hyperparameter settings are not present.

A3: Thanks for your comments. Our manuscript provides the computational complexity analysis, the interpretation of GeoF performance, the dynamic analysis of the geometric state evolution, and the hyperparameter settings. We detail each of them below.

1. Computational complexity analysis. Appendix J shows that the cost of GeoF is linear in the numbers of nodes and edges, since the block-diagonal parameterization reduces the node-wise geometric operations from cubic to linear in the hidden dimension. Appendix K.5 reports per-epoch training time and GPU memory. GeoF trains faster than ARGNN on node classification, with the advantage growing on larger graphs, and its overhead relative to ARGNN remains a moderate constant factor across datasets.

2. Interpretation of GeoF performance. Section 4.6 gives GCN, GIN, and ARGNN the same second-order residual statistics as additional features with matched parameter counts. The augmentation has mixed effects on these baselines, and GeoF exceeds the strongest augmented variant in every setting, which shows that the gains come from using these statistics as feedback that updates the propagation geometry rather than from the statistics alone. Table 5 further shows that replacing the full second-order feedback with mean-only or diagonal statistics, or with one-shot feedback computed only once, consistently degrades performance on all four datasets. This indicates that the gains come from the directional variation and within-block dependencies captured by second-order feedback, and from refining the geometry across recurrent steps. The ablations in Section 4.3 and Appendix K.3 show that removing any component reduces performance, and the visualizations in Appendix L show more compact class clusters under GeoF and under the learned geometric correction.

3. Dynamic analysis of the geometric state evolution. Section 4.5 unrolls trained models to 20 recurrent steps and shows that the state change decays quickly and remains bounded, with one-step sensitivity controlled by the regularization and coordinate bounds and stable across node degrees. Section 4.4 additionally tracks the change of the geometry over recurrent steps, which shrinks as the number of steps increases.

4. Hyperparameter settings. Appendix H.2 lists the hyperparameter settings of GeoF, including the learning rate, weight decay, number of geometric blocks and prototypes, number of recurrent steps, correction rank, coordinate bounds, and update coefficients.

Reviewer_1AkkOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GeoF, a graph neural network framework that jointly evolves node features and local geometry at each node through message-passing feedback. In their method, each node keeps a local symmetric positive-definite geometry, parameterized in block log-triangular coordinates. At each step, the geometry sets the neighborhood weights, and triangular frame transport converts each source message into the target node's local coordinates before aggregation. Experiments on node classification, graph classification, and link prediction show nice improvements of the proposed framework.

Critical strengths
  1. Neighbourhoods with identical feature means can have different second-order structure. Using this information to update subsequent propagation is a nice way to improve the geometric learning aspect of graphs.

  2. The experiments are broad, and the paper has necessary control experiments. Further, the baseline selection is good, even though some important baselines such as sheaf GNNs are missing.

  3. The theory is not very in-depth or surprising, yet they are correctly written and adds some value to the narrative.

Critical weaknesses
  1. The authors should include Neural Sheaf Diffusion as a baseline, as it also learns local maps to transform messages between node spaces. This comparison would help establish whether GeoF’s second-order feedback offers benefits beyond learned message transformations.

Bodnar, Cristian, et al. "Neural sheaf diffusion: A topological perspective on heterophily and oversmoothing in gnns." Advances in Neural Information Processing Systems 35 (2022): 18527-18541.

  1. The convergence results assume fixed node features, whereas the proposed method updates both features and geometry during propagation. So, there is a theory-empirical gap, and it's not clear how this impacts the claims of the paper. The guarantees also require stability or contraction conditions that are not verified experimentally.

  2. The node-classification datasets are mainly homophilic. Evaluation on heterophilic graphs is needed to understand whether the learned geometry remains useful when neighbouring nodes have different labels. Authors can use Platanov heterophilic datasets for this.

Platonov, Oleg, et al. "A critical look at the evaluation of GNNs under heterophily: Are we really making progress?." arXiv preprint arXiv:2302.11640 (2023).

  1. The largest single graph tested has only around 34k nodes. The authors should discuss scalability to substantially larger networks, such as ogbn-arxiv and ogbn-products. I don't think adding large OGB graph-classification datasets would help address this concern, as their individual graphs remain small.
Questions for authors

The idea introduced in this paper is somewhat novel and interesting, but its advantages and practical scope are not yet sufficiently established. Currently, I consider this paper borderline. Addressing the weaknesses would strengthen my assessment.

7 comments
AuthorsOpenReview ↗

Response to Reviewer 1Akk

We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your comments in the following:

W1: The authors should include Neural Sheaf Diffusion as a baseline, as it also learns local maps to transform messages between node spaces. This comparison would help establish whether GeoF’s second-order feedback offers benefits beyond learned message transformations.

A1: Thanks for your comments. To examine whether the second-order feedback of GeoF offers benefits beyond learned message transformations, we include Neural Sheaf Diffusion (NSD) as an additional baseline [1]. Like GeoF, NSD learns local maps that transform messages between node spaces, but these maps are recomputed at each layer from the current node features, whereas GeoF maintains a persistent node-wise geometry that is updated by second-order statistics of aligned residuals. We conduct experiments on all task settings under the same experimental settings.

Node classification

MethodCiteSeerPubMedCSPhysicsPhotoComputers
NSD73.5±2.087.3±0.494.1±0.2OOM94.8±0.390.1±0.7
GeoF77.7±1.289.6±0.395.8±0.397.2±0.196.0±0.491.2±0.5

Link prediction

MethodCiteSeerPubMedCSPhysicsPhotoComputers
NSD92.2±0.996.1±2.296.4±0.2OOM96.3±0.495.9±0.9
GeoF94.9±0.898.5±0.297.7±0.397.9±0.398.4±0.298.0±0.4

Graph classification

MethodPROTEINSMutag.NCI1FRANK.BBBPmolhiv
NSD77.2±2.081.1±2.580.3±1.970.8±2.390.5±2.080.8±1.5
GeoF79.4±1.684.9±1.783.0±1.573.3±1.893.4±1.882.9±1.1

As shown in the above tables, GeoF consistently outperforms NSD across all three tasks, while NSD runs out of memory (OOM) on Physics. This indicates that learned message transformations alone do not explain the gains of GeoF; the persistent geometry updated by second-order residual feedback provides additional benefits. We have added NSD to Tables 1 and 2 of our manuscript.

W2: The convergence results assume fixed node features, whereas the proposed method updates both features and geometry during propagation. So, there is a theory-empirical gap, and it's not clear how this impacts the claims of the paper. The guarantees also require stability or contraction conditions that are not verified experimentally.

A2: Thanks for your comments. We have rewritten Proposition 3 around the properties that GeoF relies on as a finite unrolled recursion and revised the proof in Appendix E of our manuscript. The new Proposition 3 holds for the joint update of features and geometry and establishes three properties. First, each recurrent step is well-posed and maps the bounded state domain into itself, with all Cholesky factors, triangular solves and log-diagonal coordinates well defined. Second, the geometric update is a projected proximal step whose deviation from the residual-driven update is bounded by the learned correction. Third, the one-step map is Lipschitz with a constant that is independent of node degrees and is governed explicitly by the regularization ϵs\epsilon_s and the coordinate bounds.

To verify these properties empirically, we unroll trained models on the Computers dataset to 20 recurrent steps, far beyond the two steps used in training, and estimate the spectral radius of the one-step Jacobian at the final state under our default setting (Appendix H.2) and under a looser setting with smaller ϵs\epsilon_s and wider coordinate bounds.

SettingFinite rolloutsRel. increment (step 20)Spectral radiusACC (%)
Loose (ϵs=10−5\epsilon_s=10^{-5}, ℓ∈[−5,10]\ell\in[-5,10], amax⁡=10a_{\max}=10)10/100.156690.9±0.5
Default (ϵs=10−2\epsilon_s=10^{-2}, ℓ∈[−5,5]\ell\in[-5,5], amax⁡=2a_{\max}=2)10/100.067.591.2±0.5

As shown in the above table, all rollouts remain finite far beyond the training depth with bounded state increments under both settings, confirming the first property. Compared with the loose setting, the default setting, with larger ϵs\epsilon_s and tighter bounds, reduces the spectral radius by an order of magnitude and further damps the late-step increments without affecting accuracy, in line with the dependence of the constant in the third property on these quantities. Additionally, we add an analysis of the coupled dynamics in Section 4.5 of our manuscript. As shown in Figure 2 of our manuscript, the one-step gain does not grow with node degree, which supports the degree-independent constant in the third property.

W3: The node-classification datasets are mainly homophilic. Evaluation on heterophilic graphs is needed to understand whether the learned geometry remains useful when neighbouring nodes have different labels. Authors can use Platanov heterophilic datasets for this.

A3: To examine whether the learned geometry remains useful when neighboring nodes have different labels, we conduct node classification experiments on four widely used heterophilic benchmarks, namely Actor, Chameleon, Cornell and Squirrel. We compare GeoF with GIN [2], WaveGC [3], SPDGNN [4] and ARGNN [5] under the same splits and evaluation protocol.

MethodActorChameleonCornellSquirrel
GIN25.62±0.9842.79±1.9144.74±1.7626.14±1.36
WaveGC28.90±0.8850.61±1.7364.74±2.2033.01±1.67
SPDGNN29.52±1.0051.73±1.9053.95±2.0940.12±1.40
ARGNN37.61±0.9053.03±2.3867.89±3.1430.07±1.95
GeoF37.19±1.1455.92±2.4469.74±2.6040.78±1.01

As shown in the above table, GeoF achieves the best accuracy on Chameleon, Cornell and Squirrel and remains on par with the strongest baseline on Actor. This indicates that the geometry learned from second-order residual feedback remains useful under heterophily, where neighboring messages differ more and their residual statistics carry richer information about local variation. We have added these results to Appendix K.1 of our manuscript.

W4: The largest single graph tested has only around 34k nodes. The authors should discuss scalability to substantially larger networks, such as ogbn-arxiv and ogbn-products. I don't think adding large OGB graph-classification datasets would help address this concern, as their individual graphs remain small.

A4: Thanks for your suggestion. We evaluated GeoF on ogbn-arxiv, which has about 169K nodes and 1.17M edges and is roughly five times larger than the largest graph in our original experiments. We use the same 60/20/20 random split and protocol as for the other node classification datasets, and compare with representative general, manifold, and adaptive baselines.

Methodogbn-arxiv
GIN56.8±1.3
WaveGC70.3±0.6
SPDGNN66.6±0.7
ARGNN66.1±0.2
GeoF71.3±0.3

GeoF achieves the best accuracy on ogbn-arxiv, and its advantage over the geometric baselines SPDGNN and ARGNN is larger than on the smaller graphs, which indicates that learning propagation geometry from message-passing feedback remains effective at this scale. We also attempted ogbn-products with about 2.4M nodes and 61.9M edges. Under full-batch training, GeoF and all geometric baselines exceed the memory of our GPU, and only GIN can be trained.

Regarding scalability beyond this size, the cost of GeoF is linear in the numbers of nodes and edges, as analyzed in Appendix J, and all of its operations are local. Neighborhood weights and frame transport are computed per edge, and residual statistics and geometry updates are computed per node from its neighbors. GeoF is therefore compatible with standard neighbor sampling and subgraph mini-batch training, which removes the full-batch memory bottleneck on graphs such as ogbn-products. We have added the ogbn-arxiv results to Appendix K.1, together with the heterophilic results.


[1] Bodnar C, Di Giovanni F, Chamberlain B, et al. Neural sheaf diffusion: A topological perspective on heterophily and oversmoothing in gnns[J]. NIPS 2022.

[2] Xu K, Hu W, Leskovec J, et al. How powerful are graph neural networks?[C]. ICLR 2019.

[3] Liu N, He X, Laurent T, et al. A general graph spectral wavelet convolution via chebyshev order decomposition[C]. ICML 2025.

[4] Wang Y, Chang Y. Enhancing graph neural networks on SPD manifolds via cholesky decomposition[J]. Pattern Recognition 2025.

[5] Wang X, Ding C, Li T, et al. Adaptive riemannian graph neural networks[C]. AAAI 2026.

I thank the authors for the detailed response. The revision improves the paper.

Resolved: NSD has been added to the main tables across all tasks. The revised stability proposition now covers the joint update of features and geometry, which resolves the mismatch between theory and method that I raised, and the new analysis of the coupled dynamics supports it empirically.

I want to provide the remaining issues below.

Theory: The revised proposition establishes well-posedness and a Lipschitz bound, not convergence, and the reported spectral radius shows that the map is not contractive. This is fine. But please state this limitation explicitly in the paper.

Heterophily: The new results use Chameleon, Squirrel and Cornell, whose flaws motivated my request for the Platonov datasets. As shown by Platonov et al., Chameleon and Squirrel contain a large number of duplicate nodes that appear in both the training and test sets, which causes data leakage and inflates the reported accuracy. Cornell is very small, so its results have high variance and are not statistically reliable. In addition, the baselines used are not designed for heterophily, so they do not provide a meaningful comparison in this setting. Please report results on the Platonov datasets against heterophily specific baselines, such as NSD, GPRGNN, ACM-GCN and GloGNN.

Scalability: ogbn-arxiv experiment is acceptable. However, please report training time and GPU memory comparison with some strong baseline, such as GCN, on this dataset. GIN is also not a strong baseline for node classification, so please compare with stronger ones, such as well tuned GCN, GraphSAGE and GAT.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revision

We sincerely thank all reviewers for their constructive comments. We have revised the manuscript accordingly, and all changes are highlighted in blue. The main revisions are summarized below.

1. Baselines and benchmarks

  • Added Neural Sheaf Diffusion (NSD) as a baseline in Tables 1 and 2. (Reviewer 1Akk)
  • Added results on four heterophilic benchmarks (Actor, Chameleon, Cornell, Squirrel) and on ogbn-arxiv in Appendix K.1. (Reviewer 1Akk)

2. Theory and dynamics

  • Rewrote Proposition 3 for the joint update of features and geometry, with the revised proof in Appendix E. (Reviewer 1Akk)
  • Added an analysis of the coupled dynamics in Section 4.5 and Figure 2. (Reviewers 1Akk, 3CJF)
  • Added the default value of ϵs\epsilon_s to Appendix H.2.

3. Statistical tests and parameter counts

  • Replaced the Friedman and Holm analysis with per-setting paired t-tests, Wilcoxon signed-rank tests, 95% confidence intervals and parameter counts for all 18 settings in Appendix K.2 (Table 8). (Reviewer ygao)

4. Ablations and analyses

  • Added the No persistence and Fixed-frame variants to Table 5 in Section 4.6. (Reviewer ygao)
  • Added the Whitened and Aligned target conventions to Table 5, and revised the description of the target in Section 3.3 and Appendix F. (Reviewer ygao)
  • Added prototype usage statistics to Table 3 in Section 4.4. (Reviewer ygao)
  • Revised the description of the w/o SF variant in Section 4.3. (Reviewer ygao)

5. Presentation and related work

  • Defined key terms at first use and unified terminology across the manuscript. (Reviewer 3CJF)
  • Added a note after Eq. 4 in Section 3.2 that the graph topology is fixed. (Reviewer 3CJF)
  • Revised Section 2 to include graph rewiring methods, SPDGNN, ARGNN and the log-Cholesky parameterization. (Reviewers 3CJF, ygao)
  • Revised the description of the prototype atlas in the abstract, the introduction and Section 3.1. (Reviewer ygao)

We hope these revisions address the reviewers' concerns, and we are happy to provide further clarification during the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.