acceptodds
Under review as a conference paper at ICLR 2027

OCG: Orthogonal Consensus Graph Learning for Robust Heterogeneous Graph Representations

Abstract

Heterogeneous graph representation learning uses typed relations and node attributes, both of which can be corrupted. We propose Orthogonal Consensus Graph Learning (OCG), which augments contrastive alignment between matched nodes with a residual cross-Gram penalty and stochastic-path predictive consistency. We derive an exact decomposition of the residual penalty into a spectral shrinkage term for the semantic residual and a coupling term for its interaction with relational disagreement. We derive bounds on corruption-induced changes in the semantic residual and the fused representation under directional separation conditions. On ACM, DBLP, Yelp, and MAG, OCG achieves the highest mean Macro-F1 and Micro-F1 among the evaluated baselines. We evaluate robustness under edge and node-feature perturbations, assess individual objectives through ablation, and examine residual geometry in a controlled experiment.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
heterogeneous graph representation learning, graph contrastive learning, multi-view learning, residual orthogonality, robustness

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_BrWQOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Orthogonal Consensus Graph Learning (OCG) for robust heterogeneous graph representation learning. OCG constructs multiple schema-induced relational views and an attribute-derived semantic hypergraph, followed by view-specific encoding and iterative representation refinement. Its central component is a cross-Gram regularizer that penalizes interactions between relational and semantic residuals centered at their empirical consensus. The framework further combines cross-view contrastive alignment and stochastic-path predictive consistency. Experiments on ACM, DBLP, Yelp, and MAG evaluate node classification and clustering, with additional robustness tests, ablations, and residual-geometry analyses.

Critical strengths
  1. The motivation is clearly connected to a limitation of conventional cross-view alignment. Different views may agree while sharing corruption-induced errors, so simply maximizing agreement does not necessarily produce robust representations. Examining residual interactions provides a meaningful additional perspective.

  2. The framework integrates relational and attribute-derived information using complementary graph constructions. Schema-specific relational graphs capture typed connectivity, while the KNN-based semantic hypergraph provides feature-space relations that may be absent from the original graph. This is a reasonable design for multiplex representation learning.

  3. The empirical evaluation covers several relevant baseline families. Beyond conventional GNNs, Table 2 includes graph structure learning, robust representation learning, and multiplex graph learning methods.

Critical weaknesses
  1. A central motivation is that different views may contain correlated corruption. However, Proposition 1 establishes that adding the same matrix to every view leaves the residual penalty unchanged, and Proposition 3 explicitly demonstrates that shared errors can remain arbitrarily large even when the loss is zero. This creates a gap between the motivating problem and what the regularizer can actually control.

  2. In Table 2, the Macro-F1 improvements over InfoMGF are most not very significant. While the ranking is consistent, these margins are modest compared with the complexity of the proposed framework. I would encourage the author to measure whether the improvements remain consistent under matched computational budgets.

  3. Table 7 shows that coupling-only regularization achieves a slightly higher mean Macro-F1 than the complete residual penalty, while spectral-only regularization produces nearly the same performance as the complete penalty. Although these differences are small relative to the three-seed variability, the results do not establish that combining both components is necessary.

  4. OCG adds view-specific structure learning, iterative refinement, cross-Gram regularization, and predictive consistency on top of multi-view encoding. Table 9 shows that its ACM training step is approximately 60% slower than InfoMGF, with substantially higher peak GPU memory. I would encourage the authors to add experimental analysis on large-scale datasets.

  5. There are only 4 very classical heterogeneous graph datasets used in this paper. I think that there should be more datasets, especially large-scale ones I mentioned before.

Questions for authors

Please answer my problems in Critical Weaknesses. I may rise my rating if these problems can be well addressed.

Reviewer_xCqsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes OCG for robust heterogeneous graph representation learning, combining cross-view contrastive learning, residual orthogonality regularization, and predictive consistency. Theoretical analysis and experiments on four datasets are provided to evaluate its effectiveness and robustness.

Critical strengths
  1. The paper provides a theoretical analysis of the proposed residual regularizer, including its mathematical decomposition and robustness bounds.

2.It addresses a meaningful problem by recognizing that cross-view alignment may preserve shared noise while suppressing useful view-specific information.

Critical weaknesses
  1. The proposed framework closely resembles InfoMGF (NeurIPS 2024), particularly in its graph structure learning and representation encoding processes. The main differences lie in the introduction of an additional semantic view and several regularization strategies, rather than a fundamentally new learning framework. Moreover, the performance improvements over InfoMGF are relatively modest, raising concerns about whether these additional components provide substantial benefits.

  2. The ablation results in Table 7 do not convincingly demonstrate the necessity of combining the two components of the proposed residual regularizer. The coupling-only variant achieves a higher Macro-F1 than the full objective, while the spectral-only variant performs similarly.

  3. Compared with InfoMGF, OCG achieves only marginal performance improvements while introducing substantially higher computational costs.

Questions for authors
  1. The proposed framework closely resembles InfoMGF (NeurIPS 2024), particularly in its graph structure learning and representation encoding processes. The main differences lie in the introduction of an additional semantic view and several regularization strategies, rather than a fundamentally new learning framework. Moreover, the performance improvements over InfoMGF are relatively modest, raising concerns about whether these additional components provide substantial benefits.

  2. The ablation results in Table 7 do not convincingly demonstrate the necessity of combining the two components of the proposed residual regularizer. The coupling-only variant achieves a higher Macro-F1 than the full objective, while the spectral-only variant performs similarly.

  3. Compared with InfoMGF, OCG achieves only marginal performance improvements while introducing substantially higher computational costs.

Reviewer_nwB2OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies robust heterogeneous graph representation learning when structural and attribute corruption can affect multiple views. Its motivation is that cross-view agreement may coexist with shared errors, while disagreement can contain useful view-specific information. OCG combines schema-induced relational views with an attribute-derived semantic view, encodes and refines their representations, and penalizes cross-Gram interactions between residuals centered at the empirical view mean. This residual regularizer is combined with cosine-based contrastive alignment and predictive consistency across independently masked embedding paths. The final representation uses gated fusion of row-normalized views. The paper provides an exact decomposition of the residual penalty and conditional bounds connecting residual changes to fused-representation changes. Experiments report improved mean classification performance on four benchmarks and robustness under the evaluated perturbations.

Critical strengths
  1. Since relational and attribute-derived views depend on the same node features, their corruption effects need not be independent. The Introduction therefore identifies a concrete limitation of relying on cross-view agreement alone, while recognizing that suppressing all disagreement could discard complementary information.
  2. Proposition 1 explicitly separates the cross-Gram penalty into semantic-residual spectral shrinkage and coupling with relational disagreement. This makes the objective easier to interpret and clarifies that residual orthogonality is not equivalent to independent prediction errors. The conditional analysis and discussion of shared-error nonidentifiability also help delineate the scope of the method.
Critical weaknesses
  1. Equation (6) operates on representations before the final row normalization. At the representation level, scaling every final view by a common positive factor s multiplies this penalty by s to the fourth power, while leaving cosine similarities and normalized fusion unchanged, provided norms remain above the numerical normalization threshold. No fixed-norm or variance constraint on all final refined views is specified. Thus, a smaller raw penalty does not by itself establish reduced interactions in the geometry used for prediction. Moreover, Proposition 1 shows that zero penalty forces the semantic view to equal the relational mean; the objective does not identify whether removed semantic-specific information is corruption or useful signal. The conditional bounds do not resolve this distinction: they assume directional separation, depend on the loss on corrupted inputs, and retain uncontrolled anchor and normalization terms. Appendices E.3 and G acknowledge related limitations, but the training mechanism connecting residual reduction to meaningful robustness remains unclear.
  2. The auxiliary prediction head in Section 4.4 is trained only through agreement between masked paths. A constant-output head achieves zero symmetric KL divergence regardless of the embeddings; for a linear head, zero weights and a fixed bias provide such a solution and eliminate the consistency gradient to the encoder. Contrastive learning can keep node embeddings discriminative without preventing this separate head from becoming constant. Appendix G acknowledges this possibility, but no constraint or optimization argument explains why the head should provide informative predictive supervision. The reported ablation supports an empirical contribution, yet does not establish the proposed mechanism. This is an unresolved design issue, rather than evidence that the trained head necessarily collapses.
Questions for authors

see Weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.