acceptodds
Under review as a conference paper at ICLR 2027

UniCO: Fostering Cross-Domain and Cross-Modal Generalization for Graph Foundation Models via Unified Covariance Operators

Abstract

Input feature heterogeneity across domains severely impairs the transferability of graph foundation models. Although natural language instructions provide task priors, existing cross-domain heterogeneous graph models are restricted to the graph-only modality, while current graph-text models suffer from either topological distortion induced by graph serialization or rigid dependency on domain-specific alignment. To this end, we propose UniCO, a cross-domain graph-text framework based on unified covariance operators. UniCO conceptualizes text instructions as "text graphs", thereby casting the dual cross-domain and cross-modal challenge into a singular heterogeneous graph generalization problem. Building upon this, we construct unified multi-order covariance operators for fine-grained cross-modal interactions, and design an asymmetric mask to preserve instruction temporal order while isolating the graph from instruction noise. Theoretically, we prove that these operators exhibit joint distributional and expectation invariance under input feature basis transformations, and reconcile "macroscopic modal decoupling" with "microscopic graph-text interactions" in expectation, eliminating dependence on domain-specific feature spaces and laying a theoretical foundation for cross-domain transfer. Experiments on multiple molecular graph and text-attributed graph benchmarks demonstrate that UniCO achieves superior zero-shot cross-domain generalization and efficient few-shot fine-tuning performance.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph Foundation Models, Covariance Operators, Cross-Domain Generalization, Cross-Modal Generalization

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_yjXcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes UniCO, a unified graph-text framework for cross-domain and cross-modal graph foundation models. The method models natural-language instructions as text graphs and jointly processes them with input graphs. To address heterogeneous feature spaces across domains, UniCO employs stochastic Gaussian projections and multi-order covariance operators, while an asymmetric masking mechanism controls graph-text information flow. The method is evaluated on molecular graph and text-attributed graph tasks in zero-shot and few-shot settings.

Critical strengths
  1. The paper presents an interesting formulation that represents instructions as text graphs, allowing graph structures and natural-language task descriptions to be processed in a unified framework. This design may preserve graph topology more naturally than direct graph serialization.
  2. The proposed use of stochastic projections and multi-order covariance operators is a meaningful attempt to address feature heterogeneity across graph domains. The experimental study also covers a broad range of tasks, including graph-level prediction, node classification, link prediction, and few-shot adaptation.
Critical weaknesses

W1. Although the authors establish the distributional and expectation invariance of the proposed covariance operators under feature basis transformations, these results do not necessarily guarantee generalization across heterogeneous graph domains. In particular, real-world domain shifts may involve changes in feature distributions, structural patterns, and feature–label relationships that cannot be characterized solely by coordinate transformations. The connection between the established invariance properties and improved downstream cross-domain generalization remains insufficiently justified.

W2. The proposed operator-enhanced attention essentially incorporates multi-order covariance operators as additive biases into standard Transformer attention. While the construction of these operators is interesting, the overall attention mechanism follows the established paradigm of structure-aware attention. The methodological distinction from existing covariance-based graph models and structure-aware Transformers could be better clarified.

W3. Although the experiments demonstrate promising zero-shot and few-shot performance, it remains unclear whether these improvements primarily arise from the proposed covariance operators or from incorporating additional multi-order structural information into Transformer attention. More controlled comparisons with conventional topology-aware attention biases, under the same backbone and propagation orders, would strengthen the empirical justification for the proposed design.

Questions for authors
  1. The theoretical analysis establishes invariance under feature basis transformations. Under what additional assumptions can these invariance properties lead to improved cross-domain generalization, particularly when source and target domains exhibit substantial differences in feature distributions and graph structures?

  2. The proposed cross-modal covariance operators are shown to have zero expectation while remaining nonzero in individual stochastic realizations. How can the authors justify that these stochastic cross-modal interactions capture meaningful graph–text semantic relationships rather than random statistical correlations?

  3. The method explicitly constructs dense covariance operators, potentially incurring substantial computational and memory overhead as graph size or instruction length increases. Although Appendix E discusses a possible linear-time reformulation, its practical effectiveness remains unverified. Experiments on larger graphs and longer instructions, together with runtime and memory comparisons against existing methods, would strengthen the scalability evaluation.

4 comments
AuthorsOpenReview ↗

Kind Reminder: Discussion Period Ending Soon (Submission 12003 / Reviewer yjXc)

Dear Reviewer yjXc,

Thank you again for your constructive comments on our paper "UniCO: Fostering Cross-Domain and Cross-Modal Generalization for Graph Foundation Models via Unified Covariance Operators".

As the discussion period is nearing its end with less than two days remaining, we wanted to kindly check if you have had the opportunity to review our responses.

To address your concerns, we have provided detailed clarifications and additional experimental evidence in our initial rebuttal:

(Note: We have updated our manuscript PDF to incorporate all these detailed clarifications, revised texts, and additional experimental tables.)


Invariance vs. Generalization Assumptions (W1 & Q1): We clarified that basis invariance serves as a foundation, and detailed key cross-domain transfer assumptions in Appendix B.2.3 that bridge invariance to cross-domain generalization.


Methodological Distinction of Operator-Enhanced Attention (W2): We highlighted that unlike pure-graph covariance models or traditional structure-aware Transformers tied to fixed scales, UniCO introduces a parameter-free cross-modal statistical conduit (CtgC_{tg}) and stochastic covariance biases that eliminate coordinate basis dependencies.


Controlled Comparison with Conventional Structural Biases (W3): We conducted rigorous controlled experiments replacing Cuni(k)C_{uni}^{(k)} with standard multi-order adjacency matrices AkA^k and RWSE under identical settings (K=5K=5). UniCO consistently outperformed these variants across benchmarks, confirming the unique efficacy of our covariance formulation (Table 4).


Semantic Correlation vs. Projection Randomness (Q2): We conducted instruction perturbation tests and Gaussian seed sensitivity analyses, showing that semantic disruptions cause severe performance drops while low seed variance confirms CtgC_{tg} captures true instruction semantics rather than noise (Appendix F, Tables 11 & 12).


Scalability & Practical Efficiency (Q3): We clarified that subgraph sampling limits practical sizes to ∣V∣≤100|\mathcal{V}| \le 100. Furthermore, empirical scalability tests demonstrated that our optimized variant, UniCO-Efficient, scales efficiently while fully maintaining predictive accuracy (Appendix E.3, Table 10).


We hope these responses have addressed your concerns satisfactorily. If you have any remaining questions or additional feedback, please let us know—we would be more than happy to discuss further.

Thank you very much for your time and dedication!

Best regards,

The Authors of Submission 12003

AuthorsOpenReview ↗

Kind Reminder: Final Hours of Discussion Period (Submission 12003 / Reviewer yjXc)

Dear Reviewer yjXc,

Thank you for your time and effort in reviewing our paper. As the discussion period is nearing its end with less than 6 hours remaining, we note that we have not yet received any follow-up response from you. We wanted to kindly reach out to ensure that we have addressed your concerns satisfactorily.

If there are any additional points or feedback you'd like us to consider, please let us know. We remain available for further discussion before the deadline.

Best regards,

The Authors of Submission 12003

Thank you for the clarification. However, two concerns remain.

The connection between feature basis invariance and cross-domain generalization remains unclear. In particular, there is a lack of theoretical analysis (e.g., a generalization bound) establishing how the proposed invariance, under the stated assumptions, can reduce cross-domain distribution discrepancy while preserving task-relevant information.

The first two assumptions appear relatively strong. Consistent topology–feature interaction rules may not hold under substantial structural shifts, particularly homophily shifts. Furthermore, nodes with similar structural roles do not necessarily exhibit similar feature co-variation patterns across domains. Assuming such consistency effectively incorporates part of the desired cross-domain alignment into the assumption itself.

Therefore, I maintain my score.

AuthorsOpenReview ↗

Response to Reviewer yjXc

We thank the reviewer for raising these two pointed concerns. We hope to offer a final clarification of our work, and below is a response clarifying the theoretical role of our invariance results and refining the framing of our assumptions.


1.From Basis Invariance to Cross-Domain Generalization

We agree that orthogonal basis invariance alone does not constitute a formal cross-domain generalization bound or eliminate distribution shift. In UniCO, basis invariance serves as a prerequisite rather than a sufficient condition: it removes incompatible coordinate bases and feature dimensions across domains so that cross-domain transfer relies on relational co-variation and instruction conditioning rather than raw feature alignment.

1)Distribution discrepancy in operator space: Because heterogeneous graphs lack a shared feature coordinate space, projecting onto Rdh\mathbb{R}^{d_h} and centering (Πg,Πt\Pi_g, \Pi_t) maps them into a common covariance space where orthogonal transformations leave the distribution and expectation unchanged (Propositions 4.1.2, Theorem 4.2.1), while Theorem 4.4.1 and Proposition 4.4.2 bound finite-dhd_h approximation and stochastic training risk. Remaining cross-domain discrepancy in expectation then reflects differences in centered propagated outer products (ΠgAgkΠgXgXg⊤Πg(Agk)⊤Πg\Pi_g A_g^k \Pi_g X_g X_g^\top \Pi_g (A_g^k)^\top \Pi_g). Deriving a formal domain-adaptation generalization bound under arbitrary non-orthogonal and label shifts remains an important direction for future work.

2)Preserving task-relevant information: Centering removes global means while retaining relative node/token variations and multi-hop contrasts (Agk,AtkA_g^k, A_t^k). Although E[Ctg(k)]=0\mathbb{E}[C_{tg}^{(k)}] = 0 avoids systematic cross-modal bias, each forward pass yields a non-zero Ctg(k)C_{tg}^{(k)} modulated by learnable head weights (αk,βk\alpha_k, \beta_k) and asymmetric masks, while task-specific semantics are carried by the instruction TτT^\tau and decoded autoregressively by the LM.


2.Refining the Assumptions to Avoid Circularity

We agree that treating cross-domain similarity of co-variation patterns as a universal assumption risks circularity, and that homophily shifts or structural shifts can weaken these regularities.

In Appendix B.2.3, we will reframe these points not as universal axioms, but as scope conditions indicating when zero-shot transfer is favorable versus when shifts or negative transfer occur:

1)Transfer is favored when source and target share compatible relational/attribute-topology patterns (whether homophilic, heterophilic, or neighborhood-contrastive) and compatible mesoscopic/latent graph structure (e.g., block/community or graphon-like structure).

2)When homophily, structural roles, or feature spaces shift sharply—as reflected in non-orthogonal/nonlinear perturbations (Table 5) or mild negative transfer in Leave-One-Domain-Out experiments (Appendix H, Table 14)—zero-shot transfer can be constrained, motivating few-shot adaptation of αk,βk\alpha_k, \beta_k, LoRA, and the prediction head.

We will revise Appendix B.2.3 accordingly to make this distinction between exact orthogonal invariance and empirical transfer scope explicit.

Reviewer_LTkCOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies the generalization of cross-domain and cross-modal Graph Foundation Model (GFM) and proposes the framework UniCO. UniCO models text into graphs, which turns the cross-domain and cross-model problem into a single cross-domain graph generalization problem. The design of multi-order covariance operators serves as the key mechanism, theoretically proved to rely less on domain-specific feature spaces. Experimental results on diverse domains and baselines show the effectiveness of the proposed method.

Critical strengths
  • The figures are well-designed and visually appealing.
  • Extensive experiments on datasets across various domains under zero-shot and few-shot settings demonstrate the superior performance of UniCO.
Critical weaknesses

Insufficient motivation: In paragraph two and three, the limitations of existing works are simply listed and do not show a main research challenge. For example, the second paragraph discusses three kinds of deficiency of existing methods of graph-text modeling, while the third paragraph lists techniques like"specialized encoders", "structuralization", "feature projections", "order statistics" and "second-order statistics". However, these discussions read more like a list of related works, rather than a coherent analysis of the core research problem. As a result, the motivation for proposed framework UniCO is not sufficiently established.

Questions for authors
  • Graph serialization does not necessarily disrupt high-order connectivity, since text edge lists are able to preserve the original structural information (in Lines 44-46).
  • This paper mistakenly uses citation commands (i.e., \citep{} or \citet{}), which reduces the readability and presentation of the paper.
3 comments
AuthorsOpenReview ↗

Kind Reminder: Discussion Period Ending Soon (Submission 12003 / Reviewer LTkC)

Dear Reviewer LTkC,

Thank you again for your constructive comments on our paper "UniCO: Fostering Cross-Domain and Cross-Modal Generalization for Graph Foundation Models via Unified Covariance Operators".

As the discussion period is nearing its end with less than two days remaining, we wanted to kindly check if you have had the opportunity to review our responses.

To address your concerns, we have provided detailed clarifications and revisions in our initial rebuttal:

(Note: We have updated our manuscript PDF to incorporate all these detailed clarifications and revised texts.)


Motivation Clarification: We restructured Sec. 1 to elevate our motivation from a list of technical limitations to a cohesive analysis of the central bottleneck in GFMs, explicitly formulating the core challenge of balancing cross-domain feature spaces and fine-grained cross-modal instruction interactions.


Graph Serialization Limitations vs. Multi-Order Covariance (Q1): We clarified that while text edge lists preserve graph topology, LMs suffer from severe token length explosion and computational bottlenecks during autoregressive multi-hop path inference. We detailed how UniCO captures high-order KK-hop connectivity through parameter-free covariance operators (Cuni(k)C_{uni}^{(k)}) without autoregressive generation overhead, and updated the manuscript phrasing accordingly.


LaTeX Citation Corrections (Q2): We performed a thorough audit of our LaTeX source code and corrected all instances of misapplied citation commands, significantly improving the manuscript's readability and formatting.


We hope these responses have addressed your concerns satisfactorily. If you have any remaining questions or additional feedback, please let us know—we would be more than happy to discuss further.

Thank you very much for your time and dedication!

Best regards,

The Authors of Submission 12003

Thank you for your responses. I will raise my score to 3.

AuthorsOpenReview ↗

Thank You for the Positive Score Upgrade and Constructive Feedback (Reviewer LTkC)

Dear Reviewer LTkC,

Thank you very much for taking the time to review our responses and for upgrading your rating to a positive score!

We are deeply grateful for your constructive feedback. Your suggestions have been invaluable in improving the structural logical flow of our Introduction and significantly enhancing the overall readability of the manuscript.

We will carefully follow your guidance to further polish and optimize the paper, as well as seamlessly integrate all the rebuttal discussions into the final version.

Thank you again for your time, dedication, and thoughtful support throughout the review process!

Best regards,

The Authors of Submission 12003

Reviewer_8TGLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

UniCO addresses instruction-conditioned cross-domain graph transfer by modeling text instructions as token-level “text graphs” and combining them with ordinary graphs through multi-order covariance operators. Independently sampled Gaussian projections map graph and text features into a shared dimension, while asymmetric masking restricts the direction of graph–text information flow. The paper derives invariance results and evaluates zero-shot and few-shot performance on molecular and text-attributed graph tasks. The reported accuracy and ablation improvements are promising.

Critical strengths

S1. Representing instructions as graphs is a useful design perspective that can preserve native graph connectivity without first serializing the input graph into text (Sec. 3.2; Fig. 1). S2. The unified covariance construction and direction-specific masking form a coherent architectural proposal. The paper also analyzes orthogonal-basis invariance and expected properties of the stochastic cross-modal blocks (Secs. 3–4). S3. The evaluation includes different prediction granularities and informative component ablations. Table 4 reports BACE ROC-AUC of 80.86 for UniCO versus 68.24 without the graph–text cross-modal covariance block (w/o Ctg), providing evidence that the module affects performance.

Critical weaknesses

W1. The interpretation of the cross-modal covariance as semantic alignment needs stronger support. Graph and text projections are independently resampled, and Theorem 4.2.1 shows that their cross-modal blocks vanish in expectation. A nonzero sample realization alone does not establish stable instruction-grounded correlation. Instruction-shuffle/contradiction tests and seed-sensitivity analysis would clarify what the operator learns (Secs. 3.2, 4.2). W2. The scope of the invariance claim should be stated more precisely. Proposition 4.1.2 proves a distributional statement for orthogonal changes of basis; it does not by itself imply invariance under feature scaling, nonlinear transforms, or shifts in feature semantics. A controlled non-orthogonal shift test would establish a clearer boundary (Sec. 4.1). W3. The scalability argument should distinguish operator–value multiplication from full attention. Appendix E notes O(N²) attention-bias storage and proposes a linear-in-N associative covariance–value product, but this does not automatically make the stated softmax attention implementation linear in N. End-to-end time/memory measurements on larger graphs are needed (Appendix E). W4. The few-shot comparisons do not fully match adaptation capacity. UniCO updates the prediction head, LoRA attention parameters, and covariance coefficients, while Appendix C.4.3 specifies head-only updating for multiple baselines. Matched trainable-parameter or compute budgets, together with a head-only UniCO ablation, would improve attribution of the reported gains (Appendix C.3–C.4).

Questions for authors

Q1. What evidence shows that independent random cross-modal covariance reflects task semantics rather than projection randomness? Can the authors provide instruction perturbation and random-seed tests? Q2. How robust is the method under non-orthogonal input transformations and equalized few-shot adaptation budgets? Q3. Does the deployed softmax-attention implementation actually achieve linear scaling, or only the proposed covariance–value multiplication? Please report end-to-end complexity.

2 comments
AuthorsOpenReview ↗

Kind Reminder: Discussion Period Ending Soon (Submission 12003 / Reviewer 8TGL)

Dear Reviewer 8TGL,

Thank you again for your constructive comments on our paper "UniCO: Fostering Cross-Domain and Cross-Modal Generalization for Graph Foundation Models via Unified Covariance Operators".

As the discussion period is nearing its end with less than two days remaining, we wanted to kindly check if you have had the opportunity to review our responses.

To address your concerns, we have provided detailed clarifications and additional experimental evidence in our initial rebuttal:

(Note: We have updated our manuscript PDF to incorporate all these detailed clarifications, revised texts, and additional experimental tables.)


Instruction Semantics & Projection Randomness (W1 & Q1): We conducted instruction perturbation tests and Gaussian seed sensitivity analyses, showing that semantic perturbations cause severe performance drops, while low seed variance confirms that Ctg(k)C_{tg}^{(k)} captures true instruction semantics rather than projection noise (Appendix F, Tables 11 & 12).


Non-Orthogonal Feature Shifts & Invariance Boundaries (W2 & Q2): We outlined cross-domain transfer assumptions (Appendix B.2.3) and demonstrated UniCO's strong robustness over UniGraph under non-orthogonal/nonlinear feature shifts in equalized 32-shot settings (Table 5).


Scalability & End-to-End Complexity (W3 & Q3): We clarified that our practical complexity is kept low via standard subgraph sampling (∣V∣≤100|\mathcal{V}| \le 100). Furthermore, empirical tests confirmed that our optimized variant, UniCO-Efficient, scales efficiently while fully maintaining predictive accuracy (Appendix E.3, Table 10).


Matched Adaptation Budget & Head-Only Ablation (W4): We clarified baseline adaptation protocols and evaluated a head-only variant (UniCO-ho). Under identical 16-shot budgets, UniCO-ho still comfortably outperformed UniGraph on both graph-level and node-level tasks (Appendix D, Table 9).


We hope these responses have addressed your concerns satisfactorily. If you have any remaining questions or additional feedback, please let us know—we would be more than happy to discuss further.

Thank you very much for your time and dedication!

Best regards,

The Authors of Submission 12003

AuthorsOpenReview ↗

Kind Reminder: Final Hours of Discussion Period—Submission 12003

Dear Reviewer 8TGL,

Thank you for your time and effort in reviewing our paper. As the discussion period is nearing its end with less than 6 hours remaining, we note that we have not yet received any follow-up response from you. We wanted to kindly reach out to ensure that we have addressed your concerns satisfactorily.

If there are any additional points or feedback you'd like us to consider, please let us know. We remain available for further discussion before the deadline.

Best regards,

The Authors of Submission 12003

Reviewer_4i3XOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces UniCO, a graph foundation model framework that uses unified covariance operators to support cross-domain and cross-modal transfer. By representing textual instructions as “text graphs”, UniCO formulates graph–text learning across domains as a unified heterogeneous graph generalization problem. The framework incorporates multi-order covariance operators and asymmetric attention masks to integrate graph structure with textual instructions. Experiments evaluate its zero-shot and few-shot transfer performance on molecular and text-attributed graph tasks.

Critical strengths
  1. The paper introduce a “text graph” paradigm that reduces the intricate dual cross-modal and cross-domain challenge into a singular cross-domain heterogeneous graph generalization problem.
  2. The covariance operators provide relational attention biases whose dimensions depend on the number of graph nodes and instruction tokens rather than the original feature dimensions, enabling a common architecture across heterogeneous input spaces.
  3. The paper reports results across graph classification, graph regression, node classification, and link prediction on molecular and text-attributed graphs.
Critical weaknesses
  1. UniCO combines covariance-based feature processing with unified graph–text modeling, both of which have close precedents [1] [2]. The authors are required to specify difference between the aforementioned papers to defend their novelty.

  2. I noticed that in the few-shot fine-tuning setting, UniCO was fine-tuned on its output head, the LoRA parameters within its attention mechanisms, and its operator coefficients, whereas other baseline models were restricted to fine-tuning only their output heads. I wonder if this is a fair comparison?

  3. The text-attributed graph pre-training datasets primarily consist of academic citation networks and a social network. However, the downstream evaluation datasets also largely belong to similar domains. Could the authors further evaluate UniCO on text-attributed graphs from different domains, such as e-commerce and recommendation networks, to better demonstrate its cross-domain generalization capability?

  4. Multi-domain pre-training may suffer from negative transfer, where knowledge from certain source domains adversely affects generalization to unseen target domains. Although UniCO jointly learns from multiple graph domains, the paper does not investigate whether all source domains contribute positively to downstream performance. Could the authors provide an analysis of potential negative transfer across different source domains?

[1] GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learning, NeurIPS 2023.

[2] Bridging Input Feature Spaces Towards Graph Foundation Models, ICLR 2026.

Questions for authors

Please see weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.