- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes UniCO, a unified graph-text framework for cross-domain and cross-modal graph foundation models. The method models natural-language instructions as text graphs and jointly processes them with input graphs. To address heterogeneous feature spaces across domains, UniCO employs stochastic Gaussian projections and multi-order covariance operators, while an asymmetric masking mechanism controls graph-text information flow. The method is evaluated on molecular graph and text-attributed graph tasks in zero-shot and few-shot settings.
Critical strengths
- The paper presents an interesting formulation that represents instructions as text graphs, allowing graph structures and natural-language task descriptions to be processed in a unified framework. This design may preserve graph topology more naturally than direct graph serialization.
- The proposed use of stochastic projections and multi-order covariance operators is a meaningful attempt to address feature heterogeneity across graph domains. The experimental study also covers a broad range of tasks, including graph-level prediction, node classification, link prediction, and few-shot adaptation.
Critical weaknesses
W1. Although the authors establish the distributional and expectation invariance of the proposed covariance operators under feature basis transformations, these results do not necessarily guarantee generalization across heterogeneous graph domains. In particular, real-world domain shifts may involve changes in feature distributions, structural patterns, and feature–label relationships that cannot be characterized solely by coordinate transformations. The connection between the established invariance properties and improved downstream cross-domain generalization remains insufficiently justified.
W2. The proposed operator-enhanced attention essentially incorporates multi-order covariance operators as additive biases into standard Transformer attention. While the construction of these operators is interesting, the overall attention mechanism follows the established paradigm of structure-aware attention. The methodological distinction from existing covariance-based graph models and structure-aware Transformers could be better clarified.
W3. Although the experiments demonstrate promising zero-shot and few-shot performance, it remains unclear whether these improvements primarily arise from the proposed covariance operators or from incorporating additional multi-order structural information into Transformer attention. More controlled comparisons with conventional topology-aware attention biases, under the same backbone and propagation orders, would strengthen the empirical justification for the proposed design.
Questions for authors
-
The theoretical analysis establishes invariance under feature basis transformations. Under what additional assumptions can these invariance properties lead to improved cross-domain generalization, particularly when source and target domains exhibit substantial differences in feature distributions and graph structures?
-
The proposed cross-modal covariance operators are shown to have zero expectation while remaining nonzero in individual stochastic realizations. How can the authors justify that these stochastic cross-modal interactions capture meaningful graph–text semantic relationships rather than random statistical correlations?
-
The method explicitly constructs dense covariance operators, potentially incurring substantial computational and memory overhead as graph size or instruction length increases. Although Appendix E discusses a possible linear-time reformulation, its practical effectiveness remains unverified. Experiments on larger graphs and longer instructions, together with runtime and memory comparisons against existing methods, would strengthen the scalability evaluation.
4 comments
Kind Reminder: Final Hours of Discussion Period (Submission 12003 / Reviewer yjXc)
Dear Reviewer yjXc,
Thank you for your time and effort in reviewing our paper. As the discussion period is nearing its end with less than 6 hours remaining, we note that we have not yet received any follow-up response from you. We wanted to kindly reach out to ensure that we have addressed your concerns satisfactorily.
If there are any additional points or feedback you'd like us to consider, please let us know. We remain available for further discussion before the deadline.
Best regards,
The Authors of Submission 12003
Thank you for the clarification. However, two concerns remain.
The connection between feature basis invariance and cross-domain generalization remains unclear. In particular, there is a lack of theoretical analysis (e.g., a generalization bound) establishing how the proposed invariance, under the stated assumptions, can reduce cross-domain distribution discrepancy while preserving task-relevant information.
The first two assumptions appear relatively strong. Consistent topology–feature interaction rules may not hold under substantial structural shifts, particularly homophily shifts. Furthermore, nodes with similar structural roles do not necessarily exhibit similar feature co-variation patterns across domains. Assuming such consistency effectively incorporates part of the desired cross-domain alignment into the assumption itself.
Therefore, I maintain my score.
Response to Reviewer yjXc
We thank the reviewer for raising these two pointed concerns. We hope to offer a final clarification of our work, and below is a response clarifying the theoretical role of our invariance results and refining the framing of our assumptions.
1.From Basis Invariance to Cross-Domain Generalization
We agree that orthogonal basis invariance alone does not constitute a formal cross-domain generalization bound or eliminate distribution shift. In UniCO, basis invariance serves as a prerequisite rather than a sufficient condition: it removes incompatible coordinate bases and feature dimensions across domains so that cross-domain transfer relies on relational co-variation and instruction conditioning rather than raw feature alignment.
1)Distribution discrepancy in operator space: Because heterogeneous graphs lack a shared feature coordinate space, projecting onto and centering () maps them into a common covariance space where orthogonal transformations leave the distribution and expectation unchanged (Propositions 4.1.2, Theorem 4.2.1), while Theorem 4.4.1 and Proposition 4.4.2 bound finite- approximation and stochastic training risk. Remaining cross-domain discrepancy in expectation then reflects differences in centered propagated outer products (). Deriving a formal domain-adaptation generalization bound under arbitrary non-orthogonal and label shifts remains an important direction for future work.
2)Preserving task-relevant information: Centering removes global means while retaining relative node/token variations and multi-hop contrasts (). Although avoids systematic cross-modal bias, each forward pass yields a non-zero modulated by learnable head weights () and asymmetric masks, while task-specific semantics are carried by the instruction and decoded autoregressively by the LM.
2.Refining the Assumptions to Avoid Circularity
We agree that treating cross-domain similarity of co-variation patterns as a universal assumption risks circularity, and that homophily shifts or structural shifts can weaken these regularities.
In Appendix B.2.3, we will reframe these points not as universal axioms, but as scope conditions indicating when zero-shot transfer is favorable versus when shifts or negative transfer occur:
1)Transfer is favored when source and target share compatible relational/attribute-topology patterns (whether homophilic, heterophilic, or neighborhood-contrastive) and compatible mesoscopic/latent graph structure (e.g., block/community or graphon-like structure).
2)When homophily, structural roles, or feature spaces shift sharply—as reflected in non-orthogonal/nonlinear perturbations (Table 5) or mild negative transfer in Leave-One-Domain-Out experiments (Appendix H, Table 14)—zero-shot transfer can be constrained, motivating few-shot adaptation of , LoRA, and the prediction head.
We will revise Appendix B.2.3 accordingly to make this distinction between exact orthogonal invariance and empirical transfer scope explicit.
Kind Reminder: Discussion Period Ending Soon (Submission 12003 / Reviewer yjXc)
Dear Reviewer yjXc,
Thank you again for your constructive comments on our paper "UniCO: Fostering Cross-Domain and Cross-Modal Generalization for Graph Foundation Models via Unified Covariance Operators".
As the discussion period is nearing its end with less than two days remaining, we wanted to kindly check if you have had the opportunity to review our responses.
To address your concerns, we have provided detailed clarifications and additional experimental evidence in our initial rebuttal:
(Note: We have updated our manuscript PDF to incorporate all these detailed clarifications, revised texts, and additional experimental tables.)
Invariance vs. Generalization Assumptions (W1 & Q1): We clarified that basis invariance serves as a foundation, and detailed key cross-domain transfer assumptions in Appendix B.2.3 that bridge invariance to cross-domain generalization.
Methodological Distinction of Operator-Enhanced Attention (W2): We highlighted that unlike pure-graph covariance models or traditional structure-aware Transformers tied to fixed scales, UniCO introduces a parameter-free cross-modal statistical conduit (Ctg) and stochastic covariance biases that eliminate coordinate basis dependencies.
Controlled Comparison with Conventional Structural Biases (W3): We conducted rigorous controlled experiments replacing Cuni(k) with standard multi-order adjacency matrices Ak and RWSE under identical settings (K=5). UniCO consistently outperformed these variants across benchmarks, confirming the unique efficacy of our covariance formulation (Table 4).
Semantic Correlation vs. Projection Randomness (Q2): We conducted instruction perturbation tests and Gaussian seed sensitivity analyses, showing that semantic disruptions cause severe performance drops while low seed variance confirms Ctg captures true instruction semantics rather than noise (Appendix F, Tables 11 & 12).
Scalability & Practical Efficiency (Q3): We clarified that subgraph sampling limits practical sizes to ∣V∣≤100. Furthermore, empirical scalability tests demonstrated that our optimized variant, UniCO-Efficient, scales efficiently while fully maintaining predictive accuracy (Appendix E.3, Table 10).
We hope these responses have addressed your concerns satisfactorily. If you have any remaining questions or additional feedback, please let us know—we would be more than happy to discuss further.
Thank you very much for your time and dedication!
Best regards,
The Authors of Submission 12003