acceptodds
Withdrawn from ICLR 2027

ReCoG: Reciprocal Co-Evolution for Multimodal Graph Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal graph learning requires jointly training over graph structure and heterogeneous node attributes, yet existing methods largely decouple these processes: prior multimodal graph neural networks (GNNs) focus on aligning modalities in a shared embedding space while operating on fixed or weakly adapted graph structures, and graph structure learning approaches infer topology from unimodal node representations without accounting for multimodal interactions. This separation fundamentally limits the ability of GNNs to capture semantically meaningful relationships in multimodal settings, where observed edges are often noisy, incomplete, or misaligned with underlying semantics. We propose ReCoG (Reciprocal Co-Evolution for Multimodal Graph Learning), a new learning paradigm that tightly couples graph structure learning and multimodal representation learning through end-to-end reciprocal interaction. Concretely, ReCoG integrates (i) a multimodal graph refiner that infers and corrects edges using cross-modal semantic evidence, and (ii) a coupled cross-modal message passing mechanism that performs joint intra- and inter-modality propagation over the refined graph. This unified design yields greater expressiveness than decoupled or two-stage formulations and allows dynamic interaction between topology and representation learning. Across diverse benchmarks for node classification and link prediction, the best ReCoG variant outperforms strong multimodal graph learning baselines, including graph foundation models, on all nine benchmarks. Our results demonstrate that reciprocal co-evolution of structure and semantics is important for effective multimodal graph learning, challenging the prevailing separation between topology and representation learning. Across diverse benchmarks for node classification and link prediction, ReCoG consistently outperforms strong multimodal graph structure learning baselines, including graph foundation models. Our results demonstrate that reciprocal co-evolution of structure and semantics is important for effective multimodal graph learning, challenging the prevailing separation between topology and representation learning.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
multimodal graph learning, cross-modal message passing, graph refinement, graph neural networks

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_pJ1KOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ReCoG, a framework for multimodal graph learning that jointly refines graph topology and learns multimodal node representations through end-to-end reciprocal interaction. It consists of a multimodal graph refiner that uses cross-modal semantic information to correct existing edges and identify potential missing connections, and a coupled cross-modal message passing mechanism that enables intra- and inter-modal information propagation over the refined graph. The paper also provides a spectral analysis of the coupled propagation operator and derives a contraction bound under specific conditions. Experiments on nine benchmarks for node classification and link prediction report improvements over a range of existing baselines.

Critical strengths
  1. A new framework for multimodal graph learning. ReCoG jointly addresses graph topology refinement and multimodal representation learning, combining a multimodal graph refiner with coupled cross-modal message passing. This is a reasonable approach to handling noisy or incomplete graph structures while leveraging complementary modalities.

  2. Broad empirical evaluation across tasks and datasets. The paper evaluates ReCoG on nine benchmarks covering node classification and link prediction, with multiple feature encoder configurations. The reported improvements across a range of baselines suggest that the proposed framework has practical potential beyond a single dataset or task.

Critical weaknesses
  1. The novelty of the proposed framework appears limited. Components such as similarity-guided candidate selection, residual graph corrections, and cross-modal message passing closely parallel existing approaches in graph structure learning and multimodal fusion, making the overall methodological contribution somewhat incremental.

  2. The multimodal fusion method introduced in Section 3.3 is a node-adaptive scalar gating mechanism. Specifically, for each node ii, the textual feature zitz_i^t and visual feature zivz_i^v are concatenated and passed through an MLP followed by a Sigmoid function to derive a scalar gate gig_i. This gate then modulates the weighted sum of both modalities before applying Layer Normalization. This formulation is conceptually identical to the Gated Multimodal Unit (GMU) proposed by Arevalo et al. (Gated Multimodal Units for Information Fusion, ICLR 2017 Workshop). The authors didn't cite this baseline work or discuss how their proposed fusion mechanism differs from or improves upon such standard gated fusion units.

Questions for authors
  1. Could the authors explain why the multimodal graph refiner improves node classification and link prediction?

  2. What enables the proposed multimodal graph refiner to achieve better results compared to existing refinement/GSL approaches you mentioned in the Related Work, such as IDGL (Chen et al., 2020b), SLAPS (Fatemi et al., 2021), ProGNN (Jin et al., 2020), and NodeFormer (Wu et al., 2022a)? I also suggest including a discussion and comparison with recent relevant literature from 2025–2026 to better position the paper within the current state of the art.

Reviewer_j3vAOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the joint learning of graph structure and multimodal node representations, and proposes ReCoG to train multimodal graph refinement and cross-modal message passing end-to-end. Specifically, the graph refiner uses node text and image features to adjust existing edge weights and add semantic neighbors. Then, the text and image branches exchange both self and neighbor information at each propagation layer, followed by an adaptive fusion module for final prediction. The authors conduct node classification and link prediction experiments on nine datasets, and provide ablations and controlled studies to analyze the role of each module.

Critical strengths
  • The problem studied here is meaningful, and the method design fits the motivation well. The original edges in a graph do not always reflect the multimodal semantics of nodes, nor are they necessarily optimal for downstream tasks. Putting graph structure refinement and cross-modal propagation into the same training framework lets the connection weights and message passing adapt together toward the prediction target. This is a practical direction to explore.

  • The experimental coverage is broad and includes targeted controlled comparisons. The evaluation covers node classification, link prediction, multiple datasets, and different encoder setups, with competitive results. Table 4 checks the individual contributions of graph refinement and cross-modal propagation, while Appendix G compares different edge scoring ways and tests single-stream vs. two-stream propagation on the exact same refined graph. These experiments provide good support for the actual benefit of each module.

Critical weaknesses
  • The exact computation mechanism of "reciprocal co-evolution" still needs clearer explanation. Section 3.2 states that updated representations iteratively improve the topology, but the refiners in Eq. 2 and Eq. 5-8 mainly take the pretrained node features (x_i^t, x_i^v) as input, without clearly showing how the current GNN hidden representations are fed back into graph construction. While the current equations support optimizing the refiner and GNN jointly through the task loss, the connection between this and the iterative representation-topology feedback emphasized in the main text remains unclear. Proposition 2 in Appendix M shows that the joint optimization space includes frozen refiners as a special case, which also does not directly show this feedback process. This makes it hard for me to judge the mechanistic difference and novelty of "co-evolution" compared to standard end-to-end joint training.

  • The inclusion proof in Appendix L needs additional conditions or revised phrasing. The pairwise feature phi(i, j) in Eq. 5 does not contain the original connection state, whereas the target correction u(i, j) defined in Eq. 40 depends on A_orig(i, j). Therefore, when two node pairs have the exact same pairwise features but different original edge states, the target is not a single-valued function of the pairwise features, and the universal approximation argument cannot be applied directly.

    For example, suppose two node pairs both have text and image similarities of 0.5 and identical pairwise features phi(i, j), while their original edge weights in A_orig are 1 and 0, respectively. Taking alpha = 0.5, the target edge weights produced by the similarity refiner are 0.75 and 0.25, which correspond to required corrections of -0.25 and +0.25. This satisfies the correction magnitude bound when delta = 0.5, but a residual scorer that only takes the identical pairwise features as input cannot output both corrections at the same time. Thus, the current proof regarding the inclusion relationship between F_sim and F_res at least requires an extra separability assumption or a revision. This issue limits the completeness of the theoretical claim, though it does not directly negate the reported empirical gains.

Questions for authors
  • Does the graph refiner only use fixed pretrained features and update its parameters via task gradients, or does it also use GNN representations updated during training? Please clarify the exact order of computation and updates for each module, and explain how it relates to the claim in the main text that "updated representations improve topology".

  • How does Appendix L handle the case where two node pairs have the same pairwise features phi(i, j) but different original connection states A_orig(i, j)? What additional assumptions are needed for the inclusion conclusion to hold? Please clarify the function families being compared and fix the proof accordingly.

Reviewer_nZYXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents ReCoG, a multimodal graph learning framework that combines graph refinement with cross-modal message passing over frozen text and image features. It introduces similarity-based and residual edge refiners, maintains separate modality streams during propagation, and combines their outputs through a learned gate. Experiments cover node classification and link prediction on nine datasets, with improvements over the reported baselines and ablations examining the contributions of graph refinement and cross-modal propagation.

Critical strengths

S1. With the same CLIP features, ReCoG-res improves Amazon-Sports MRR from SAGE’s 33.83 to 38.57, showing a clear gain on link prediction (Table 5).

S2. The Fashion ablation drops from 88.60 to 87.09 without refinement and to 86.41 without cross-modal propagation, supporting the contribution of both components (Table 4).

Critical weaknesses

W1. Equations (2)–(8) refine topology from frozen encoder features without explicitly feeding back the updated node representations, leaving the claimed distinction between reciprocal co-evolution and ordinary joint training unclear.

W2. The experiments lack matched comparisons with graph structure learning methods such as IDGL and NodeFormer, which already learn topology together with node representations, making the added contribution difficult to judge.

Questions for authors

Q1. Where do updated node representations enter the refiner, and how does performance change when a trained refiner is frozen during subsequent GNN training?

Q2. How is the coarse projection (W_c) trained when its scores only determine hard TopK indices and the fine scorer uses the original features?

Q3. How does Proposition 1 handle pairs with identical (\phi(i,j)) but different (A_{\mathrm{orig}}(i,j)), which can require opposite residual corrections despite giving the MLP identical inputs?

Q4. Can you provide a comparison with IDGL or NodeFormer on a representative dataset using the same encoder features, data splits, and tuning budget?

Reviewer_2pwfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Summary

This paper proposes ReCoG, a multimodal graph learning framework that jointly optimizes graph structure refinement and cross-modal representation learning. The method consists of two main components: (1) a multimodal graph refiner that modifies the observed topology using textual and visual semantic features, with either a cosine-similarity-based or residual MLP-based implementation; and (2) a coupled cross-modal message passing mechanism that enables intra-modal aggregation and cross-modal information exchange at each GNN layer. The paper additionally provides theoretical analyses of the expressiveness and spectral properties of the proposed architecture. Experiments on nine multimodal graph datasets covering node classification and link prediction show that ReCoG achieves improvements over conventional GNNs and several recent multimodal graph learning methods.

Critical strengths

Strengths

  1. Strong empirical results across multiple benchmarks. The evaluation covers seven node classification datasets and two link prediction datasets, with various pretrained feature encoders. The improvements are particularly substantial on Goodreads-NC and Reddit-M.

  2. Useful component analyses. The paper includes ablations of graph refinement and cross-modal propagation, comparisons between different edge-scoring functions, and a controlled comparison between propagation architectures on the same graph. These experiments help demonstrate the value of the individual architectural components.

Critical weaknesses

Weaknesses

  1. The claimed reciprocal co-evolution mechanism is unclear.

    The central contribution of this paper is the reciprocal co-evolution between graph topology and multimodal representations. However, according to Eqs. (2), (5), and (7), the graph refiner appears to always use features extracted from frozen text and vision encoders to compute edge similarities or residual scores, rather than the updated node representations obtained through cross-modal message passing. While the graph refiner can be jointly optimized with the GNN through task gradients, it is unclear how the refined node representations directly influence subsequent graph refinement. This raises concerns about whether the proposed architecture actually implements the claimed reciprocal co-evolution between graph structure and node representations. The authors should clarify which specific design enables the updated representations to influence graph refinement and how this differs from conventional end-to-end graph structure learning.

  2. Missing comparisons with relevant graph structure learning methods.

    The paper discusses several established graph structure learning methods in the related work, including IDGL, SLAPS, Pro-GNN, and NodeFormer. However, none of these methods are included in the experimental comparisons. Since graph structure refinement is a central component of ReCoG, comparisons with these methods would be important to establish the effectiveness of the proposed approach. Even though these methods were originally designed for unimodal features, they could be adapted to multimodal settings using concatenated or projected multimodal embeddings. Furthermore, the concurrent work TMTE studies a closely related problem of topology-representation co-evolution for multimodal graphs. Although TMTE is discussed in the related work and briefly compared in Appendix K, its results should also be included in the main comparison (Table 3) on the overlapping datasets. This would provide a more complete evaluation against the most directly relevant existing approach.

Questions for authors

Questions

  1. In Eqs. (2), (5), and (7), the graph refiner appears to use the frozen pretrained text and vision features. How do the updated node representations from cross-modal message passing influence the graph refiner? Which specific component implements the claimed reciprocal co-evolution? Would a two-stage approach that trains or fixes the graph refiner separately achieve similar results?

  2. Why are the graph structure learning methods discussed in the related work (e.g., IDGL, SLAPS, Pro-GNN, and NodeFormer) not included as experimental baselines? Could the authors also include TMTE in Table 3 to provide a more comprehensive comparison?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.