acceptodds
Under review as a conference paper at ICLR 2027

Evidence Late, Relation Early: On When to Aggregate and Condition in Multimodal Knowledge Graph Completion

Abstract

Multimodal knowledge graph completion (MMKGC) leverages multimodal information to infer missing facts in incomplete knowledge graphs. Most existing methods construct entity representations before triple scoring, leaving two fundamental design questions underexplored: *when to read out multimodal evidence* and *at what stage should relational guidance interact with multimodal evidence*. We revisit MMKGC from this perspective and identify two complementary principles: **evidence-late**, which defers the compression of fine-grained evidence to the scoring stage, and **relation-early**, which incorporates relational information during representation learning to emphasize relation-relevant evidence. We first instantiate the evidence-late principle in **Origin**, enabling a controlled comparison with early evidence compression and showing that preserving fine-grained evidence until scoring yields better performance than premature compression into a fixed vector. Building on this observation, we propose **REEL**, a lightweight framework that operationalizes both principles in a unified design. Experiments on three MMKGC benchmarks show that REEL achieves strong completion performance, while ablation studies and case analyses further validate the effectiveness of both design principles. Our code is available at https://anonymous.4open.science/r/reel-1955.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multimodal knowledge graph completion, MMKGC

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XDoEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates when to read out multimodal evidence and when to introduce relational guidance in multimodal knowledge graph completion. It proposes REEL, which implements the relation-early and evidence-late principles by introducing relation information during representation learning and retaining token-level evidence until late readout at the scoring stage. The model achieves the highest reported mean MRR on each of the three benchmarks, with additional analyses covering ablations, attention patterns, and input perturbations.

Critical strengths

S1. The timing of evidence readout and the stage at which relation information is injected are well-defined and worthwhile research questions. They directly affect whether the model can draw on different evidence for the same entity under different relations. S2. REEL achieves the highest mean MRR in Table 2 on all three benchmarks and the best results on all four metrics on MKG-Y, demonstrating competitive link prediction performance. S3. The experiments include component and relation-information ablations, modality masking, and an analysis of attention-weight stability across runs. Comparing the full REEL model in Table 2 with the "w/o Rel." variant in Table 3, removing relation information from both RGCI and RCTR reduces MRR on DB15K by 3.18 percentage points, providing empirical support for the relation-early principle on that dataset.

Critical weaknesses

W1. The contribution of late readout has not been isolated. Table 1 keeps the encoder configuration matched but compares "[CLS] + a conventional KGC scorer" with "the retained token sequence + FacetE." Although the difference in evidence granularity is inherent to the two readout strategies, the comparison also changes the scoring mechanism. Thus, Table 1 supports the performance advantage of the combined approach, but does not isolate the contribution of late readout from that of the specific FacetE design. W2. The claim of consistent gains contradicts the reported results. The scoring-function ablation claims that REEL with alternative scorers consistently outperforms the corresponding Origin variants. However, Tables 1 and 4 show that, on DB15K, ComplEx drops from 37.57% to 34.32%, RotatE from 37.23% to 35.32%, and TuckER from 35.72% to 35.05%. These decreases directly contradict the claim of consistent improvements. W3. The incremental benefit of the proposed modules over existing conditioning approaches is unclear. The relation-conditioned scaling and shifting in Equation (7) are a form of conditional affine modulation akin to FiLM. The paper discusses the link-aware visual fusion used in LAFA but does not provide an empirical comparison. It also does not specify how the "w/o Rel." variants in Table 3 handle the gating, scaling, and shifting terms, or whether relation-independent learnable parameters are retained. Without simpler relation-conditioning baselines with comparable parameter counts and identical training configurations, the results do not establish how much additional benefit comes from the specific designs of RGCI and RCTR. W4. The efficiency measurements do not reflect the training configuration used for the main results. Appendix C specifies 64 negative samples per positive example, whereas Appendix D explicitly states that Table 6 excludes negative sampling and measures only the base architectural cost. Table 6 therefore does not directly reflect the full training cost of obtaining the main results on DB15K. It also lacks an efficiency comparison between the Origin variants with early and late readout, leaving the overhead attributable to late readout itself unclear. W5. The noise experiments are underspecified and do not establish superior noise robustness. Figure 6 evaluates only MKG-W and reports the probability of applying noise, but does not specify the Gaussian noise standard deviation, its scale relative to the features, or the stage at which it is injected. The modality-missingness sampling and retraining protocols are also unclear. The noise-response curves remain nearly flat for all three methods, providing limited evidence of differences in sensitivity to noise under the evaluated setting. Since REEL already performs better without perturbations, its higher absolute MRR under noise does not by itself establish greater robustness.

Questions for authors

Q1 (W1). With the inputs, encoder architecture, training configuration, relation-conditioning setting, and final candidate-scoring formulation held fixed, does late readout still outperform [CLS]-based early readout? Q2 (W2). How do you explain the substantial degradation of some scorers relative to Origin in Table 4, and how should the claim of consistent improvements be revised? Q3 (W3). How are the gating, scaling, and shifting terms handled in the "w/o Rel." variants? Are learnable, relation-independent parameters retained? With comparable parameter counts and identical training configurations, how much do RGCI and RCTR improve over simpler relation-conditioning approaches? Q4 (W4). What are the actual runtime and GPU memory requirements of the complete training procedure used to obtain the results in Table 2? How much additional cost does it incur relative to the Origin variants with early and late readout? Q5 (W5). What Gaussian noise standard deviation is used, at what stage is noise injected, and what does the reported noise probability apply to (e.g., entities, modalities, tokens, or feature dimensions)? How are missing modalities sampled, and are the models retrained? When the noise magnitude increases at a fixed application probability, do the three methods show consistent differences in degradation relative to their own clean-input performance?

Reviewer_wmoYOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes REEL based on the principles of Evidence-Late and Relation-Early. It introduces relation-gated cross-modal interaction and relation-conditioned token refinement to incorporate relational information into multimodal representation learning. A relation-aware scoring mechanism, FacetE, is further employed to adaptively aggregate fine-grained multimodal evidence for missing entity prediction.

Critical strengths

The paper provides an interesting perspective on when to incorporate relational information and aggregate multimodal evidence, supported by strong results across several benchmarks.

Critical weaknesses
  • The motivation and novelty are insufficiently justified. Existing methods, such as MoMoK, have already incorporated relational information into modality representation learning, which overlaps with the proposed Relation-Early principle. The authors should better clarify the differences and novelty. More importantly, Section 3 mainly examines the fusion and aggregation of modality-level representations, rather than the preservation of fine-grained evidence within individual modalities. These experiments do not directly support the motivation that retaining fine-grained evidence enables better relation-specific selection. For instance, Lines 232–234 explicitly emphasize identifying relevant anatomical regions within an image, but the proposed experiments and method do not demonstrate such capability.
  • The experimental comparison in Section 3 does not fully isolate the claimed contribution. The Early-Readout variants use conventional scoring functions, whereas Late-Readout uses the proposed FacetE scorer. Consequently, performance improvements may result from differences in scoring architectures rather than delayed evidence aggregation itself. Furthermore, the modality masking experiment (Fig. 2) only evaluates the relative importance of entire modalities, without examining whether the model can identify important regions or tokens within a specific modality. This weakens the evidence supporting the proposed fine-grained evidence preservation principle.
  • The methodology is difficult to follow, and several important details are unclear. In Eq. (6), is XeX_e the same multimodal representation defined in Eq. (4)? Since XeX_e already contains xemx_e^m, using it as both keys and values in MHA may allow the attention mechanism to focus primarily on the original modality, potentially weakening the intended cross-modal interaction. The authors should clarify this design and analyze self-modal versus cross-modal attention contributions. Moreover, important implementation details are missing, such as the architectures and output dimensions of gm(⋅)g^m(\cdot), as well as how these relation-conditioned transformations are applied across tokens and layers.
  • The effectiveness of relation-guided fine-grained evidence selection needs stronger validation. Additional token-level or region-level analyses, such as visualizing relation-conditioned attention or masking relevant and irrelevant image regions, would better substantiate.
  • There are inconsistencies in the notation. In particular, nn has different meanings in Figure 3 and Section 3.1: the former refers to the number of candidate entities, whereas the latter denotes the number of evidence tokens. The notation should be made consistent or clearly distinguished throughout the paper.
Questions for authors

see Weakness.

Reviewer_69WLOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The manuscript raises an important question in the field of multimodal knowledge graph completion, which is when to introduce evidence. The manuscript suggests that evidence should be aggregated at a later stage to avoid premature aggregation and loss of multimodal information, while introducing relational information at an earlier stage to fully preserve the information required for inferring entities. Based on this viewpoint, the manuscript proposes the REEL model, which consists of three modules: RGCI, RCTR, and FacetE. These three modules are responsible for the interaction between multimodal information, introducing and integrating relational information, and finally generating ratings based on multiple independently retained modality specific information representations. The manuscript validated the proposed REEL model, surpassing all the baselines compared. In addition, the manuscript also separately verified the viewpoint of introducing evidence information later. The sub module components were also subjected to ablation experiments to verify their effectiveness.

Critical strengths

The manuscript is well-organized and easy to understand. In addition, the mechanism of multimodal information fusion should be very important, and currently most of the work in the field of multimodal knowledge graph completion focuses on how to fuse multiple types of information, rather than exploring the best fusion timing. Therefore, I believe that the research content of the manuscript is valuable in this regard. The manuscript also underwent comprehensive experimental verification of the proposed viewpoint, achieving optimal performance compared to existing methods.

Critical weaknesses
  1. The core point of the manuscript is that the late aggregation of multimodal evidence is beneficial for the completion of knowledge graph information. In order to evaluate this viewpoint, the author also conducted separate experiments for evaluation, but based on the experimental results, the performance improvement was relatively limited compared to the early aggregation case, almost on par with the best baseline for comparison.

  2. In addition, the manuscript validates the viewpoint of early introduction of relationships by examining the modal composition influenced by relational information. Can we evaluate the benefits directly through the final task benefits, similar to verifying the benefits brought by late aggregation.

Questions for authors
  1. Although late stage evidence aggregation achieved the best performance in Table 1, the improvement was relatively limited compared to the strongest early readout baseline. Do these improvements have statistical significance in independent operation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.