- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper investigates when to read out multimodal evidence and when to introduce relational guidance in multimodal knowledge graph completion. It proposes REEL, which implements the relation-early and evidence-late principles by introducing relation information during representation learning and retaining token-level evidence until late readout at the scoring stage. The model achieves the highest reported mean MRR on each of the three benchmarks, with additional analyses covering ablations, attention patterns, and input perturbations.
Critical strengths
S1. The timing of evidence readout and the stage at which relation information is injected are well-defined and worthwhile research questions. They directly affect whether the model can draw on different evidence for the same entity under different relations. S2. REEL achieves the highest mean MRR in Table 2 on all three benchmarks and the best results on all four metrics on MKG-Y, demonstrating competitive link prediction performance. S3. The experiments include component and relation-information ablations, modality masking, and an analysis of attention-weight stability across runs. Comparing the full REEL model in Table 2 with the "w/o Rel." variant in Table 3, removing relation information from both RGCI and RCTR reduces MRR on DB15K by 3.18 percentage points, providing empirical support for the relation-early principle on that dataset.
Critical weaknesses
W1. The contribution of late readout has not been isolated. Table 1 keeps the encoder configuration matched but compares "[CLS] + a conventional KGC scorer" with "the retained token sequence + FacetE." Although the difference in evidence granularity is inherent to the two readout strategies, the comparison also changes the scoring mechanism. Thus, Table 1 supports the performance advantage of the combined approach, but does not isolate the contribution of late readout from that of the specific FacetE design. W2. The claim of consistent gains contradicts the reported results. The scoring-function ablation claims that REEL with alternative scorers consistently outperforms the corresponding Origin variants. However, Tables 1 and 4 show that, on DB15K, ComplEx drops from 37.57% to 34.32%, RotatE from 37.23% to 35.32%, and TuckER from 35.72% to 35.05%. These decreases directly contradict the claim of consistent improvements. W3. The incremental benefit of the proposed modules over existing conditioning approaches is unclear. The relation-conditioned scaling and shifting in Equation (7) are a form of conditional affine modulation akin to FiLM. The paper discusses the link-aware visual fusion used in LAFA but does not provide an empirical comparison. It also does not specify how the "w/o Rel." variants in Table 3 handle the gating, scaling, and shifting terms, or whether relation-independent learnable parameters are retained. Without simpler relation-conditioning baselines with comparable parameter counts and identical training configurations, the results do not establish how much additional benefit comes from the specific designs of RGCI and RCTR. W4. The efficiency measurements do not reflect the training configuration used for the main results. Appendix C specifies 64 negative samples per positive example, whereas Appendix D explicitly states that Table 6 excludes negative sampling and measures only the base architectural cost. Table 6 therefore does not directly reflect the full training cost of obtaining the main results on DB15K. It also lacks an efficiency comparison between the Origin variants with early and late readout, leaving the overhead attributable to late readout itself unclear. W5. The noise experiments are underspecified and do not establish superior noise robustness. Figure 6 evaluates only MKG-W and reports the probability of applying noise, but does not specify the Gaussian noise standard deviation, its scale relative to the features, or the stage at which it is injected. The modality-missingness sampling and retraining protocols are also unclear. The noise-response curves remain nearly flat for all three methods, providing limited evidence of differences in sensitivity to noise under the evaluated setting. Since REEL already performs better without perturbations, its higher absolute MRR under noise does not by itself establish greater robustness.
Questions for authors
Q1 (W1). With the inputs, encoder architecture, training configuration, relation-conditioning setting, and final candidate-scoring formulation held fixed, does late readout still outperform [CLS]-based early readout? Q2 (W2). How do you explain the substantial degradation of some scorers relative to Origin in Table 4, and how should the claim of consistent improvements be revised? Q3 (W3). How are the gating, scaling, and shifting terms handled in the "w/o Rel." variants? Are learnable, relation-independent parameters retained? With comparable parameter counts and identical training configurations, how much do RGCI and RCTR improve over simpler relation-conditioning approaches? Q4 (W4). What are the actual runtime and GPU memory requirements of the complete training procedure used to obtain the results in Table 2? How much additional cost does it incur relative to the Origin variants with early and late readout? Q5 (W5). What Gaussian noise standard deviation is used, at what stage is noise injected, and what does the reported noise probability apply to (e.g., entities, modalities, tokens, or feature dimensions)? How are missing modalities sampled, and are the models retrained? When the noise magnitude increases at a fixed application probability, do the three methods show consistent differences in degradation relative to their own clean-input performance?