acceptodds
Under review as a conference paper at ICLR 2027

Role-Aware Hierarchical Sub-Entity Alignment and Difficulty-Weighted Negative Sampling for Multi-Modal Knowledge Graph Completion

Abstract

Multi-modal knowledge graph completion (MMKGC) aims to infer missing facts by jointly exploiting structural, visual, and textual evidence. However, existing methods often suffer from two limitations: (i) coarse entity-level fusion overlooks relation- and role-specific correspondences between fine-grained multimodal sub-entities, and (ii) hard-negative training may amplify false-negative bias by heavily penalizing plausible but unobserved triples. To address these issues, we propose RHDN, a unified framework for role-aware multimodal representation learning and false-negative-aware training. RHDN first introduces a role-aware hierarchical alignment module that constructs relation- and role-conditioned optimal transport couplings between visual and textual sub-entities and aggregates coupling-weighted pairwise interactions with hierarchical role prototypes, yielding fine-grained relation-aware entity representations. On the training side, RHDN explicitly decouples negative difficulty from false-negative risk: model scores characterize hardness, while multimodal role-consistency evidence provides an independent risk signal. The two factors are combined through KL-regularized reweighting to preserve informative hard negatives while reducing the influence of high-risk candidates. Experiments on three widely used MMKGC benchmarks demonstrate consistent improvements over strong unimodal, multimodal, and negative-sampling baselines, with further analyses validating the contributions of the proposed components.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Multi-Modal Knowledge Graph Completion, Optimal Transport, Role-Aware Representation Learning, False-Negative-Aware Negative Sampling, Multimodal Alignment

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_DbdLOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RHDN addresses multimodal knowledge graph completion through two modules. R-HAlign combines relation- and role-conditioned optimal transport (OT) over visual and textual sub-entities with pairwise interactions and hierarchical role prototypes. FDNS reweights negatives using model-based difficulty and multimodal role-consistency evidence. A ConvE-style decoder scores triples. Experiments on DB15K, MKG-W, and MKG-Y report the highest MRR among the listed baselines, with component ablations and pseudo-false-negative diagnostics.

Critical strengths

S1. The non-separable pairwise aggregation in Eq. (11) makes the representation sensitive to the transport coupling. The Independent Coupling ablation provides a useful control by retaining the same marginals. S2. RHDN improves MRR over the strongest listed baseline by 0.59, 1.17, and 0.88 percentage points on the three benchmarks. Removing R-HAlign or FDNS consistently reduces performance. S3. The appendix documents feature extraction, missing-modality handling, relation clustering, and evaluation. The frozen-pool and score-matched diagnostics directly examine whether risk reweighting contributes beyond model difficulty.

Critical weaknesses

W1. FDNS does not condition its risk estimate on the other entity in the query (Eqs. 17-19). It therefore measures candidate role compatibility without distinguishing which specific triples are true. The score-matched diagnostic does not control for semantic type, leaving the claimed false-negative discrimination insufficiently established. W2. The diagnostic statistics need consistent normalization. In Fig. 5, the actor column sums to approximately 0.17, while its plotted marginal is 0.20. In Fig. 6, the two largest normalized weights sum to more than one. Table 5 does not specify how average weights are computed, and Appendix D.2 omits candidate-pool selection and score-matching counts. W3. Tables 1, 2, and 8 omit variability across seeds, despite reporting three-run averages. This matters for gains such as the 0.59-point DB15K MRR improvement. Baseline result sources, feature inputs, and tuning budgets are unspecified, and no matched RGAT/ConvE structural baseline is provided. W4. The contribution needs sharper positioning against MyGO and NativE, and against the omitted OTKGE (NeurIPS 2022) and ComCon (Information Fusion 2026). Existing ablations do not isolate the benefit of cluster-level sharing or compare OT with learned cross-attention. These comparisons are needed to identify what the proposed alignment and weighting add to prior MMKGC methods. W5. The claim of controlled computational overhead is unsupported by runtime or memory measurements. Appendix D.4 identifies the substantial pairwise-encoding cost but reports only asymptotic complexity. Full-entity ranking, representation caching, and head prediction under Eq. (15) also lack implementation details. W6. Appendix D.3.3 defines Relation Selectivity, Role Selectivity, and Prototype Purity@5 without reporting their values or label sources. The mapping from the implemented 4 by 4 patch grid and BERT tokens to the semantic regions and phrases in Fig. 5 and Tables 9-10 is also unspecified.

Questions for authors

Q1. (W1) Does FDNS distinguish pseudo-FNs from negatives matched by both score and semantic type? How does its risk estimate distinguish queries with different opposite-side entities? Q2. (W2) How are the weights in Table 5 and Fig. 6 normalized? What explains the marginal discrepancy in Fig. 5? Please report pool-selection rules and matched-pair counts. Q3. (W3) Which baselines were reproduced, using which features and tuning budgets? Please report per-seed results and a matched RGAT/ConvE structural baseline. Q4. (W4) What distinguishes RHDN from OTKGE and ComCon? How does it perform with cross-attention replacing OT and with relation-level prototypes alone? Q5. (W5) What are the training time, peak memory, and full-ranking time relative to strong baselines? How are representations cached and head queries evaluated? Q6. (W6) What are the selectivity and purity results, and where do their labels come from? How are tokens and patches mapped to the displayed phrases and regions?

4 comments
AuthorsOpenReview ↗

Response to Reviewer DbdL

Response to Reviewer DbdL

We thank the reviewer for the detailed technical assessment. We have expanded the controlled diagnostics, added architecture-matched comparisons, quantified runtime and memory, and made the interpretation of the OT visualizations more explicit. The additions are highlighted in blue in the revised paper.

Q1/W1. Score-, type-, and role-matched false-negative discrimination

The reviewer is correct that the evidence-based FDNS risk in Eqs. (17)–(19) depends on the corrupted entity, relation, and head/tail role, not explicitly on the opposite-side entity. The latter instead affects the complete-triple score si=f(hi,r,ti)s_i=f(h_i,r,t_i) and thus the difficulty component of qi∝exp⁡(si/τ−λfnri)q_i\propto\exp(s_i/\tau-\lambda_{\mathrm{fn}}r_i).

To test discrimination beyond score and coarse type, we extend the DB15K diagnostic from 240 score-matched pairs (∣Δs∣≤0.005|\Delta s|\leq0.005) to two nested subsets. Group A matches type and modality availability within the same pool and corruption direction (136 pairs); Group B additionally matches mean role support (∣Δaˉ∣≤0.01|\Delta\bar a|\leq0.01, 64 pairs). Full FDNS improves paired risk concordance over the mean-support-only estimator from 61.8% to 67.6% in Group A and 50.0% to 62.5% in Group B. For Group B, the mean paired risk gap is 0.0116, and the geometric-mean weight ratio is 0.9965. This supports additional risk ordering among closely matched candidates without equating role compatibility with triple-level truth.

Manuscript additions: Appendix D.2, Type- and role-matched FDNS diagnostic and Dependence on the opposite-side entity (Tables 13–14).

Q2/W2. Weight normalization, marginal mass, and pool statistics

Weight calculation. For every frozen candidate pool CbC_b, self-adversarial and FDNS weights are normalized over all valid candidates in that pool, not separately by category:

q_{bi}=\frac{e^{s_{bi}/\tau-\lambda_{\mathrm{fn}}r_{bi}}} {\sum_{j\in C_b}e^{s_{bj}/\tau-\lambda_{\mathrm{fn}}r_{bj}}}.$$ The weight reported for a category is the **mean of its candidate-occurrence weights after full-pool normalization**. Pseudo-FNs are identified by hidden-set membership; among remaining candidates, the highest and lowest 20% by score in each pool are designated hard and easy, with the middle group containing the remainder. The corresponding DB15K category means are: | Method | Pseudo-FN | Hard | Hard/FN | |---|---:|---:|---:| | Self-adversarial | 0.03790 | 0.03506 | 0.925 | | Score-only risk | 0.03582 | 0.03325 | 0.928 | | **FDNS** | **0.03341** | **0.03477** | **1.041** | **Pool selection and matching.** The 5% hiding protocol retains **75,261 pools** from 79,222 training triples, with **4,808,912 valid candidate occurrences**, **1,248 pseudo-FN occurrences**, **1,176 positive-hit pools (1.56%)**, and **240 one-to-one score-matched pairs (19.23% coverage)**. Pools are generated by standard corruption, without enrichment based on hidden-set membership. **Fig. 5.** The marginal refers to the **full grouped Sinkhorn plan**, including the aggregated *Other* region. In the Geoffrey Rush example, the named visual regions contribute **0.171** to the *actor* column and the remaining region contributes **0.029**, giving the textual marginal **0.200**. For a reduced visualization, grouping omitted entries preserves total mass. **Fig. 6.** Training weights likewise use the **full-pool denominator**. In the reported 64-candidate pool, the two highlighted highest-weight candidates have normalized weights **0.118** and **0.096**; the remaining 62 candidates account for **0.786**, so the complete distribution sums to one. Selecting candidates for display does not change their training normalization. **Manuscript additions:** **Appendix D.2 (Tables 9–10 and 12–14)** defines the normalization, category averages, and diagnostic coverage; **Appendix D.3.3 (Tables 22–24)** documents the grouped transport masses associated with Fig. 5. The full-pool weight interpretation is clarified here for Fig. 6.
AuthorsOpenReview ↗

Response to Reviewer DbdL

Response to Reviewer DbdL (continued: Part 2)

Q3/W3. Baseline controls and seed variability

We added a reproducibility-oriented comparison using validation-MRR model selection, a stated budget of 12 configurations per method and dataset, and seeds 42, 3407, and 2027 for the controlled reruns. The matched structure-only model uses RHDN's two-layer RGAT (four heads, d=200d=200) and the same ConvE-style decoder, taking only RGAT structural embeddings and using the same self-adversarial weighting as RHDN w/o FDNS. Data splits, negative-candidate budgets, validation selection, and filtered head/tail evaluation are held consistent.

DatasetRGAT/ConvERHDN w/o FDNSRHDNReproduced comparator
DB15K34.53 ± 0.3538.56 ± 0.2440.24 ± 0.26C²RS: 39.68 ± 0.34
MKG-W34.89 ± 0.2738.45 ± 0.2840.92 ± 0.27C²RS: 39.73 ± 0.21
MKG-Y36.67 ± 0.2638.80 ± 0.2840.38 ± 0.23SNAG: 39.49 ± 0.16

MRR in %, reported as mean ± sample SD over three seeds.

Relative to the matched structural model, the multimodal pipeline without FDNS improves MRR by 4.03/3.56/2.13 points; FDNS adds 1.68/2.47/1.58 points. Against the controlled competing reruns, full RHDN improves mean MRR by 0.56/1.19/0.89 points. On DB15K, the individual RHDN–C²RS differences are +1.10/−0.08/+0.65 points. These figures make the run-to-run variability visible alongside the average improvement. We also report seed-level MRR for four FDNS variants (e.g., 39.76 ± 0.18 without evidence disagreement versus 40.24 ± 0.26 for full RHDN).

Manuscript additions: Appendix C.2, Baseline Provenance and Seed Variability, with three-seed comparisons in Tables 6–7; the matched structure-only model is also included in Appendix D.4.1 (Table 25).

Q4/W4. Distinction from prior methods and targeted OT/prototype controls

OTKGE aligns modality-level distributions through OT; RHDN instead uses relation–role-conditioned region–token transport whose entries weight non-separable pairwise interactions (Eq. 11). This makes the learned representation sensitive to the correspondence pattern itself. Relative to MyGO's fine-grained entity representation and NativE's relation-adaptive modality fusion, R-HAlign explicitly combines sub-entity correspondence with head/tail prototypes. Compared with ComCon's complementary–contradictory regularization and weighted negative sampling, FDNS separately estimates score-based difficulty and multimodal role-consistency risk before KL-regularized weighting. The Related Work now discusses OTKGE and ComCon.

The targeted controls give the following MRR (%) comparisons:

DatasetRow attentionSinkhorn (both w/o OT loss)Relation-only prototypesFull RHDN
DB15K39.6439.8340.0340.24
MKG-W40.1340.2940.5340.92
MKG-Y39.6939.8040.1940.38

The parameter-matched cross-attention and Sinkhorn variants share the pairwise costs, marginals, aggregation, and parameter count, and both set λOT=0\lambda_{\mathrm{OT}}=0. Enforcing the second marginal increases MRR by 0.19/0.16/0.11 points. The relation-only control retains relation-specific head/tail prototypes, OT, and FDNS, while removing the cluster-prototype input and relation–cluster consistency loss. Restoring cluster-level sharing adds 0.21/0.39/0.19 points over relation-only prototypes. The original Independent Coupling control also shows 0.67/0.81/0.76-point MRR losses when internal transport correspondence is removed.

Manuscript additions: Sections 2.2–2.3 for prior-work positioning; Section 4.3 (Tables 2–3) and Appendix D.1.1 (Table 8) for the OT, attention, and prototype controls.

AuthorsOpenReview ↗

Response to Reviewer DbdL

Response to Reviewer DbdL

Q5/W5. Training cost, caching, and full-entity ranking

We measured training and full-test evaluation on a single NVIDIA RTX A6000 (48 GB) against C²RS on DB15K/MKG-W, SNAG on MKG-Y, and the matched RGAT/ConvE structure-only model. The table reports RHDN values alongside the relevant strong baseline:

Datasets/epoch (RHDN / baseline)Peak train GB (RHDN / baseline)Full-test s (RHDN / baseline)
DB15K322.8 / 228.434.56 / 19.82151.9 / 32.9
MKG-W151.2 / 108.327.96 / 15.3695.3 / 20.2
MKG-Y96.8 / 73.726.85 / 13.7463.1 / 17.4

Full-test time includes cache construction and filtered head/tail ranking. For RHDN, cache construction accounts for 121.8/76.3/49.2 seconds, while subsequent head-plus-tail ranking takes 30.1/19.0/13.9 seconds. Total training times for RHDN are 45.7/21.5/13.0 hours on the three datasets, respectively. This quantifies both the end-to-end cost and the specific cache-building bottleneck.

Caching and scoring. Frozen ViT/BERT region/token features are precomputed for training. At evaluation, representations are cached one relation at a time: Qr\mathbf Q_r stores ConvE query vectors for candidate heads, and Zrtail\mathbf Z_r^{\mathrm{tail}} stores tail-role embeddings. For (?,r,t)(?,r,t), head scores are Qrzt,rtail\mathbf Q_r\mathbf z^{\mathrm{tail}}_{t,r}; for (h,r,?)(h,r,?), tail scores are Zrtailch,r\mathbf Z_r^{\mathrm{tail}}\mathbf c_{h,r}. Chunking changes memory use, not Eq. (15); the evaluator filters other known true completions while retaining the target. Batched and direct candidate-triple scoring yield identical filtered ranks under the same tie rule.

Manuscript additions: Appendix D.4.1, Runtime, Memory, and Full-Entity Ranking, including the profiling protocol, caching/scoring equations, and Table 25. The isolated FDNS training increment is reported separately in Appendix D.2 (Table 16).

Q6/W6. Selectivity, type labels, and token/region traceability

We report the numerical diagnostics on the DB15K validation set using 1,500 relation pairs, 1,800 role pairs, and 416 eligible prototypes. Relation Selectivity holds entity/input/role fixed while varying relation; Role Selectivity holds entity/input/relation fixed while varying the head/tail role. Both measure mean base-2 Jensen–Shannon divergence of visual and textual attention marginals and require no type labels.

VariantRelation SelectivityRole SelectivityPurity@5
w/o Role Prototypes0.0730.046—
Row attention (w/o OT loss)0.0940.06979.76%
Sinkhorn (w/o OT loss)0.0980.07380.48%
RHDN Full0.1030.07881.35%

Purity@5 uses entity-type assertions from a fixed source-KG snapshot, matched to DB15K entity IDs and mapped to coarse categories (e.g., person, work, organization, place, event). Ambiguous or missing types are excluded. Each relation–role slot receives its expected type from labeled training triples; the nearest five validation entities are then checked against that type. Full RHDN achieves 1,692 matching neighbors out of 2,080 retrieved occurrences.

The displayed phrases and image-region names are post-hoc labels, not additional model inputs. Phrases map to character spans and retained, non-special BERT subword indices; images use 16 pooled ViT cells in a 4×44\times4 row-major grid. For Geoffrey Rush, for instance, actor maps to token 5, Academy Award to tokens 17–18, Face to cells P2,P6,P7P_2,P_6,P_7, and Trophy to P12,P16P_{12},P_{16}. Displayed region–phrase weights sum the corresponding cells of the full transport plan, retaining all remaining mass in Other. Thus, the Geoffrey Rush actor column sums to 0.171 + 0.029 = 0.200, matching its textual marginal.

Manuscript additions: Appendix D.3.3, Dataset-Level Validation, including Table 21 (diagnostics), Table 22 (token/grid index mapping), and Tables 23–24 (grouped transport plans corresponding to Fig. 5 and the qualitative examples).

We appreciate the reviewer's careful technical suggestions, which have motivated more precise mechanism diagnostics and evaluation reporting.

Dear Reviewer,

Thank you again for your thoughtful review. We have responded to your comments and revised the manuscript accordingly.

As the discussion period is approaching its end, we would be grateful if you could review our response when convenient. If any concerns remain, we would be happy to clarify them.

Best regards,
The Authors

Reviewer_7auiOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes RHDN, a framework for multi-modal knowledge graph completion that addresses two challenges: fine-grained cross-modal alignment and false-negative bias in negative sampling. The proposed R-HAlign module employs role-aware hierarchical prototypes and Optimal Transport to capture relation-specific interactions between visual regions and textual tokens. The FDNS module estimates negative sample difficulty and false-negative risk separately, and combines them through KL-regularized reweighting to reduce potentially misleading supervision. Experiments on three benchmark datasets demonstrate competitive performance, supported by ablation studies and additional analyses of the proposed mechanisms. The work is significant and useful.

Critical strengths
  1. The paper addresses two relevant challenges in multi-modal knowledge graph completion. The combination of role-aware fine-grained alignment and false-negative-aware negative sampling is interesting and provides a meaningful extension to existing methods. In particular, separating negative sample difficulty from false-negative risk is a reasonable and potentially useful idea.
  2. The authors evaluate RHDN on three benchmark datasets against a broad range of baselines. The experimental results demonstrate consistent improvements in MRR, and the ablation studies provide evidence for the contributions of the proposed components. The additional analyses of OT-based alignment and false-negative detection also help explain the behavior of the proposed methods.
Critical weaknesses
  1. R-HAlign uses fixed-grid visual regions, which may not capture complete semantic objects. It remains unclear whether these regions can be accurately aligned with textual information through OT. Further justification of the alignment quality and the advantages of OT over alternative methods would strengthen the paper.
  2. Although FDNS effectively identifies potential false negatives, their prevalence and impact on model performance in realistic training settings remain unclear. Further analysis of the actual performance gains relative to the computational overhead would better justify the necessity of FDNS.
Questions for authors

None

1 comment
AuthorsOpenReview ↗

Response to Reviewer 7aui

Response to Reviewer 7aui

We thank the reviewer for the positive assessment and thoughtful suggestions. We added direct grounding measurements and a separate analysis of FDNS prevalence, effectiveness, and computational cost.

W1. Fixed-grid grounding and the benefit of OT

RHDN extracts frozen ViT-B/16 features and pools them into a 4×44\times4 grid of 16 regional representations. The soft, many-to-many coupling allows a textual phrase to align with multiple grid cells, including when a semantic object spans cell boundaries. To assess the quality of these alignments, we annotated 240 DB15K validation phrases, of which 192 were visually groundable, and evaluated row-normalized attention and Sinkhorn on the same phrases and regions.

AlignmentRegion Hits@1Region Hits@3
Row attention47.92% (92/192)73.96% (142/192)
Sinkhorn OT54.17% (104/192)78.65% (151/192)

Sinkhorn improves region grounding by 6.25 points at Hits@1 and 4.69 points at Hits@3. These measurements complement the link-prediction controls by evaluating the phrase-to-region correspondence itself.

Manuscript additions: Appendix D.3.2, paragraph Grid-cell grounding (Table 20), and Appendix D.3.3, which specifies the token-to-phrase, grid-to-region, and grouped-OT mappings (Tables 22–24).

W2. FDNS prevalence, performance gain, and overhead

We audited standard-split candidate pools, labeling candidates that match known validation/test positives only after pool generation. The identifiable held-out-positive occurrence rates on DB15K/MKG-W/MKG-Y are 0.0564%/0.0521%/0.0596%; the proportions of pools containing at least one such candidate are 3.36%/3.15%/3.62%. On the same frozen pools, the mean weight share assigned to these candidates decreases under FDNS from 0.285% to 0.249%, 0.251% to 0.207%, and 0.312% to 0.258%, respectively.

With the same encoder, batch size, negative-candidate budget, hardware, and precision, FDNS adds only 3.46%/3.56%/2.98% to per-epoch training time and 0.16/0.16/0.15 GB to peak GPU memory. The corresponding MRR improvements over self-adversarial weighting are +1.68/+2.47/+1.58 percentage points. Thus, the measured incremental FDNS cost is small relative to the observed ranking gains, while its weighting shifts mass away from identifiable held-out positives.

Manuscript additions: Appendix D.2, FDNS training overhead (Table 16) and Audit on the standard splits (Table 17); end-to-end profiling of the full framework is additionally given in Appendix D.4.1 (Table 25).

Reviewer_BWPxOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper addresses two key challenges in Multimodal Knowledge Graph Completion (MMKGC): insufficient fine-grained, relation-aware cross-modal alignment and false-negative bias in hard negative sampling. To tackle these issues, the authors propose RHDN, a unified framework consisting of two components. First, Role-aware Hierarchical Sub-Entity Alignment (R-HAlign) employs optimal transport to align visual regions and textual tokens under relation- and role-specific guidance, incorporating hierarchical role prototypes to enhance multimodal representations. Second, False-Negative-Aware Difficulty-Weighted Negative Sampling (FDNS) separately estimates negative sample difficulty and false-negative risk, combining them through KL-regularized reweighting to preserve informative hard negatives while suppressing potentially false negatives. Experiments on three MMKGC benchmarks (DB15K, MKG-W, and MKG-Y) demonstrate that RHDN consistently outperforms competitive baselines, with ablation studies supporting the effectiveness of both components.

Critical strengths

S1. The paper clearly identifies two important challenges in MMKGC: relation-aware multimodal evidence selection and false-negative bias, with corresponding solutions.

S2. R-HAlign employs relation- and role-conditioned optimal transport to capture fine-grained token–region correspondences beyond coarse multimodal fusion.

S3. Experiments on three MMKGC benchmarks, together with extensive ablations and controlled false-negative analyses, provide empirical support for the proposed components.

Critical weaknesses

W1. The proposed framework builds on established techniques, including OT-based alignment (OTKGE [1]), fine-grained multimodal representations (MyGO [2]), and false-negative-aware sampling (ComCon [3]). The paper does not sufficiently distinguish its contributions from these closely related methods, and OTKGE and ComCon are missing from the related work.

W2. FDNS estimates false-negative risk based on the corrupted entity's similarity to relation-role prototypes, without considering the other entity in the triple. This captures semantic compatibility rather than triple-level correctness, potentially limiting its ability to distinguish false negatives from valid hard negatives.

W3. Although Appendix D.2 presents promising results, it does not report pseudo-FN counts and prevalence in candidate pools or the coverage of score-matched pairs. These missing statistics, together with the acknowledged uncertainty in negative labels, make it difficult to fully assess the reliability and representativeness of the results.

W4. Although the paper includes OT-specific ablations, it lacks comparisons with simpler alternatives, such as parameter-matched cross-attention or learned pairwise weighting. Additionally, Appendix D.3.3 introduces relation- and role-selectivity metrics without reporting their numerical results.

[1] OTKGE: Multi-modal Knowledge Graph Embeddings via Optimal Transport, NeurIPS 2022.

[2] Tokenization, Fusion, and Augmentation: Towards Fine-grained Multi-modal Entity Representation, AAAI 2025.

[3] ComCon: Complementary-contradictory regularization for multimodal knowledge graph completion, Information Fusion 2026.

Questions for authors
  1. How does R-HAlign fundamentally differ from OTKGE beyond applying relation- and role-conditioned transport to finer-grained features? Similarly, what distinguishes FDNS from existing false-negative-aware weighting methods such as ComCon?

  2. Since Eqs. (17)–(19) use only the corrupted entity and its expected relation role, how can FDNS distinguish a true missing triple from an incorrect triple involving an equally role-compatible entity? Can the existing diagnostics be further controlled for entity type and relation-role compatibility?

  3. How many pseudo-false negatives occur under the 5% hiding protocol, what proportion of candidate pools contain them, and how many score-matched pairs satisfy Eq. (28)? Were any candidate pools enriched or selected specifically for the diagnostic experiments?

  4. Can the authors clarify whether the improvements originate from the optimal transport constraints rather than general learned cross-modal pairwise attention? Additionally, what are the numerical Relation Selectivity, Role Selectivity, and Prototype Purity@5 results introduced in Appendix D.3.3?

3 comments
AuthorsOpenReview ↗

Response to Reviewer BWPx

Response to Reviewer BWPx

We thank the reviewer for the careful assessment and constructive suggestions. We have strengthened the methodological positioning and added targeted controls for the false-negative risk estimator and the role-aware alignment mechanism. The corresponding additions are highlighted in blue in the revised manuscript.

W1/Q1. Relationship to OTKGE, MyGO, and ComCon

Distinction of R-HAlign. OTKGE uses OT to address discrepancies among modality-specific embedding distributions, whereas R-HAlign constructs relation- and head/tail-role-conditioned transport between visual regions and textual tokens. More importantly, each transport entry weights a learned non-separable pairwise interaction (Eq. 11), including element-wise products. Consequently, changing the coupling can change the representation even when both marginals remain unchanged. In the Independent Coupling control, replacing the OT plan with the outer product of its marginals decreases MRR by 0.67/0.81/0.76 points on DB15K/MKG-W/MKG-Y. Unlike MyGO's fine-grained entity encoding, R-HAlign additionally conditions the selection and matching of sub-entity evidence on the query relation and semantic role. NativE provides relation-adaptive multimodal fusion, while R-HAlign explicitly models region–token correspondence and hierarchical head/tail prototypes.

Distinction of FDNS. ComCon uses complementary–contradictory regularization and weighted negative sampling to address multimodal discrepancies and suspected false negatives. FDNS instead estimates difficulty from the complete-triple score and a separate risk proxy from multimodal relation–role evidence, combining them through KL-regularized reweighting (Eq. 21). With the same weighting rule, replacing the evidence-based risk by a score-only proxy decreases DB15K MRR from 40.24 to 39.31.

Manuscript additions: Sections 2.2–2.3 (Related Work) now discuss OTKGE and ComCon; Sections 3.4–3.5 define the coupling-sensitive representation and FDNS; further mechanism controls appear in Table 2, Appendix D.1.1 (Table 8), and Appendix D.2 (Table 15).

W2/Q2. False-negative discrimination under tighter matching

We agree that Eqs. (17)–(19) estimate relation–role compatibility rather than independently verifying triple truth. Holding the corrupted entity, relation, and role fixed leaves the evidence-based risk unchanged when the opposite entity changes; the complete-triple score still changes the difficulty component of the final weight.

We therefore extend the DB15K 5% hidden-triple diagnostic beyond score matching. Starting from 240 one-to-one pairs with ∣Δs∣≤0.005|\Delta s|\leq 0.005, Group A further matches entity type, modality availability, pool, and corruption side (136 pairs). Group B additionally matches average multimodal role support with ∣Δaˉ∣≤0.01|\Delta\bar a|\leq 0.01 (64 pairs). Full FDNS versus the mean-support-only control (β=0\beta=0) achieves 67.6% vs. 61.8% pairwise risk concordance in Group A, and 62.5% vs. 50.0% in Group B. The full-FDNS mean risk gap in Group B is 0.0116, with a geometric-mean within-pool weight ratio of 0.9965. These results show additional, albeit modest, discrimination from cross-modal disagreement after controlling for score and semantic compatibility.

Manuscript additions: Appendix D.2, paragraphs Type- and role-matched FDNS diagnostic and Dependence on the opposite-side entity, with matching coverage and risk results in Tables 13–14.

AuthorsOpenReview ↗

Response to Reviewer BWPx (continued: Part 2)

W3/Q3. Pseudo-FN prevalence and matched-pair coverage

We now report a complete candidate-pool audit for the DB15K 5% hiding protocol. From 79,222 training triples, 3,961 are hidden, leaving 75,261 retained triples and candidate pools. With 64 corruption draws per pool, within-pool deduplication, and filtering of observed training positives, there are 4,808,912 valid candidate occurrences. Among these, 1,248 are identifiable pseudo-FN occurrences corresponding to 712 distinct hidden triples. They appear in 1,176/75,261 pools (1.56%), with a candidate-level prevalence of 0.0260%. The 240 one-to-one score-matched pairs cover 19.23% of pseudo-FN occurrences.

The pools are generated by standard corruption before hidden-set lookup. No hidden triples are inserted or oversampled, and no pools are regenerated according to pseudo-FN membership. Matching uses only the eligible candidates already present in those pools.

Manuscript additions: Appendix D.2, Candidate-pool prevalence and score-match coverage (Table 9) and Exact-score-matched analysis (Table 12).

W4/Q4. OT constraints, grounding, and selectivity

To distinguish two-marginal OT from ordinary attention, we evaluate a parameter-matched row-normalized cross-attention variant using identical features, pairwise costs, role-conditioned marginals, and pairwise aggregation. Both variants set λOT=0\lambda_{\mathrm{OT}}=0. Row attention preserves the visual marginal and uses the textual marginal as a prior; Sinkhorn enforces both. Sinkhorn improves MRR by 0.19/0.16/0.11 points on DB15K/MKG-W/MKG-Y, respectively. This complements the Independent Coupling ablation by isolating the benefit of the second marginal constraint.

We also report the previously defined DB15K diagnostics on 1,500 relation pairs, 1,800 role pairs, and 416 eligible prototypes. Full RHDN obtains Relation Selectivity 0.103, Role Selectivity 0.078, and Prototype Purity@5 81.35%, versus 0.094/0.069/79.76% for row attention and 0.098/0.073/80.48% for Sinkhorn without the auxiliary OT loss. These results quantify relation/role sensitivity and prototype coherence.

Manuscript additions: Section 4.3 (Table 3) and Appendix D.1.1 (Table 8) present the attention control; Appendix D.3.3 (Table 21) reports the selectivity and purity results.

The authors’ response has partially addressed my concern. I will therefore maintain my original score.

Comments on the paper

1 comment

Dear Reviewers,

There is still time to review the authors' rebuttal and indicate whether your concerns have been addressed. If any concerns remain, please raise them in the discussion as soon as possible.

Thank you.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.