acceptodds
Under review as a conference paper at ICLR 2027

PRISM: A Unified Heterogeneous-Expert Agent Framework for Knowledge Graph Completion

Abstract

Knowledge graph completion (KGC) aims to infer missing facts and alleviate the inherent incompleteness of knowledge graphs. Existing KGC methods capture complementary information at different levels, including structural patterns, path and rule information, and local or global graph semantics, while multimodal KGC further incorporates textual and visual modalities. However, heterogeneous models differ in prediction reliability, output scales, and inference costs, and simply combining them cannot adaptively determine which models should be invoked or when further computation is necessary. We formulate KGC as an information-driven sequential decision problem and propose PRISM, a unified framework for coordinating heterogeneous reasoning models. PRISM standardizes model outputs and maintains a continuously updated shared state. The Profiler summarizes relation, entity, and modality characteristics; the Adapter normalizes model outputs; the Verifier evaluates structural support and prediction agreement; the Judge performs reliability-aware rank fusion; and the Router estimates the utility of additional model invocation and decides whether to Call or Stop. Together, these components form a traceable Call–Verify–Stop loop for sequential model selection and stopping. On the MKG-W dataset, PRISM reduces the average number of model invocations from 6 to 2.09 and lowers the configured inference cost by approximately 67.10%, while largely preserving the performance of the full-model ensemble. These results demonstrate that PRISM can effectively coordinate heterogeneous reasoning models while substantially reducing redundant computation. Our code and data are available at https://anonymous.4open.science/r/PRISM-C687

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Multimodal Knowledge Graph Completion, Model Routing, Adaptive Computation.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1jwuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper is about knowledge graph completion, which is an interesting topic. The authors formulate KGC as an information-driven sequential decision problem and propose PRISM, a unified framework for coordinating heterogeneous reasoning models. The paper is well written and well organized. However, there are several concerns in the current version of the paper that addressing them will increase the quality of this paper.

Critical strengths

2 The improvements resulting from the experiment are exciting, as they effectively address existing problems.

3 The paper’s code has been made open source, which helps facilitate the dissemination of the paper and its underlying ideas.

Critical weaknesses

1 As it stands, the paper appears to offer a formalized framework; however, it is worth clarifying whether these various components have already been implemented in existing work—or if they have simply been given new names.

2 Figure 2 could be further improved; while the current structure is clear, its visual presentation could be refined—in particular, the text color in some places is too light, making it difficult to read.

3 Would like to see a more comprehensive computational complexity discussion, Section D2 is a bit brief; each of its components warrants further discussion.

4 The datasets currently in use are quite limited in terms of both domain and scale, and may be insufficient to support the conclusions.

Questions for authors

1 The experimental results in Table 1 also raise questions: why are the results for many of the models presented incompletely? Have the authors actually reproduced these models themselves?

2 Tables 1 and 2 use different datasets; must the comparison methods also be categorized into "multimodal" and "unimodal" types? Is it possible to compare them all together?

5 comments
AuthorsOpenReview ↗

Response to Reviewer 1jwu Part 3/3 (Questions 1, 2)

Q1. Incomplete baseline results in Table 1

Thank you very much for the comments. The baseline results reported in Table 1 of the paper are taken from the cited papers. We independently reproduced the six expert models used in the PRISM expert pool. The corresponding results are presented in Table C4 of the paper's appendix. Following your suggestion, we reproduced OTKGE, VISTA, and MoMoK with their publicly released code to fill in the missing baseline results in Table 1. For brevity, Table 3 only includes the baselines with newly obtained results. All experiments follow the same evaluation protocol as PRISM on every dataset.

Table 3. Each cell reports MRR / Hits@1 / Hits@10 in percent. A dash indicates that no verified result is available under this protocol.

MethodMKG-WMKG-YYAGO15K
OTKGE34.36 / 28.85 / 44.8835.51 / 31.97 / 41.3838.93 / 32.38 / 50.72
VISTA35.02 / 28.09 / 48.0030.23 / 24.35 / 41.5523.64 / 15.71 / 39.09
MoMoK35.08 / 29.75 / 45.2937.75 / 34.85 / 42.6022.89 / 15.61 / 37.13
PRISM (ours)38.57 / 32.67 / 49.7039.12 / 35.62 / 45.2841.91 / 33.22 / 55.46

We will fill in the missing entries in Table 1 of the paper and label each baseline result as either reported in the original paper or reproduced in our experiments. Thank you for your suggestion.

Q2. Comparing unimodal and multimodal methods

Thank you very much for the comments. Table 1 of the paper covers three multimodal KGC benchmarks, MKG-W, MKG-Y, and YAGO15K, which provide entity descriptions and images alongside knowledge graph triples. The standalone baselines in Table 1 of the paper were designed to use these modalities for link prediction. PRISM likewise includes multimodal experts alongside structural experts, allowing it to draw on both kinds of evidence. Table 2 of the paper covers two unimodal KGC benchmarks, WN18RR and FB15K-237, with only structural information. The baselines infer missing entities from graph triples alone. PRISM uses a pool consisting solely of structural experts for Table 2, with graph triples as the only input for training and prediction.

Following your suggestion, we bring both groups of methods together in Table 4. On the three multimodal benchmarks, we trained three representative structural baselines using their released code. On the two unimodal benchmarks, we retrained MyGO and AdaMF-MAT. These supplementary runs use PRISM’s filtered head and tail ranking protocol with average ranks for tied scores.

Table 4. Structural and multimodal KGC methods across five benchmarks. Each cell reports MRR / Hits@1 / Hits@10 in percent.

MethodMKG-WMKG-YYAGO15KWN18RRFB15K-237
HAKE34.32 / 28.08 / 46.0137.50 / 34.00 / 43.8041.51 / 34.09 / 55.1349.70 / 45.20 / 58.2034.60 / 25.00 / 54.20
HousE21.25 / 7.48 / 42.7730.21 / 23.34 / 40.5233.91 / 24.56 / 50.8451.10 / 46.50 / 60.2036.10 / 26.60 / 55.10
CompoundE34.34 / 28.66 / 44.8637.07 / 33.95 / 42.3640.94 / 33.80 / 54.1249.10 / 45.00 / 57.6035.70 / 26.40 / 54.50
MyGO36.10 / 29.78 / 47.7538.44 / 35.01 / 44.1931.10 / 22.18 / 48.5844.18 / 40.99 / 49.9735.48 / 26.14 / 54.12
AdaMF-MAT35.85 / 29.04 / 48.4238.57 / 34.34 / 45.7634.40/26.20/48.6046.72 / 43.04 / 53.7032.41 / 22.99 / 51.11
PRISM (ours)38.57 / 32.67 / 49.7039.12 / 35.62 / 45.2841.91 / 33.22 / 55.4655.26 / 50.02 / 65.9142.85 / 34.67 / 60.48

On MKG-W and MKG-Y, PRISM exceeds the strongest of the three structural baselines by 4.23 and 1.62 percentage points in MRR, respectively. These baselines can learn relational patterns from triples alone, which leaves entity descriptions and images unused when those features could help distinguish plausible candidates.

On WN18RR and FB15K-237, PRISM outperforms the structure-only adaptations of MyGO and AdaMF-MAT. Its Hits@10 exceeds the better of the two baselines by 12.21 and 6.36 percentage points, respectively. On WN18RR, it leads the stronger of the two adaptations by 8.54 MRR points.

These comparisons support PRISM’s use of query-level expert selection and rank fusion to exploit complementary predictions. On multimodal graphs, those predictions can incorporate image and text evidence. On unimodal graphs, they come from experts with different ways of reasoning over the same triples.

Once again, thank you very much for all of your comments. We would like to know whether our responses have addressed your concern. Please feel free to comment if there are any further confusions.

AuthorsOpenReview ↗

Response to Reviewer 1jwu Part 2/3 (Weakness 4)

W4. Dataset domain and scale

Thank you very much for the comments. The paper evaluates PRISM on five KGC benchmarks that span lexical and encyclopedic domains and vary in graph size and relation coverage.

  • Multimodal KGC. MKG-W is based on Wikidata, and MKG-Y and YAGO15K are based on YAGO. They provide entity images and text alongside graph triples and cover encyclopedic facts about people, places, and cultural works. Each contains about 15,000 entities, with 28–169 relation types and approximately 21,000–86,000 training triples.
  • Unimodal KGC. WN18RR captures lexical-semantic relations among WordNet synsets, with 40,943 entities, 11 relations, and 86,835 training triples. FB15K-237 covers general knowledge from Freebase, including film, sports, and awards, with 14,541 entities, 237 relations, and 272,115 training triples. Both evaluations use graph triples as the sole input.

We extended the evaluation to a larger graph using the official CoDEx-L split, a Wikidata-based encyclopedic benchmark with 77,951 entities, 69 relations, and 551,193 training triples. The released checkpoints were unavailable at evaluation time, so we trained the embedding-based PairRE and the rule-based AnyBURL on the official training split. We evaluated both experts individually and as a two-expert PRISM pool on all 61,244 head and tail prediction queries in Table 2.

Table 2. CoDEx-L structural KGC evaluation with filtered average-tie ranking.

MethodMRR (%)H@1 (%)H@10 (%)
PairRE20.0313.6432.20
AnyBURL19.4616.9023.32
PRISM (ours)26.2421.0436.13

Among the two standalone experts, PairRE leads in MRR and Hits@10, and AnyBURL leads in Hits@1. PRISM exceeds the better standalone score on each metric by 6.21 percentage points in MRR, 4.14 in Hits@1, and 3.93 in Hits@10.

Together, the results on the five benchmarks in the manuscript and on CoDEx-L support our conclusion that PRISM can exploit complementary expert predictions across domains and input settings. The CoDEx-L experiment extends this evidence to structural KGC on a substantially larger graph.

AuthorsOpenReview ↗

Response to Reviewer 1jwu Part 1/3 (Weaknesses 1,2,3)

Thank you for asking us to clarify PRISM's contribution, explain its computational cost, and document the scope and sources of its comparisons.

W1. PRISM’s decision-making strategy for KGC

Thank you very much for the comments. Even queries with the same relation can benefit from different reasoning mechanisms. This raises a decision problem for knowledge graph completion (KGC): given the predictions available so far, which expert should the system call next, and when should it stop? PRISM addresses this problem by formulating KGC as sequential expert selection under a per-query budget.

Our contribution is the design of the Profiler, Adapter, Verifier, Judge, and Router as an integrated feedback loop. The Profiler initializes a shared state for the query. After each successful expert call, the Adapter puts the new prediction on a common scale, the Verifier assesses structural support and agreement, and the Judge updates the candidate ranking. These updates feed back into the Router, which estimates how much each available expert could improve the current ranking. The Router uses that estimate and the remaining budget to decide whether to call another expert or return the current ranking.

Prior KGC ensembles already learn relation-dependent weights (RelEns-DSC) or query-dependent weights from the scores of the constituent models (DynaSemble). PRISM’s Call–Verify–Stop loop makes evidence verification and ranking updates part of the decision about which expert predictions to obtain next. As a result, the sequence of expert calls can differ even for queries with the same relation.

W2. Text contrast in Figure 2

We will improve the contrast of the light-colored text and further clarify the arrows, shared states, and CALL/STOP loop to enhance the visual presentation of Figure 2. In addition, we will explicitly clarify the execution order of the framework: Judge generates the current fused ranking, Verifier performs structural validation of that ranking, and the subsequent Router decision is made based on the resulting signals.

W3. Computational complexity in Section D.2

Thank you very much for the comments. We will expand Section D.2 to analyze computational complexity component by component, covering Profiler preprocessing and query feature extraction, Adapter score alignment and ranking, Judge fusion, Verifier graph support and agreement checks, Router action evaluation, and expert inference.

Table 1. Instrumented per-query timing on MKG-W with synchronized, warm, batch-size-one serving on an RTX 4090.

ComponentMean time (ms)Share (%)
Engine and measurement0.320.87
Profiler0.371.01
Adapter4.7312.83
Judge2.135.78
Verifier1.474.00
Router4.0711.03
Expert inference23.7664.47

Expert inference is the largest source of latency in Table 1, taking 23.76 ms per query (64.47%). All other components and measurement together take 13.09 ms, including 4.07 ms (11.03%) for the Router. PRISM uses these routing decisions to target the dominant cost directly. It calls experts selectively and stops when the estimated benefit of another call no longer justifies its cost.

Reviewer_QFb1OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes PRISM, a unified heterogeneous-expert agent framework for MMKGC, aiming to balance predictive performance and computational cost through query-adaptive expert coordination. PRISM integrates independently trained unimodal and multimodal KGC models into a shared decision-making framework consisting of five components: Profiler, Adapter, Verifier, Judge, and Router. Specifically, the framework characterizes query-level information, standardizes expert predictions, evaluates structural support and prediction agreement, and performs reliability-aware rank fusion. The Router estimates the potential benefits of additional expert invocations and dynamically determines whether to call another expert or stop inference under budget constraints. Experiments on five unimodal and multimodal KGC benchmarks demonstrate competitive link prediction performance while reducing expert invocations and inference costs compared with exhaustive expert ensembles.

Critical strengths
  1. The paper addresses the interesting problem of multimodal knowledge graph completion and provides a relatively comprehensive review of the relevant literature.

  2. Unlike traditional KGC studies that primarily focus on improving MRR, PRISM considers both predictive performance and computational cost. In particular, when the expert pool includes GNN-based, rule-based, and multimodal models, invoking every expert for every query may be unnecessary. This makes the proposed approach practically meaningful.

Critical weaknesses
  1. The query-level heterogeneity investigated in this paper is meaningful, but the research motivation is not sufficiently justified. Specifically, in Fig. 1, the authors argue that structural reasoning models may be more reliable than multimodal models when visual information lacks distinctive features. However, the limited contribution of visual information does not necessarily imply a negative impact on the predictive performance of multimodal models, as existing multimodal knowledge graph completion methods can often adaptively adjust the contributions of different modalities through attention mechanisms or adaptive fusion strategies.

  2. I find it difficult to reconcile the overall architecture illustrated in Figure 2 with the technical details presented in the methodology section.

  3. PRISM appears to adopt a multi-path decision-making strategy. However, its predictive performance does not show a substantial improvement over All-expert equal RRF. This raises concerns about the effectiveness of the proposed strategy. Could the authors further clarify its advantages beyond inference efficiency?

  4. The paper reports competitive performance on conventional KGC benchmarks but does not sufficiently explain the sources of these improvements. Moreover, PRISM continues to achieve leading performance on these datasets even without multimodal information, which appears to be inconsistent with the research motivation emphasizing the importance of coordinating unimodal and multimodal information.

Questions for authors

See Weaknesses.

6 comments
AuthorsOpenReview ↗

Response to Reviewer QFb1 Part 3/3 (Weakness 4)

W4. Performance on conventional KGC benchmarks

Thank you very much for the comments. PRISM is motivated by query-level heterogeneity in the relative strengths of different experts. On benchmarks that provide graph triples, images, and text, we compare PRISM with multimodal KGC baselines to assess the benefit of selecting and combining complementary predictions from structural and multimodal experts. On conventional KGC benchmarks such as WN18RR and FB15K-237, graph triples are the sole input. Comparisons with conventional KGC baselines assess whether PRISM can also benefit from coordinating experts that reason over the same graph using different mechanisms, such as embeddings, graph paths, and rules.

To sufficiently explain the sources of these improvements, we designed two sets of experiments. First, on conventional KGC benchmarks, we compared exclusive rank-1 hit counts for each structural expert in PRISM’s pool and for PRISM itself. A hit is exclusive when only one of the compared methods ranks the target entity rank-1 on a query, as shown in Table 4A. Second, on multimodal KGC benchmarks, we compared the full expert pool with structural-only and multimodal-only pools under the same nominal routing budget within each dataset to assess the contribution of making both expert types available, as shown in Table 4B.

Table 4A. Total and exclusive rank-1 hits on conventional KGC benchmarks (FB15K-237).

Methodtotal hitsexclusive hits
PairRE9,869716 (1.75%)
TuckER10,659790 (1.93%)
NBFNet12,9821,040 (2.54%)
PRISM (ours)14,191176 (0.43%)

Each standalone expert records exclusive hits, showing that different structural reasoning mechanisms succeed on different queries over the same graph. For all 176 instances of FB15K-237, PRISM fuses multiple expert outputs to place the target at the top, despite every standalone expert ranking it below this position.

Table 4B. Expert-pool comparisons under the same nominal routing budget within each dataset.

DatasetExpert poolMRR (%)Mean callsMean configured cost
MKG-WStructural only36.902.221.52
MKG-WMultimodal only37.891.541.94
MKG-WPRISM (ours)38.572.091.84
MKG-YStructural only38.562.101.39
MKG-YMultimodal only37.251.501.83
MKG-YPRISM (ours)39.122.011.63

On MKG-W, it improves over the structural-only and multimodal-only pools by 1.67 and 0.68 percentage points, respectively. On MKG-Y , the corresponding gains are 0.56 and 1.87 points. The stronger restricted pool is multimodal-only on MKG-W and structural-only on MKG-Y. On each dataset, giving PRISM access to both expert types yields higher MRR than the stronger restricted pool, supporting the premise that structural and multimodal experts provide complementary evidence. Compared with the multimodal-only pool, full PRISM achieves higher MRR at a lower mean configured cost on both datasets (1.84 vs. 1.94 on MKG-W and 1.63 vs. 1.83 on MKG-Y).

These results support PRISM’s design rationale of selecting and combining complementary experts for each query and sufficiently explain the sources of these improvements. On conventional benchmarks, that complementarity comes from different ways of reasoning over the same graph. On multimodal benchmarks, the router can also draw on experts that use visual and textual information.

Once again, thank you very much for all of your comments. We would like to know whether our responses have addressed your concern. Please feel free to comment if there are any further confusions.

AuthorsOpenReview ↗

Response to Reviewer QFb1 Part 2/3 (Weakness 3)

W3. Predictive value of sequential expert routing

Thank you very much for the comments. Beyond its gains in inference efficiency, PRISM’s multi-path decision-making strategy uses intermediate predictions to decide which expert to call next and refines the ranking with complementary evidence from subsequent calls. To clarify these advantages, we conducted two experiments. First, we compared PRISM with LearnedRouter-K. Second, we compared the rankings produced by PRISM’s full sequence of expert calls with those based on the first expert’s prediction alone. We also replayed an individual query to compare the ranking produced by PRISM’s selected route with that obtained through exhaustive fusion.

Table 3A-B. Controls for expert selection and stopping.

ComparisonDatasetEvaluated rankingMRR (%)Mean callsMean configured cost
A. Equal RRFMKG-WPRISM (ours)38.512.061.83
A. Equal RRFMKG-WLearnedRouter-K38.292.272.151
A. Equal RRFMKG-YPRISM (ours)38.881.981.405
A. Equal RRFMKG-YLearnedRouter-K38.441.981.465
B. Routing checksMKG-WPRISM (one expert call)37.291.000.99
B. Routing checksMKG-WPRISM (ours)38.572.091.84

PRISM uses the predictions received so far to choose the next expert or stop. LearnedRouter-K chooses its entire query-specific subset before any expert predicts. In Table 3A, we keep each policy's selected subsets fixed and apply equal RRF to both. PRISM achieves higher MRR at lower mean configured cost on both datasets. Table 3B shows that using only the expert from the first call reduces MRR by 1.28 percentage points, indicating that subsequent calls can retrieve valuable evidence.

Table 3C. Ranking replay for one MKG-W tail query.

Evaluation conditionExpert predictions usedFiltered rank of correct tail
First call onlyVISTA3
Exhaustive referenceAll six experts2
PRISM (ours)VISTA and PairRE1

Table 3C illustrates how the selected expert models affect entity rankings for an individual query. The query is (Q2998046, P106, ? ), and the correct tail entity is Q82955. VISTA ranks the correct entity third. After PRISM invokes PairRE, it rises to first place; however, when all six predictions are fused, it ranks second.

AuthorsOpenReview ↗

Response to Reviewer QFb1 Part 1/3 (Weaknesses 1, 2)

Thank you very much for taking the time to review our work. We appreciate your constructive comments. Below, we address each point in detail.

W1. Research motivation and adaptive fusion

Thank you very much for the comments. PRISM addresses query-level heterogeneity in the relative effectiveness of different experts. To sufficiently justify PRISM’s research motivation, we designed two experiments. First, we compared the predictive performance of structural and multimodal experts on the same queries. Second, we evaluated how PRISM and multimodal baselines with adaptive fusion respond to changes in different input modalities.

Table 1. RulE and MoMoK are the structural expert and multimodal expert, respectively. A method is considered the winner if it assigns a better rank to the target entity than the other method for the same query.

DatasetTest queriesMethod AMethod BA winsB winsSame rank
MKG-W8,548RulEMoMoK3,237 (37.87%)2,839 (33.21%)2,472 (28.92%)
MKG-W8,548PRISMRulE3,624 (42.40%)1,906 (22.30%)3,018 (35.31%)
MKG-W8,548PRISMMoMoK4,002 (46.82%)1,913 (22.38%)2,633 (30.80%)
MKG-Y5,326RulEMoMoK1,579 (29.65%)1,892 (35.52%)1,855 (34.83%)
MKG-Y5,326PRISMRulE2,357 (44.25%)930 (17.46%)2,039 (38.28%)
MKG-Y5,326PRISMMoMoK2,040 (38.30%)1,074 (20.17%)2,212 (41.53%)

Table 1 illustrates the query-level heterogeneity that motivates PRISM. On the MKG-Y test set, RulE ranked the target entity higher than MoMoK on 29.65% of queries, and MoMoK ranked it higher than RulE on 35.52%. Each expert thus provided better rankings on a different subset of queries. Across the four pairwise comparisons on MKG-W and MKG-Y, PRISM ranked the target entity higher than the corresponding standalone expert on 38.30%–46.82% of queries. The standalone expert ranked it higher than PRISM on 17.46%–22.38% of queries. These results support PRISM’s design rationale of selecting and combining experts for each query to leverage their complementary strengths.

We compared PRISM with AdaMF-MAT, MoMoK, MyGO, and APKGC on MKG-W and MKG-Y in Table 2. These baselines use attention mechanisms or adaptive fusion to adjust the contributions of different modalities. For each dataset, we ran three trials. In each trial, we randomly selected 50% of the entities and masked only their visual features. We assessed how PRISM and the adaptive multimodal baselines responded to this change in available evidence by measuring each model’s change in MRR from unmasked to masked inputs.

Table 2. Modality sensitivity under entity-level masking. Clean MRR is in percent. Changes are averaged over three masks and reported in percentage points.

DatasetMethodClean MRRΔ visual mask
MKG-WAdaMF-MAT (COLING 2024)35.70−7.36
MKG-WMoMoK (ICLR 2025)35.08−1.33
MKG-WMyGO (AAAI 2025) 36.22−1.52
MKG-WAPKGC (AAAI 2025)36.53−26.08
MKG-WPRISM (ours)38.57−0.19
MKG-YAdaMF-MAT (COLING 2024)39.08−6.62
MKG-YMoMoK (ICLR 2025)37.75−0.47
MKG-YMyGO (AAAI 2025)36.97−8.91
MKG-YAPKGC (AAAI 2025)38.76−28.26
MKG-YPRISM (ours)39.12−0.14

PRISM retains the highest MRR after masking and records the smallest drop on both datasets, at 0.19 points on MKG-W and 0.14 on MKG-Y. On MKG-Y, AdaMF-MAT and APKGC start within 0.4 percentage points of PRISM on clean inputs. Under the same masks, PRISM retains 38.98% MRR, compared with 32.46% for AdaMF-MAT and 10.50% for APKGC. PRISM complements modality adaptation within a model by choosing among structural and multimodal experts for each query. This gives it access to predictors that rely solely on graph structure when visual evidence is unavailable.

The paired rankings and robustness results support PRISM’s research motivation of addressing query-level heterogeneity through expert selection. Selecting among experts with different reasoning mechanisms complements the modality adaptation already performed within individual multimodal models.

W2. Figure 2 and the methodology

Thank you very much for the comments. We will revise Figure 2 to clarify the interactions between components and show how they correspond to the technical details in the methodology section. The revised figure will also show that each attempted expert call consumes budget and that fusion uses only the expert opinions returned by successful calls.

Reviewer_7FsaOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PRISM, a framework for multimodal knowledge graph completion (KGC) that treats various unimodal or multimodal KGC models as experts. To aggregate predictions from diverse experts, PRISM’s adapter utilizes rank-based representations. Subsequently, the Verifier estimates the reliability of the current prediction, and the Judge merges the predictions to generate a refined output. Finally, the Router determines whether to invoke additional experts. Experimental results demonstrate that PRISM achieves competitive or superior performance compared to utilizing all experts simultaneously, while reducing the number of expert calls.

Critical strengths
  1. Treating diverse (multimodal) KGC methods as individual experts and aggregating their predictions using rank information seems to be a novel approach.
  2. The method demonstrates strong performance on multimodal KGC benchmarks while maintaining a small number of expert calls.
Critical weaknesses
  1. On unimodal KGC benchmarks, the paper lacks direct comparisons with the models employed as experts, e.g.,NBFNet, as well as other competitive baselines such as KnowFormer [1].
  2. Ablation studies on individual components are insufficient, making it difficult to assess the exact contribution of each module. For instance, further analysis is needed to show how performance changes when the Verifier is excluded or when prediction uncertainty is not taken into account.

[1] J. Liu et al., "KnowFormer: Revisiting Transformers for Knowledge Graph Reasoning", ICML 2024.

Questions for authors
  • Could the authors provide more comprehensive ablation studies on the various model components, such as the Verifier or the prediction uncertainty, to clarify their individual contributions to the overall performance?
  • Could the authors justify the selection criteria for the specific experts used in the framework? Additionally, please report how the model's performance scales and varies with different numbers and configurations of experts.
2 comments
AuthorsOpenReview ↗

Response to Reviewer 7Fsa Part 2/2 (Question 2)

Q2. Expert selection and scaling

Thank you for your comments. The primary selection criteria for the specific experts used in the framework was coverage of complementary scoring mechanisms. Structural information is central to link prediction, so we selected PairRE and TuckER for their embedding-based scores. We included NBFNet and RulE to provide additional structural evidence from graph paths and rules, respectively, and MoMoK and VISTA to incorporate visual and textual information. This combination allows PRISM to draw on different types of evidence by selecting experts for each query.

To examine how the model's performance scales and varies with different numbers and configurations of experts, we evaluated 10 pools containing two to six experts (Table 4). We constructed the pools using greedy selection on the validation set, random sampling, and predefined combinations of expert types. For each pool, we refitted the routing policy with a budget cap of 2.50 and a maximum of three expert calls per query.

Table 4. PRISM performance with different expert pool sizes and configurations on MKG-W. Calls and configured costs are averaged per query.

Pool sizePool constructionAvailable expertsMRR (%)Mean expert callsMean configured cost
2Validation greedyMoMoK + VISTA37.791.511.91
2Random, Structural reasoningNBFNet + RulE36.751.521.89
2RandomTuckER + VISTA37.441.581.57
3Validation greedyMoMoK + PairRE + VISTA37.931.611.44
3RandomNBFNet + RulE + TuckER36.701.671.59
3Predefined heterogeneousMoMoK + NBFNet + TuckER36.541.651.69
4Validation greedyMoMoK + PairRE + RulE + VISTA38.461.851.79
4RandomMoMoK + NBFNet + PairRE + TuckER36.872.161.54
5Validation greedyMoMoK + NBFNet + PairRE + RulE + VISTA38.321.831.84
6PRISM (ours)MoMoK + NBFNet + PairRE + RulE + TuckER + VISTA38.572.091.84

At a fixed pool size and budget cap, MRR varies with the expert mix. The two-expert pools span 36.75–37.79% MRR, and the three-expert pools span 36.54–37.93%. Both four-expert pools contain MoMoK and PairRE. The pool with RulE and VISTA achieves 38.46% MRR at a mean configured cost of 1.79, compared with 36.87% at 1.54 for the pool with NBFNet and TuckER. The roughly 1.6-point difference between these four-expert pools exceeds the gain from expanding the validation-selected pool from two to four experts.

Along the validation-selected sequence, MRR rises from 37.79% with two experts to 37.93% with three and 38.46% with four. It dips to 38.32% with five and reaches 38.57% with six. The four-expert pool captures most of the observed gain while averaging 1.85 calls per query versus 2.09 for the six-expert run. The full pool gives PRISM more experts to choose from within the same budget and three-call limit. A well-chosen four-expert pool therefore offers accuracy close to the full pool, while keeping all six experts available gives PRISM more options for each query under the same budget and call limit.

Once again, thank you very much for all of your comments. We would like to know whether our responses have addressed your concern. Please feel free to comment if there are any further confusions.

AuthorsOpenReview ↗

Response to Reviewer 7Fsa Part 1/2 (Weaknesses 1, 2, Question 1)

Thank you very much for taking the time to review our work. We appreciate your constructive comments. Below, we address each point in detail.

W1. Direct comparisons on unimodal KGC benchmarks

Thank you very much for the comments. Following your suggestion, we added direct comparisons on unimodal KGC benchmarks with the models employed as experts (e.g., NBFNet), as well as with the competitive baseline KnowFormer in Table 1. We evaluated all baselines using the same protocol as PRISM.

Table 1. Structural KGC accuracy on WN18RR and FB15k-237. MRR and Hits are percentages.

DatasetMethodMRRH@1H@10
WN18RRPairRE44.3439.9353.40
WN18RRTuckER45.8643.0051.18
WN18RRRulE48.4044.7055.42
WN18RRNBFNet55.0649.4666.11
WN18RRKnowFormer54.4348.9065.19
WN18RRPRISM (ours)55.2650.0265.91
FB15k-237PairRE33.7624.1153.25
FB15k-237TuckER35.2826.0453.59
FB15k-237RulE36.2026.6055.30
FB15k-237NBFNet40.9531.7259.11
FB15k-237KnowFormer41.4832.1159.83
FB15k-237PRISM (ours)42.8534.6760.48

We added comparisons on WN18RR and FB15k-237 in Table 1, including standalone results for the structural experts used by PRISM and a reproduced KnowFormer baseline. PRISM improves on NBFNet, the strongest standalone expert in both pools, by 0.20 and 1.90 MRR points on WN18RR and FB15k-237, respectively. These results show that combining structural experts can improve on the strongest constituent expert even when the inputs are purely structural.

Table 2. A hit indicates that the target entity is ranked at 1. Only PRISM hits counts test triples where PRISM ranks the target entity at 1 but KnowFormer does not.

DatasetTest queriesOnly PRISM hits
WN18RR6,268176 (2.81%)
FB15k-23740,9321,863 (4.55%)

Compared with KnowFormer, PRISM offers complementary advantages in query coverage. As shown in Table 2, PRISM places the target entity at rank 1 on 176 test queries (2.81%) on WN18RR and 1,863 test queries (4.55%) on FB15k-237 for which KnowFormer assigns the target entity a rank greater than 1. This pattern supports PRISM’s design rationale of selecting experts for each query to draw on their complementary strengths.

W2 and Q1. Ablation experiments for each component

Thank you very much for the comments. As shown in Table 3, we conducted more comprehensive ablation studies on the various model components, including the Verifier and prediction uncertainty, to clarify the individual contributions of each component to the model's overall performance.

Table 3. Component ablations on MKG-Y.

ConfigurationLowest MRR (%)Mean expert callsMean configured cost
w/o Verifier38.741.951.68
w/o prediction uncertainty38.531.951.54
w/o risk mechanism38.851.831.40
w/o contextual utility residuals38.802.081.58
w/o adaptive stopping38.952.822.07
PRISM (ours)39.122.011.63

Excluding the Verifier and disabling prediction uncertainty lower MRR by 0.38 and 0.59 percentage points, respectively. With the Verifier excluded, the Router still uses structural information from the query profile and uncertainty signals from expert predictions. When prediction uncertainty is disabled, the Verifier still assesses graph support and expert agreement, and the Judge continues to weight predictions using relation-specific expert competence. These remaining signals may help explain why PRISM retains most of its ranking quality in both ablations.

PRISM uses structural support, expert agreement, and prediction uncertainty to assess intermediate predictions and estimate which expert can add the most value for each query. Adaptive stopping lets PRISM maintain comparable ranking quality with fewer expert calls.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.