- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper is about knowledge graph completion, which is an interesting topic. The authors formulate KGC as an information-driven sequential decision problem and propose PRISM, a unified framework for coordinating heterogeneous reasoning models. The paper is well written and well organized. However, there are several concerns in the current version of the paper that addressing them will increase the quality of this paper.
Critical strengths
2 The improvements resulting from the experiment are exciting, as they effectively address existing problems.
3 The paper’s code has been made open source, which helps facilitate the dissemination of the paper and its underlying ideas.
Critical weaknesses
1 As it stands, the paper appears to offer a formalized framework; however, it is worth clarifying whether these various components have already been implemented in existing work—or if they have simply been given new names.
2 Figure 2 could be further improved; while the current structure is clear, its visual presentation could be refined—in particular, the text color in some places is too light, making it difficult to read.
3 Would like to see a more comprehensive computational complexity discussion, Section D2 is a bit brief; each of its components warrants further discussion.
4 The datasets currently in use are quite limited in terms of both domain and scale, and may be insufficient to support the conclusions.
Questions for authors
1 The experimental results in Table 1 also raise questions: why are the results for many of the models presented incompletely? Have the authors actually reproduced these models themselves?
2 Tables 1 and 2 use different datasets; must the comparison methods also be categorized into "multimodal" and "unimodal" types? Is it possible to compare them all together?
5 comments
Response to Reviewer 1jwu Part 2/3 (Weakness 4)
W4. Dataset domain and scale
Thank you very much for the comments. The paper evaluates PRISM on five KGC benchmarks that span lexical and encyclopedic domains and vary in graph size and relation coverage.
- Multimodal KGC. MKG-W is based on Wikidata, and MKG-Y and YAGO15K are based on YAGO. They provide entity images and text alongside graph triples and cover encyclopedic facts about people, places, and cultural works. Each contains about 15,000 entities, with 28–169 relation types and approximately 21,000–86,000 training triples.
- Unimodal KGC. WN18RR captures lexical-semantic relations among WordNet synsets, with 40,943 entities, 11 relations, and 86,835 training triples. FB15K-237 covers general knowledge from Freebase, including film, sports, and awards, with 14,541 entities, 237 relations, and 272,115 training triples. Both evaluations use graph triples as the sole input.
We extended the evaluation to a larger graph using the official CoDEx-L split, a Wikidata-based encyclopedic benchmark with 77,951 entities, 69 relations, and 551,193 training triples. The released checkpoints were unavailable at evaluation time, so we trained the embedding-based PairRE and the rule-based AnyBURL on the official training split. We evaluated both experts individually and as a two-expert PRISM pool on all 61,244 head and tail prediction queries in Table 2.
Table 2. CoDEx-L structural KGC evaluation with filtered average-tie ranking.
| Method | MRR (%) | H@1 (%) | H@10 (%) |
|---|---|---|---|
| PairRE | 20.03 | 13.64 | 32.20 |
| AnyBURL | 19.46 | 16.90 | 23.32 |
| PRISM (ours) | 26.24 | 21.04 | 36.13 |
Among the two standalone experts, PairRE leads in MRR and Hits@10, and AnyBURL leads in Hits@1. PRISM exceeds the better standalone score on each metric by 6.21 percentage points in MRR, 4.14 in Hits@1, and 3.93 in Hits@10.
Together, the results on the five benchmarks in the manuscript and on CoDEx-L support our conclusion that PRISM can exploit complementary expert predictions across domains and input settings. The CoDEx-L experiment extends this evidence to structural KGC on a substantially larger graph.
Response to Reviewer 1jwu Part 1/3 (Weaknesses 1,2,3)
Thank you for asking us to clarify PRISM's contribution, explain its computational cost, and document the scope and sources of its comparisons.
W1. PRISM’s decision-making strategy for KGC
Thank you very much for the comments. Even queries with the same relation can benefit from different reasoning mechanisms. This raises a decision problem for knowledge graph completion (KGC): given the predictions available so far, which expert should the system call next, and when should it stop? PRISM addresses this problem by formulating KGC as sequential expert selection under a per-query budget.
Our contribution is the design of the Profiler, Adapter, Verifier, Judge, and Router as an integrated feedback loop. The Profiler initializes a shared state for the query. After each successful expert call, the Adapter puts the new prediction on a common scale, the Verifier assesses structural support and agreement, and the Judge updates the candidate ranking. These updates feed back into the Router, which estimates how much each available expert could improve the current ranking. The Router uses that estimate and the remaining budget to decide whether to call another expert or return the current ranking.
Prior KGC ensembles already learn relation-dependent weights (RelEns-DSC) or query-dependent weights from the scores of the constituent models (DynaSemble). PRISM’s Call–Verify–Stop loop makes evidence verification and ranking updates part of the decision about which expert predictions to obtain next. As a result, the sequence of expert calls can differ even for queries with the same relation.
W2. Text contrast in Figure 2
We will improve the contrast of the light-colored text and further clarify the arrows, shared states, and CALL/STOP loop to enhance the visual presentation of Figure 2. In addition, we will explicitly clarify the execution order of the framework: Judge generates the current fused ranking, Verifier performs structural validation of that ranking, and the subsequent Router decision is made based on the resulting signals.
W3. Computational complexity in Section D.2
Thank you very much for the comments. We will expand Section D.2 to analyze computational complexity component by component, covering Profiler preprocessing and query feature extraction, Adapter score alignment and ranking, Judge fusion, Verifier graph support and agreement checks, Router action evaluation, and expert inference.
Table 1. Instrumented per-query timing on MKG-W with synchronized, warm, batch-size-one serving on an RTX 4090.
| Component | Mean time (ms) | Share (%) |
|---|---|---|
| Engine and measurement | 0.32 | 0.87 |
| Profiler | 0.37 | 1.01 |
| Adapter | 4.73 | 12.83 |
| Judge | 2.13 | 5.78 |
| Verifier | 1.47 | 4.00 |
| Router | 4.07 | 11.03 |
| Expert inference | 23.76 | 64.47 |
Expert inference is the largest source of latency in Table 1, taking 23.76 ms per query (64.47%). All other components and measurement together take 13.09 ms, including 4.07 ms (11.03%) for the Router. PRISM uses these routing decisions to target the dominant cost directly. It calls experts selectively and stops when the estimated benefit of another call no longer justifies its cost.
Response to Reviewer 1jwu Part 3/3 (Questions 1, 2)
Thank you very much for the comments. The baseline results reported in Table 1 of the paper are taken from the cited papers. We independently reproduced the six expert models used in the PRISM expert pool. The corresponding results are presented in Table C4 of the paper's appendix. Following your suggestion, we reproduced OTKGE, VISTA, and MoMoK with their publicly released code to fill in the missing baseline results in Table 1. For brevity, Table 3 only includes the baselines with newly obtained results. All experiments follow the same evaluation protocol as PRISM on every dataset.
Table 3. Each cell reports MRR / Hits@1 / Hits@10 in percent. A dash indicates that no verified result is available under this protocol.
We will fill in the missing entries in Table 1 of the paper and label each baseline result as either reported in the original paper or reproduced in our experiments. Thank you for your suggestion.
Thank you very much for the comments. Table 1 of the paper covers three multimodal KGC benchmarks, MKG-W, MKG-Y, and YAGO15K, which provide entity descriptions and images alongside knowledge graph triples. The standalone baselines in Table 1 of the paper were designed to use these modalities for link prediction. PRISM likewise includes multimodal experts alongside structural experts, allowing it to draw on both kinds of evidence. Table 2 of the paper covers two unimodal KGC benchmarks, WN18RR and FB15K-237, with only structural information. The baselines infer missing entities from graph triples alone. PRISM uses a pool consisting solely of structural experts for Table 2, with graph triples as the only input for training and prediction.
Following your suggestion, we bring both groups of methods together in Table 4. On the three multimodal benchmarks, we trained three representative structural baselines using their released code. On the two unimodal benchmarks, we retrained MyGO and AdaMF-MAT. These supplementary runs use PRISM’s filtered head and tail ranking protocol with average ranks for tied scores.
Table 4. Structural and multimodal KGC methods across five benchmarks. Each cell reports MRR / Hits@1 / Hits@10 in percent.
On MKG-W and MKG-Y, PRISM exceeds the strongest of the three structural baselines by 4.23 and 1.62 percentage points in MRR, respectively. These baselines can learn relational patterns from triples alone, which leaves entity descriptions and images unused when those features could help distinguish plausible candidates.
On WN18RR and FB15K-237, PRISM outperforms the structure-only adaptations of MyGO and AdaMF-MAT. Its Hits@10 exceeds the better of the two baselines by 12.21 and 6.36 percentage points, respectively. On WN18RR, it leads the stronger of the two adaptations by 8.54 MRR points.
These comparisons support PRISM’s use of query-level expert selection and rank fusion to exploit complementary predictions. On multimodal graphs, those predictions can incorporate image and text evidence. On unimodal graphs, they come from experts with different ways of reasoning over the same triples.
Once again, thank you very much for all of your comments. We would like to know whether our responses have addressed your concern. Please feel free to comment if there are any further confusions.