- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies inductive knowledge graph completion, focusing on the fully inductive setting in which neither entities nor relations are shared between the training and test graphs. The authors propose RuleTopo, which encodes Horn rules mined by AMIE+ into relation-level descriptors and incorporates them into a relation-graph attention network with six types of topological patterns. The resulting relation representations guide entity-level message passing and triple scoring. The method also introduces a rule-consistency loss that encourages relation compositions derived from the mined rules to satisfy additive constraints in the embedding space.
The main contribution is the integration of symbolic rules and global relation topology during relation representation learning, rather than applying rules only after prediction.
The authors evaluate the method through baseline comparisons and component ablations on several inductive splits of FB15K-237 and NELL995. The reported results show improvements over InGram and TARGI in the fully inductive setting, although these improvements do not extend to all other settings. There are unexplained discrepancies between the main results and the ablation results, and the rule encoding procedure and test-time pipeline are insufficiently specified. Consequently, the experiments provide preliminary support for the approach but do not yet establish the effectiveness of its core mechanism or the robustness of its advantages.
Critical strengths
- A well-motivated mechanism for relation-level integration
Fully inductive knowledge graph completion requires generalization to unseen relations, not merely unseen entities. The proposed method incorporates rule descriptors directly into relation-graph attention, allowing symbolic information to influence representation learning rather than functioning solely as a separate post-processing module.
This constitutes a concrete methodological contribution. The relation level is a sensible interface for combining graph structure with relational composition patterns, and the design offers potential for further extension.
However, I regard this as a plausible and concrete modeling approach, rather than an established mechanism for general-purpose logical transfer.
- Promising reported results in the most relevant fully inductive settings
Compared with TARGI, RuleTopo improves MRR from 0.251 to 0.272 on FB-100 and from 0.315 to 0.334 on NL-100, corresponding to relative improvements of approximately 8.4% and 6.0%, respectively.
These results directly address the challenging setting emphasized by the paper and use a strong topology-aware relation-graph method as the comparator. If the experimental settings are consistent, the results are reproducible, and the gains are stable, these findings would provide the main empirical basis for recommending acceptance.
At present, however, the absence of statistics across multiple runs and the discrepancies between the reported tables make this strength conditional.
Critical weaknesses
- The core mechanism for generalization to unseen relations is insufficiently specified
The paper argues that rules help the model handle entirely unseen relations. However, being independent of entity identities does not, by itself, imply that a rule representation can transfer to an unseen relation vocabulary.
The current manuscript does not sufficiently specify:
- How a consistent rule-feature space is constructed across graphs;
- Whether rule encoding depends on specific relation identifiers;
- How rules and representations for unseen relations are constructed in the test graph.
Furthermore, the rule signal used in attention is formed through symmetric summation, while the consistency loss models composition through commutative vector addition. Horn rules and relational path compositions generally contain directional and order-sensitive information. The manuscript does not adequately explain which logical information is preserved by these representations.
These issues bear directly on the central claim: whether the model transfers logical structure or instead benefits from additional statistical descriptors of the target graph. Without a sufficiently precise operational specification, the method cannot be reliably reproduced, and it is difficult to assess whether it achieves the claimed neuro-symbolic synergy.
- The experimental evidence is insufficient to support the strength of the conclusions
The most immediate concern is the inconsistency in the reported performance of the full model on FB-100:
| Metric | Main Results: Table 1 | Ablation Results: Table 3 |
|---|---|---|
| MRR | 0.272 | 0.253 |
| Hits@10 | 0.433 | 0.429 |
| Hits@1 | 0.194 | 0.172 |
The paper does not explain these discrepancies. If different settings were used, the ablation results do not directly account for the main results. If the settings were identical, the reporting needs to be corrected and the source of the discrepancies explained.
In addition, the results on NL-50 and NL-0 do not support the broad claim that the method consistently outperforms strong baselines. The manuscript also lacks variability estimates across multiple runs, controls using randomized or shuffled rule information, and empirical comparisons with ULTRA and AdaRPT, both of which are discussed in the related work.
The paper’s contribution rests primarily on empirical validation. Experimental consistency and appropriate controls are therefore essential to establishing the effectiveness of the proposed mechanism, rather than merely matters of presentation. The current evidence does not adequately distinguish stable methodological gains from differences in experimental settings or random variation.
Questions for authors
Question 1: Please specify the complete rule encoding procedure and test-time pipeline
Is AMIE+ run again on the test support graph? What are the exact inputs, encoding procedure, and parameterization of ? How are entirely unseen relations handled, and how is feature construction made independent of training-specific relation identifiers? Does the encoding preserve rule-head versus rule-body roles, atom order, and variable-sharing structure? How should the descriptions of rule embeddings in the main text and appendix be reconciled?
Question 2: Please explain the discrepancies between the main results and the ablation results
Were the two sets of full-model results on FB-100 obtained using the same data, model configuration, checkpoint selection procedure, and evaluation protocol? If not, what motivated the different settings? Do the gains attributed to the individual components remain when evaluated under a consistent experimental setup?
Question 3: What evidence shows that the gains arise from rule structure rather than additional features or random variation?
Under matched training settings, do genuine rules outperform shuffled rules or non-logical features with comparable statistical properties? Are the improvements stable across multiple runs? Does the proposed mechanism retain an advantage when compared with relevant fully inductive methods?