- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SIRLM, a framework for knowledge graph reasoning that aims to mitigate reasoning evidence perception drift between LLM parametric knowledge and KG structural information. The method combines a structural KG tokenizer, a Structure-Internalized Rule Generator (SIRG) equipped with Structural Relation Memory (SRM), and a Rule-Constraint Message Propagation (RCMP) reasoner. SIRLM supports both supervised training and GRPO-based post-training. Experiments on 36 datasets demonstrate competitive performance across transductive and inductive KG reasoning settings.
Critical strengths
S1. Integration of structural knowledge and LLM-based reasoning.
The paper presents an interesting attempt to integrate structural KG representations with LLM-based rule generation. The SRM and RCMP modules provide a potentially useful approach to incorporating structural information into neural reasoning.
S2. Extensive evaluation across diverse reasoning settings.
The experiments cover 36 datasets, including challenging fully inductive settings, and compare against a broad range of existing approaches. The results demonstrate competitive performance, while the ablation studies provide evidence for the effectiveness of the proposed components.
Critical weaknesses
W1. Limited methodological novelty and unclear distinctions from existing approaches.
The proposed KG tokenizer largely builds upon existing structural representation learning techniques, such as those used in ULTRA and MOTIF, while LLM-based KG reasoning and rule generation have been explored in KRLM and ChatRule. The main contributions appear to be relation-memory-augmented attention and the incorporation of generated rule representations into NBFNet. However, the methodological distinctions and advantages over these existing approaches are not sufficiently established.
W2. Insufficient justification of the proposed rule-based reasoning mechanism.
The SRM module selects Top-K relations using learned relevance scores without explicitly establishing their logical relevance to the query. Moreover, RCMP incorporates the hidden states of generated rules into GNN initialization rather than explicitly enforcing the generated relation sequences during message propagation. Therefore, it remains unclear whether the improvements result from faithful rule-based reasoning or simply from additional structural representations.
Questions for authors
Questions For Authors
Q1. Novelty compared with KRLM.
KRLM already incorporates a structural KG tokenizer, a dynamic knowledge memory mechanism, and a structure-aware next-entity predictor. What are the fundamental methodological differences between SIRLM and KRLM, particularly regarding the proposed SRM and RCMP modules? Could the authors provide more direct evidence of their additional benefits?
Q2. Comparison with FtG.
FtG employs a structure-text adapter and ego-graph information to incorporate KG structural knowledge into LLMs. How does SIRLM's structural tokenization and rule generation provide advantages over FtG's structure-text fusion approach? Could the authors further clarify the differences and evaluate their respective effectiveness under comparable settings?
Q3. Necessity and faithfulness of rule generation.
Unlike FtG and KRLM, which predict target entities, SIRLM introduces an intermediate rule generation stage. However, RCMP incorporates rule hidden states into GNN message passing rather than explicitly executing the generated rules. What evidence demonstrates that this additional stage provides faithful reasoning rather than merely improving neural representations?
Q4. More targeted experimental comparisons.
Could the authors provide controlled comparisons with FtG and KRLM, including matched experimental settings and computational costs? In particular, ablations replacing SRM with KRLM-style knowledge memory or removing the rule generation stage would help establish the unique contributions of SIRLM.
General Response to Reviewers, ACs, SACs, and PCs
We sincerely thank all reviewers as well as ACs, SACs, and PCs for their careful handling of our submission. We also appreciate the positive feedback from the reviewers on research motivation and practical relevance (
LFt6), model design (LFt6, 96GR, and Y6ye), reasoning reliability and interpretability (96GR), and extensive experiments and generalization evaluation (LFt6, 96GR, and Y6ye). Based on the constructive suggestions provided by the reviewers, we have prepared point-by-point responses below. All revisions are highlighted in blue in the updated PDF. Here, we highlight our major responses and hope they can properly address the reviewers' concerns.Reviewers LFt6, 96GR, and Y6ye):Section 5.6andAppendix J.5).-RCMPand newly addedSRM Initablations demonstrate the contribution of generated rule semantics beyond additional structural representations. The two-stage WL analysis inAppendix Efurther supports the enhanced structural distinguishability and expressive power of RCMP. Detailed analysis is highlighted in blue inAppendix J.5.Reviewer Y6ye):Table 1and the theoretical analysis inAppendix Esupport the effectiveness of this design.Random SRMandSRM Initablations to validate the contributions of relation memory in SIRG and generated rule semantics.Reviewers LFt6 and 96GR):Reviewer 96GR): We correct the error in Eq. (14) and reformulate Eq. (15). The modified content has been highlighted in blue at Eqs. (14) and (15).Reviewer 96GR): We clarify target-edge filtering during rule mining and training to prevent leakage (Appendix H). An identity-rule ablation further shows that removing identity-rule supervision brings no consistent performance gains, supporting its retention as a fallback (Appendix J.4).Reviewers LFt6 and 96GR): We corrected the definition of the structural subspace in Proposition 1 by excluding memory value vectors and revised its proof accordingly, establishing the non-triviality of semantic drift reduction. We also refined Assumption B2 by explicitly introducing a unique-maximum (tie-free) condition and revised the corresponding scope and proof of Proposition 2(I) to ensure theoretical rigor. The above modifications are highlighted in blue inAppendix DandE.2, respectively.Reviewer 96GR): We recalculate relative gains for each training mode of SIRLM against the strongest baseline. We also conduct three-seed experiments on six representative datasets and report mean ± std for MRR and Hit@10, supporting the reliability of our results.We appreciate your time and careful consideration of our submission. We hope this summary assists you in navigating our rebuttal situation. The revised PDF containing all these improvements has been uploaded.
Best regards,
on behalf of the authors of Paper 14701