- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This work aims to reduce the LLM calls during the search process in multi-hop reasoning for KGQA. TriFOR runs PUCT search over question-specific Freebase subgraphs, guided by a lightweight forecaster. A PLM encoder of path prefixes, a hop-level Transformer and cross-attention to the question form three heads to predict an action prior, a continuation value and a stop probability. The forecaster is pretrained on LLM-validated gold paths. It also introduces DeepHopQA as a synthetic benchmark for multi-hop KGQA, built from CWQ subgraphs with LLM path selection, question generation and validation.
Critical strengths
The motivation of removing in-loop LLM calls from KG tree search is practical and clear, the improvement is also confirmed in the evaluation.
Critical weaknesses
-
The conceptual novelty is modest, as policy-guided MCTS for graph walking was already introduced by prior works such as M-Walk, which the paper cites only in the appendix. RL-based KG path finding approaches such as DeepPath (Xiong et al., EMNLP 2017), MINERVA (Das et al., ICLR 2018) and MultiHopKG (Lin et al., EMNLP 2018), which includes learned stopping via a NO_OP action, are not discussed. The paper should state precisely what is new relative to these works.
-
DeepHopQA subgraphs are built by explicitly injecting the gold path and expanding neighbourhoods around gold-path nodes, which can leave structural hints (e.g., density or degree signatures along the gold path) that a learned forecaster trained on the same distribution can exploit.-- this should not be allowed in practice.
Questions for authors
N/A
Thank you for highlighting these relevant works. While prior studies have explored reinforcement learning and MCTS for graph traversal, TriFOR differs from prior graph-search methods in objective, supervision, and search guidance. M-Walk combines learned policy and action-value guidance with MCTS for relation-query graph walking, with termination represented as a STOP action. DeepPath$^{[1]}$ learns relational paths through embedding-based reinforcement learning, while MINERVA$^{[2]}$ and MultiHopKG$^{[3]}$ use reinforcement learning to navigate knowledge graphs for relation-query answering.
TriFOR differs in both its learning supervision and its evidence-oriented search objective. Rather than learning graph traversal primarily from reward signals for structured relation queries, TriFOR targets semantically relevant evidence-path retrieval for natural-language KGQA and constructs step-level supervision from LLM-validated gold reasoning paths. Specifically, we jointly train an action prior, a state-level continuation value, and an explicit answer-readiness signal to guide PUCT-based action selection and expansion, leaf evaluation and value-based pruning, and adaptive termination, respectively. By conditioning these complementary forecasts on the natural-language question and partial reasoning path, TriFOR retrieves ranked evidence paths for final LLM answering, replacing repeated in-loop LLM decisions with lightweight forecaster evaluations while retaining a single final-answer LLM call.
We further strengthen the empirical evidence for this design. As reported in Tables R.1–R.2 of our response to Reviewer 5Nm5 [W1, Q1], removing the policy, value, or stop head on DeepHopQA reduces Hits@1 by 12.8, 7.6, and 16.3 percentage points, respectively, using value-threshold termination for the stop-head ablation. Moreover, two comparably trained alternatives, a joint state classifier and a learned STOP-action formulation, remain numerically weaker than TriFOR across WebQSP, CWQ, and DeepHopQA, with differences ranging from 0.7 to 2.5 percentage points.
These results support the benefit of the complementary forecasting signals within TriFOR's lightweight KGQA search framework. We will expand the discussion of these prior approaches in the main manuscript and clarify TriFOR's methodological distinctions and contributions.
[1] Xiong et al. DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning. EMNLP, 2017. [2] Das et al. Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning. ICLR, 2018. [3] Lin et al. Multi-Hop Knowledge Graph Reasoning with Reward Shaping. EMNLP, 2018.
Thank you for raising this concern. DeepHopQA guarantees the inclusion of LLM-validated gold reasoning paths to support controlled long-hop evaluation. We agree that this construction could introduce structural cues correlated with the annotated gold paths.
To investigate whether TriFOR's performance depends on these cues, we conducted additional experiments on 1,000 DeepHopQA questions. Specifically, we reconstructed question-specific subgraphs without using annotated gold paths for edge injection or neighborhood expansion and evaluated TriFOR and ReaRev under identical graph conditions. This intervention changes subgraph construction, not the use of gold reasoning paths as training supervision.
Table R.11. Evaluation on gold-blind reconstructed subgraphs. Values indicate TriFOR minus ReaRev Hits@1 (percentage points).
In the first setting, we evaluated the existing checkpoints on reconstructed test subgraphs without gold-path-aware edge injection or expansion. In the second, both methods were retrained using matched gold-blind subgraph conditions. These results show that TriFOR's relative advantage on the reconstructed graphs does not require gold-path-aware test-subgraph construction and persists when both methods are retrained under the same conditions.
We additionally evaluated robustness in increasing graph complexity. As detailed in our response to Reviewer idQ2 [Q6], TriFOR exhibits substantially smaller performance degradation than ReaRev when additional connected distractor edges are introduced.
These comparisons address robustness under reconstructed graphs rather than establishing superiority under every graph construction. Together, they provide evidence that TriFOR remains effective without gold-path-aware test-subgraph construction. We will incorporate these results and clarify the scope of DeepHopQA's controlled evaluation in the revised manuscript.