acceptodds
Under review as a conference paper at ICLR 2027

IKBQA-R1: Reinforcing Large Language Models for Question Answering over Incomplete Knowledge Graphs

Abstract

Knowledge Base Question Answering (KBQA) aims to answer natural language questions through reasoning over structured knowledge graphs (KGs), offering explicit and traceable inference that is valuable in hallucination-sensitive domains. However, real-world KGs are inherently incomplete, causing even semantically correct logical forms to return empty or incomplete answers when required evidence is missing. Existing approaches mitigate KG incompleteness by generating missing facts from LLMs or incorporating retrieved text into multi-step reasoning, but such textual evidence is often fragmented, transient, and difficult to accumulate and reuse across multiple reasoning steps. To address these limitations, we propose , an interactive reinforcement learning framework that formulates incomplete KBQA as a multi-turn decision process and expands the structured action space with \texttt{Search\\_Web} and actions for evidence integration into the logical form. To mitigate the cold-start problem, we first initialize the policy through Referenced Rejection Sampling and hint-free multi-turn supervised fine-tuning; we then introduce into GRPO, aligning generated actions with gold action anchors and projecting local action quality into bounded token-level credit adjustments. We further construct the IKBQA benchmarks WebQSP-IKG and GraphQ-IKG from the widely used WebQSP and GraphQuestions datasets to support multi-turn policy learning and evaluation under controlled KG incompleteness. Experimental results show that IKBQA-R1 outperforms the baseline on WebQSP-IKG by 8.1% in relative F1 improvement, while achieving 32.7 and 16.1 F1-point gains over gold execution without external evidence on WebQSP-IKG and GraphQ-IKG, respectively.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Knowledge Base Question Answering, Incomplete Knowledge Graphs, Large Language Models, Reinforcement Learning, Interactive Reasoning, External Evidence Recovery, Action-Aware Credit Assignment

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_MZz1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper adapts the KBQA‑R1 agent‑reinforcement‑learning paradigm to incomplete knowledge‑base question answering (IKBQA). The authors formulate incomplete‑KBQA as a multi‑turn Markov decision‑making process. Building upon the existing KBQA‑R1 pipeline (RRS trajectory sampling, multi‑turn SFT cold‑start, GRPO reinforcement learning, S‑Expression executable reasoning), they add two new atomic actions: Search Web (mapped to WEB‑JOIN) for retrieving external evidence and Combine (mapped to UNION) to merge knowledge‑graph outputs with external retrieved results into a unified executable logical form that supports downstream reasoning operations. Two new benchmarks WebQSP‑IKG and GraphQ‑IKG are constructed by controlled hiding of KG triples to simulate knowledge incompleteness for multi‑turn agent training and evaluation. Empirical results show clear F1 improvements over prior incomplete‑KBQA baselines (GoG, DoM) on their curated benchmarks.

Critical strengths
  1. Solid executable integration of external evidence
  2. Action‑aware credit assignment is a useful incremental RL improvement
  3. New task‑specific benchmarks for trainable IKBQA agents
  4. Strong empirical performance on the proposed benchmarks
Critical weaknesses
  1. Nearly the full training pipeline is inherited from KBQA‑R1: multi‑turn MDP formulation, RRS referenced‑rejection sampling, hint‑free multi‑turn SFT, base GRPO setup and S‑Expression execution engine. Major modifications are confined to two new atomic actions plus the GRPO credit‑assignment tweak.
  2. There is no ablation where the pre‑trained KBQA‑R1 model is directly prompted with descriptions of Search‑Web / Combine without re‑training via SFT+RL. This experiment would help quantify how much performance gain comes purely from prompt engineering versus the full re‑training pipeline.
  3. All external evidence is pre‑cached offline, rather than retrieved from open‑web corpora at runtime. The setup avoids real‑world challenges: open‑domain retrieval noise, entity‑relation alignment failures, cache misses.
Questions for authors

Please see above weaknesses. The limited novelty of the whole framework and the missing comparison with zero‑shot prompting of original KBQA‑R1 are major concerns.

6 comments
AuthorsOpenReview ↗

[Response to Weakness 1(a)] Clarifying Contributions beyond the Inherited Training Framework

Thank you for identifying the training components inherited from KBQA-R1.

Initializing a policy with supervised trajectories and then applying reinforcement learning through environment interaction is a general approach to training multi-turn tool-use policies. We adopt KBQA-R1's specific training framework, retaining its multi-turn MDP formulation, RRS, hint-free multi-turn SFT, base GRPO, and S-Expression execution foundations. Our method turns external evidence into program state that supports continued computation and uses action structure to refine GRPO's token-level advantage allocation. The interface comparison in Table 8 and allocation-rule comparison in Table 15 provide performance evidence for these two designs, respectively.

First, we construct data and an environment for policy learning. We replay gold programs on the complete KG, collect direct triples and CVT evidence for relation steps, hide supporting facts at one selected step, and produce a reference program with external retrieval and integration. The same rules apply to training and test splits: Table 10 reports 2,929/1,566 examples for WebQSP-IKG and 2,220/2,151 for GraphQ-IKG. RRS uses reference-action hints to prepare warm-start trajectories, with acceptance rates of 75.6%/73.2% (Table 6). Hints are removed from SFT dialogues and are absent from RL policy inputs and test inputs; gold annotations remain available internally for training rewards and action alignment. The sampling–SFT–RL procedure is inherited, while the data, environment, and execution supervision support learning under KG incompleteness.

Second, external evidence becomes part of a program state that supports continued execution. Search_Web creates a WEB_JOIN branch, and Combine integrates it with the KG branch through UNION, enabling further queries, intersections, filters, ordering, and counting. Figure 9 (Appendix C.1) shows continued city querying after retrieving an intermediate entity, yielding Fairfax; Figure 20 (Appendix G.1) shows subsequent type filtering and MAX ordering. In Table 8, executable integration improves the two GPT models by 7.32/10.39 F1 over textual context with shared questions and evidence, alongside the corresponding call and token costs. These results support using external evidence as input to subsequent program operations.

To maintain this continuity, Lineage-Preserving Lazy Execution in Appendix E.2 retains the KG branch's source expression, relation path, direction, hidden-edge exclusions, and subsequent constraints alongside external results. The model receives a bounded preview, while later execution and trajectory rewards use the complete semantics, avoiding treating a truncated preview as the full entity set. This also supports filtering, intersection, and counting after high-fan-out queries. Precomputation and on-demand loading of large JOINs further reduce repeated-query costs; lazy state maintenance and query acceleration respectively support execution correctness and efficiency. Figure 19 (Appendix E.1) describes external-evidence construction and runtime integration.

Third, action-aware credit assignment refines base GRPO. Algorithm 3 (Appendix B.5) monotonically aligns generated actions with training gold-action anchors, allowing skips on both sides. Local quality based on tools, relations, and arguments is normalized by span length, projected to action and paired reasoning tokens, and incorporated through bounded advantage refinement. Trajectory rewards and group normalization are preserved; unmatched actions retain the trajectory-level advantage. Table 15 reports 71.4/42.4 F1 for base GRPO, 73.5/46.2 for uniform local allocation, and 75.1/49.9 for aligned allocation. Local supervision first adds 2.1/3.8 points, and action-specific allocation adds another 1.6/3.7. Figure 10 (Appendix C.2) illustrates allocation differences with fixed total credit, while the table provides performance evidence.

Together, these designs support reasoning quality and interaction efficiency. Table 5 reports 4.77/5.35 calls for the trained 8B policy on WebQSP/GraphQ, compared with 12.33/15.06 for DoM. In the same-environment comparison of Table 7, the 8B policy reaches 75.05 F1 on WebQSP versus 68.22 and 66.16 for the two GPT models, while using fewer input tokens. Building on the inherited training framework, our additions provide traceable data for policy learning, executable integration and lazy state maintenance for continued computation, and action-level credit for optimizing multi-step decisions.

We appreciate this observation, which helped distinguish the inherited framework from our execution and credit-assignment designs.

AuthorsOpenReview ↗

**[Response to Weakness 1(b)] Clarifying Contributions beyond the Inherited Training Framework**

Table 6: Trajectory acceptance and SFT initialization F1 (%).

\text{Dataset} & \text{Pipeline} & \text{Accepted/Total} & \text{Acceptance (\\%)} & \text{SFT Init F1} \\\\ \hline \text{WebQSP} & \text{KBQA-R1} & 1505/2929 & 51.4 & 72.1 \\\\ \text{WebQSP-IKG} & \text{IKBQA-R1} & 2213/2929 & 75.6 & 69.2 \\\\ \hline \text{GraphQ} & \text{KBQA-R1} & 1562/2332 & 67.0 & 47.8 \\\\ \text{GraphQ-IKG} & \text{IKBQA-R1} & 1626/2220 & 73.2 & 41.7 \\\\ \end{array}$$ **Table 8: Textual versus executable evidence integration on 256 WebQSP-IKG test questions.** $$\begin{array}{lllrrr} \text{Evidence integration} & \text{Setting} & \text{Model} & \text{F1 (\\%)} & \text{LLM calls} & \text{Tokens In / Out} \\\\ \hline \text{Executable program} & \text{IKBQA-R1 harness} & \text{GPT-5.6-Sol} & 72.39 & 4.82 & 13{,}502 / 622 \\\\ \text{Textual context} & \text{Adapted harness} & \text{GPT-5.6-Sol} & 65.07 & 4.12 & 7{,}834 / 638 \\\\ \Delta\text{ (textual - executable)} & & & -7.32 & -0.70 & -5{,}668.1 / {+}15.5 \\\\ \hline \text{Executable program} & \text{IKBQA-R1 harness} & \text{GPT-6-Sol} & 72.42 & 5.03 & 14{,}352 / 513 \\\\ \text{Textual context} & \text{Adapted harness} & \text{GPT-6-Sol} & 62.02 & 4.62 & 9{,}350 / 659 \\\\ \Delta\text{ (textual - executable)} & & & -10.39 & -0.41 & -5{,}001.6 / {+}146.0 \\\\ \hline \text{Executable program} & \text{IKBQA-R1 (RL)} & \text{Llama-3.1-8B} & 73.79 & 5.21 & 14{,}482 / 592 \\\\ \text{Textual context} & \text{IKBQA-R1 (RL)} & \text{Llama-3.1-8B} & 49.91 & 6.23 & 11{,}716 / 701 \\\\ \Delta\text{ (RL textual - executable)} & & & -23.88 & +1.02 & -2{,}765.9 / {+}109.2 \\\\ \end{array}$$ **Table 15: Effect of local credit allocation on answer F1 (%) for WebQSP-IKG and GraphQ-IKG.** $$\begin{array}{lrr} \text{Credit assignment} & \text{WebQSP-IKG} & \text{GraphQ-IKG} \\\\ \hline \text{Trajectory-level GRPO} & 71.4 & 42.4 \\\\ \text{Uniform local allocation} & 73.5 & 46.2 \\\\ \text{Aligned action-aware allocation} & 75.1 & 49.9 \\\\ \end{array}$$ **Table 5: Average LLM calls per sample during inference. Lower values indicate fewer policy decisions.** $$\begin{array}{lrr} \text{Method} & \text{WebQSP} & \text{GraphQ} \\\\ \hline \text{DoM} & 12.33 & 15.06 \\\\ \text{KBQA-o1 (Complete KG)} & 28.8 & 78.0 \\\\ \text{KBQA-R1 (Complete KG)} & 2.65 & 3.16 \\\\ \text{IKBQA-R1 (Incomplete KG)} & 4.77 & 5.35 \\\\ \end{array}$$ **Table 7: F1 and inference costs of generalist models and learned policies in IKBQA-R1. “Harness” uses the IKBQA-R1 action space and feedback format with an off-the-shelf model, while IKBQA-R1 uses the learned RL policy.** $$\begin{array}{llrrrrrrrr} \text{Harness} & \text{Model} & \text{F1 (\\%)} & & & \text{Turns} & & & \text{Tokens In / Out} & \\\\ & & \text{WebQSP} & \text{GraphQ} & \text{Avg.} & \text{WebQSP} & \text{GraphQ} & \text{Avg.} & \text{WebQSP} & \text{GraphQ} \\\\ \hline \text{IKBQA-R1 Harness} & \text{GPT-5.6-Sol} & 68.22 & 54.59 & 61.41 & 4.92 & 5.67 & 5.36 & 14{,}338 / 659 & 16{,}587 / 988 \\\\ \text{IKBQA-R1 Harness} & \text{GPT-6-Sol} & 66.16 & 51.14 & 58.65 & 5.18 & 5.90 & 5.60 & 15{,}185 / 535 & 17{,}483 / 763 \\\\ \hline \text{IKBQA-R1 Harness} & \text{DeepSeek-V4.1-Flash} & 54.04 & 49.54 & 51.79 & 5.32 & 5.76 & 5.57 & 17{,}097 / 476 & 18{,}674 / 683 \\\\ \text{SFT policy} & \text{Llama-3.1-8B} & 69.17 & 41.73 & 55.45 & 5.21 & 4.73 & 4.93 & \text{—} & \\\\ \text{IKBQA-R1} & \text{Llama-3.1-8B} & 75.05 & 49.93 & 62.49 & 4.77 & 5.35 & 5.11 & 12{,}550 / 347 & 13{,}763 / 773 \\\\ \end{array}$$
AuthorsOpenReview ↗

[Response to Weakness 2(a)] Tool Descriptions and the Contribution of Policy Training

Thank you for suggesting a comparison between tool prompting and policy training.

Descriptions of Search_Web and Combine explain how the tools are used, but attributing performance gains to prompting requires comparisons with policy training. We provide evidence through the same-environment comparison of generalist models, SFT, and RL in Table 7, together with training-stage ablations in Table 4. Same-backbone SFT-to-RL gains are 5.88/8.20 F1 points on WebQSP-IKG/GraphQ-IKG, showing that further environment-interaction training improves performance after an initial tool-use policy has been established.

The KBQA-R1 checkpoint has been specialized through SFT and RL for interaction over complete KGs, with a training action space that excludes Search_Web and Combine. New tool descriptions provide operational rules, while our training further optimizes the policy for these actions and subsequent execution. To examine these roles, we report generalist models directly using the tool interface and same-backbone training comparisons within our task.

In main-text Table 7, DeepSeek-V4.1-Flash, GPT-5.6-Sol, and GPT-6-Sol act directly as agents without additional task-specific training. All models share the tools, execution feedback, KG, evidence cache, and eight-turn interaction limit. Each generates one trajectory per question without gold reference-action hints. This setting allows generalist models to use their existing capabilities and tool descriptions, while the same 8B backbone's SFT and RL results characterize gains from further policy optimization.

The performance results show that task training provides further gains after tool descriptions are supplied. On WebQSP-IKG, DeepSeek-V4.1-Flash, GPT-5.6-Sol, and GPT-6-Sol obtain 54.04, 68.22, and 66.16 F1, respectively. Llama-3.1-8B reaches 69.17 after SFT and 75.05 after RL, exceeding the two GPT models by 6.83 and 8.89 percentage points. Same-backbone SFT-to-RL improvements are from 69.17 to 75.05 on WebQSP-IKG and from 41.73 to 49.93 on GraphQ-IKG, gains of 5.88 and 8.20 points. On GraphQ-IKG, DeepSeek, GPT-5.6-Sol, and GPT-6-Sol obtain 49.54, 54.59, and 51.14, respectively; the trained 8B obtains 49.93, slightly above DeepSeek but below both GPT models. Its two-dataset average F1 is 62.49, compared with 61.41 for GPT-5.6-Sol, 58.65 for GPT-6-Sol, and 51.79 for DeepSeek. In this two-dataset comparison, the trained 8B policy therefore has higher average F1 than the three direct generalist agents, while retaining the performance differences across datasets.

Table 4 further examines training stages within the task. SFT alone obtains 69.2/41.7 F1 on WebQSP-IKG/GraphQ-IKG; RL without SFT initialization obtains 70.5/43.5; full training obtains 75.1/49.9. Combining SFT initialization and subsequent RL outperforms either stage alone on both datasets, providing same-backbone evidence for the training procedure.

The performance gains are not obtained through more interaction turns. The RL policy uses 4.77 and 5.35 turns on WebQSP-IKG and GraphQ-IKG, respectively. GPT-5.6-Sol uses 4.92 and 5.67, GPT-6-Sol uses 5.18 and 5.90, and DeepSeek uses 5.32 and 5.76. The trained policy averages fewer interaction turns than all three generalist models on both datasets.

For token usage in Table 7, the RL policy's WebQSP-IKG input/output counts are 12,550/347, compared with 14,338/659 for GPT-5.6-Sol, 15,185/535 for GPT-6-Sol, and 17,097/476 for DeepSeek. On GraphQ-IKG, the RL policy uses 13,763/773, compared with 16,587/988, 17,483/763, and 18,674/683 for the three generalist models in the same order. The trained policy uses fewer input tokens on both datasets and fewer output tokens on WebQSP-IKG. Its GraphQ-IKG output count is below GPT-5.6-Sol, close to GPT-6-Sol, and above DeepSeek. The trained policy thus maintains fewer interaction turns and input tokens, while output-length differences depend on the dataset and comparison model.

Figure 14 (Appendix D.3) and Figure 15 (Appendix D.5) further characterize behavioral changes after training. Figure 14 shows improved outcome categories for 203 WebQSP-IKG questions versus declines for 95, and improvements for 399 GraphQ-IKG questions versus declines for 211. In Figure 15, nonempty-evidence coverage on GraphQ-IKG increases from 36.12% to 69.13%, and successful-integration coverage from 31.24% to 65.32%. These results connect same-backbone SFT-to-RL F1 gains with improvements in evidence acquisition and integration, supporting further task-policy training beyond tool descriptions.

We appreciate this suggestion, which helped organize the evidence on tool interfaces and task-specific training.

Reviewer_Sz6KOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes IKBQA-R1, a reinforcement learning framework for question answering over incomplete knowledge graphs, where missing facts can prevent otherwise correct queries from producing complete answers. Building on an executable KBQA framework, the method introduces Search_Web and Combine actions to obtain external evidence and merge it with KG-derived results, allowing subsequent reasoning operations to use the integrated information.

The training pipeline combines referenced rejection sampling, supervised fine-tuning, and GRPO. Its action-aware credit assignment mechanism aligns generated actions with reference action sequences and adjusts token-level policy updates according to local action quality. The authors construct WebQSP-IKG and GraphQ-IKG by selectively hiding supporting facts and rewriting the corresponding query programs. Under a controlled setting with a preconstructed external-evidence cache, IKBQA-R1 achieves 75.1 and 49.9 answer F1 on the two benchmarks, respectively, outperforming DoM by 5.6 points on WebQSP-IKG. The paper further presents training ablations and reports fewer inference-time LLM calls than DoM on WebQSP-IKG.

Critical strengths

1. Executable integration of external evidence. Instead of passing retrieved text to the LLM, the paper represents external evidence as a WEB_JOIN branch that is merged into the logical form via UNION. Retrieved results are grounded to Freebase identifiers and can be reused directly by subsequent operations, so the whole reasoning process remains executable and traceable, which is valuable in hallucination-sensitive settings. The resulting agent is also efficient, requiring 4.70 and 4.28 LLM calls per question on WebQSP-IKG and GraphQ-IKG, compared with 22.02 for DoM (Table 5).

2. A training-ready benchmark with detailed documentation. The paper constructs WebQSP-IKG and GraphQ-IKG through executable-program replay and controlled evidence hiding. As the paper notes, existing incomplete-KG QA benchmarks mainly target training-free evaluation. These new benchmarks instead provide training splits together with the original and rewritten programs and evidence provenance, which makes it possible to learn and analyze multi-turn policies under KG incompleteness. The construction procedure, data statistics, and training configurations are documented in detail, and the paper states that data-processing, training, and evaluation scripts are included in the supplementary material.

Critical weaknesses

1. Reliance on the teacher LLM and choice of backbone. The paper targets KG incompleteness, but the proposed solution essentially distills the tool-use behavior of a stronger LLM (DeepSeek-V4-Flash) into Llama-3.1-8B. The capability therefore still comes from large language models rather than from a new way of handling KG incompleteness. This raises two questions. First, if DeepSeek-V4-Flash can already perform the task, why not use it directly as the agent? The paper reports DeepSeek-V4-Flash only inside ToG, GoG, and DoM, and during trajectory collection it is guided by gold reference action sequences. A baseline in which DeepSeek-V4-Flash acts as the agent in the same environment, with the same action space but without reference hints, is needed to show what the training actually adds. Second, if a smaller model is preferred for deployment, why use Llama-3.1-8B, which is now fairly outdated? Recent models such as the Qwen3 series (4B, 8B, 14B) already have strong tool-calling abilities. Although Llama-3.1-8B matches the backbone of KBQA-o1 and KBQA-R1, results with such recent models would show whether the proposed training still brings gains.

2. Limited methodological novelty. The pipeline of collecting teacher trajectories and training a smaller model with SFT and GRPO follows KBQA-R1, and teaching an LLM to call a search tool via RL has also been explored (e.g., Search-R1). The only algorithm-level addition, action-aware credit assignment, is a relatively small modification that relies on gold programs. Overall, the contribution appears to be primarily data-level engineering rather than a methodological advance in SFT or RL. An algorithmic contribution at the SFT or RL level would make the paper considerably more convincing.

3. Limited experiments. The evaluation covers only two Freebase datasets, WebQSP and GraphQuestions, whose questions involve relatively short reasoning chains (the median number of actions in the constructed benchmarks is 3 and 5, respectively), and no competing method is evaluated on GraphQ-IKG. The method should also be evaluated on more complex multi-hop KBQA datasets, such as ComplexWebQuestions (CWQ) or GrailQA, to which the same construction pipeline should apply.

Questions for authors

see weaknesses

10 comments
AuthorsOpenReview ↗

[Response to Weakness 1(a)] The Role of the Teacher and the Benefits of Policy Training

Thank you for asking why a smaller trained policy is preferable to using a large model directly.

Using a large model directly is feasible, but successful teacher rollouts guided by reference actions do not imply comparable performance without reference hints. The teacher provides cold-start data, while subsequent RL further optimizes the policy through environment interaction. Teacher-trajectory initialization and subsequent policy optimization serve distinct roles: training a smaller policy improves performance beyond initialization while reducing deployment-time interaction overhead.

Direct-agent comparisons without reference hints address why one should not simply use a large model directly. Table 7 adds DeepSeek-V4.1-Flash and further evaluates GPT-5.6-Sol and GPT-6-Sol. The three models share the IKBQA-R1 tool interface, execution feedback, KG, evidence cache, and eight-turn interaction limit with the trained policy. Each generates one trajectory per question without gold reference-action hints. On WebQSP-IKG, the three direct agents obtain 54.04, 68.22, and 66.16 F1, respectively, while the trained 8B policy reaches 75.05, exceeding the two GPT models by 6.83 and 8.89 percentage points. On GraphQ-IKG, the three models obtain 49.54, 54.59, and 51.14, respectively; the trained 8B reaches 49.93, slightly above DeepSeek but below both GPT models. Averaged across the two datasets in Table 7, the trained policy obtains 62.49 F1, compared with 51.79, 61.41, and 58.65 for DeepSeek, GPT-5.6-Sol, and GPT-6-Sol. These results support training a smaller policy while retaining the differences in performance across datasets.

Same-backbone SFT-to-RL comparisons further show that the gains extend beyond teacher-trajectory initialization. F1 increases from 69.17 after SFT to 75.05 after RL on WebQSP-IKG, and from 41.73 to 49.93 on GraphQ-IKG, gains of 5.88 and 8.20 percentage points. Figure 14 (Appendix D.3) shows improved outcome categories for 203 WebQSP questions versus declines for 95; the corresponding GraphQ counts are 399 and 211. Figure 15 (Appendix D.5) shows successful-integration coverage on GraphQ increasing from 31.24% to 65.32%. These results connect subsequent RL gains with improved evidence-integration behavior, showing that hint-free environment interaction can further optimize a policy initialized from teacher trajectories.

The teacher uses training-time reference hints to generate cold-start trajectories. Gold programs guide DeepSeek Flash's stepwise interaction, and RRS filters the resulting rollouts by execution quality. Table 6 reports acceptance of 2,213/2,929 WebQSP-IKG trajectories (75.6%) and 1,626/2,220 GraphQ-IKG trajectories (73.2%). Hints are removed before SFT and remain absent from RL policy inputs and testing; gold programs are used internally for training rewards and action alignment. Teacher acceptance rates therefore do not establish hint-free test performance.

The smaller trained policy also reduces interaction overhead. In Table 7, the 8B policy averages 4.77/5.35 calls on WebQSP/GraphQ, compared with 4.92/5.67 for GPT-5.6-Sol, 5.18/5.90 for GPT-6-Sol, and 5.32/5.76 for DeepSeek. The trained policy uses fewer model calls on both datasets, so its performance is not obtained by taking more interaction turns. Together with the same-backbone training gains, these results support using a teacher to prepare cold-start data and then optimizing a deployable smaller policy through environment interaction.

Table 7: F1 and inference costs of generalist models and learned policies in IKBQA-R1. “Harness” uses the IKBQA-R1 action space and feedback format with an off-the-shelf model, while IKBQA-R1 uses the learned RL policy.

\text{Harness} & \text{Model} & \text{F1 (\\%)} & & & \text{Turns} & & & \text{Tokens In / Out} & \\\\ & & \text{WebQSP} & \text{GraphQ} & \text{Avg.} & \text{WebQSP} & \text{GraphQ} & \text{Avg.} & \text{WebQSP} & \text{GraphQ} \\\\ \hline \text{IKBQA-R1 Harness} & \text{GPT-5.6-Sol} & 68.22 & 54.59 & 61.41 & 4.92 & 5.67 & 5.36 & 14{,}338 / 659 & 16{,}587 / 988 \\\\ \text{IKBQA-R1 Harness} & \text{GPT-6-Sol} & 66.16 & 51.14 & 58.65 & 5.18 & 5.90 & 5.60 & 15{,}185 / 535 & 17{,}483 / 763 \\\\ \hline \text{IKBQA-R1 Harness} & \text{DeepSeek-V4.1-Flash} & 54.04 & 49.54 & 51.79 & 5.32 & 5.76 & 5.57 & 17{,}097 / 476 & 18{,}674 / 683 \\\\ \text{SFT policy} & \text{Llama-3.1-8B} & 69.17 & 41.73 & 55.45 & 5.21 & 4.73 & 4.93 & \text{—} & \\\\ \text{IKBQA-R1} & \text{Llama-3.1-8B} & 75.05 & 49.93 & 62.49 & 4.77 & 5.35 & 5.11 & 12{,}550 / 347 & 13{,}763 / 773 \\\\ \end{array}$$ We appreciate this suggestion, which helped distinguish teacher initialization, policy optimization, and direct-agent evaluation.
AuthorsOpenReview ↗

[Response to Weakness 1(b)] Backbone Selection and Training Benefits on Qwen3-8B

Thank you for suggesting that we test training gains on a newer backbone.

We chose Llama-3.1-8B to retain the backbone used by KBQA-o1 and KBQA-R1, allowing us to examine changes to the environment and training mechanisms within the established executable KBQA framework. To test whether a newer backbone still benefits, Table 19 (Appendix C.5) adds a same-backbone SFT/RL comparison with Qwen3-8B: RL increases its WebQSP-IKG F1 from 70.23 to 76.42, a gain of 6.19 percentage points. This result shows that the training gains also occur on the newer Qwen3-8B and are not limited to Llama-3.1-8B.

The backbone choice first supports comparison with prior methods. Both KBQA-o1 and KBQA-R1 use Llama-3.1-8B. Retaining this backbone helps examine the roles of the incomplete-KG environment, external evidence integration, and policy optimization on the same underlying model. We agree that results on this backbone alone do not establish whether the training remains useful for recent models, which motivated the additional Qwen3-8B experiment.

The Qwen3-8B results directly test whether a newer model continues to benefit from RL after supervised initialization. In Table 19, Qwen3-8B obtains SFT/RL F1 of 70.23/76.42, a gain of 6.19 percentage points; the corresponding Llama-3.1-8B scores are 69.17/75.05, a gain of 5.88 points. Qwen3-8B already scores higher after SFT, yet improves further through RL. A newer backbone's general tool-calling ability can therefore coexist with gains from task-specific policy training. On WebQSP-IKG, both 8B backbones show consistent SFT-to-RL improvements.

Table 19: backbone comparison on WebQSP-IKG (F1 %)

\text{Backbone} & \text{SFT} & \text{RL} & \text{Gain} \\\\ \hline \text{Llama-3.1-8B} & 69.17 & 75.05 & +5.88 \\\\ \text{Qwen3-8B} & 70.23 & 76.42 & +6.19 \\\\ \end{array}$$ We appreciate this suggestion, which motivated the Qwen3-8B comparison and clarified our backbone choice.
AuthorsOpenReview ↗

[Response to Weakness 2(a)] Action-Aware Credit Assignment beyond the Inherited Training Pipeline

Thank you for asking us to clarify the algorithmic contribution.

Initializing a policy with supervised trajectories and then applying reinforcement learning through environment interaction is a general approach to training multi-turn tool-use policies. We retain KBQA-R1's RRS, SFT, and basic GRPO training pipeline. Furthermore, our contributions focus on executable evidence integration and action-aware credit assignment. The former turns evidence outside the KG into intermediate results usable by subsequent program operations; the latter converts action structure from training gold programs into local token-level advantages, refining updates to different actions within a trajectory. In Table 15, aligned allocation improves over uniform local allocation by a further 1.6/3.7 F1 points on WebQSP-IKG/GraphQ-IKG. This comparison provides more direct algorithmic evidence for the action-level allocation rule than overall SFT-to-RL gains alone.

The RL addition incorporates structured action supervision into GRPO advantage allocation. Standard GRPO broadcasts the same trajectory-level advantage across the response, although a trajectory may contain correct evidence retrieval, incorrect relation selection, and valid integration operations. We use monotonic dynamic programming with skips on both sides to align generated actions with gold action anchors, then compute matched-action quality from tool type, canonical relations, and arguments. Local quality is normalized by action-span length and projected to the corresponding tokens, with a lower weight for paired reasoning spans. Gating and a bounded residual then incorporate this signal into the original GRPO advantage. The mechanism preserves trajectory-level rewards and group normalization while refining update strength for local decisions. Unmatched actions retain the trajectory-level advantage, and disabling the local term recovers basic GRPO. Algorithm 3 (Appendix B.5) specifies the computation and alignment procedure.

Table 15 further separates the benefit of adding local supervision from allocating it by action. Trajectory-level GRPO obtains 71.4/42.4 F1 on WebQSP-IKG/GraphQ-IKG, uniform local allocation obtains 73.5/46.2, and aligned action allocation obtains 75.1/49.9. Uniform allocation improves over trajectory-level GRPO by 2.1/3.8 percentage points; alignment adds a further 1.6/3.7 points. These results support both adding local signals and determining their allocation locations and strengths from matched-action quality.

Figure 10 (Appendix C.2) illustrates the two allocation rules on the same GraphQ trajectory. Combine and Merge each contain nine tokens, and total action credit remains 1.5958. Uniform allocation assigns 0.7979 to each, whereas aligned allocation retains 1.0000 and 0.5958, respectively. The example shows how the same total credit is distributed across different actions. Figure 8 (Appendix B.5) presents the overall computation, and Figure 11 (Appendix C.2) provides an offline credit analysis of recorded trajectories. Table 16 reports alignment coverage: overall coverage is 75.16%/29.33% on WebQSP/GraphQ, and Combine coverage is 95.84%/95.12%. These analyses explain the mechanism and its coverage; performance evidence comes from the allocation-rule comparison in Table 15.

Table 15: Effect of local credit allocation on answer F1 (%) for WebQSP-IKG and GraphQ-IKG.

\text{Credit assignment} & \text{WebQSP-IKG} & \text{GraphQ-IKG} \\\\ \hline \text{Trajectory-level GRPO} & 71.4 & 42.4 \\\\ \text{Uniform local allocation} & 73.5 & 46.2 \\\\ \text{Aligned action-aware allocation} & 75.1 & 49.9 \\\\ \end{array}$$ **Table 16: Action-alignment coverage on full-test RL rollouts, with matched/parsed action counts and coverage rates (%).** $$\begin{array}{lrrrr} \text{Action} & \text{WebQSP-IKG} & & \text{GraphQ-IKG} & \\\\ & \text{Matched/parsed} & \text{Rate (\\%)} & \text{Matched/parsed} & \text{Rate (\\%)} \\\\ \hline \text{Find_relation} & \text{1,716/2,765} & 62.06 & \text{574/4,351} & 13.19 \\\\ \text{Search_Web} & \text{1,175/1,341} & 87.62 & \text{303/2,101} & 14.42 \\\\ \text{Combine} & \text{1,313/1,370} & 95.84 & \text{1,502/1,579} & 95.12 \\\\ \text{Merge} & \text{142/257} & 55.25 & \text{125/440} & 28.41 \\\\ \text{Order} & \text{11/42} & 26.19 & \text{7/59} & 11.86 \\\\ \text{Time_constraint} & \text{48/76} & 63.16 & \text{—} & \text{—} \\\\ \text{Count} & \text{—} & \text{—} & \text{49/81} & 60.49 \\\\ \text{Compare} & \text{0/3} & 0.00 & \text{0/53} & 0.00 \\\\ \text{Answer} & \text{0/7} & 0.00 & \text{0/65} & 0.00 \\\\ \hline \text{All parsed actions} & \text{4,405/5,861} & 75.16 & \text{2,560/8,729} & 29.33 \\\\ \end{array}$$ We appreciate this suggestion; it helped clarify our contributions.
Reviewer_27taOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
3
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes IKBQA-R1, an interactive reinforcement learning framework for question answering over incomplete knowledge bases. It extends executable KG reasoning with two actions: Search_Web, which obtains relation-specific external evidence, and Combine, which incorporates that evidence into the current executable state. Training consists of referenced rejection sampling, hint-free supervised fine-tuning, and GRPO with answer/trace rewards and action-aware credit assignment. The authors construct WebQSP-IKG and GraphQ-IKG by hiding supporting evidence from existing KGQA examples. Under an annotation-driven, cached-evidence protocol, the method achieves 75.1 and 49.9 answer F1, respectively. Ablations support the contributions of RL, trace rewards, and action-aware credit refinement.

Critical strengths
  1. Clear motivation and meaningful evidence integration. The paper addresses an important failure mode: even a correct query can fail when necessary facts are missing. Incorporating external evidence into executable states, rather than simply appending retrieved text, allows subsequent filtering, intersection, and other operations to reuse that evidence.

  2. Traceable benchmark construction. By executing gold programs to identify and hide supporting facts, the authors can control where knowledge is missing while preserving the original questions and answers. This provides a clear experimental setting for studying evidence acquisition and integration.

Critical weaknesses
  1. The retrieval setting is substantially simplified. Search_Web queries a cache constructed from hidden-fact annotations and containing no unrelated distractors. The environment also signals insufficient knowledge at the designated gap. The experiments therefore primarily test querying and integrating clean evidence, rather than handling retrieval noise, conflicting facts, or identifying knowledge gaps in realistic settings. Although acknowledged, this limitation substantially narrows the conclusions.

  2. Limited evidence for broader applicability and generalization. Evaluation covers only two datasets using the same incompleteness construction, with one hiding location per question. Given the broader goal of incomplete-KB question answering, it remains unclear whether the method transfers to more complex datasets or knowledge-gap patterns not encountered during training.

Questions for authors
  1. Is the method robust to retrieval noise? Real-world web search involves irrelevant results, insufficient evidence, and conflicting facts, whereas the current cache contains no unrelated distractors. Could the authors introduce these conditions or evaluate with actual retrieval results to test whether the method remains effective beyond integrating preselected correct evidence?

  2. Can the evaluation cover more datasets? Could the authors construct comparable incomplete-KG settings on datasets such as CWQ to test whether the gains persist on more complex multi-hop questions? Results on the current two datasets provide limited evidence about the method’s broader applicability.

  3. Can OOD experiments establish generalization? Training and testing currently use the same incompleteness construction. Could the authors evaluate on a new dataset or examples with multiple knowledge gaps without retraining? This would help distinguish transferable evidence acquisition and integration from adaptation to the current setup.

  4. Do the baselines have equivalent evidence access? Do ToG, GoG, and DoM have access to equivalent external evidence under comparable interaction budgets? Could the authors also include a control that automatically executes Search_Web and Combine after a missing-evidence signal to isolate the additional benefit of learning this behavior?

6 comments
AuthorsOpenReview ↗

[Response to Question 1] Evaluating Robustness to Retrieval Noise

Thank you for raising the question of retrieval noise and evidence quality.

We added a controlled retrieval-noise experiment in Appendix Table 25. On WebQSP-IKG, retaining correct evidence while adding distractor entities yields F1 close to the unperturbed result and higher than evidence removal or replacement. This difference suggests a retrieval strategy that prioritizes retaining potentially useful evidence while tolerating some additional candidates, rather than filtering so aggressively that useful evidence is lost. Under the tested conditions, preserving valid evidence is more beneficial to final-answer performance than eliminating all distractors.

With GPT-6-Sol fixed as the policy, we evaluate 256 questions from each of WebQSP-IKG and GraphQ-IKG, keeping the prompts, KG, evidence cache, and action budget unchanged while testing missing evidence, additive distractors, and evidence replacement. Perturbations modify the sets actually returned by WEB_JOIN and affect subsequent Combine and query execution; complete trajectories are regenerated for each condition. Let S denote the original retrieval result. Missing evidence sets the selected query's result to the empty set; additive distractors return S∪D, retaining the original evidence; replacement returns D, removing the original evidence, with |D|=|S|. We select D from cached results for other questions and source entities within the same dataset, matching the relation and direction and excluding known correct targets for the current query. A query is eligible for addition or replacement only when its original result is nonempty and sufficient distractor candidates exist; otherwise, its result is preserved. The 10%/30% settings specify the probability that each eligible unique query is perturbed, rather than a proportion of test questions or returned entities; repeated executions of the same query receive consistent results.

On WebQSP-IKG, unperturbed F1 is 64.44. Additive distractors at 10%/30% yield F1 of 63.84/64.02, decreases of only 0.60/0.42 percentage points. Replacing the original evidence yields F1 of 60.75/56.99, decreases of 3.69/7.45 points. This comparison supports the conclusion above: under the tested conditions, addition that preserves correct evidence performs better than replacement.

The additive condition preserves the original evidence while introducing candidate entities from other questions and source entities with the same relation and direction, requiring the model to continue reasoning over results containing distractors. On WebQSP-IKG, F1 under both additive conditions remains close to the unperturbed result and above the replacement conditions. In this single-seed subset evaluation, additional distractors do not substantially reduce final-answer performance, indicating some tolerance to distractors when valid evidence is retained.

As an additional comparison, Table 25 also reports missing-evidence conditions. At 30% missing evidence, F1 decreases from 64.44 and 61.80 to 55.05 and 49.49 on WebQSP-IKG and GraphQ-IKG, respectively, reductions of 9.39 and 12.30 percentage points.

Table 25 — WebQSP-IKG (256 questions; EM/F1 %)

\text{Condition} & \text{p (\\%)} & \text{EM} & \text{F1} & \text{Delta F1} \\\\ \hline \text{Clean} & 0 & 55.08 & 64.44 & 0.00 \\\\ \text{Missing} & 10 & 46.48 & 57.72 & -6.72 \\\\ \text{Missing} & 30 & 42.58 & 55.05 & -9.39 \\\\ \text{Additive} & 10 & 53.52 & 63.84 & -0.60 \\\\ \text{Additive} & 30 & 51.95 & 64.02 & -0.42 \\\\ \text{Replacement} & 10 & 50.78 & 60.75 & -3.69 \\\\ \text{Replacement} & 30 & 44.14 & 56.99 & -7.45 \\\\ \end{array}$$ **Table 25 — GraphQ-IKG (256 questions; EM/F1 %)** $$\begin{array}{lrrrr} \text{Condition} & \text{p (\\%)} & \text{EM} & \text{F1} & \text{Delta F1} \\\\ \hline \text{Clean} & 0 & 57.81 & 61.80 & 0.00 \\\\ \text{Missing} & 10 & 50.78 & 56.64 & -5.16 \\\\ \text{Missing} & 30 & 41.02 & 49.49 & -12.30 \\\\ \text{Additive} & 10 & 53.52 & 58.31 & -3.48 \\\\ \text{Additive} & 30 & 52.73 & 57.95 & -3.84 \\\\ \text{Replacement} & 10 & 54.69 & 58.68 & -3.12 \\\\ \text{Replacement} & 30 & 55.47 & 60.04 & -1.76 \\\\ \end{array}$$ We appreciate this suggestion, which helped clarify the effects of distractors and evidence availability.
AuthorsOpenReview ↗

[Response to Question 2] Extending Evaluation to GrailQA-IKG

Thank you for suggesting evaluation on additional datasets with more complex reasoning. We chose GrailQA to examine compositional reasoning beyond the original two datasets. Its executable logical-form annotations support the same evidence-extraction, masking, and program-rewriting pipeline, while its I.I.D., compositional, and zero-shot splits allow us to examine in-distribution questions, new combinations of known schema items, and unseen schema items separately. We therefore construct GrailQA-IKG and report results on all 5,704 test questions in Table 9.

The additional dataset includes comparisons with generalist models using the same environment and an external method, alongside our trained policy. In Table 9, DoM with DeepSeek-V4.1-Flash obtains overall EM/F1 of 35.54/54.74; the IKBQA-R1 harness with DeepSeek-V4.1-Flash and GPT-6-Sol obtains 63.25/68.00 and 69.41/72.07, respectively. Both harness models are evaluated on the full test set, with one trajectory per question and no reference-action hints. The evaluation therefore broadens dataset coverage and compares different reasoning methods on GrailQA-IKG.

The grouped results further characterize performance across question structures. We retain GrailQA's I.I.D., compositional, and zero-shot groups, which assess in-distribution questions, new combinations of known schema items, and questions involving unseen schema items, respectively. Here, “seen” and “unseen” are defined relative to GrailQA's original training split. IKBQA-R1 obtains EM/F1 of 73.20/75.70, 69.35/71.86, and 72.57/75.09 across these three groups. Table 9 retains complete grouped results for all methods to expose differences beyond the overall average. Table 20 in Appendix C.6 additionally lists published complete-KG results on the GrailQA development set as background references.

In addition, Figure 20 (Appendix G.1) presents a recorded GraphQ trajectory for selecting the largest area unit in the SI system. After integrating KG and external evidence, the model performs type filtering and MAX ordering to obtain the correct answer. This case illustrates how external evidence participates in subsequent compositional operations as intermediate results, complementing the full-set and grouped GrailQA-IKG evaluation with further evidence of applicability.

Table 9: EM and F1 (%) on GrailQA-IKG across I.I.D., compositional, and zero-shot questions. “Harness” uses the IKBQA-R1 action space and feedback format with an off-the-shelf model, while IKBQA-R1 uses the learned RL policy.

\text{Method} & \text{LLM} & \text{I.I.D.} & & \text{Compositional} & & \text{Zero-shot} & & \text{Overall} & \\\\ & & \text{EM} & \text{F1} & \text{EM} & \text{F1} & \text{EM} & \text{F1} & \text{EM} & \text{F1} \\\\ \hline \text{DoM} & \text{DeepSeek-V4.1-Flash} & 32.81 & 54.28 & 25.45 & 40.15 & 40.87 & 60.82 & 35.54 & 54.74 \\\\ \text{IKBQA-R1 harness} & \text{DeepSeek-V4.1-Flash} & 61.65 & 67.23 & 56.48 & 60.85 & 66.58 & 71.12 & 63.25 & 68.00 \\\\ \text{IKBQA-R1 harness} & \text{GPT-6-Sol} & 70.77 & 73.52 & 66.48 & 69.80 & 69.96 & 72.34 & 69.41 & 72.07 \\\\ \mathbf{IKBQA\text{-}R1} & \text{Llama-3.1-8B} & 73.20 & 75.70 & 69.35 & 71.86 & 72.57 & 75.09 & 72.03 & 72.89 \\\\ \end{array}$$ We appreciate this suggestion, which motivated broader evaluation and grouped comparisons.
AuthorsOpenReview ↗

[Response to Question 3] Cross-Dataset and Multi-Gap Transfer without Retraining

Thank you for suggesting cross-dataset and multi-gap evaluation without retraining.

We added cross-dataset transfer and multi-gap evaluation without retraining in Table 21 (Appendix C.7). We freeze the Llama-3.1-8B RL policies trained on WebQSP-IKG and GraphQ-IKG and directly evaluate them on GrailQA-IKG. The two policies obtain 50.23 and 57.71 F1 on the fixed 256-question single-gap subset, showing that learned evidence-acquisition and executable-integration behavior can transfer to a new dataset without using its training data. For context, DoM obtains 54.74 F1 on all 5,704 GrailQA-IKG questions in Table 20. The frozen policies’ subset scores are in a similar range, although they remain below the policy trained specifically on GrailQA-IKG. These results provide evidence of cross-dataset generalization, while the declines on the same questions with two gaps identify multi-gap integration as a remaining challenge.

We select the 256 GrailQA-IKG questions by proportional stratified random sampling from the 5,704-question test set, using seed 42. The subset contains 60 I.I.D., 55 compositional, and 141 zero-shot questions, preserving the original group proportions. Both frozen policies are evaluated on exactly the same question IDs. The 61-question two-gap subset includes all sampled questions with an additional JOIN whose recorded support facts are disjoint from the original gap; selection does not depend on model predictions or correctness.

For cross-dataset transfer, the WebQSP-trained policy obtains overall EM/F1 of 48.05/50.23 on the same 256 GrailQA-IKG questions, while the GraphQ-trained policy obtains 54.69/57.71. The GraphQ-trained policy performs better across the I.I.D., compositional, and zero-shot groups, with overall EM/F1 higher by 6.64/7.48 percentage points and the largest difference on compositional questions. This shows that transfer performance varies with the source dataset. One possible explanation is that the type constraints and program composition patterns in GraphQ are more similar to those in GrailQA. Table 21 provides the complete grouped results.

For multiple knowledge gaps, we retain the original gap and add a second at another JOIN with recorded, non-overlapping support, including its evidence in the cache. This yields 195 single-gap and 61 two-gap questions, with mixed-set EM/F1 of 46.48/49.07 for the WebQSP policy and 52.73/55.34 for GraphQ. On the same 61 questions, single-gap versus two-gap EM/F1 is 22.95/28.62 versus 14.75/22.64 for WebQSP and 19.67/28.50 versus 16.39/23.99 for GraphQ. The subset contains 56 two-JOIN and five three-JOIN questions; its low single-gap scores also indicate difficulty transferring compositional programs. Single-gap runs use eight turns, while mixed/two-gap runs use ten. Despite the larger budget, two-gap performance remains lower, identifying multi-gap integration as a remaining challenge.

Table 21: Cross-dataset transfer of frozen Llama-3.1-8B RL policies to GrailQA-IKG under single-gap and mixed-gap settings.

The mixed setting adds a second gap to 61 of the same 256 questions; results on these 61 questions are also reported under both single-gap and two-gap settings. Maximum turns are 8 for the single-gap setting and 10 for the mixed setting. EM and F1 are reported in percent.

\text{Source RL policy} & \text{I.I.D.} & & \text{Compositional} & & \text{Zero-shot} & & \text{Overall} & \\\\ & \text{EM} & \text{F1} & \text{EM} & \text{F1} & \text{EM} & \text{F1} & \text{EM} & \text{F1} \\\\ \hline \rlap{\text{Single gap (256 questions, Turn 8)}} & & & & & & & & \\\\ \text{WebQSP-IKG} & 41.67 & 45.76 & 30.91 & 33.62 & 57.45 & 58.61 & 48.05 & 50.23 \\\\ \text{GraphQ-IKG} & 43.33 & 48.84 & 49.09 & 53.88 & 61.70 & 62.97 & 54.69 & 57.71 \\\\ \hline \rlap{\text{Mixed: 195 single-gap + 61 two-gap questions (Turn 10)}} & & & & & & & & \\\\ \text{WebQSP-IKG} & 45.00 & 47.96 & 29.09 & 33.17 & 53.90 & 55.74 & 46.48 & 49.07 \\\\ \text{GraphQ-IKG} & 48.33 & 53.40 & 36.36 & 39.22 & 60.99 & 62.45 & 52.73 & 55.34 \\\\ \hline \rlap{\text{Single gap on the same 61-question subset (Turn 8)}} & & & & & & & & \\\\ \text{WebQSP-IKG} & 12.50 & 23.85 & 29.41 & 33.33 & 25.00 & 28.47 & 22.95 & 28.62 \\\\ \text{GraphQ-IKG} & 12.50 & 24.82 & 17.65 & 33.14 & 25.00 & 27.80 & 19.67 & 28.50 \\\\ \hline \rlap{\text{Two-gap subset of the mixed setting (61 questions, Turn 10)}} & & & & & & & & \\\\ \text{WebQSP-IKG} & 12.50 & 19.17 & 17.65 & 26.09 & 14.29 & 22.52 & 14.75 & 22.64 \\\\ \text{GraphQ-IKG} & 18.75 & 30.63 & 5.88 & 15.11 & 21.43 & 25.60 & 16.39 & 23.99 \\\\ \end{array}$$ We appreciate this suggestion, which helped distinguish cross-dataset transfer from multi-gap capability.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to Reviewers, ACs, SACs, and PCs

We thank the reviewers, ACs, SACs, and PCs for their careful assessment of our submission. We have prepared point-by-point responses and revised the manuscript to clarify the methodological contributions, broaden the evaluation, and examine evidence integration under different retrieval conditions. Selected revisions are highlighted in blue in the updated manuscript. We summarize the main responses below.

Methodological contributions beyond KBQA-R1 (Reviewers Sz6K and MZz1). We clarify that IKBQA-R1 retains KBQA-R1’s RRS, SFT, and basic GRPO training framework. Our method makes external evidence available to subsequent executable operations and uses structured action supervision to refine token-level advantage allocation. Table 15 separates trajectory-level GRPO, uniform local allocation, and aligned action-aware allocation: alignment improves over uniform allocation by 1.6/3.7 F1 points on WebQSP-IKG/GraphQ-IKG. Appendix B.5 and Algorithm 3 specify the alignment, local quality, and token-projection procedures.

Broader datasets, external baselines, and reasoning structures (Reviewers 27ta and Sz6K). We extend the construction and evaluation to GrailQA-IKG, covering all 5,704 test questions with I.I.D., compositional, and zero-shot results, including DoM and generalist-model comparisons (Table 9). We also add ToG, GoG, and DoM on GraphQ-IKG (Table 3). Tables 22–23 analyze performance by remaining operations and logical operator type. RL improves F1 by 17.68 points on WebQSP questions requiring at least three subsequent operations and by 10.97/10.86 points on WebQSP/GraphQ questions containing multiple JOINs. GraphQ ordering and counting remain weaknesses.

Transfer without retraining and multiple knowledge gaps (Reviewer 27ta). Table 21 evaluates frozen WebQSP-IKG and GraphQ-IKG policies on the same 256 GrailQA-IKG questions, without GrailQA training data or reference-action hints. They obtain 50.23 and 57.71 F1. We further introduce a second gap in 61 eligible questions and compare the same questions under single- and two-gap conditions. Performance decreases with two gaps despite a larger interaction budget, identifying multi-gap integration as a remaining challenge.

Teacher guidance, policy training, and newer backbones (Reviewers Sz6K and MZz1). Table 7 compares hint-free generalist agents with SFT and RL policies under the same IKBQA-R1 interface. The trained Llama-3.1-8B obtains 75.05/49.93 F1 on WebQSP-IKG/GraphQ-IKG, improving over same-backbone SFT by 5.88/8.20 points and using fewer model calls than the three generalist agents. We retain the dataset-specific differences: both GPT models outperform the trained 8B on GraphQ. Table 19 adds Qwen3-8B, whose WebQSP F1 improves from 70.23 after SFT to 76.42 after RL. We also clarify that reference hints are used for teacher trajectory collection and removed before policy training and testing.

Evidence access, integration, and interaction overhead (Reviewer 27ta). The main comparisons retain each baseline’s native workflow. Table 8 therefore adds a shared-evidence comparison between textual and executable interfaces: executable integration improves GPT-5.6-Sol and GPT-6-Sol by 7.32 and 10.39 F1 points, with calls and token costs also reported. Table 18 analyzes retrieval policies and automatic retrieval–integration chaining. Estimates from existing trajectories indicate similar F1 with fewer calls under automatic chaining. We explain the execution-inspection and action-supervision benefits of retaining a separate Combine action.

Offline evidence and retrieval distractors (Reviewers 27ta and MZz1). We clarify the pre-collection, alignment, and runtime lookup process in Appendix E.1 and Figure 19, and state the controlled evidence-access scope in Limitations. Pre-collection supports concurrent RL rollouts while reducing repeated requests, network instability, and rate-limit effects. Table 25 tests missing evidence, additive distractors, and replacement. On WebQSP, additive distractors yield 63.84/64.02 F1 versus 64.44 without perturbation, while replacement yields 60.75/56.99. These single-seed results indicate some tolerance to distractors when valid evidence is retained; GraphQ additive/replacement coverage is limited.

We appreciate the reviewers’ constructive suggestions. The detailed responses provide the corresponding tables, settings, and manuscript locations for each concern.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.