acceptodds
Under review as a conference paper at ICLR 2027

Reward on Path: Learning Intermediate Supervision Signals for Knowledge Graph Question Answering

Abstract

Knowledge Graph Question Answering (KGQA) aims to answer user questions by reasoning over Knowledge Graphs (KGs). Recent methods use supervision derived from answer labels or refined by Large Language Models (LLMs) to train models that retrieve KG evidence for LLM-based answer reasoning. However, answer-derived supervision treats every answer-reaching path as correct and thus yields noisy training signals, whereas LLM-refined supervision mitigates this noise at substantial cost. To address these limitations, we propose Reward on Path (RoP), a framework to learn a lightweight, question-conditioned path reward from answer labels with an asymmetric objective. Paths reaching the same answer are supervised jointly as a bag, allowing the reward model to learn their relative contributions, while each path is penalized individually for retrieving non-answer entities. The learned reward then trains an LLM-based relation path generator in two stages: reward distillation transfers reward-induced preferences over candidate paths into the generator, and on-policy optimization with GRPO further refines the policy on self-generated paths. Generated paths are grounded in the KG to retrieve evidence for answer reasoning. Experiments on multiple KGQA datasets show that RoP improves F1 over answer-derived methods by at least 2.6% while outperforming LLM-refined methods without costly supervision construction.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Knowledge Graph, Knowledge Graph Question Answering

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_8w9bOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Reward on Path (ROP), a framework to learn a lightweight, question-conditioned path reward from answer labels with an asymmetric objective. The asymmetric objective combines positive supervision over bags of paths reaching the same answer with individual penalties based on the fraction of retrieved entities outside the annotated answer set. The learned reward then trains an LLM-based relation path generator in two stages: reward distillation transfers reward-induced preferences over candidate paths into the generator, and on-policy optimization with GRPO further refines the policy on self-generated paths. Generated relation paths are then grounded in the knowledge graph to provide evidence for an answer LLM.

Critical strengths

The paper addresses a concrete supervision problem: paths that reach the correct answer can still follow irrelevant relations. The proposed reward learning offers a practical way to improve this supervision without additional LLM annotation.

The evaluation connects final-answer improvements with path retrieval and reward-ranking quality. The development-set ranking results provide useful evidence that the learned path preferences generalize beyond the training questions.

The learned supervision improves both RoG and SubgraphRAG while retaining their retrieval architectures. These transfer results strengthen the practical value of the contribution.

Critical weaknesses

The ablations change how positive paths are grouped, how incorrect retrievals are penalized, and how the losses are weighted. The paper lacks a controlled comparison that isolates the effect of grouping.

The ranking tests mix paths that retrieve different correct and incorrect entities. Testing paths that return the same entities would better show whether the reward can distinguish correct reasoning from shortcuts.

Path-generation errors can leave the answer model with missing or irrelevant evidence, and the pipeline provides no feedback to revise retrieval. A controlled comparison using reference paths would show how much correcting these errors improves final-answer accuracy.

Questions for authors

How much of the reward-model gain remains when only positive aggregation changes?

What is the reward model's ranking accuracy when the reference path and competing shortcuts retrieve identical entity sets?

What is the total training cost of RoP compared with RAPL/ReG, including GRPO?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 8w9b

We appreciate your thoughtful review and your recognition of our method’s practical value and effectiveness across different retrieval frameworks. We address each point below.

W1 & Q1. Ablation of bag-level aggregation and the impurity penalty

Response: We isolate the contribution of positive bag aggregation and impurity weighting while keeping all other settings unchanged. w/o positive bag aggregation replaces the loss on each bag’s aggregate prediction with the mean positive loss over its constituent paths. w/o impurity weighting retains positive bag aggregation but penalizes only paths retrieving no annotated answer, removing the fractional penalty on paths retrieving both answer and non-answer entities.

Reward objectiveWebQSP Dev Path Hits@1WebQSP Test Retrieval F1WebQSP Test QA F1CWQ Dev Path Hits@1CWQ Test Retrieval F1CWQ Test QA F1
w/o positive bag aggregation77.665.278.749.824.059.8
w/o impurity weighting84.566.980.065.825.961.1
Full RoP88.369.881.970.329.162.8

Removing either component reduces path ranking, retrieval, and KGQA performance on both datasets, with larger drops from removing positive bag aggregation. Replacing the entire asymmetric objective with pointwise BCE or pairwise ranking (Table 4) yields even lower QA F1 than either component ablation on both datasets. These results highlight the importance of learning relative path utility through bag-level supervision instead of assigning uniform positive credit to all answer-reaching paths, with impurity weighting further refining the supervision by penalizing paths that retrieve non-answer entities.

We will include these additional results and the accompanying analysis in the revised manuscript.

W2 & Q2. Testing ranking on shortcuts or paths with identical entity sets

Response: We evaluate path ranking on a subset of development questions (19% of WebQSP and 22% of CWQ) where the SPARQL reference path and competing shortcuts retrieve identical entity sets from the same question entity, giving them identical answer coverage and impurity.

ScorerWebQSP Hits@1WebQSP MRRCWQ Hits@1CWQ MRR
Answer F137.762.925.537.9
Pointwise BCE64.676.531.045.0
Pairwise ranking65.877.232.546.5
RM w/o question52.169.827.042.8
RM w/o positive bag aggregation67.778.433.648.2
Full RoP RM89.693.868.779.4

RoP outperforms all compared scorers by a large margin on both datasets. Removing question conditioning substantially lowers performance, supporting the importance of question-dependent path utility. The gains over pointwise BCE, pairwise ranking, and the variant without positive bag aggregation further support the role of the asymmetric objective. These results support RoP’s ability to distinguish SPARQL reference paths from shortcuts with identical execution results.

We will include these additional results and the accompanying analysis in the revised manuscript.

W3. Impact of path generation errors on final-answer accuracy

Response: The table below shows the effect of replacing RoP-generated paths with SPARQL reference paths on final answer accuracy, while keeping the KG grounding procedure and answer LLM unchanged.

Path sourceWebQSP F1WebQSP HitWebQSP Hits@1CWQ F1CWQ HitCWQ Hits@1
RoP-generated paths81.991.086.062.871.165.3
SPARQL reference paths90.195.793.168.275.870.4
Improvement+8.2+4.7+7.1+5.4+4.7+5.1

The gains from using SPARQL reference paths indicate that path generation remains an important source of error, while the remaining errors also highlight limitations in answer reasoning, consistent with Appendix I.

We will include these results and the accompanying analysis in the revised manuscript.

Q3. Comparison of total training cost

Response: The table below reports the training time of RoP, including reward model training, reward distillation, and on-policy optimization with GRPO, alongside RAPL and ReG.

Method / Training stageWebQSP (h)CWQ (h)
RoP: Reward model training2.123.7
RoP: Reward distillation (10 epochs)2.516.2
RoP: On-policy optimization (3 epochs)3.628.5
RoP: Total8.268.4
RAPL9.785.3
ReG5.157.2

RoP requires less total training time than RAPL and more than ReG. Although not the fastest to train, it requires no additional LLM annotation for supervision construction (Table 3) and achieves better KGQA performance than both methods on both datasets (Table 1).

We will include this training-time breakdown and the accompanying discussion in the revised manuscript.

Dear Reviewer 8w9b,

Thank you again for your thoughtful review and valuable time. In our previous responses, we clarified the points raised in your comments with additional experimental results. As the rebuttal period is ending soon, we kindly ask whether your concerns have been adequately addressed.

Best regards,

The Authors

Thank you for providing the additional experiments. Most of may concerns raised have been solved. I will keep my positive score.

Thank you again for your great efforts in reviewing our manuscript!

Reviewer_7RaBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on Knowledge Graph Question Answering, which aims to answer users' questions based on existing Knowledge Graphs. Specifically, it proposes Reward on Path (RoP), a framework that learns a lightweight reward model to provide fine-grained supervision for relation path generation. The learned rewards are then used to train an LLM-based path generator through reward distillation and GRPO. The experimental results on widely used KGQA datasets demonstrate the effectiveness of the proposed method.

The proposed asymmetric objective is reasonable, and the experimental results demonstrate the effectiveness of the method. However, several aspects of the paper remain insufficiently justified or discussed, particularly the necessity of a separate reward model, the differences from closely related work, and the motivation for the chosen technical approach. Overall, this reviewer leans toward a negative score.

Critical strengths
  1. The core design of the asymmetric objective is reasonable and effective.
  2. The proposed method achieves strong performance on widely used KGQA benchmarks.
  3. The paper provides extensive experiments and analyses to validate the effectiveness of the proposed method.
Critical weaknesses
  1. The necessity of an separate lightweight reward model is insufficiently justified.
    The current ablation studies only compare the learned reward with answer-based rewards, rather than rewards directly derived from the proposed asymmetric supervision (with necessary adaptations). Therefore, it remains unclear whether the improvements come from the fine-grained supervision signals themselves or the additional reward modeling process.
  2. The dicussion with closely related work is insufficient.
    For example, the paper characterizes DP as treating every relation path connecting a question entity to an annotated answer as a positive training instance (Lines 120–121), which oversimplifies DP's efforts to improve intermediate supervision for relation path generation. Moreover, the paper does not sufficiently explain how the proposed asymmetric objective differs from existing supervision and preference optimization strategies, or why it is more effective.
  3. The motivation for adopting the KG-RAG paradigm is insufficiently discussed. Although the paper distinguishes KG-RAG methods from agent-based methods, it does not clearly explain the rationale for choosing KG-RAG paradigm, or the potential advantages and limitations of this choice.
Questions for authors

None

6 comments
AuthorsOpenReview ↗

Response to Reviewer 7RaB [1/2]

We are grateful for your constructive feedback and your recognition of our method’s effectiveness and strong performance on KGQA benchmarks. We address your concerns with point-by-point responses below.

W1. Necessity of a separate reward model

Response: We provide two comparisons to clarify the contribution of reward modeling beyond directly using scores derived from the asymmetric objective.

(1) Gains over direct scoring. We replace the learned reward model with the following fixed score in both generator training stages:

\sum_{a\in\mathcal{A}_{q,e}^{+}} \frac{\mathbb{1}\\!\left[a\in\mathcal{E}_{z_i}(e)\right]} {|\mathcal{Z}_{q,e}^{a}|} - \xi_{z_i}.$$ This adaptation of Equation (4) replaces learned weighting within each bag with uniform credit allocation for bag-level positive supervision while retaining the path-level impurity penalty. We substitute this score for $s_\phi(q,z)$ in the distillation target (Equation (5)) and GRPO reward (Equation (7)). To account for score scale differences, we select the distillation temperature from $\\{0.1,0.3,0.5,0.7,1.0\\}$ based on the distilled generator’s development retrieval F1, choosing 0.3 for WebQSP and 0.1 for CWQ. All other training settings remain unchanged. | Variant | WebQSP QA F1 | WebQSP QA Hit | WebQSP Retrieval F1 | CWQ QA F1 | CWQ QA Hit | CWQ Retrieval F1 | |---|---:|---:|---:|---:|---:|---:| | Direct supervision | 78.2 | 90.1 | 63.8 | 59.1 | 65.9 | 22.5 | | Full RoP | **81.9** | **91.0** | **69.8** | **62.8** | **71.1** | **29.1** | Full RoP consistently improves KGQA and retrieval performance on both datasets, demonstrating the value of learning path rewards over directly using scores derived from the asymmetric objective. **(2) Distinguishing paths with identical execution results.** We further evaluate path ranking on a subset of development questions (19% of WebQSP and 22% of CWQ) where the SPARQL reference path and competing shortcuts retrieve identical entity sets from the same source entity. The direct score assigns equal scores to these candidates, yielding random ranking when ties are broken randomly. | Scorer | WebQSP Hits@1 | WebQSP MRR | CWQ Hits@1 | CWQ MRR | |---|---:|---:|---:|---:| | Direct score (random ties) | 37.7 | 62.9 | 25.5 | 37.9 | | Full RoP RM | **89.6** | **93.8** | **68.7** | **79.4** | The learned reward model distinguishes paths that receive identical direct scores. Through shared parameters across training examples, it learns question–path associations to estimate relative path utility, providing more informative supervision for path generation. We will include these additional results and the accompanying analysis in the revised manuscript.
AuthorsOpenReview ↗

Response to Reviewer 7RaB [2/2]

W2. Discussion with closely related work

Response:

(1) Clarification of the related work discussion. Given the limited space in the paper, Lines 121–123 summarize existing methods according to how positive supervision is defined, a key distinction between RoP and these methods. DP uses answer-reaching paths as positive examples for SFT and KTO-based preference optimization, with additional synthetic negatives constructed through targeted perturbations for the latter. Our discussion focuses on this definition of positive supervision, and we will expand the discussion of DP’s supervision refinements in the revised manuscript.

(2) Differences from existing supervision strategies and why RoP is more effective. As outlined in Lines 87–91, 131–134, and 205–211, RoP learns a lightweight reward model to provide fine-grained supervision, unlike existing methods that treat all answer-reaching paths as positive examples or rely on costly LLM-refined supervision. Bag-level positive supervision allows the model to learn relative path contributions instead of assigning uniform positive credit to every answer-reaching path, while the path-level impurity penalty further refines supervision by discouraging paths that retrieve non-answer entities. The learned relative utility then provides finer-grained supervision for the path generator through soft targets in reward distillation and graded rewards in on-policy optimization.

The ranking analysis in our response to W1 shows that the learned rewards distinguish paths with identical execution results, supporting their ability to capture utility beyond answer reachability. Table 4 further supports the advantages of this supervision design over baselines. Together, these findings help explain how learning relative path contributions while penalizing impure retrieval provides more informative supervision for path generation, consistent with the improvements in retrieval and downstream KGQA performance (Tables 2 and 1).

We will expand the related work discussion and clarify these distinctions and their supporting evidence in the revised manuscript.

W3. Motivation for adopting the KG-RAG paradigm

Response:

(1) Rationale and advantages. RoP adopts KG-RAG with a trained relation path generator that produces multiple candidate paths in a single LLM call per question entity, reducing repeated LLM calls compared with agent-based methods that iteratively call LLM for graph exploration. Furthermore, grounding these paths in the KG provides compact, structured evidence for answer reasoning, while more precise retrieval reduces token consumption.

RoP achieves higher answer F1 than ToG and DoG on both datasets (Table 1), while reducing runtime by over 85%, LLM calls by over 78%, and token consumption by over 93%, as shown below.

MethodWebQSP Runtime (s)WebQSP CallsWebQSP TokensCWQ Runtime (s)CWQ CallsCWQ Tokens
ToG12.615.67,102.715.623.19,692.8
DoG13.110.47,541.617.012.08,817.5
RoP (ours)1.82.0382.02.12.6593.8

(2) Limitations and trade-offs. As discussed in Appendix J, error propagation is a common limitation of KG-RAG methods that adopt a staged pipeline, including RoP: errors in retrieval can propagate to downstream answer reasoning. Agent-based approaches offer greater flexibility to revise exploration based on intermediate evidence, potentially mitigating such errors. Incorporating such feedback into KG-RAG framework is a promising direction for future work.

We will expand the discussion of this design choice and its advantages and limitations in the revised manuscript.

Dear Reviewer 7RaB,

Thank you again for your thoughtful review and valuable time. In our previous responses, we clarified the points raised in your comments with additional experimental results. As the rebuttal period is ending soon, we kindly ask whether your concerns have been adequately addressed.

Best regards,

The Authors

Reviewer_XNQXOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose ROP, a framework to improve KGQA by generating reliable intermediate supervision signals instead of spurious paths or costly using LLMs to refine. ROP solves this by training a lightweight, question-conditioned reward model directly from answer labels to evaluate the true reasoning utility of candidate paths.
Contributions:
1. uses a unique loss function that evaluates paths reaching the same answer to learn their relative contributions, penalizing individual paths that retrieve non-answer entities; 2. leverages an LLM-based relation path generator using path reward distillation, followed by GRPO 3. experiments on WebQSP and CWQ, showing higher F1 and Hits@1 with low inference cost, and also have ablation study.

Critical strengths
  1. ROP, a novel asymmetric objective for path reward learning that evaluates paths reaching the same answer, while strictly penalizing individual paths for retrieving non-answer entities. It mitigates the severe supervision noise in standard answer-derived labels and solves high cost of relying on manual annotations or LLMs. The transfer experiment is strong to show that the benifits comes from supervision signals instead of pipeline itself.
  2. Most KGQA papers report only final answer accuracy, which makes it hard to see where gains come from. Retrieval-side SPARQL path Hits@1 and precision improve clearly as well. Such results link cleaner supervision to better downstream performance.
Critical weaknesses
  1. The answer LLMs in Table 1 are not controlled. RoG and GNN-RAG use a fine-tuned LLaMA2-7B, RPO-RAG uses a fine-tuned LLaMA3.1-8B, and ROP uses GPT-4o. The GRPO stage adds only 1.1 to 1.3 F1 points over distillation alone, while the standard deviation is about 0.5 to 0.6 over just three runs. So the claim that both stages are essential is not well supported.
  2. The asymmetric objective is the core contribution, but the ablation is not enough. The preference for SPARQL-consistent paths does not have analysis. And seems no ablation for bag-level aggregation and impurity penalty. Also It is unclear whether the reward model’s preference for correct paths comes from logical filtering by the objective or from statistical question–relation priors learned across training questions.
Questions for authors
  1. Can you provide more evidence to show that GRPO stage is important. Seems GRPO stage adds only 1.1 to 1.3 F1 points over distillation alone, while the standard deviation is about 0.5 to 0.6.
  2. Need to control same GPT-4o answer LLM for all baselines and the same Freebase subgraphs as ROP. Otherwise, it is hard to distinguish whether the improvements come from supervision or answerer.
  3. Tables 5 and Figure 2 evaluate the reward model on the questions it was trained on, while GPT-4o scores them zero-shot. It seems unfair to do such comparisons.
  4. Does the generation policy learning to abstain or hallucinate when the retrieved paths missed the target end entities?
  5. Can author provide abalation of bag-level aggregation and impurity penalty.
7 comments
AuthorsOpenReview ↗

Response to Reviewer XNQX [1/2]

We thank you for your thoughtful review and for recognizing our method’s novelty and its benefits for retrieval and downstream KGQA performance. Below, we address your concerns point-by-point.

W1 & Q2. Fair comparison under matched answer models and KG subgraphs

Response: For RoP and the baselines evaluated in our implementation, we use the same preprocessed Freebase subgraphs from RoG-WebQSP and RoG-CWQ. For RoG, GNN-RAG, and RPO-RAG, Table 1 retains the fine-tuned answer LLMs used in their original experimental configurations, where answer reasoning is trained as part of the overall pipeline.

The table below compares RoP and the baselines using the same GPT-4o answer LLM and the same Freebase subgraphs:

MethodWebQSP QA F1WebQSP QA HitWebQSP Retrieval F1CWQ QA F1CWQ QA HitCWQ Retrieval F1
RoG72.586.648.152.064.721.1
GNN-RAG74.287.653.158.268.021.2
RPO-RAG77.990.746.159.469.923.1
RoP (ours)81.991.069.862.871.129.1

With the same answer LLM, RoP consistently achieves better KGQA and retrieval performance than all three baselines on both datasets, with a larger KGQA performance gap over RPO-RAG than in Table 1, supporting the contribution of improved retrieval to its KGQA gains.

We will clarify the comparison settings and include these additional results in the revised manuscript.

W1 & Q1. Contribution of the GRPO stage

Response: We provide two additional analyses to clarify the contribution of the on-policy optimization (GRPO) stage.

(1) Consistent gains across runs. The table below reports the mean and standard deviation of paired gains from GRPO over the distillation-only variant across five runs, including two more runs than reported in the paper.

DatasetΔ QA F1 (mean ± SD)Δ QA Hit (mean ± SD)Δ Retrieval F1 (mean ± SD)Answer Input Tokens: Distillation only → Full
WebQSP+1.2 ± 0.2+0.3 ± 0.1+3.7 ± 0.6180.2 → 158.5 (−12.0%)
CWQ+1.3 ± 0.2+1.1 ± 0.2+3.8 ± 0.9320.3 → 231.8 (−27.6%)

The paired KGQA performance gains have substantially smaller standard deviations relative to their means, supporting consistent improvements across runs. GRPO also improves retrieval performance with fewer answer input tokens, suggesting more precise evidence retrieval.

(2) Improved generalization across datasets. We further compare the distillation-only variant and full RoP by training on one dataset and evaluating on the other.

Train → TestDistillation only QA F1Full QA F1Δ QA F1Distillation only Retrieval F1Full Retrieval F1Δ Retrieval F1
CWQ → WebQSP75.1 ± 0.678.6 ± 0.5+3.5 ± 0.350.6 ± 0.761.4 ± 0.8+10.8 ± 0.9
WebQSP → CWQ47.4 ± 0.750.5 ± 0.6+3.1 ± 0.414.7 ± 0.620.2 ± 0.5+5.5 ± 0.3

GRPO improves retrieval and KGQA performance in both transfer directions, with larger gains than when training and testing on the same dataset, supporting better generalization beyond reward distillation.

We will include these additional results and the accompanying analysis in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer XNQX [2/2]

W2 & Q5. Ablation and analysis of the asymmetric reward objective

Response: We provide additional ablations and analyze the source of the learned path preferences.

(1) Ablations of positive bag aggregation and impurity weighting. We isolate the contribution of each component while keeping all other settings unchanged. w/o positive bag aggregation replaces the loss on each bag’s aggregate prediction with the mean positive loss over its constituent paths. w/o impurity weighting retains positive bag aggregation but applies the negative penalty only to paths that retrieve no annotated answer, removing the fractional penalty on paths that retrieve both answer and non-answer entities.

Reward objectiveWebQSP Dev Path Hits@1WebQSP Test Retrieval F1WebQSP Test QA F1CWQ Dev Path Hits@1CWQ Test Retrieval F1CWQ Test QA F1
w/o positive bag aggregation77.665.278.749.824.059.8
w/o impurity weighting84.566.980.065.825.961.1
Full RoP88.369.881.970.329.162.8

Removing either component reduces path ranking, retrieval, and KGQA performance on both datasets, with larger drops from removing positive bag aggregation. Replacing the entire asymmetric objective with pointwise BCE or pairwise ranking (Table 4) yields even lower QA F1 than either component ablation on both datasets. These results highlight the importance of learning relative path utility through bag-level supervision instead of assigning uniform positive credit to all answer-reaching paths, with impurity weighting further refining the supervision by penalizing paths that retrieve non-answer entities.

(2) Source of the learned path preferences. The reward model learns question-dependent path preferences through shared parameters across training examples. We examine these preferences on a subset of development questions (19% of WebQSP and 22% of CWQ) where the SPARQL reference path and competing shortcuts retrieve identical entity sets from the same question entity, giving them identical answer coverage and impurity.

ScorerWebQSP Hits@1WebQSP MRRCWQ Hits@1CWQ MRR
Answer F1 (random)37.762.925.537.9
Pointwise BCE64.676.531.045.0
Pairwise ranking65.877.232.546.5
RM w/o question52.169.827.042.8
RM w/o positive bag aggregation67.778.433.648.2
Full RoP RM89.693.868.779.4

RoP outperforms all compared scorers by a large margin on both datasets. Removing question conditioning substantially lowers performance, supporting the importance of question-dependent path utility. The gains over pointwise BCE, pairwise ranking, and the variant without positive bag aggregation further support the role of the asymmetric objective. Together, these findings suggest that the asymmetric objective guides the learning of question–path associations across training examples, supporting path preferences beyond logical filtering based on answer coverage and impurity.

We will include these additional results and the accompanying analysis in the revised manuscript.

Q3. Fair evaluation of the reward model against GPT-4o

Response: The training set analysis in Table 5 and Figure 2 evaluates the supervision provided to the path generator. To assess effectiveness on unseen questions, Table 12 reports development set results, where RoP exceeds GPT-4o by 5.0 and 10.6 SPARQL path Hits@1 points on WebQSP and CWQ, respectively. We will clarify the purposes of these evaluations and highlight the development set results in the revised manuscript.

Q4. Abstention and hallucination when retrieved paths miss the answer

Response: The answer LLM uses zero-shot prompting without fine-tuning or explicit abstention instructions. When the retrieved evidence contains no annotated answer, it abstains in only 1.55% of cases on WebQSP and 0.72% on CWQ, and unsupported answers remain possible. Adding explicit instructions to abstain when evidence is insufficient is a promising direction for future work to reduce hallucinations. We will include these results and discuss this future direction in the revised manuscript.

Dear Reviewer XNQX,

Thank you again for your thoughtful review and valuable time. In our previous responses, we clarified the points raised in your comments with additional experimental results. As the rebuttal period is ending soon, we kindly ask whether your concerns have been adequately addressed.

Best regards,

The Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

Changes in Manuscript

Dear Area Chair and Reviewers,

Thank you again for your constructive feedback. Below, we outline the key updates in the latest manuscript:

  • Component ablations (8w9b: W1/Q1; XNQX: W2/Q5). We added separate ablations of bag-level aggregation and impurity weighting in Section 5.3, with detailed definitions and complete results in Appendix F.4.
  • Reward learning versus direct scoring (7RaB: W1). We added a comparison against fixed scores derived from the asymmetric objective in Section 5.3, with the scoring formulation provided in Appendix F.4.
  • Paths with identical retrieval results (8w9b: W2/Q2; XNQX: W2). We added a path-ranking analysis in Section 5.4 to examine the learned question–path associations when candidate paths retrieve identical entity sets.
  • Efficiency and training cost (7RaB: W3; 8w9b: Q3). We expanded the inference-cost comparison with ToG and DoG in Section 5.2 and report RoP’s total training time and its breakdown by stage in Appendix F.7.
  • Related-work clarifications (7RaB: W2/W3). We expanded the discussion of KG-RAG and agent-based methods and clarified DP’s supervision construction in Section 2.

We appreciate your time and welcome any further questions or comments on the revised manuscript. The layout of the overall content will be updated after careful arrangement in the future.

Best regards,
The Authors

AuthorsOpenReview ↗

Rebuttal Summary

Dear Area Chair,

We sincerely thank you and all reviewers for your time and constructive feedback. Below, we provide a concise summary of the strengths recognized by the reviewers and the additional evidence and clarifications addressing their concerns.

Strengths recognized by the reviewers:

  1. Methodological novelty and effectiveness. The asymmetric objective for reward learning is novel (XNQX) and provides a practical, effective approach to addressing the problem of noisy path supervision in KGQA (8w9b, 7RaB).

  2. Strong performance and supporting analyses. RoP achieves strong KGQA performance, supported by extensive experiments and analyses connecting reward quality, retrieval effectiveness, and final answer accuracy (7RaB, 8w9b, XNQX).

  3. Practical value and transferability. Transfer experiments show that the learned rewards benefit other frameworks, supporting the value of the supervision signals beyond RoP’s own pipeline (8w9b, XNQX).

Additional evidence and clarifications addressing reviewers’ concerns:

  1. Reward modeling and supervision design. To assess whether the gains come from reward learning rather than the underlying supervision signals alone, we compared RoP with fixed scores derived from the same asymmetric objective. The learned rewards improve KGQA and retrieval performance while reducing answer reasoning token consumption, supporting the contribution of reward modeling (7RaB). Controlled ablations further establish the contributions of positive bag aggregation and the path-level impurity penalty (8w9b, XNQX).

  2. Analysis of learned path rewards. To examine whether the learned preferences extend beyond answer coverage and impurity, we evaluated paths with identical retrieval results. RoP distinguishes SPARQL paths from shortcuts even in this setting, while further analysis and ablations support learning question–path associations that capture question-dependent relative path utility (7RaB, 8w9b, XNQX).

  3. Additional evaluation. For baselines whose original pipelines include answer LLM fine-tuning, comparisons using the same answer LLM confirm RoP’s advantages in KGQA and retrieval performance. Paired comparisons across independent runs and cross-dataset evaluations further support on-policy optimization(GRPO)’s consistent benefits beyond reward distillation and its contribution to generalization (XNQX).

  4. Positioning and practical efficiency. We revised the related-work section to improve the discussion of prior methods and clarify RoP’s differences from existing supervision strategies, supported by additional experiments (7RaB). Expanded discussion and inference efficiency comparisons with agent-based methods clarify the rationale for adopting KG-RAG (7RaB). We also reported total training time by stage and compared it with baselines (8w9b).

  5. Error analysis and answer behavior. Experiments using SPARQL paths quantify the impact of path generation errors on KGQA performance (8w9b), while the abstention analysis examines answer behavior when retrieval misses the annotated answers (XNQX).

Following the discussion, Reviewer 8w9b confirmed that most concerns were resolved and maintained a positive score. Reviewer XNQX raised their score and confirmed that our follow-up clarified the remaining question. Reviewer 7RaB maintained the original rating without specifying which concerns remained unresolved.

The main manuscript updates are summarized in our earlier Changes in Manuscript comment.

We thank you and the reviewers again for your time and careful consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.