- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Reward on Path (ROP), a framework to learn a lightweight, question-conditioned path reward from answer labels with an asymmetric objective. The asymmetric objective combines positive supervision over bags of paths reaching the same answer with individual penalties based on the fraction of retrieved entities outside the annotated answer set. The learned reward then trains an LLM-based relation path generator in two stages: reward distillation transfers reward-induced preferences over candidate paths into the generator, and on-policy optimization with GRPO further refines the policy on self-generated paths. Generated relation paths are then grounded in the knowledge graph to provide evidence for an answer LLM.
Critical strengths
The paper addresses a concrete supervision problem: paths that reach the correct answer can still follow irrelevant relations. The proposed reward learning offers a practical way to improve this supervision without additional LLM annotation.
The evaluation connects final-answer improvements with path retrieval and reward-ranking quality. The development-set ranking results provide useful evidence that the learned path preferences generalize beyond the training questions.
The learned supervision improves both RoG and SubgraphRAG while retaining their retrieval architectures. These transfer results strengthen the practical value of the contribution.
Critical weaknesses
The ablations change how positive paths are grouped, how incorrect retrievals are penalized, and how the losses are weighted. The paper lacks a controlled comparison that isolates the effect of grouping.
The ranking tests mix paths that retrieve different correct and incorrect entities. Testing paths that return the same entities would better show whether the reward can distinguish correct reasoning from shortcuts.
Path-generation errors can leave the answer model with missing or irrelevant evidence, and the pipeline provides no feedback to revise retrieval. A controlled comparison using reference paths would show how much correcting these errors improves final-answer accuracy.
Questions for authors
How much of the reward-model gain remains when only positive aggregation changes?
What is the reward model's ranking accuracy when the reference path and competing shortcuts retrieve identical entity sets?
What is the total training cost of RoP compared with RAPL/ReG, including GRPO?
4 comments
Dear Reviewer 8w9b,
Thank you again for your thoughtful review and valuable time. In our previous responses, we clarified the points raised in your comments with additional experimental results. As the rebuttal period is ending soon, we kindly ask whether your concerns have been adequately addressed.
Best regards,
The Authors
Thank you for providing the additional experiments. Most of may concerns raised have been solved. I will keep my positive score.
Thank you again for your great efforts in reviewing our manuscript!
Response to Reviewer 8w9b
We appreciate your thoughtful review and your recognition of our method’s practical value and effectiveness across different retrieval frameworks. We address each point below.
Response: We isolate the contribution of positive bag aggregation and impurity weighting while keeping all other settings unchanged. w/o positive bag aggregation replaces the loss on each bag’s aggregate prediction with the mean positive loss over its constituent paths. w/o impurity weighting retains positive bag aggregation but penalizes only paths retrieving no annotated answer, removing the fractional penalty on paths retrieving both answer and non-answer entities.
Removing either component reduces path ranking, retrieval, and KGQA performance on both datasets, with larger drops from removing positive bag aggregation. Replacing the entire asymmetric objective with pointwise BCE or pairwise ranking (Table 4) yields even lower QA F1 than either component ablation on both datasets. These results highlight the importance of learning relative path utility through bag-level supervision instead of assigning uniform positive credit to all answer-reaching paths, with impurity weighting further refining the supervision by penalizing paths that retrieve non-answer entities.
We will include these additional results and the accompanying analysis in the revised manuscript.
Response: We evaluate path ranking on a subset of development questions (19% of WebQSP and 22% of CWQ) where the SPARQL reference path and competing shortcuts retrieve identical entity sets from the same question entity, giving them identical answer coverage and impurity.
RoP outperforms all compared scorers by a large margin on both datasets. Removing question conditioning substantially lowers performance, supporting the importance of question-dependent path utility. The gains over pointwise BCE, pairwise ranking, and the variant without positive bag aggregation further support the role of the asymmetric objective. These results support RoP’s ability to distinguish SPARQL reference paths from shortcuts with identical execution results.
We will include these additional results and the accompanying analysis in the revised manuscript.
Response: The table below shows the effect of replacing RoP-generated paths with SPARQL reference paths on final answer accuracy, while keeping the KG grounding procedure and answer LLM unchanged.
The gains from using SPARQL reference paths indicate that path generation remains an important source of error, while the remaining errors also highlight limitations in answer reasoning, consistent with Appendix I.
We will include these results and the accompanying analysis in the revised manuscript.
Response: The table below reports the training time of RoP, including reward model training, reward distillation, and on-policy optimization with GRPO, alongside RAPL and ReG.
RoP requires less total training time than RAPL and more than ReG. Although not the fastest to train, it requires no additional LLM annotation for supervision construction (Table 3) and achieves better KGQA performance than both methods on both datasets (Table 1).
We will include this training-time breakdown and the accompanying discussion in the revised manuscript.