- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes AoG, a recoverability-aware multi-hop retrieval framework that formulates retrieval as a sequential decision process. AoG explicitly models the recoverability of intermediate retrieval states. To be specific, AoG converts answer-reaching paths into step-wise state-action and prefix-state supervision when training, and during inference, AoG adopts an uncertainty-aware beam search for reliable retrieval while reducing redundant exploration. Extensive experiments show the effectiveness of the proposed methods.
Critical strengths
- This paper identifies two retrieval failure types across baselines, cascading error propagation and path explosion, which may limit the LLM reasoning.
- Experiments on several datasets demonstrate the effectiveness of the proposed method, and ablation study shows the effectiveness of each component.
- This paper is well-written and easy to follow.
Critical weaknesses
- This paper may lack some representative baselines, such as GCR [1] that also finetuned a Llama-3-8B model, and PoG [2].
- This paper defines the recoverability as whether a gold answer is structurally reachable within the remaining hop budget. However, structural reachability does not guarantee that the path provides useful reasoning path. It would be beneficial to include a further analysis to validate these paths.
- For uncertainty-aware beam search, same-entities merging may discard meaningful reasoning prefixes. It would be beneficial to clarify this.
[1] Luo, Linhao, et al. "Graph-constrained reasoning: Faithful reasoning on knowledge graphs with large language models." ICML 2025. [2] Chen, Liyi, et al. "Plan-on-graph: Self-correcting adaptive planning of large language model on knowledge graphs." Advances in Neural Information Processing Systems 37 (2024): 37665-37691.
Questions for authors
Please refer to Weaknesses section.
Dear Reviewer,
I hope this message finds you well. As the discussion period is nearing its end, I wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we’re eager to address any remaining issues to improve our work.
Thank you for your time and effort in reviewing our paper!