- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes counterfactual continuation-value supervision for sequential evidence composition in retrieval-augmented multi-hop QA. On HotpotQA and zero-shot 2WikiMultiHopQA, it outperforms BGE Top-4, full eight-passage context, and a MORSE-style proxy, showing learned evidence composition improves answer quality and efficiency.
Critical strengths
One critical strength of this paper is that it derives exact ADD/STOP targets from exhaustive frozen-reader evaluations over a small candidate pool. This captures higher-order evidence complementarity and avoids relying on handcrafted or proxy objectives, while requiring no reader calls during selection.
Critical weaknesses
One major issue is that the method is unscalable and evaluated under an unrealistic setup. Offline target construction requires 2^K reader evaluations per question (256 when K=8), which becomes intractable for larger candidate pools. Another issue is that the 2Wiki candidate pools are explicitly constructed to always contain annotated support passages, so the evaluation only tests composition given gold evidence already in the pool. This bypasses the real retrieval challenge in open-domain multi-hop QA. As a result, the reported gains over BGE Top-4 do not demonstrate end-to-end effectiveness in realistic settings where supports may not be retrieved.
Questions for authors
What is the scalability for the proposed method?