- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes ReGain, a framework for improving retrieval-augmented reasoning through information-gain-guided rollback. ReGain uses changes in answer confidence to evaluate each retrieval step and actively removes low-quality searches from the trajectory, prompting the model to reformulate its query. The framework combines rollback-aware SFT with GRPO training using both process-level information gain and outcome rewards. At inference time, a self-probe mechanism enables rollback without ground-truth answers. Experiments on seven QA benchmarks show improvements over existing RL-based search methods, particularly on multi-hop tasks.
Critical strengths
- The rollback mechanism provides a useful extension to existing process-reward methods. This addresses error propagation during multi-turn reasoning and helps to get better trajectory.
- ReGain improves average F1 from 48.8 to 50.6 over IGPO, with a larger improvement on multi-hop tasks. The evaluation across two model scales and the reported computational overhead also help assess the method's practical value.
Critical weaknesses
- The current ablations lack the setting of training without rollback but enabling it at inference. This makes it difficult to distinguish improvements from better learned policies versus inference-time intervention.
- Although Appendix C.1 shows that self-probe and GT-based rollback achieve similar overall F1 scores, this does not establish whether they make consistent step-level rollback decisions.
Questions for authors
- Could the authors provide a controlled ablation, particularly training without rollback but enabling self-probe rollback at inference?
- Please report the step-level agreement between GT-based and self-probe rollback decisions, with an analysis of cases where the two disagree.
- Could the authors provide actual rollback case studies and quantitative statistics on how often rollback corrects or worsens final answers? An analysis of whether query reformulation improves retrieval quality would also help verify that the model learns to use rollback feedback effectively.