- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes Evidence-in-the-Loop, a framework for training search agents through knowledge-graph-guided self-play. It verifies that every relation in a sampled KG path is supported by retrievable corpus passages before using the path for question generation. These passages also provide intermediate supervision through two rewards: evidence acquisition, which measures retrieval of verified supporting passages, and evidence utilization, which measures consecutive retrieval transitions involving newly discovered bridge entities. Process rewards provide partial credit only for incorrect answers, while correct answers receive full reward.
Critical strengths
- A concrete improvement to intermediate search supervision. Associating KG relations with verified corpus passages ties rewards to observable retrieval behavior rather than entity mentions alone. Applying partial rewards only to incorrect trajectories also preserves full credit for alternative successful search strategies.
- Consistent results with relevant baselines and a useful transfer experiment. The method improves average scores across five configurations compared with SSP and KG-Path. Training a fresh Solver on equally sized generated datasets with outcome-only rewards further supports the value of the generated questions independently of the proposed process rewards
Critical weaknesses
- Evidence acquisition may miss valid supporting passages and evidence utilization is not sufficiently validated. The reward matches a small set of predefined passage IDs, potentially overlooking alternative evidence that supports the same relations. This may bias supervision toward selected passages rather than evidence relevance. Entity reuse in consecutive queries does not establish actual dependence on retrieved evidence. The utilization reward adds only 0.3 points in average EM and F1, without repeated-run results or uncertainty estimates.
- Limited comparison with stronger search agents. Comparisons focus on SSP and KG-Path, while continued-training experiments use only Search-R1, but Search-R1 is not a robust search agent. Broader evaluation with more capable search-agent frameworks under matched budgets is needed to establish practical significance.
- Limited novelty beyond existing frameworks. The method primarily extends SSP [1] and KG-Path [2] with corpus verification and evidence-based rewards. The incremental contribution requires stronger empirical support, particularly for the proposed utilization reward.
[1] Lu et al. Search Self-Play: Pushing the Frontier of Agent Capability without Supervision. ICLR 2026.
[2] Wu et al. Knowledge-Graph Paths as Intermediate Supervision for Self-Evolving Search Agents. 2026.
Questions for authors
-
How sensitive is the acquisition reward to the predefined passage set? How often do trajectories retrieve valid supporting evidence that receives no credit because its passage ID is not included?
-
Can you provide behavioral evidence that the utilization reward captures actual dependence on retrieved information, rather than entity repetition? Is its 0.3-point gain reproducible across training seeds?
-
Do the improvements persist for larger models or more recent model generations? How does the benefit vary with the underlying model’s search capability?
-
The continued-training evaluation relies solely on Search-R1, leaving applicability to stronger search agents unclear. Can you evaluate stronger search-agent checkpoints and additional search frameworks under matched retrieval and training budgets?
-
Relative to SSP and KG-Path, what benefit does the full framework provide beyond evidence-acquisition reward alone? Can a comparison using identical KG paths and generated questions isolate the contributions of corpus verification and evidence utilization?