- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
EVE-Agent adds evidence rewards to self-training search agents, aiming to improve answers and the supporting passages they return.
Critical strengths
The writing is clear, the figures are simple and easy to follow, and the overall idea makes sense.
Critical weaknesses
- The main trained baseline is Dr. Zero. Broader comparisons with recent RAG and search agents would better establish the contribution.
- Training curves are missing. Performance across training steps and repeated runs would help distinguish consistent improvements from random fluctuations.
Questions for authors
see above
2 comments
Thank you for the response. I will maintain my original positive rating.
Response to Reviewer Vrkq
We thank the reviewer for appreciating the clarity of the presentation, the figures, and the overall idea. We have expanded the baseline evaluation and training analysis as requested.
Q1. Additional baselines
We added a direct RAG comparison in the completed 7B evaluation.
All methods are evaluated on the same 6,125 questions:
EVE achieves the highest average answer accuracy and Joint performance among these methods.
This comparison provides additional evidence that the improvement is not explained simply by retrieving supporting passages. Evidence-aware self-training improves the joint production of correct answers and supporting evidence over both initial-model RAG and difficulty-based self-evolution in this setting.
An additional 3B rubric evaluation supports this finding: correct answers accompanied by sufficiently rated evidence increase from 13.4% for RAG to 16.8% for EVE, with improvements on six of seven benchmarks.
We also evaluated frozen RAG and query rewriting in the reduced 3B pilot, with complete results included in the supplementary material.
Dr. Zero remains our primary trained baseline because EVE directly builds on its proposer–solver framework. We will further expand the comparison with recent trained search agents in the camera-ready version, explicitly accounting for differences in training supervision and computation.
Q2. Training curves and repeated runs
We completed three-seed training analyses and evaluated intermediate checkpoints at updates 0, 4, 8, and 12.
Every checkpoint is evaluated using the same fixed 32-question subset, and no checkpoint is selected based on evaluation performance.
We additionally report between-seed standard deviations for the phase/reward ablations and evaluate the saved models on 256 new questions, providing an independent check of the reduced pilot's results.
Across the pilot, we evaluated 2,944 final outputs and 768 intermediate outputs, with zero API failures or invalid support labels.
We further assessed evaluator variability: only 12 of 540 repeated-prompt groups produced differing labels despite temperature 0. These disagreements are retained and analyzed rather than excluded.
These additions directly address the requests for training trajectories, repeated runs, broader baseline coverage, and evaluation robustness. We will incorporate the corresponding analyses into the camera-ready version.