acceptodds
Under review as a conference paper at ICLR 2027

EVE-Agent: Evidence-Verifiable Self-Evolving Agents

Abstract

Self-evolving search agents learn from the questions and answers they generate themselves; however, existing approaches typically use question difficulty as the primary training signal, providing little incentive to produce evidence that actually supports an answer. We ask whether self-generated evidence can serve as useful feedback for learning to answer questions both correctly and with justification. We introduce EVE-Agent, a self-evolving search agent that generates a candidate evidence span along with each question and answer. The proposer is rewarded based on how much the span increases a fixed solver's agreement with the generated answer when the span is provided versus when it is withheld, with search disabled in both conditions. This model-relative utility signal requires no external answer labels and directly measures whether the generated evidence is useful to the solver. The same span is subsequently reused as an evidence target for solver training, turning evidence generation into a reusable source of supervision. Across seven question-answering benchmarks, EVE-Agent consistently outperforms Dr. Zero in joint answer-and-evidence performance under the same backbone and retrieval pipeline. It also improves both average answer accuracy and judged evidence support. These results show that self-generated evidence can provide an effective training signal for self-evolving search agents rather than serving merely as an explanation appended to the final answer.

Primary area:
foundation or frontier models, including LLMs
Keywords:
self-evolving agents, search agents, evidence verification, question answering

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_VrkqOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

EVE-Agent adds evidence rewards to self-training search agents, aiming to improve answers and the supporting passages they return.

Critical strengths

The writing is clear, the figures are simple and easy to follow, and the overall idea makes sense.

Critical weaknesses
  1. The main trained baseline is Dr. Zero. Broader comparisons with recent RAG and search agents would better establish the contribution.
  2. Training curves are missing. Performance across training steps and repeated runs would help distinguish consistent improvements from random fluctuations.
Questions for authors

see above

2 comments
AuthorsOpenReview ↗

Response to Reviewer Vrkq

We thank the reviewer for appreciating the clarity of the presentation, the figures, and the overall idea. We have expanded the baseline evaluation and training analysis as requested.

Q1. Additional baselines

We added a direct RAG comparison in the completed 7B evaluation.

All methods are evaluated on the same 6,125 questions:

MethodAnswer EMJoint
Initial-model RAG16.8%13.5%
Dr. Zero25.5%20.9%
EVE-Agent26.8%22.8%

EVE achieves the highest average answer accuracy and Joint performance among these methods.

This comparison provides additional evidence that the improvement is not explained simply by retrieving supporting passages. Evidence-aware self-training improves the joint production of correct answers and supporting evidence over both initial-model RAG and difficulty-based self-evolution in this setting.

An additional 3B rubric evaluation supports this finding: correct answers accompanied by sufficiently rated evidence increase from 13.4% for RAG to 16.8% for EVE, with improvements on six of seven benchmarks.

We also evaluated frozen RAG and query rewriting in the reduced 3B pilot, with complete results included in the supplementary material.

Dr. Zero remains our primary trained baseline because EVE directly builds on its proposer–solver framework. We will further expand the comparison with recent trained search agents in the camera-ready version, explicitly accounting for differences in training supervision and computation.

Q2. Training curves and repeated runs

We completed three-seed training analyses and evaluated intermediate checkpoints at updates 0, 4, 8, and 12.

Every checkpoint is evaluated using the same fixed 32-question subset, and no checkpoint is selected based on evaluation performance.

We additionally report between-seed standard deviations for the phase/reward ablations and evaluate the saved models on 256 new questions, providing an independent check of the reduced pilot's results.

Across the pilot, we evaluated 2,944 final outputs and 768 intermediate outputs, with zero API failures or invalid support labels.

We further assessed evaluator variability: only 12 of 540 repeated-prompt groups produced differing labels despite temperature 0. These disagreements are retained and analyzed rather than excluded.

These additions directly address the requests for training trajectories, repeated runs, broader baseline coverage, and evaluation robustness. We will incorporate the corresponding analyses into the camera-ready version.

Thank you for the response. I will maintain my original positive rating.

Reviewer_ZQ3rOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces EVE-Agent, which incorporates evidence-based feedback into the proposer–solver self-training framework of Dr. Zero. The proposer generates a question, a candidate answer, and an evidence span. Its evidence utility reward measures the increase in a frozen solver’s agreement with the candidate answer when the evidence is provided versus withheld, with search disabled in both conditions. The generated answers and evidence spans subsequently serve as pseudo-references for solver training, using a weighted combination of answer exact match and evidence token F1.

The evaluation uses Qwen2.5-3B-Instruct on seven QA benchmarks and reports answer EM, evidence support judged by GPT-4.1, and a Joint metric requiring both answer correctness and evidence support on the same instance. The full system improves Joint over the reported Dr. Zero baseline on all seven benchmarks, while also improving average answer EM and evidence support.

Critical strengths
  1. The paper addresses a meaningful problem through a coherent two-stage design. Answering correctly and providing supporting evidence are distinct capabilities. EVE-Agent incorporates evidence into both curriculum construction and solver supervision, giving it a substantive role in learning beyond an additional output field.

  2. The reported improvements in joint performance are substantial. Relative to the reported Dr. Zero baseline, average Joint increases from 4.4% to 16.7%, answer EM from 11.5% to 22.1%, and evidence support from 19.5% to 31.3%. Joint improves on all seven benchmarks, providing promising initial evidence for evidence-oriented self-training.

Critical weaknesses
  1. The Dr. Zero baseline performs substantially below its originally reported results, and the paper provides insufficient detail to explain this discrepancy. Its average answer EM is 11.5% here, compared with 30.4% after the first training iteration in the original Dr. Zero paper using the same Qwen2.5-3B-Instruct backbone. On NQ, the corresponding scores are 6.9% and 38.1%. The original first iteration also trains the proposer and solver for 50 steps each, so the number of self-evolution iterations alone does not explain the gap. Cross-paper differences in evaluation subsets and protocols may contribute, but the manuscript does not provide the full evaluation prompts, tool-interaction protocol, answer-parsing rules, or baseline checkpoint information needed to assess this possibility. Nor does it compare answer-only and answer-plus-evidence evaluation to isolate the effect of the additional output requirement. Until this discrepancy is explained, it is difficult to determine how much of the improvement reflects the proposed method and how much reflects the baseline’s compatibility with the evaluation protocol.

  2. The central rewards are indirect proxies for evidence support, but neither their validity nor their individual contributions is adequately established. The proposer’s p+−p−p_+ - p_- measures whether evidence increases the solver’s agreement with a generated answer; it does not verify factual correctness, entailment, or source provenance. The solver’s token F1 rewards overlap with a generated span rather than whether the output provides sufficient evidence for the answer. Indeed, the example in Appendix C shows that a span missing a necessary reasoning link can still receive an evidence reward of 2/32/3. The authors acknowledge these limitations, but do not measure the relationship between the proxy scores and actual evidence support, or isolate the effects of proposer utility, the brevity bonus, and the solver’s evidence reward. The gains could therefore be driven primarily by direct evidence supervision of the solver. Furthermore, the GPT-4.1 evaluator does not verify provenance, and its full prompt is not provided. These gaps limit the evidence for the paper’s central mechanism and its goal of producing verifiable evidence.

Questions for authors
  1. Which checkpoint was used for the Dr. Zero baseline, or was it retrained by the authors? Please specify the training iterations, full evaluation prompts, search-interaction protocol, answer and evidence parsing rules, and evaluation-subset selection procedure.

  2. Among the evidence spans ultimately used for solver training, what proportion can be matched to their sources, and what proportion fully supports the generated answers? Is utility recomputed or used for filtering after generating the final training dataset? Do higher utility scores correlate with higher evidence-support rates under human assessment?

  3. Please provide the full GPT-4.1 judge prompt and decision criteria. Must the judge assess support solely from the supplied evidence, or may it use its own knowledge to fill missing reasoning steps? Have you evaluated agreement with human judgments or conducted controls involving answer restatements and incomplete evidence?

3 comments
AuthorsOpenReview ↗

Response to Reviewer ZQ3r (1/2)

We thank the reviewer for the detailed and constructive assessment. We particularly appreciate the recognition that EVE addresses a meaningful problem through a coherent two-stage design and achieves substantial improvements in joint answer-and-evidence performance.

We have conducted additional analyses to clarify baseline comparability, evidence utility, and evaluation reliability.

Q1. Dr. Zero baseline and evaluation discrepancy

Our key contribution is to address a limitation of Dr. Zero's learning objective, rather than merely modify its output format.

Dr. Zero optimizes question difficulty and answer agreement, but its rewards do not directly assess whether generated evidence is useful or encourage the solver to recover supporting evidence. Consequently, successful answer-oriented self-evolution does not necessarily translate into reliable evidence-grounded answering.

EVE addresses this limitation through two complementary mechanisms:

  • Evidence-aware proposer: Rewards evidence according to its incremental contribution to a frozen solver's answer agreement.
  • Evidence-aware solver: Explicitly rewards recovery of both the generated answer and its associated evidence.

This creates a direct connection between evidence usefulness during task generation and evidence production during solver learning.

Regarding the discrepancy with the original Dr. Zero results, we have recovered the experiment report and verified that all 84 per-benchmark metric entries correspond to our submitted Table 2. We also recovered the evaluation specifications, including prompts, retrieval settings, parsing rules, and benchmark selection procedures.

An important distinction is that our evaluation explicitly requires both an answer and supporting evidence, whereas the original Dr. Zero results were obtained under its own answer-oriented evaluation protocol. Our primary claims therefore concern comparisons under the same backbone, retrieval pipeline, and evidence-output evaluation, rather than absolute scores compared across papers.

We agree that a complete reconciliation of the original Dr. Zero EM values requires a native-protocol comparison. We will address this in the camera-ready version through a dedicated baseline reproduction and protocol-compatibility analysis, documenting any differences and their implications for the reported results.

Importantly, the completed 7B experiment provides additional evidence under a shared protocol. EVE improves macro-average Joint by 1.89 points and evidence support by 5.07 points, with support gains across all seven benchmarks.

We also examined whether requesting evidence affects answer accuracy by evaluating identical pilot checkpoints with answer-only and answer-plus-evidence prompts. The observed differences have confidence intervals spanning zero, providing a controlled assessment of the output requirement.

Thus, our central finding is that explicitly optimizing evidence addresses a missing objective in answer-centric self-evolution and improves joint answer-and-evidence performance under the evaluated conditions.

AuthorsOpenReview ↗

Response to Reviewer ZQ3r (2/2)

Q2. Evidence provenance, utility, and reward contributions

We agree that evidence utility, semantic support, and source provenance are distinct properties. Our framework uses model-relative utility as a training signal and evaluates evidence support independently.

As explained in Section 3.1, the utility reward measures whether supplying a candidate span increases a frozen solver's agreement with the generated answer. It does not require external answer labels and is not intended as a certificate of factual correctness.

To further investigate this mechanism, we audited all 576 generated solver-training examples across four proposer conditions and three seeds.

For the full EVE proposer:

  • 98/144 generated evidence passages match their original source body under strict normalized matching.
  • 114/144 (79.2%) are judged to support the generated answer.
  • We additionally evaluated all proposer conditions on 256 new source documents, reporting provenance and semantic support separately.

We also clarify that utility is used to train the proposer, not as a final data-selection filter. Final examples are selected using the format criterion. The utility reward shapes evidence generation during learning, while the solver's token-F1 reward encourages recovery of the generated evidence target.

These signals serve different purposes. In particular, token overlap measures recovery of a pseudo-reference rather than semantic entailment, as explicitly illustrated in Appendix C.

We further examined the relationship between utility and support using GPT-4.1 and GPT-5.1. The resulting correlations are positive but sensitive to answer parsing, motivating the distinction between strict output agreement and answer-content recovery.

The contribution of EVE is to introduce an explicit evidence-dependent learning signal without requiring externally labeled training data. The full-system experiments then evaluate whether this training approach improves correct answers accompanied by supporting evidence.

The new audits provide additional insight into this mechanism, while the individual reward contributions will be examined further under the full training schedule in the camera-ready evaluation.

Q3. Evidence judge and evaluation reliability

The GPT-4.1 judge is explicitly instructed to assess whether the supplied evidence supports the reference answer, without relying on outside knowledge to fill missing reasoning steps.

We provide the complete judge instructions and decision criteria in the supplementary response.

To evaluate the judge's behavior, we conducted controlled experiments on 64 multi-hop bridge questions:

Evidence conditionGPT-4.1 acceptance
Complete evidence59/64
Nonempty deletion52/64
Answer restatement23/64
Irrelevant evidence0/64

Complete evidence also improves frozen-model answer-content accuracy by 43.8 percentage points over no evidence (95% CI [+30.9, +56.2]).

These experiments distinguish evidence that helps recover an answer from evidence that independently establishes the required reasoning chain.

We additionally constructed 32 symbolic reasoning examples. GPT-4.1 accepts all 32 complete mappings and rejects all missing-link, swapped-link, and restatement controls.

To further assess evaluator reliability, we conducted 480 independent GPT-5.1 judgments. On the 96 training-evidence audit items, GPT-4.1 and GPT-5.1 achieve 96.9% agreement (κ=0.890\kappa= 0.890).

The second-model analysis also identifies evaluator sensitivity to certain real-QA restatements; we retain these disagreements rather than selecting favorable judgments. These results establish automated cross-model consistency, while human calibration remains a separate evaluation question.

These additional experiments substantially strengthen the evidence evaluation by testing sufficiency, incomplete reasoning, and consistency across independent evaluator models.

We will incorporate the judge specifications, controls, and limitations into the camera-ready manuscript.

Thank you for the additional analyses, which address most of my concerns. However, the baseline discrepancy remains unresolved: the original Dr. Zero reports 30.4% average EM after one iteration, versus 11.5% here, with particularly large gaps on NQ (38.1% vs. 6.9%) and TriviaQA (52.6% vs. 25.7%). Checking the experiment logs and noting protocol differences do not explain these gaps. This unresolved issue directly affects my confidence in the claimed improvements, so I maintain my original rating.

Reviewer_3VNnOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces EVE-Agent, a label-free self-evolving framework for search agents that extends Dr. Zero by incorporating evidence into both task generation and answer generation phases. In particular, the proposer generates a question, a candidate answer along with the evidence docs. And the proposer is rewarded by task solvability, difficulty and evidence, which is measured by how much the evidence increases the solver's agreement with the proposed answer compared to withholding it. With the generated tasks, the trained proposer is frozen to generate a dataset train a solver model via answer exact match and token-level F1 on the evidence. Experiment results demonstrate that EVE-Agent consistently outperforms baseline methods in answer-evidence alignment across all benchmarks, while also improving answer exact match and LLM judge performance.

Critical strengths
  1. The paper identifies an important limitation in existing data-free search agents, namely reasoning over the retrieved evidence that can justify the answer.

  2. The method measures incremental evidence utility via a frozen solver. In solver then it reuses these spans as F1 targets by rewarding these relevant information, leading to improve search agent performance.

  3. The evaluation is extensive and comprehensive, showing consistent performance gains over the baseline methods: Under the same backbone and exp settings, EVE-Agent outperforms Dr. Zero on the primary joint answer-and-evidence metric along with other metrics like EM across QA benchmarks.

Critical weaknesses
  1. All experiments are conducted on a single small backbone, leaving it unclear whether the proposed method remains effective when scaled to larger models.

  2. The evaluation is restricted to relatively simple, short-form QA benchmarks, omitting more challenging settings such as long-form QA, STEM QA and deep-research benchmarks like HLE / BrowseComp.

  3. The paper only evaluates EVE-Agent as a complete system without ablation studies, the effectiveness of individual components / design choices remain unclear

Questions for authors
  1. Could the authors provide additional experiments on larger backbones (e.g., 7B) or another model family (e.g., Llama-3)?

  2. Could the authors include a brief ablation isolating the Phase A / B like evidence-overlap objective, as well as a compute-matched comparison against Dr. Zero?

1 comment
AuthorsOpenReview ↗

Response to Reviewer 3VNn

We thank the reviewer for recognizing the importance of the problem, the value of evidence-utility feedback, and the comprehensive experimental evaluation.

We have directly addressed the requests for larger-backbone validation and additional ablation and compute analyses.

Q1. Larger-backbone evaluation

Yes. We completed a Qwen2.5-7B-Instruct experiment using full fine-tuning, with 50 proposer updates and 50 solver updates.

The evaluation covers the same seven benchmarks and 6,125 questions, using a shared backbone and retrieval pipeline.

MetricDr. ZeroEVE-Agent
Joint20.9%22.8%
Answer EM25.5%26.8%
Evidence support38.4%43.5%

Evidence support improves on all seven benchmarks. Joint and answer accuracy improve on HotpotQA, 2Wiki, MuSiQue, and Bamboogle.

The macro-average Joint improvement is +1.89 percentage points, with a paired bootstrap 95% confidence interval of [+0.54, +3.24]. We also report pooled results and uncertainty estimates in Supplement E.

The completed experiment further includes an initial-model RAG baseline, over which EVE achieves higher Joint performance (22.8% versus 13.5%).

These results extend the evidence for EVE's effectiveness beyond the original 3B backbone and demonstrate consistent improvements in judged evidence support at 7B scale.

Q2. Ablations and compute-aware comparison

We completed the requested phase/reward ablations with three independent training seeds, separately examining evidence utility in Phase A, evidence overlap in Phase B, and the brevity reward.

The experiments use a reduced 3B LoRA configuration to study component behavior. All variants are evaluated on 128 fixed questions, with the saved checkpoints subsequently tested on 256 additional questions.

These experiments provide component-level diagnostics, while the shortened training schedule does not by itself establish the individual rewards' contributions under the submitted full-training regime. We will extend this analysis in the camera-ready version.

To assess the additional verification cost, we also introduced a training-time-matched answer-reward control.

At approximately equal measured training time, EVE achieves:

  • 50.3% evidence support versus 42.4% for the extended control (+7.8 points, 95% CI [+3.1, +12.5]).
  • 28.1% Joint versus 25.5% for the extended control.

On 256 additional questions, the evidence-support difference remains positive (+3.1 points), with the corresponding uncertainty reported in Supplement B.

We distinguish this answer-reward control from a native Dr. Zero reproduction. The 7B comparison uses matched training-update counts but not identical total computation; detailed accounting is provided in Supplement E.

The completed full-training experiments establish the main system-level improvements, while the additional ablations and compute-aware controls clarify the roles and costs of evidence-oriented learning.

We agree that long-form QA, STEM reasoning, HLE, and BrowseComp are valuable extensions. Our current evaluation focuses on controlled short-answer and multi-hop QA, enabling direct assessment of evidence-aware self-evolution. We will clarify this scope and discuss broader evaluation settings in the camera-ready version.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Overall Response

We sincerely thank all three reviewers for their thoughtful and constructive feedback. We are encouraged that the reviewers recognize the importance of evidence-grounded self-evolution, the coherent two-stage design, and the consistent improvements demonstrated in our experiments.

Our central contribution is to address a fundamental limitation of existing self-evolving search agents: learning to produce correct answers does not necessarily teach an agent to provide evidence supporting those answers. Dr. Zero provides an effective foundation for self-evolution but does not directly optimize evidence utility or evidence recovery. EVE-Agent addresses this missing learning signal by (1) rewarding evidence that improves a frozen solver's answer agreement and (2) reusing that evidence to supervise the solver. Importantly, this requires no externally annotated training answers or evidence.

The submitted 3B results demonstrate substantial improvements: EVE increases average Joint answer-and-evidence performance from 4.4% to 16.7% (+12.3 percentage points) over Dr. Zero, improving on all seven benchmarks. Average answer accuracy increases from 11.5% to 22.1%, while judged evidence support rises from 19.5% to 31.3%.

In response to the reviewers, we have completed extensive additional evaluations:

  1. Larger backbone: A completed Qwen2.5-7B-Instruct experiment on 6,125 questions improves macro-average Joint from 20.9% to 22.8% and evidence support from 38.4% to 43.5%. Support improves on all seven benchmarks, and the paired 95% confidence interval for the macro-average Joint improvement is [+0.54, +3.24] points.
  2. Ablations and robustness: Three-seed phase/reward ablations, compute-aware comparisons, learning trajectories, and validation on 256 additional questions.
  3. Evidence verification: Source-matching and support audits covering all 576 generated pilot training examples, together with controlled evidence perturbations.
  4. Evaluator reliability: Independent GPT-4.1 and GPT-5.1 assessments achieve 96.9% agreement (κ = 0.890) on 96 training-evidence items.
  5. Reproducibility: Recovered evaluation specifications and verification of all 84 per-benchmark metric entries in the submitted 3B results.

Together, these results strengthen our main finding: self-generated evidence can provide a reusable learning signal for self-evolving agents, rather than merely serving as an explanation appended to the final answer.

We address the reviewers' specific concerns below.

AuthorsOpenReview ↗

Overall Response

We sincerely thank the reviewers for their constructive feedback, which has helped us substantially strengthen the empirical evaluation and clarify the contribution.

Our central message is that evidence should be a learning objective, not merely an output field. EVE-Agent addresses a fundamental limitation of answer-centric self-evolution by making evidence useful during task generation and explicitly supervising its recovery during solver learning.

The submitted 3B results demonstrate Joint improvements across all seven benchmarks, while the completed 7B experiments provide further evidence through higher macro-average Joint and consistent evidence-support gains.

The additional ablations, compute-aware comparisons, source audits, and independent evaluator controls provide a substantially more comprehensive assessment of the approach.

We believe these new results and clarifications meaningfully strengthen the paper and address the reviewers' central concerns. We would be grateful if the reviewers would reconsider their assessments and consider raising their ratings in light of this additional evidence.

Thank you again for your time and thoughtful feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.