- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
EVESkillDebate integrates evidence-grounded debate with the updating of external natural language agent strategies. Before synthesizing the final output, the system links candidate arguments to document snippets, graph edges, or tool outputs, which are then verified by a validation component. Experiments cover many domains such as recommendation systems, rare disease ranking, mathematics, and education, and involve a variety of foundation models.
Critical strengths
- The problem is clearly stated and well motivated. Integrating evidence-grounded multi-agent debate with persistent skill evolution addresses both unsupported reasoning and recurring procedural errors.
- An extensive evaluation of five benchmarks and three model families supports the claim of their generalisability. In addition, the contributions of each model are analysed through a large number of ablation experiments.
Critical weaknesses
- The respective contributions of verifier tuning and skill evolution to the final results have not been sufficiently disentangled. Although the verifier remains fixed during the evaluation phase, this does not account for how its prompts, scoring rules, and thresholds are originally selected or calibrated. Furthermore, repeatedly updating skills under the guidance of a fixed verifier may lead to the skills becoming overfitted to the verifier's specific scoring preferences. The authors could employ an independent verifier or human evaluation to assess the final generated skills.
- The additional benefits provided by the debate module are limited, and it remains unclear whether these are sufficient to justify the extra computational cost. In Table 3 on page 8, the debate function yielded improvements of only 0.7 and 0.3% with GPT-5.4-Nano, respectively. As debate often entails additional communication and inference costs, such minimal gains are insufficient.
- Many baseline models are variants of MAD. Constructing a robust "single-agent retrieval-plus-verification" baseline will help clarify the marginal value provided by MAD within the constraints of evidence verification.
Questions for authors
- How were the verifier's prompts, scoring rules, and thresholds selected and calibrated? Do the evolved skills retain their improvements when assessed by an independent verifier or human evaluators?
- Do the debate module's performance gains justify the additional computational cost relative to the no-debate variant?
Gentle Reminder: ICLR Rebuttal Discussion Deadline – November 18
Dear Reviewers,
Thank you very much for your time, thoughtful reviews, and valuable contributions to the evaluation process. Thank you to the reviewers who have already interacted and provided additional feedback to the authors!
The authors have carefully addressed your comments and concerns, and the discussion period will conclude on November 18. We would greatly appreciate it if you could take a few moments to review their responses and results, engage in the discussion, and indicate whether their clarifications or additional evidence affect your assessment.
Your continued engagement is essential to ensuring a rigorous, constructive, and fair evaluation process.
Thank you again for your time, expertise, and dedication!
Best regards, Area Chair