- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper presents AgentDisCo, a multi-agent framework for open-ended deep research that separates information exploration and exploitation between a critic agent and a generator agent. The critic identifies gaps in the current outline and proposes targeted search queries, while the generator retrieves relevant evidence and refines the outline accordingly. The paper also introduces adaptive retrieval budgets and a bi-level harness optimization framework. Experiments on DeepResearchBench, DeepConsult, DeepResearchGym, and the newly proposed GALA benchmark demonstrate consistent improvements over existing systems. In addition, the paper incorporates a rendering agent to generate visually rich reports.
Critical strengths
- Separating the critic and generator is a reasonable way to mitigate self-evaluation bias and improve evidence coverage. The adaptive retrieval budget is particularly useful for managing context and allocating resources across different research topics.
- The evaluation covers multiple benchmarks and includes ablation studies, cross-model experiments, and human evaluation. Overall, the results provide reasonable evidence that the proposed framework improves report quality.
Critical weaknesses
- The critic–generator design is intuitive and potentially useful, but role separation and iterative critique are already common in existing multi-agent systems. Although adaptive retrieval budgets and harness optimization offer additional value, the paper could do more to clarify what fundamentally distinguishes AgentDisCo from prior approaches.
- The framework combines multiple agents, retrieval components, and an additional harness optimization loop. While the overall design appears effective empirically, the added complexity may make joint optimization challenging. In particular, it remains unclear how end-to-end policy learning methods, such as RLVR, could be effectively applied across these interacting components. A more explicit discussion of this limitation and the trade-offs associated with the additional complexity would strengthen the paper.
Questions for authors
- Beyond separating exploration and exploitation, what is the main methodological contribution that distinguishes AgentDisCo from existing critic–generator or iterative refinement frameworks? Which component contributes most to its novelty?
- How could the bi-level optimization framework be extended to support end-to-end policy learning, such as RLVR? Are there fundamental challenges in credit assignment across the critic, generator, and downstream agents, or is the current harness-based optimization intended to remain the primary approach?
4 comments
Response to Reviewer 8daF Part 2
W2: The framework combines multiple agents, retrieval components, and an additional harness optimization loop. While the overall design appears effective empirically, the added complexity may make joint optimization challenging. In particular, it remains unclear how end-to-end policy learning methods, such as RLVR, could be effectively applied across these interacting components. A more explicit discussion of this limitation and the trade-offs associated with the additional complexity would strengthen the paper.
Q2: How could the bi-level optimization framework be extended to support end-to-end policy learning, such as RLVR? Are there fundamental challenges in credit assignment across the critic, generator, and downstream agents, or is the current harness-based optimization intended to remain the primary approach?
Thank you for this question. We have added a discussion of policy-learning extensions in Appendix A.
-
Harness optimization is our current approach because it supports proprietary models without access to trainable weights. It modifies prompts and orchestration code, while the policy bank stores reusable research traces. This flexibility comes with additional configuration-search and evaluation costs. We therefore position it as the approach validated in this paper, while treating policy learning as a future extension.
-
End-to-end RLVR across interacting agents would face delayed rewards, credit-assignment ambiguity, and changing partner policies. The shared potential defines a common objective but does not determine which agent or action deserves credit for improved report quality. One practical direction is to distill multi-agent research trajectories into a single trainable policy through supervised imitation, followed by RLVR on subtasks with independently verifiable outcomes, such as numerical extraction and unit-normalized calculations. This would avoid credit assignment across separately trained agents, although temporal credit assignment over long trajectories would remain. We discuss this extension and its limitations in Appendix A.
Thank you for your response. It addressed my concerns, and I will keep my score at 3.
Follow-up Response to Reviewer 8daF
Thank you for the follow-up. We are glad that our responses addressed your concerns, and we appreciate your careful review and constructive suggestions, which helped us improve the manuscript.
Response to Reviewer 8daF Part 1
We thank you for recognizing the reasonable separation of the critic and generator, the utility of adaptive retrieval budgets for context management, and the comprehensive evaluation with multiple benchmarks, ablation studies, and human evaluation. Below we address each weakness and question in turn. We hope that our responses resolve your concerns, and we would very much welcome any further discussion.
Thank you for this question. We agree that role separation and iterative critique alone are not new. Our methodological contribution lies in combining a structured blueprint interface with bi-level harness optimization.
Beyond role separation, AgentDisCo introduces a blueprint interface that turns critique into actionable evidence acquisition. Each blueprint binds coverage requirements, targeted search queries, and a per-gap retrieval budget (Section 2.2). The generator uses the retrieved evidence to refine the outline and its references, which then inform subsequent critique. This connects gap identification, retrieval control, and evidence integration through an explicit intermediate representation.
The bi-level harness is the main optimization contribution distinguishing AgentDisCo from a fixed critic–generator refinement framework. The inner loop refines query-specific research artifacts, while the outer loop alternately optimizes critic and generator configurations, holding the other fixed. To directly assess this contribution, we conduct an additional alternating-versus-joint ablation (see Appendix D.13, Table 19). With the architecture and configuration-search settings held fixed, alternating updating achieves 54.11 RACE, compared with 53.35 for joint updating and 52.97 without harness optimization. The advantage supports the contribution of the alternating schedule motivated by the potential-game formulation.
The largest observed performance increment comes from the architectural change, whereas methodological novelty rests on the structured interface and optimization mechanism. Replacing the single-agent ReAct workflow with disentangled outline refinement improves RACE from 49.77 to 51.84; adaptive retrieval budgets further increase it to 52.97, and the harness to 54.11 (Table 1). Separately, targeted optimization achieves 54.11 versus 53.12 for a Meta-Harness-style end-to-end variant under the same optimization-round budget (Appendix D.6, Table 12), supporting the targeted optimization scope in our evaluated configuration.