acceptodds
Under review as a conference paper at ICLR 2027

AgentDisCo: Towards Disentanglement and Collaboration in Open-ended Deep Research Agents

Abstract

Open-ended deep research agents have emerged as a promising paradigm for autonomously performing comprehensive information gathering and synthesis. However, existing approaches typically integrate information exploration and exploitation into a single unified module—such as an outline generator or a report generator—thereby limiting their flexibility and optimization potential. In this paper, we introduce AgentDisCo, a novel Disentangled and Collaborative agentic architecture that formulates deep research as an adversarial optimization problem between information exploration and exploitation. Specifically, a critic agent is optimized to evaluate and critique the generated outlines (serving as information exploitation states) and subsequently refine the search queries (serving as information exploration states), while a generator agent is optimized to retrieve updated search results based on the refined search queries (serving as information exploration states) and accordingly update the generated outlines (serving as information exploitation states). The resulting outline, progressively refined through iterative adversarial optimization, is subsequently delivered to a downstream report writer module. This module leverages the structured outline alongside the accumulated search results to synthesize a comprehensive, coherent, and well-grounded research report.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Open-ended Deep Research Agents, AI Life Agent, AI Harness

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_8daFOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents AgentDisCo, a multi-agent framework for open-ended deep research that separates information exploration and exploitation between a critic agent and a generator agent. The critic identifies gaps in the current outline and proposes targeted search queries, while the generator retrieves relevant evidence and refines the outline accordingly. The paper also introduces adaptive retrieval budgets and a bi-level harness optimization framework. Experiments on DeepResearchBench, DeepConsult, DeepResearchGym, and the newly proposed GALA benchmark demonstrate consistent improvements over existing systems. In addition, the paper incorporates a rendering agent to generate visually rich reports.

Critical strengths
  • Separating the critic and generator is a reasonable way to mitigate self-evaluation bias and improve evidence coverage. The adaptive retrieval budget is particularly useful for managing context and allocating resources across different research topics.
  • The evaluation covers multiple benchmarks and includes ablation studies, cross-model experiments, and human evaluation. Overall, the results provide reasonable evidence that the proposed framework improves report quality.
Critical weaknesses
  • The critic–generator design is intuitive and potentially useful, but role separation and iterative critique are already common in existing multi-agent systems. Although adaptive retrieval budgets and harness optimization offer additional value, the paper could do more to clarify what fundamentally distinguishes AgentDisCo from prior approaches.
  • The framework combines multiple agents, retrieval components, and an additional harness optimization loop. While the overall design appears effective empirically, the added complexity may make joint optimization challenging. In particular, it remains unclear how end-to-end policy learning methods, such as RLVR, could be effectively applied across these interacting components. A more explicit discussion of this limitation and the trade-offs associated with the additional complexity would strengthen the paper.
Questions for authors
  • Beyond separating exploration and exploitation, what is the main methodological contribution that distinguishes AgentDisCo from existing critic–generator or iterative refinement frameworks? Which component contributes most to its novelty?
  • How could the bi-level optimization framework be extended to support end-to-end policy learning, such as RLVR? Are there fundamental challenges in credit assignment across the critic, generator, and downstream agents, or is the current harness-based optimization intended to remain the primary approach?
4 comments
AuthorsOpenReview ↗

Response to Reviewer 8daF Part 1

We thank you for recognizing the reasonable separation of the critic and generator, the utility of adaptive retrieval budgets for context management, and the comprehensive evaluation with multiple benchmarks, ablation studies, and human evaluation. Below we address each weakness and question in turn. We hope that our responses resolve your concerns, and we would very much welcome any further discussion.

W1: The critic–generator design is intuitive and potentially useful, but role separation and iterative critique are already common in existing multi-agent systems. Although adaptive retrieval budgets and harness optimization offer additional value, the paper could do more to clarify what fundamentally distinguishes AgentDisCo from prior approaches.

Q1: Beyond separating exploration and exploitation, what is the main methodological contribution that distinguishes AgentDisCo from existing critic–generator or iterative refinement frameworks? Which component contributes most to its novelty?

Thank you for this question. We agree that role separation and iterative critique alone are not new. Our methodological contribution lies in combining a structured blueprint interface with bi-level harness optimization.

  1. Beyond role separation, AgentDisCo introduces a blueprint interface that turns critique into actionable evidence acquisition. Each blueprint binds coverage requirements, targeted search queries, and a per-gap retrieval budget (Section 2.2). The generator uses the retrieved evidence to refine the outline and its references, which then inform subsequent critique. This connects gap identification, retrieval control, and evidence integration through an explicit intermediate representation.

  2. The bi-level harness is the main optimization contribution distinguishing AgentDisCo from a fixed critic–generator refinement framework. The inner loop refines query-specific research artifacts, while the outer loop alternately optimizes critic and generator configurations, holding the other fixed. To directly assess this contribution, we conduct an additional alternating-versus-joint ablation (see Appendix D.13, Table 19). With the architecture and configuration-search settings held fixed, alternating updating achieves 54.11 RACE, compared with 53.35 for joint updating and 52.97 without harness optimization. The advantage supports the contribution of the alternating schedule motivated by the potential-game formulation.

  3. The largest observed performance increment comes from the architectural change, whereas methodological novelty rests on the structured interface and optimization mechanism. Replacing the single-agent ReAct workflow with disentangled outline refinement improves RACE from 49.77 to 51.84; adaptive retrieval budgets further increase it to 52.97, and the harness to 54.11 (Table 1). Separately, targeted optimization achieves 54.11 versus 53.12 for a Meta-Harness-style end-to-end variant under the same optimization-round budget (Appendix D.6, Table 12), supporting the targeted optimization scope in our evaluated configuration.

AuthorsOpenReview ↗

Response to Reviewer 8daF Part 2

W2: The framework combines multiple agents, retrieval components, and an additional harness optimization loop. While the overall design appears effective empirically, the added complexity may make joint optimization challenging. In particular, it remains unclear how end-to-end policy learning methods, such as RLVR, could be effectively applied across these interacting components. A more explicit discussion of this limitation and the trade-offs associated with the additional complexity would strengthen the paper.

Q2: How could the bi-level optimization framework be extended to support end-to-end policy learning, such as RLVR? Are there fundamental challenges in credit assignment across the critic, generator, and downstream agents, or is the current harness-based optimization intended to remain the primary approach?

Thank you for this question. We have added a discussion of policy-learning extensions in Appendix A.

  1. Harness optimization is our current approach because it supports proprietary models without access to trainable weights. It modifies prompts and orchestration code, while the policy bank stores reusable research traces. This flexibility comes with additional configuration-search and evaluation costs. We therefore position it as the approach validated in this paper, while treating policy learning as a future extension.

  2. End-to-end RLVR across interacting agents would face delayed rewards, credit-assignment ambiguity, and changing partner policies. The shared potential defines a common objective but does not determine which agent or action deserves credit for improved report quality. One practical direction is to distill multi-agent research trajectories into a single trainable policy through supervised imitation, followed by RLVR on subtasks with independently verifiable outcomes, such as numerical extraction and unit-normalized calculations. This would avoid credit assignment across separately trained agents, although temporal credit assignment over long trajectories would remain. We discuss this extension and its limitations in Appendix A.

Thank you for your response. It addressed my concerns, and I will keep my score at 3.

AuthorsOpenReview ↗

Follow-up Response to Reviewer 8daF

Thank you for the follow-up. We are glad that our responses addressed your concerns, and we appreciate your careful review and constructive suggestions, which helped us improve the manuscript.

Reviewer_BMbaOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AgentDisCo, a deep research architecture that separates exploration and exploitation into two agents: a critic that probes the evolving outline and issues gap-aware search queries with adaptive per-key-point retrieval budgets, and a generator that retrieves evidence and refines the outline. The authors formalize the interaction as a dual-agent cooperative MDP with a bi-level potential game structure, report consistent gains over leading deep research agents on DeepResearchBench, DeepConsult, and DeepResearchGym, and additionally introduce the GALA benchmark for daily-life research needs and a rendering agent for visually rich reports.

Critical strengths
  1. The motivation is clear. Assigning exploration and exploitation to separate roles is an intuitive idea.
  2. The improvement on DeepResearchBench is substantial, which demonstrates the effectiveness of the proposed method.
  3. The experiments are comprehensive. In addition, the paper introduces the GALA benchmark, which fills a gap in existing datasets.
Critical weaknesses
  1. Is Gemini-2.5-Pro also used as the judge model in the evaluation? If so, the judge shares the backbone of AgentDisCo, which may introduce self-preference bias. The authors should clarify this.
  2. How is the "w/ Single-agent Architecture" variant in the main experiments implemented? Please describe its experimental setup.
  3. GALA is built from real user interaction data. I suggest that the authors disclose this in the Ethics Statement and describe what processing was applied to the data (e.g., anonymization and filtering).

Minor:

  1. In Figure 2, the word following the wrench icon should be "denotes" rather than "demotes".
Questions for authors

Please refer to the weakness section

3 comments
AuthorsOpenReview ↗

Response to Reviewer BMba

We thank you for recognizing the clear motivation, the substantial improvements on DeepResearchBench, and the comprehensive experiments, including the GALA benchmark. Below, we address each weakness and question in turn. We hope that our responses resolve your concerns, and we would very much welcome any further discussion.

W1: Is Gemini-2.5-Pro also used as the judge model in the evaluation? If so, the judge shares the backbone of AgentDisCo, which may introduce self-preference bias. The authors should clarify this.

Thank you for raising this concern. Gemini-2.5-Pro is indeed the DeepResearchBench judge and shares our main system backbone. We clarify the evaluation setup and provide complementary evidence across judge models, system backbones, and human assessments.

  1. The improvements extend to benchmarks evaluated by judges that do not share our main backbone. DeepResearchBench uses Gemini-2.5-Pro, DeepConsult uses GPT-4.1, DeepResearchGym uses GPT-4.1-mini, and GALA uses Gemini-3-Flash (Appendix C). Under GPT-4.1 and GPT-4.1-mini, harness optimization improves DeepConsult Overall from 6.91 to 7.12 and DeepResearchGym Overall from 95.63 to 96.42, respectively (Table 2). These results support our benefits beyond Gemini-based evaluation.

  2. On DeepResearchBench itself, the improvements also hold for system backbones from different model families. Under the same Gemini-2.5-Pro judge, harness optimization improves Claude-Opus-4.6 from 54.72 to 55.60 and Qwen3-235B from 51.82 to 52.75 (Appendix D.3, Table 5). Thus, our improvements are also observed for Claude and Qwen systems evaluated by the Gemini judge.

  3. Independent human evaluation also supports improvements in report quality. Table 10 (Appendix D.5) reports 76.0% pairwise LLM–human agreement on 50 DeepResearchBench report pairs. Separately, in a blind comparison of 40 round-0 versus round-20 report pairs, optimized reports were preferred in 70.0% of cases, with Fleiss’ κ = 0.58. These findings indicate that the human evaluation results are consistent with the automatic evaluation.

W2: How is the "w/ Single-agent Architecture" variant in the main experiments implemented? Please describe its experimental setup.

Thank you for this question. We have added the implementation and experimental setup of the single-agent variant to Appendix C.4.

The “w/ Single-agent Architecture” variant is a ReAct agent that performs research and report writing within a single interaction trajectory. Given a user query, it interleaves reasoning, search-tool calls, and inspection of retrieved evidence. The agent itself identifies information gaps, formulates subsequent search queries, and decides when to produce the final report. It uses no separate critic, explicit intermediate-outline refinement, or downstream writer agent. The single-agent and disentangled variants use the same backbone model, evaluation queries, and retrieval source. Their principal difference is the research workflow: the single-agent variant directly searches and writes, whereas the disentangled variant uses a critic–generator loop to refine an intermediate outline before report synthesis.

W3: GALA is built from real user interaction data. I suggest that the authors disclose this in the Ethics Statement and describe what processing was applied to the data (e.g., anonymization and filtering).

Thank you for this suggestion. We have updated the Ethics Statement to clarify that GALA is built from anonymized user interactions, including clicks, comments, and browsing records, with personally identifying information filtered out.

M1: In Figure 2, the word following the wrench icon should be "denotes" rather than "demotes".

Thank you for catching this typo. We have corrected “demotes” to “denotes” in Figure 2.

Thank you for the response. After reading the full rebuttal, I will keep my original positive score.

AuthorsOpenReview ↗

Follow-up Response to Reviewer BMba

Thank you for reading our full rebuttal and for your continued support. We appreciate your careful review and constructive feedback, which helped us improve the manuscript.

Reviewer_d6PAOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This submission addresses the challenge of balancing exploration and exploitation in open-ended deep-research agents, where existing approaches may suffer from self-evaluation bias, premature convergence, and exploration collapse. The paper proposes AgentDisCo, a dual-agent framework in which a critic agent examines an evolving research outline and proposes gap-aware search queries with adaptive retrieval budgets, while a generator agent retrieves evidence and iteratively refines the outline. The coordination between the two agents is formulated as a bi-level potential game to guide search strategy optimization. The method is evaluated on three public deep-research benchmarks (DeepResearchBench, DeepConsult, and DeepResearchGym) and a newly introduced lifestyle-oriented benchmark, GALA (General AI Life Assistants).

After rebuttal: The authors responses partially address my concerns in review, and I have updated my overall score accordingly.

Critical strengths

S1. An interesting game-theoretic formulation of search strategy optimization.

Formulating the coordination between exploration and exploitation as a bi-level potential game offers an interesting perspective on deep-research agent optimization. The framework provides a theoretically motivated way to coordinate the critic and generator, rather than relying solely on heuristic search and refinement strategies.

S2. A useful new benchmark targeting realistic research scenarios.

The proposed GALA benchmark is a potentially valuable contribution, as it extends evaluation beyond conventional deep-research tasks to more realistic, everyday lifestyle queries. This could help assess the practical usefulness and generalizability of deep-research agents.

Critical weaknesses

W1. Reliance on an internal scoring agent raises concerns about evaluation bias and reward reliability.

The framework relies on an internal scoring agent (SummaryQA) to guide search strategy optimization. However, it is unclear how reliably this agent assesses research quality and whether its scores correlate with actual evidence coverage, factual correctness, and final report quality. This is particularly concerning given that self-evaluation bias is one of the key limitations the paper aims to address. Without independent validation, the framework may simply shift evaluation bias from the generator to the scoring agent.

W2. The separation between outline optimization and final report generation may limit end-to-end effectiveness.

If I understand correctly, the final report writer remains frozen and is not jointly optimized with the critic and generator. This implicitly assumes that improving the research outline will translate into better final reports. However, a frozen writer may struggle to effectively synthesize AgentDisCo's detailed, multi-blueprint outlines, potentially limiting the benefits of the proposed optimization. Consequently, it remains unclear whether improvements in outline quality consistently lead to corresponding gains in end-to-end report quality.

Questions for authors

Q1 Reliability of the scoring agent: How is SummaryQA validated against independent human judgments or external evaluators? To what extent do its scores correlate with factual correctness, evidence coverage, and final report quality?

Q2 Sensitivity to scoring errors: How robust is AgentDisCo to inaccurate or biased feedback from SummaryQA? Could optimizing against this scoring agent lead to reward hacking or reinforce the self-evaluation bias that the framework aims to mitigate?

Q3 Impact of the frozen writer: Have the authors investigated whether improvements in outline quality consistently translate into better final reports? In particular, could the frozen writer become a bottleneck when processing complex, multi-blueprint outlines?

Q4 Contribution of the game-theoretic formulation: How much of the performance improvement can be attributed specifically to the bi-level potential game formulation, rather than simply introducing a separate critic agent, gap-aware search, and adaptive retrieval budgets? A comparison with a similarly configured dual-agent system without game-theoretic optimization would help clarify this contribution.

Q5 Generalizability across writers: Would the improvements remain consistent when using different report-writing models, particularly models with weaker synthesis capabilities?

4 comments
AuthorsOpenReview ↗

Response to Reviewer d6PA Part 1

We thank you for recognizing the game-theoretic formulation of search-strategy optimization and the value of the GALA benchmark. Below, we address each weakness and question in turn. We hope that our responses and the additional experiments resolve your concerns, and we would very much welcome any further discussion.

W1: The framework relies on an internal scoring agent (SummaryQA) to guide search strategy optimization. However, it is unclear how reliably this agent assesses research quality and whether its scores correlate with actual evidence coverage, factual correctness, and final report quality. This is particularly concerning given that self-evaluation bias is one of the key limitations the paper aims to address. Without independent validation, the framework may simply shift evaluation bias from the generator to the scoring agent.

Q1: How is SummaryQA validated against independent human judgments or external evaluators? To what extent do its scores correlate with factual correctness, evidence coverage, and final report quality?

We distinguish the scoring mechanism, evidence of downstream usefulness, and direct validation of SummaryQA as follows.

  1. To clarify whether optimization relies solely on SummaryQA’s self-evaluation, the scoring pipeline comprises two distinct components. As described in Section 2.3 and Appendices F.1–F.2, SummaryQA performs document-level relevance scoring and snippet extraction. A separate evaluator, implemented as JudgeSearchQueryDiversityAPI, uses relevance distributions and snippets to assess retrieval support and cross-query complementarity. Its code and prompt remain frozen throughout harness optimization, preventing direct modification of its scoring rules. The optimization target is therefore this downstream search-coverage judgment, rather than SummaryQA’s mean relevance score.

  2. To examine whether improvements in the internal signal translate into better final reports, we compare internal and external metrics across optimization checkpoints. As shown in Table 7 (Appendix D.4), across harness rounds 0–20, Search Coverage increases from 6.25 to 8.21, while RACE Overall increases from 52.97 to 54.11, with gains at every reported checkpoint. Citation accuracy also improves from 77.86% to 82.12%, and effective citations increase from 84.94 to 92.34 (Table 1). These findings support the downstream usefulness of the internal signal.

  3. To assess whether the report improvements are also recognized by people rather than only automated evaluators, we conduct a blind human comparison. As reported in Appendix D.5, three annotators evaluated 40 pairs of reports from harness rounds 0 and 20. Optimized reports were preferred in 70.0% of cases, with Fleiss’ κ = 0.58.

Although these results support the downstream usefulness of the internal scoring signal, they do not directly validate SummaryQA’s document-level judgments. To address this gap, we have added the following new evaluations:

  1. To directly test whether SummaryQA’s relevance judgments are corroborated by evaluators from other model families, we evaluate 100 query–document pairs using GPT-4.1 and Claude-Opus-4.6. Both evaluators use the same relevance rubric and task context, without access to SummaryQA’s outputs (Appendix D.4, Table 8). Binary agreement with SummaryQA reaches 78.0% and 79.0%, with Spearman’s ρ = 0.76 and 0.75, respectively. These results support cross-family consistency at the document level. They complement the report-level improvements under GPT-4.1 and GPT-4.1-mini: DeepConsult Overall increases from 6.91 to 7.12, and DeepResearchGym Overall from 95.63 to 96.42 (Section 4.1, Table 2).

  2. To directly answer your question about validation against independent human judgments, we ask three annotators to assess relevance on the same 100 query–document pairs. They use the same rubric and source-page evidence, without access to model scores or one another’s labels (Appendix D.5, Table 11). Pairwise SummaryQA–human agreement reaches 85.0%, compared with 94.0% human–human agreement. These results provide independent human corroboration of SummaryQA’s relevance judgments, while the remaining gap identifies room for improvement.

Together, the document-level evaluations and external report metrics address the reliability and downstream usefulness of the scoring signal.

AuthorsOpenReview ↗

Response to Reviewer d6PA Part 2

Q2: How robust is AgentDisCo to inaccurate or biased feedback from SummaryQA? Could optimizing against this scoring agent lead to reward hacking or reinforce the self-evaluation bias that the framework aims to mitigate?

Our approaches include architectural safeguards, a modification audit, and a new controlled feedback-noise experiment, as detailed below.

  1. To limit direct manipulation of the optimization objective, the final evaluator, JudgeSearchQueryDiversityAPI, and its prompt remain frozen throughout harness optimization (Section 2.3 and Appendix F.1). The audit in Table 6 (Appendix D.4) reports that approximately 68% of modifications target search-query guidelines, 26% target blueprint updates, and 6% involve SummaryQA’s relevance criteria or snippet formatting. The reviewed changes show no clear evidence of deliberate score inflation or content fabrication. However, the frozen evaluator still consumes upstream scores and summaries, so this safeguard does not eliminate possible exploitation of its inputs.

  2. To directly assess sensitivity to inaccurate feedback, we repeat the 20-round harness optimization on 100 DeepResearchBench queries with uniform noise of ±0.1 and ±0.2 added to SummaryQA’s 0–1 relevance scores, clipping the results to [0,1] while keeping other settings fixed (Appendix D.4, Table 9). External final-report RACE decreases from 54.11 to 53.84 and 53.31 for Gemini-2.5-Pro, and from 55.60 to 55.41 and 55.06 for Claude-Opus-4.6. Both remain above their reported no-harness references of 52.97 and 54.72, with a margin of 0.34 points each under the stronger perturbation. These results support tolerance to the tested random scoring errors, although increasing noise reduces the observed harness benefit.

W2: If I understand correctly, the final report writer remains frozen and is not jointly optimized with the critic and generator. [...] a frozen writer may struggle to effectively synthesize AgentDisCo's detailed, multi-blueprint outlines [...] it remains unclear whether improvements in outline quality consistently lead to corresponding gains in end-to-end report quality.

Q3: Have the authors investigated whether improvements in outline quality consistently translate into better final reports? In particular, could the frozen writer become a bottleneck when processing complex, multi-blueprint outlines?

We summarize our evidence and additional experiments addressing the potential limitations of a frozen writer as follows:

  1. Improvements in the research stage translate into better final reports with the writer held fixed in our evaluated configurations. Harness optimization improves DeepResearchBench RACE from 52.97 to 54.11, DeepConsult Overall from 6.91 to 7.12, and DeepResearchGym Overall from 95.63 to 96.42 (Tables 1–2). These metrics evaluate completed reports after writer synthesis, rather than intermediate outlines. Appendix D.1 also evaluates outline refinement while keeping the subsequent writing strategy unchanged.

  2. The writer processes complex, multi-blueprint research outputs incrementally through section-wise synthesis. As described in Appendix B.2, it expands each section using its associated references from the document bank, allowing relevant evidence to be incorporated section by section rather than requiring all blueprints and retrieved documents to be synthesized in a single pass.

  3. To directly test whether improved research inputs benefit different frozen writers, we conduct an additional writer-only ablation (Appendix D.12, Table 18). We preserve the same baseline and optimized outlines and evidence banks from the Gemini research pipeline, then substitute only the writer, keeping each writer’s configuration unchanged across the two input conditions. | Writer | Round-0 RACE | Round-20 RACE | Gain | |---|---:|---:|---:| | Claude-Opus-4.6 | 54.20 | 55.31 | +1.11 | | Gemini-2.5-Pro | 52.97 | 54.11 | +1.14 | | Qwen3-235B | 52.02 | 53.26 | +1.24 | RACE improves from 54.20 to 55.31 for Claude-Opus-4.6 (+1.11), from 52.97 to 54.11 for Gemini-2.5-Pro (+1.14), and from 52.02 to 53.26 for Qwen3-235B (+1.24). The gains persist across all three tested writers, including the relatively lower-scoring Qwen3 model.

AuthorsOpenReview ↗

Response to Reviewer d6PA Part 3

Q4: How much of the performance improvement can be attributed specifically to the bi-level potential game formulation, rather than simply introducing a separate critic agent, gap-aware search, and adaptive retrieval budgets? A comparison with a similarly configured dual-agent system without game-theoretic optimization would help clarify this contribution.

Thank you for this question. We distinguish the benefit of harness optimization from the specific contribution of its alternating update schedule.

  1. The existing no-harness baseline already includes the separate critic, gap-aware search, and adaptive retrieval budgets. Adding the bi-level harness improves RACE from 52.97 to 54.11 (Table 1), demonstrating gains beyond these architectural components. The component ablation further reduces RACE from 54.11 to 53.69 when removing the policy bank, and to 53.33 when additionally removing the iterative loop (Table 4, Appendix D.2). These comparisons support the harness components but do not independently isolate the update schedule.

  2. To directly address the requested comparison, we conduct an additional alternating-versus-joint optimization experiment (Appendix D.13, Table 19). Both variants retain the same dual-agent architecture, gap-aware search, adaptive retrieval budgets, and configuration-search settings. Joint updating allows each candidate to modify both critic and generator configurations, whereas alternating updating modifies one while holding the other fixed. Joint updating achieves 53.35 RACE, compared with 54.11 for alternating updating and 52.97 without harness optimization. | Optimization condition | RACE | |---|---:| | No harness | 52.97 | | Joint updates | 53.35 | | Alternating updates | 54.11 | The additional gain over joint updating supports the contribution of the alternating schedule beyond introducing a second agent or performing configuration search. A possible explanation is that holding one agent fixed makes beneficial changes easier to identify, whereas coupled updates may settle on locally suboptimal configurations.

Q5: Would the improvements remain consistent when using different report-writing models, particularly models with weaker synthesis capabilities?

Our existing evidence and additional experiments addressing generalization across report-writing models are detailed below.

  1. The existing cross-backbone results show that the improvements extend to weaker models. Appendix D.3 reports relative RACE gains of 1.6%–2.3% across five backbones, including Qwen2.5-72B improving from 46.51 to 47.58. Since these comparisons vary the entire system backbone rather than the writer alone, we additionally conduct a writer-only ablation.

  2. To directly assess generalization across writers, we conduct an additional writer-only ablation (Appendix D.12, Table 18). Claude-Opus-4.6, Gemini-2.5-Pro, and Qwen3-235B receive the same baseline and optimized outlines and associated evidence, while their writing configurations remain fixed. RACE improves from 54.20 to 55.31 for Claude (+1.11), from 52.97 to 54.11 for Gemini (+1.14), and from 52.02 to 53.26 for Qwen (+1.24). These results show that research-stage optimization improves final-report quality across all tested writers, including the lower-scoring Qwen3-235B.

AuthorsOpenReview ↗

Follow-up Response to Reviewer d6PA

Thank you for reviewing our responses and additional experiments, and for updating your assessment. We appreciate your constructive feedback and would be happy to clarify any remaining questions.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Response to all reviewers

We sincerely thank all reviewers for their careful and constructive feedback. We appreciate your recognition of the interesting game-theoretic formulation and valuable GALA benchmark (Reviewer d6PA), the clear motivation and substantial improvements on DeepResearchBench (Reviewer BMba), and the reasonable critic–generator design and comprehensive evaluation (Reviewer 8daF).

Our contribution goes beyond assigning separate critic and generator roles. (i) AgentDisCo makes critique actionable through structured blueprints that bind coverage requirements, targeted search queries, and per-gap retrieval budgets, connecting evidence-gap identification to retrieval and outline refinement. (ii) We formulate harness optimization as an outer potential game, in which critic and generator configurations are alternately optimized toward a shared objective evaluated by a frozen downstream evaluator. This distinguishes cross-task configuration optimization from iterative refinement of an individual task’s output. (iii) We also introduce GALA, a benchmark of everyday research needs reformulated from anonymized user-interaction histories.

Here is a list of new experiments:

  1. Document-level validation of the internal scorer (Reviewer d6PA, W1/Q1; Appendices D.4–D.5, Tables 8 and 11). On 100 query–document pairs, SummaryQA achieves 78.0% and 79.0% relevance-label agreement with GPT-4.1 and Claude-Opus-4.6, with Spearman’s ρ of 0.76 and 0.75, respectively. Pairwise SummaryQA–human agreement is 85.0%, compared with 94.0% human–human agreement (Appendices D.4–D.5). These results provide independent corroboration of SummaryQA’s document-level relevance judgments across model families and human annotators.

  2. Feedback-noise robustness (Reviewer d6PA, Q2; Appendix D.4, Table 9). We repeat the 20-round optimization with uniform perturbations of ±0.1 and ±0.2 added to SummaryQA’s relevance scores and clipped to [0,1]. External RACE changes from 54.11 to 53.84/53.31 for Gemini-2.5-Pro and from 55.60 to 55.41/55.06 for Claude-Opus-4.6. Both remain above their respective no-harness references of 52.97 and 54.72, supporting tolerance to the tested random scoring errors (Appendix D.4).

  3. Writer-replacement ablation (Reviewer d6PA, W2/Q3/Q5; Appendix D.12, Table 18). Holding the baseline and optimized research artifacts fixed across writers, the harness gain persists for Claude-Opus-4.6 (+1.11), Gemini-2.5-Pro (+1.14), and Qwen3-235B (+1.24 RACE points). The lower-scoring Qwen writer retains the research-stage benefit, supporting transfer across the tested synthesis models (Appendix D.12).

  4. Alternating-versus-joint optimization (Reviewer 8daF, W1/Q1; Reviewer d6PA, Q4; Appendix D.13, Table 19). With the architecture and configuration-search settings held fixed, alternating updating achieves 54.11 RACE, compared with 53.35 for joint updating and 52.97 without harness optimization. The advantage over joint updates supports the contribution of the alternating schedule beyond the critic–generator architecture alone (Appendix D.13). One possible explanation is that joint updates couple changes in both agents, making beneficial modifications harder to identify and potentially leading to locally suboptimal configurations.

For your convenience, all modifications in the updated PDF are highlighted in blue color.

For all revierers: as the discussion period is approaching its end, please join the discussion and put acknowledge or feedback on authors' rebuttal, thank you!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.