- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents GateTree, a multi-agent collaboration framework that integrates selective context sharing with adaptive planning. GateTree maintains two concurrent branches with complementary conservative and exploratory strategies. A GateAgent manages shared information through provenance tracking, applicability filtering, and recipient-specific context selection, while an adaptive Planner updates branch objectives and verification requirements based on shared evidence and execution feedback.
The authors evaluate GateTree on SWE-bench Pro, text-only HLE, and GAIA2 using two backbone models. GateTree achieves the highest benchmark-averaged Pass@2 across the compared methods, with an average inference cost reduction of 30.17% relative to the strongest Pass@2 baseline in each setting. The paper also presents extensive component ablations, context-selection studies, and analyses of agent coordination.
Critical strengths
-
The research question is critical and practical. Managing shared information across concurrent agents is an important challenge, particularly when evidence becomes stale, contradictory, or irrelevant as different branches evolve. The paper identifies a meaningful problem in multi-agent collaboration.
-
The experiments are comprehensive. The experiments cover three substantially different benchmarks and two backbone models. GateTree consistently achieves competitive or superior task-level coverage while maintaining relatively low inference costs.
-
The controlled experiments varying context length and relevant-evidence density provide useful insights into when selective context delivery is beneficial.
Critical weaknesses
-
The overall novelty is limited, and the differentiation from prior work is insufficient. The proposed framework largely combines established techniques, including structured shared memory, provenance-based filtering, recipient-aware context selection, heterogeneous agent roles, and feedback-driven replanning. In particular, RCR-Router [1] already investigates role- and task-stage-aware context routing over structured shared memory, with iterative memory updates and token-budget constraints. The paper also builds on ideas closely related to DeLM's feedback-driven coordination and AgentFugue's selective sharing of peer information. Although GateTree introduces more explicit applicability checks and plan-conditioned evidence delivery, it remains unclear whether these extensions constitute a substantial methodological advance.
-
The paper compares against several established agent frameworks, including DeLM and AgentFugue. However, it lacks a sufficiently strong, lightweight baseline that directly tests whether the proposed architecture is necessary. For instance, a simpler framework combining provenance-aware shared memory, plan-conditioned embedding retrieval, and the same adaptive Planner. Moreovew, although Section 5.2 compares GateAgent against Recent, Priority, and All Context selection, Recent and fixed-priority ranking are relatively weak retrieval strategies. These comparisons do not sufficiently establish the advantage of the proposed LLM-based selection mechanism over a competent semantic-retrieval baseline. Similarly, AOrchestra is discussed in related work but not experimentally compared, despite its relevance to adaptive agent orchestration.
-
The paper provides meaningful ablation evidence. However, the results do not establish whether the improvements arise from a genuinely distinctive coordination mechanism or simply from combining better information retrieval with standard feedback-driven replanning. For instance, conditioning retrieval on an updated plan naturally increases the relevance of delivered evidence. It remains unclear whether a standard plan-aware retrieval mechanism would recover most of the observed improvement. Furthermore, removing adaptive planning causes a substantial 16.5-point reduction in Pass@2, suggesting that a considerable portion of the gains may come from feedback-driven planning, which is already well established.
-
The paper emphasizes Pass@2 as its primary performance metric. However, improvements in task-level coverage do not consistently translate into higher average success rates across individual submissions. A more detailed analysis separating individual success, solution diversity, and coordination benefits would help clarify the source of the improvements.
[1] RCR-Router: Efficient Role-Aware Context Routing for Multi-Agent LLM Systems with Structured Memory, https://arxiv.org/abs/2508.04903
Questions for authors
- To what extent do improvements arise from complementary branch behavior rather than improved individual execution quality? Could the authors provide comparisons against independent conservative/exploratory attempts under matched token budgets and verification procedures?
3 comments
Thanks for your response. Some of my concerns are addressed and I would increase the score to 2.
Thank you for your feedback and for reconsidering your score. We sincerely appreciate it.
If you have any remaining questions or concerns, we would be happy to provide further clarification. We hope our responses and additional evidence may support further reconsideration of the score.
Thank you again for your time and constructive feedback.
Dear Reviewer,
Thank you again for your constructive feedback. We have provided detailed responses to your concerns, including additional experiments and analyses. As the discussion period approaches its deadline, we would appreciate it if you could review our responses. We would be happy to address any remaining questions or concerns.
Thank you for your time and consideration.