- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies when multi-agent collaboration is beneficial relative to a strong single-agent harness. It argues that the benefit of collaboration depends on task dependency structure: multi-agent systems can help on long-horizon tasks with sparse dependencies, while decomposition may hurt tightly coupled sequential workflows because of coordination overhead and dependency errors. The paper formalizes execution trajectories as dependency DAGs, identifies bridge edges as candidate subtask boundaries, and proves a context-cost lemma under oracle decomposition assumptions. It then proposes SAIGE, which incrementally spawns agents when semantic dependency boundaries are detected and retrieves inter-agent information on demand. Experiments across four long-horizon benchmarks show that SAIGE performs particularly well on sparsely dependent tasks, while avoiding much of the degradation of generic multi-agent baselines on tightly coupled tasks.
Critical strengths
The paper focuses on an important question: when does multi-agent collaboration provide meaningful benefits over a strong single-agent harness? Its dependency-aware perspective is also useful: modeling execution trajectories as dependency graphs and using incremental delegation with on-demand message fetching offers a principled way to relate task structure to context isolation and coordination costs.
Critical weaknesses
1. Overstated theoretical contributions. The paper’s characterization of its results as “core theoretical contributions” overstates what the analysis establishes. It provides a graph-based formalization and a context-cost lemma under oracle decomposition assumptions, but these do not establish when collaboration benefits outweigh coordination costs or substantiate the claimed capability boundaries of multi-agent systems. The analysis also does not justify the claimed advantage of incremental orchestration: Eq. (6) follows from event inclusion rather than demonstrating improved prediction accuracy from additional execution evidence. The theoretical claims should therefore be narrowed to match the results actually established.
2. Insufficient engagement with prior work on when and why MAS helps. The related work focuses primarily on recent LLM-based architectures and topologies, without adequately positioning the contribution against established research on task decomposition, coordination, and the benefits and costs of multi-agent problem solving. Consequently, it remains unclear which insights are new to LLM agents and which restate existing principles.
Questions for authors
-
In Table 1, SAIGE sometimes uses fewer input tokens than the single-agent baseline, despite introducing additional agents, delegation steps, and inter-agent communication. This seems a little surprising, so maybe clarifying how token usage is measured and aggregated across agents is necessary.
-
All methods are evaluated with the same foundation model, DeepSeek-V4-Pro. Are the observed advantages of SAIGE robust across different foundation models, or could they be specific to the model’s instruction-following, context-management, or tool-use behavior? Cross-model experiments would be important for theory establishment.
-
The paper uses
Nmax = 4as the default configuration, but does not explain why this value was selected. The ablation study only comparesNmax = 4, 5, 6, and therefore does not establish whether four agents are actually optimal or simply the smallest tested budget.
Reminder For Discussion
Dear Reviewer VQ6T, We hope this message finds you well. We would greatly appreciate it if you could take a moment to review our response and consider providing your feedback before the discussion period ends on Nov.18th. Thank you very much for your time and valuable feedback. We look forward to hearing from you.