- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper studies coordination failures in LLM-agent populations caused by agents competing for shared resources. It proposes BRIA, a bilevel information allocation framework that improves coordination by controlling which truthful information each agent receives, without modifying the acting agents. BRIA alternates between learning a history-aware population response model and optimizing joint information allocation using model-based evaluation. Experiments on three datasets demonstrate improvements in response prediction, task success, and reduced crowd-out compared to various baselines.
Critical strengths
-
An important and practical problem. Joint information allocation is a meaningful approach to improving coordination, especially in large-scale LLM multi-agent systems where independently reasonable decisions may lead to poor collective outcomes. I believe this is a valuable research direction with practical relevance.
-
A reasonable optimization framework. The proposed bilevel framework is well-motivated. Learning a population response model to evaluate candidate allocations can potentially reduce expensive LLM-agent executions, while alternating updates allow the model to adapt to evolving interaction histories. The experimental results provide encouraging evidence for this approach.
Critical weaknesses
-
Insufficient clarity in the methodology. Several important components are not clearly defined or explained. For example, the allocation policy and its relationship to the learned proposal and final selector are unclear, the concrete round-level task loss is unspecified, and the prior distribution is insufficiently described. Section 4.1 also lacks a clear explanation of the response model architecture and its implementation. Figure 2 is overly complicated and uses terminology inconsistent with the main text. These issues make the method difficult to understand and reproduce.
-
Insufficient experimental and implementation details. The benchmark descriptions should clarify the number of agents, their tasks, and how information allocation is constructed for each dataset. The paper also compares cooperative MARL algorithms, information selection methods, and LLM prompting methods, but their relationships to BRIA and their adaptations to this setting deserve more explanation. Additionally, although computational efficiency is an important motivation, the paper does not provide sufficient analysis of actual training and inference costs. These missing details make it difficult to fully assess the experimental comparisons and practical efficiency of the proposed method.
-
More comprehensive evaluation of coordination failures. While the reduction in Crowd-out provides direct evidence for improved coordination, the introduction discusses broader coordination failures, including duplicated effort and underutilized resources. Additional analysis of these failure modes and population-level action distributions would better support the paper's claims. Furthermore, evaluating different population sizes would help demonstrate the scalability and practical relevance of the proposed approach.
Questions for authors
See Critical Weaknesses