- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper targets the scenario in which the evidence required to solve a task is distributed across multiple context shards, such that no single model instance observes all relevant information. The paper proposes post-training with communication, where identical, parameter-shared, role-free agents each access one shard, exchange free-text messages through a shared board, and then generate answers using both their local context and the board. The method is trained with verifier-based RL and introduces two credit-assignment terms: answer credit, which encourages agents to benefit from the board, and message credit, which aims to reward messages that improve other agents’ answers. Experiments on reduction-max, HotpotQA and LongMemEval-mid show that a team trained at a small team size can be evaluated at larger team sizes.
Critical strengths
- The paper targets a relatively underexplored setting in which the evidence required for answering a question is distributed across agents. This setting is different from existing setting, such as multi-agent debate, in which each agent observes the full context.
- The proposed method is clear: identical, parameter-shared, role-free agents each observe only a private context shard and communicate through a shared free-text board. This design makes it clearer whether the performance gain comes from agents sharing their different pieces of information, rather than from manually assigned roles or using different agent models.
Critical weaknesses
- It seems that the message-credit proxy does not isolate the contribution of an individual message. Instead of comparing peers' performance before and after removing only the sender's message, it compares different rollouts where all agents' messages may differ. The credit may therefore reflect other messages or overall rollout quality, rather than the sender's causal contribution.
- Experiments are limited in scope. All main results use Qwen3-8B-Base and are evaluated on the same task constructions used for training. For example, the model trained on HotpotQA is evaluated only on HotpotQA, so it remains unclear whether the method learns a transferable ability to communicate and pool distributed evidence, rather than adapting to a specific task format. For example, training on HotpotQA should be also evaluated on some factual QA benchmark, such as TriviaQA, NQ, SimpleQA, ChineseSimpleQA.
- The evaluated application scenarios are limited. Although the proposed framework could potentially apply to many settings with distributed information, the experiments cover only synthetic aggregation, multi-hop QA, and one long-context memory setting. In particular, for the long-context application, the paper compares mainly with context extension and map-reduce, but not with retrieval- or memory-based systems such as MemAgent. These methods are particularly relevant to the LongMemEval setting and represent a different approach to handling long-context information.
Questions for authors
- Could the authors provide additional ablations or analyses to validate the cross-rollout message-credit proxy?
- For the long-context memory setting, could the authors discuss or compare against retrieval- or memory-based approaches?
- Could the authors improve the paper's presentation and layout? For example, Page 8 contains a substantial amount of unused space. Using the space more effectively could improve readability.