- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This work presents PreACT, a framework for cooperative multi-agent reinforcement learning that uses self-supervised intent signals to enable coordination within a single environment timestep. The framework adopts a two-stage policy: agents first generate and broadcast messages from local observations and policy representations, then condition their final actions on the collected messages while preserving simultaneous execution. The communication module is trained through observation reconstruction and action-distribution prediction, with task-driven policy gradients withheld to promote semantic stability. Experiments on Multi-Agent Particle Environments, Level-Based Foraging, and Google Research Football demonstrate competitive coordination and task performance relative to the evaluated baselines.
Critical strengths
- The experiments are extensive, covering diverse scenarios and a wide range of baselines, with evaluations of several aspects of the method.
- The paper focuses on a well-motivated problem that is relevant to practical multi-agent coordination.
Critical weaknesses
Scalability: The method relies on team-wide broadcasting, which may limit its scalability. In the current implementation, every agent sends messages to the entire team, without learning when to communicate, whom to communicate with, or how much information to send. In larger teams, many messages may be irrelevant to the current decision, increasing the communication burden. Moreover, the experiments are limited to simulated environments, so the method’s applicability to real-world settings remains unverified.
Evaluation: I have some concerns about whether the messages maintain consistent semantics throughout training. The messages are 16-dimensional continuous vectors. Although the paper examines their semantics through a decoder and visualizations, it does not clarify what particular dimensions or regions of the message space represent. Additional evidence showing whether similar messages retain the same meaning across different stages of training would help support this claim.
Questions for authors
- How are the tentative actions in Figure 7 computed? Are they obtained from the message decoder or from a policy evaluated without teammates’ messages?
- How are the learned opponents evaluated across methods? Are teams tested against a common set of fixed opponents, or only against their own co-trained opponents?
Dear authors and reviewers,
The author-reviewer discussion period closes on November 18 (AoE). Please engage in response periods so that we have our issues discussed thoroughly.
AC