- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes CA-RAPU, a training-free message scheduling method for MARL systems with frozen policies and communication networks. The method determines which messages to transmit under bandwidth constraints by measuring how each message changes the receiver's action distribution and whether that change aligns with a reference constructed from other agents' messages.
The method is primarily evaluated on a modified Simple Spread benchmark with simulator-generated noisy target-location clues, alongside a smaller exploratory CommNet Predator–Prey experiment. Results show improvements over KL-based scheduling in the modified Spread environment, but mixed results in CommNet.
Critical strengths
Practical motivation: Addresses communication constraints in pretrained MARL systems without requiring policy retraining.
Critical weaknesses
W1. Limited generalizability due to highly task-specific assumptions. The method requires a meaningful reference derived from other agents' messages. In the primary experiment, the simulator explicitly provides noisy estimates of other agents' target locations, which are encoded and decoded to construct this reference. This is a highly customized communication setting, unlike conventional MARL methods that exchange learned latent representations without explicit target-location information. It remains unclear how CA-RAPU can be applied to general pretrained communication policies where such information is unavailable. The CommNet experiment provides insufficient evidence to establish broad applicability.
W2. Insufficient justification and ablation of the encoder-decoder architecture. The proposed experimental architecture uses an encoder to transform two-dimensional target coordinates into two-dimensional messages, followed by a decoder that reconstructs target coordinates. The motivation and necessity of this architecture are unclear, particularly since transmitting the original coordinates directly would be a simpler alternative. The paper lacks a direct comparison against raw-coordinate transmission without the encoder-decoder, making it difficult to understand the benefit of this design and its relationship to the proposed scheduling mechanism.
W3. Insufficient comparisons with established communication-efficiency methods. Most primary baselines are heuristic variations or paper-specific learned comparators rather than established MARL communication algorithms. Although the paper discusses attention-based, belief-based, and learned communication scheduling methods, it provides limited empirical comparisons with them. In particular, Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Control (NeurIPS 2019) is relevant prior work that is not cited nor compared against. While full comparisons with jointly trained communication algorithms may require adapting the experimental protocol, stronger baselines incorporating established uncertainty-, variance-, attention-, or belief-based message-selection principles are necessary to establish the proposed method's advantages.
W4. Insufficient benchmark diversity and limited evidence of generalization. The experimental evaluation relies predominantly on one heavily modified Simple Spread benchmark, with three- and four-agent variants representing essentially the same task. The additional CommNet Predator–Prey experiment covers only six trained policies and produces mixed results. CA-RAPU underperforms KL-based scheduling on five of six policies under fixed communication capacity, and the planned larger experiment was not conducted. Consequently, the experimental evidence is insufficient to demonstrate the effectiveness of the proposed method across diverse MARL tasks and communication architectures.
Questions for authors
-
General applicability: How can CA-RAPU construct a meaningful action reference for conventional MARL communication architectures where messages are learned latent representations rather than explicitly decodable task information? What happens when simulator-generated target clues are unavailable?
-
Encoder-decoder necessity: Why is an encoder-decoder required when the underlying information consists of two-dimensional coordinates? Could the authors provide an ablation comparing against direct coordinate transmission without encoding?
-
Stronger baselines: How does CA-RAPU compare with established communication-efficiency methods? Could relevant message-selection heuristics from these methods be evaluated under the same frozen-policy setting?
-
Broader evaluation: Could the authors demonstrate effectiveness on additional established MARL benchmarks and communication architectures, particularly those using naturally learned latent messages without task-specific reference information?
-
Communication overhead and deployment assumptions: The method assumes a centralized scheduler with access to candidate messages and receiver-policy states. How are these obtained in a distributed MARL system? Would gathering this information incur additional communication costs, and how would these affect the reported bandwidth savings?