- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes MACWM, a multi-robot cooperative learning method that combines a contrastive goal-conditioned critic with an action-conditioned latent world model. The critic learns goal reachability from actual trajectories and hindsight goals, while the world model predicts the latent representations used by the critic and provides the policy with training signals that account for the consequences of joint actions through short-horizon differentiable imagination. The training does not rely on hand-designed task rewards, and at deployment time each robot generates actions based only on its own observations and goals, without running the world model or exchanging policy messages.
Overall, I have a positive view of this work and consider it a reasonable and effective cooperative learning scheme; the remaining issues mainly concern clarification of implementation details and explanation of some experimental phenomena.
Critical strengths
-
MACWM uses a critic to evaluate short-horizon imagined trajectories, enabling policy updates to account for the effects of continuous joint actions and reasonably addressing the weak learning signal from single-step actions in high-frequency control.
-
On nine dual-robot tasks, MACWM achieves a higher average success rate than MACRL and ICRL, supporting the overall effectiveness of this design; removing the imagination term on PushBox reduces the success rate, indicating that the world model provides valuable training signals for strongly cooperative tasks; MACWM also maintains its lead in tasks with an increased number of robots.
-
The paper cross-examines the world model and exploration noise, and analyzes the critic's ability to distinguish actions, providing richer evidence for understanding when the imagination term is effective.
-
The world model and centralized critic are used only during training, and each robot runs its policy independently at execution time; compared with the baselines, this further improves physical deployability.
Critical weaknesses
- The actions in the imagination phase are all sampled from the same replay rather than generated step by step based on predicted observations. In my view, this is an intentional design choice by the authors, but there seems to be a lack of experiments validating this design.
2.The default imagination horizon is five steps, and the paper has not yet adequately explained why such a short horizon can significantly improve tasks requiring sustained cooperation, such as PushBox.
- in Table 12, the average success rate under pink noise increased from 0.629 to 0.655, but the result under Gaussian noise decreased from 0.521 to 0.451. The analysis of this significant contrast seem still insufficient.
Questions for authors
-
Are the imagined actions across the entire segment sampled from the same policy distribution using independent noise, and is it intentional not to use the decoded predicted observations to update the action distribution, so as to reduce model error or training overhead? Could the authors add experiments to clarify this?
-
It is suggested to add an ablation study on the horizon.
-
Under the two types of exploration noise, are there observed differences in world model prediction error or in the number of successful cooperative trajectories? This could help explain the reversal in the direction of the imagination term's benefit in Table 12.