- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper addresses post-training robustness in multi-agent reinforcement learning (MARL) policies, where observations may be noisy due to faulty sensors. It proposes a recovery mechanism, called SIR-MARL, which uses the joint observation of all agents to infer group perceptions errors in order to reconstruct valid observations. SIR-MARL is evaluated in three SMAC tasks and compared with some baselines.
Critical strengths
The paper addresses a practically relevant problem in cooperative MARL.
Critical weaknesses
Problem formulation
While the problem is important, it is not new and already known from distributed systems [1,2,3]. However, the introduction does not contain many references (the first paragraph does not even have any), which is important for scientific integrity.
The paper focuses on Dec-POMDP, which already features noisy observations [2,4,5]. SMAC, on the other hand, represents a special case of Dec-POMDPs with deterministic observations and initial states [5,6]. Thus, the paper seems to specialize more on the SMAC setting rather than the general Dec-POMDP.
Limited significance
On the one hand, the paper assumes noisy observations at inference time, which is a realistic assumption. However, the recovery mechanism relies on the joint observation, which requires perfectly reliable global communication, which is rather unrealistic [1,2].
I also have concerns regarding the evaluation:
- SMAC is known to be outdated (due to its special case traits [5,6]).
- Only three tasks have been presented, despite SMAC having more than 10 different maps.
For a better evaluation, it might make sense to use SMAC as an idealistic training setup, while using SMACv2 [6] as the test setup (and demonstration of the recovery method).
Literature
[1] Tanenbaum et al., "Distributed Systems: Principles and Paradigms"
[2] Oliehoek et al., "A Concise Introduction to Decentralized POMDPs", 2016
[3] Stone et al., "Multiagent Systems: A Survey from a Machine Learning Perspective", Autonomous Robots 2000
[4] Lyu et al., "A Deeper Understanding of State-Based Critics in Multi-Agent Reinforcement Learning", AAAI 2022
[5] Phan et al., "Attention-Based Recurrence for Multi-Agent Reinforcement Learning under Stochastic Partial Observability", ICML 2023
[6] Ellis et al., "SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learning", NeurIPS Benchmarks 2023
Questions for authors
None
Response to rebuttal
Thank you for the new experiment. I have updated my assessment based on the most recent provided information.