- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies stability regulation in cooperative MARL under external disturbances. They propose STAB, which combines three parts including (1) incorporating disturbance-related information into policy and value-function inputs, (2) using sliding-window statistics to characterize historical deviations and their trends, and (3) designing a recovery-oriented reward based on the duration of abnormal operation and potential-based reward shaping.
The proposed approach is evaluated with MAPPO across three environments, including MultiWalker, power-grid voltage control, and traffic signal control. The experimental results show improvements in the reported unstable rate compared with the considered baselines.
Critical strengths
-
It is a meaningful problem setting. Improving the robustness and recovery capability of cooperative MARL systems under external disturbances is an important practical problem. The paper also raises a reasonable concern that conventional tracking-error objectives may not always align with the goal of rapid recovery.
-
Simple and practical design (but I think it is just three engineering tricks ). The proposed components are relatively straightforward to implement and can be integrated into existing MARL frameworks without substantial architectural changes. This may be useful from an engineering perspective.
-
The experiments cover three different cooperative control scenarios, and the proposed method consistently improves the reported unstable rate over the considered baselines. The observation ablations also provide some evidence for the usefulness of disturbance information and temporal features.
Critical weaknesses
-
From a methodological perspective, STAB appears to be primarily a combination of several relatively standard design choices including incorporating additional disturbance information, using sliding-window statistics to summarize historical dynamics, and introducing a recovery-oriented reward with potential-based shaping. Each component is intuitive, but the paper does not sufficiently clarify what fundamentally new methodological insight emerges from their combination.
-
The experimental design makes it difficult to identify which components are actually responsible for the reported improvements. Although the paper includes ablations of disturbance information and temporal features, it does not independently evaluate the effectiveness of the proposed recovery-oriented reward. In particular, a natural baseline would be MAPPO with the original observations but the proposed reward function. Another important comparison would retain the proposed observation design while using the original environment reward. Without these comparisons, it is unclear whether the performance improvements primarily come from the additional observations, the reward redesign, or their combination. This is particularly problematic because reward design is presented as one of the central contributions of the paper.
-
The experimental evaluation is not sufficiently comprehensive given the relatively incremental nature of the proposed methodology. The comparisons are primarily based on MAPPO variants, and the paper lacks stronger alternatives involving recurrent policies, different temporal representations, or alternative reward and cost formulations.
Questions for authors
See in the Weaknesses