- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies corrupted cost feedback in constrained multi-agent reinforcement learning. It analyzes how biased reports affect dual updates and proposes RCE, which combines trimmed aggregation with a conservative margin based on median absolute deviation. Experiments on six-agent ManySegmentAnt, using three seeds and several attack and control settings, show that persistent underreporting increases true cost. RCE reportedly reduces the attack-induced cost increase by approximately 80%. Additional experiments examine how corruption propagates through dual mixing and whether it escapes detection.
Critical strengths
The paper addresses a meaningful failure mode: corrupted constraint feedback can weaken safety enforcement without directly changing rewards or policy observations. This is a useful perspective on the robustness of constrained multi-agent learning.
The experiments provide an initial demonstration of the vulnerability and a substantial reduction in attack-induced cost under robust aggregation. The released code and detailed logs also make the reported behavior inspectable, providing a useful basis for further investigation.
Critical weaknesses
-
Theorem 5 overlooks corrupted reports surviving trimming and the selection bias introduced among retained honest reports, leaving its error bound unsupported. Proposition 7 and the empirical MAD calibration do not resolve this gap.
-
Eq. (2) omits each policy’s effects on other agents’ costs, while training and cost estimation handle termination differently. The mechanism gap also compares reports and evaluations from different policies, conflating reporting error with policy changes.
Questions for authors
-
Provide a corrected proof of Theorem 5 that accounts for corrupted reports surviving trimming and the selection bias among retained honest reports? Which guarantees for RCE and Proposition 7 remain valid under the corrected bound?
-
What precise constrained objective does the implementation optimize? How is Eq. (2) justified when an agent’s policy affects other agents’ costs, and how are the different termination treatments in GAE and cost estimation reconciled?
-
Do the mechanism-gap conclusions hold when reports and oracle evaluations use the same frozen policy?