- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper shows the effectiveness of KW-DRO in saving the computational complexity of tolerating agent failures in MARL.
Critical strengths
The introduced sub modularity idea sounds somewhat interesting, while the results shows its effectiveness over nominal solutions.
Critical weaknesses
-
The authors mainly test the advantage of KW-DRO over nominal, full backup and show the computational savings and performance improvement. But I am afraid that some simpler alternatives, including DrIGM, shall be incorporated. More alternatives can be adding some randomness to the training process.
-
The description of the paper is somewhat misleading, as the Wasserstein distance can be simply regarded as a budget. So, the authors shall clarify the implications of adopting Wasserstein distance here.
Minor: The paper is not so easy to follow, with inconsistent paragraph statements. More polishing is required.
Questions for authors
-
The authors mainly test the advantage of KW-DRO over nominal, full backup and show the computational savings and performance improvement. But I am afraid that some simpler alternatives, including DrIGM, shall be incorporated. More alternatives can be adding some randomness to the training process.
-
The description of the paper is somewhat misleading, as the Wasserstein distance can be simply regarded as a budge. So, the authors shall clarify the implications of adopting Wasserstein distance here.
Minor: The paper is not so easy to follow, with inconsistent paragraph statements. More polishing is required.
4 comments
Thanks for your input. In my opinion, if the Wasserstein distance can be simplified as a budget, the authors are suggested compare it with this simple idea and show its advantages. Also the readability issue remains.
Thank you for following up with us. Your comment highlights an important misunderstanding that our shorthand word "budget" in the previous message. We clarify why Wasserstein DRO differs from a simple discrete budget, where sits in the complexity spectrum, and how the two compare directly in our experiments.
1. Wasserstein distance (over distributions) vs. a simple budget (like Hamming distance on vectors). A discrete fault budget and a Wasserstein ball act on two different mathematical spaces:
- Simple discrete budget (Hamming ball on ): This is a deterministic worst-case constraint on individual fault vectors, . Because is an integer, even the smallest non-zero budget () forces a worst-case robot stall on 100% of Bellman backups, and cannot represent rare or fractional hardware fault rates.
- Wasserstein DRO ( on ): Here, Hamming distance is only the ground metric between realizations, while measures transport cost between probability distributions. KW-DRO optimizes continuous adversarial weights across fault patterns via a 2D lower convex hull (Theorem 7). Around (), optimizes a distribution mixing fault-free and faulted outcomes to match realistic fault rates (e.g., or ) while caps concurrent stalls; for any empirical nominal , requires full optimal-transport coupling and does not reduce to any scalar budget (counterexample in App. A.1).
- Why is already the minimal DRO setup: Within distributional transport, first-order is the simplest linear formulation, keeping the inner backup convex. Richer relational transport metrics — such as Gromov–Wasserstein distance across agent interaction structures — turn the coupling problem into a non-convex quadratic program, which would be prohibitive to solve at every Bellman backup. is thus the simplest distributional model that avoids deterministic over-conservatism while preserving an exact, fast inner solver.
- A nice, but also a bit long to read and study about DRO:
- Kuhn D, Shafiee S, Wiesemann W. Distributionally robust optimization. Acta Numerica. 2025;34:579-804. doi:10.1017/S0962492924000084
- A nice, but also a bit long to read and study about DRO:
2. Direct empirical comparison against "simple budget" baselines. Because the candidate curve is decreasing and convex for VDN, the simple deterministic Hamming-budget backup (stalling up to worst-case robots without distributional weighting) is algebraically identical to setting integer () in Table 1 and Figure 4:
| Method | Formulation | VDN Survival (Table 1 / Fig. 4) | Fault Deployment Throughput (Sec. 5.2) |
|---|---|---|---|
| KW-DRO () | Distributional over | 20/20 (Table 1) / 15/15 (Fig. 4) | 9.2 deliveries / 1k steps |
| Simple Hamming budget () | Deterministic () | 9/20 (11/20 fail to learn) | 3.9 deliveries / 1k steps |
| Simple Hamming budget () | Deterministic () | 0/15 (Fig. 4; none learn) | — |
Without distributional weighting, even a discrete budget assumes a robot fails at every step, making targets overly pessimistic and preventing learning on most seeds (9/20 at , 0/15 at ), whereas distributional KW-DRO keeps all 20/20 runs alive (14/20 nominal, ). The same holds on QMIX under persistent outages (App. I): lets us pair a small per-step onset probability () with concurrency cap , doubling throughput over nominal QMIX (38.0 vs. 19.7 under targeted outages, ) and beating a fixed rollout fault-rate budget (fault-randomized training, 21.0, ). Even on the simplex itself, replacing our exact 2D lower-hull solver with a simple greedy budget heuristic (ratio-greedy, sorting candidates by damage-per-cost) is suboptimal under unequal actuator costs on our 2,000-LP benchmark (Sec. 5.1, App. C.10).
3. Readability. We appreciate your points to feedback on readability. During the rebuttal window, we prioritized spelling out every assumption, derivation, and control experiment in full detail so all reviewers could verify the technical claims rigorously, which made the revision denser rather than shorter. For the final version, we will streamline the main text — moving secondary technical caveats and diagnostic details to the appendix and tightening Sections 3–5 so the core intuition and method read cleanly from start to finish.
Thanks for the comment and the efforts. I will maintain my score, due to the remaining concern on readability.
Response to Reviewer oMRc
Thank you for your review and for noting the appeal of our submodular formulation and its gains over nominal training.
We have uploaded a revised PDF for rebuttal, and highlighted the edits in blue.
Q1. Simpler alternatives: DrIGM, Double Q, and adding randomness to training. We agree that KW-DRO should be tested against simpler robust and randomized baselines. The revision evaluates all three across VDN (Sec. 5.2, Table 1; App. C.6) and a new 16-seed-per-arm QMIX study (
162runs at 107 steps; Sec. 5.3, App. I):DrIGM (Qu et al., 2026) and Double Q on VDN (Table 1, 10 seeds/layout): Under DrIGM's vanishing-minimal-value condition, its ρ-contamination target (Eq. 8 of its arXiv version) is r+γ(1−ρ)E[maxa′Qtot(s′,a′)], which for VDN equals our scalar-shrinkage backup r+(1−c)γVtot(s′) at ρ=c. Table 1 marks these rows (
DrIGM-contamination, ρ∈{0.0036,0.0072}). At ρ=0.5, KW-DRO keeps20/20runs alive vs.14/20nominal (p=0.020) and18/20DrIGM-contamination (p=0.50), with Double Q also at20/20. At ρ=1.0, KW-DRO's state-dependent penalty stalls early exploration (9/20vs.20/20for uniform shrinkage): moderate pessimism stabilizes VDN, while state-dependent targets need a calibrated radius (Fig. 4).Adding randomness during training (Fault-Randomized Training) — VDN and new 16-seed QMIX study: In App. C.6 (Table 3), VDN trained with random stalls and the nominal backup kept
3/5runs alive onsmall-8ag(7/10pooled vs.10/10for KW-DRO, p=0.211). To test this with higher power on nonlinear mixing, we ran a 16-seed-per-arm QMIX study at 107 steps (Sec. 5.3, App. I, Tables 13–15) comparing nominal QMIX, transient KW-DRO (ρ=0.5), persistent KW-DRO (D=10, dose-matched ρ=0.05), and fault-randomized training (random multi-step stalls during rollouts with nominal QMIX):tiny-4ag(K=2), fault-randomized training barely improves over nominal QMIX (24.0vs.22.4deliveries/1k steps fault-free,4/16vs.3/16competent seeds;21.0vs.19.7under targeted 20-step K-robot outages). Both transient KW-DRO (45.3clean,35.7under outages,11/16competent) and persistent KW-DRO (48.9clean,38.0under outages,11/16competent) double throughput, and persistent KW-DRO beats fault-randomized training by +17.0 deliveries/1k steps under outages (p=0.005).small-8ag(K=3), persistent KW-DRO also leads nominal QMIX and fault-randomized training in mean throughput under outages (34.9vs.24.6and28.0), though not resolved at 16 seeds.Q2. Viewing the Wasserstein distance as a budget, and its implications. We agree: around P^=δ0 under the Hamming metric, Lemma 5 gives W1(P,δ0)=EP[∥ξ∥1], so ρ budgets expected stalled actuators per step. We revised Sec. 1 and added a paragraph after Definition 1 stating this up front. The support-constrained W1 set has four implications:
Minor: Presentation. We polished Sec. 1, Sec. 3.1, and Sec. 5 for consistent terminology and added a Notation Table in Appendix G (Table 12). Again, new edits are highlighted blue in revised PDF.