- Rating
- 3
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies decentralized MAMDP with Byzantine agents and rewards learned from private preference comparisons. However, I am not very familiar with this topic.
Critical strengths
This paper studies a complex task, combining preference learning, multi-agent games, and Byzantine attacks, and seems to address it well.
Critical weaknesses
-
For readers not familiar with the topic, it is helpful to provide a detailed MAMDP formulation to make the problem definition clearer.
-
The experimental setup is wholly set by the authors. It is not clear why these scenarios are useful and meaningful in practice.
Questions for authors
What is the practical meaning (e.g., the application scenario) of this paper?
2 comments
Thank you for your reply. Since I'm not familiar with the topic of this paper, I maintain my score and confidence.
Response to Reviewer oAhn
Thank you for the review.
W1. Formalization
The revised Sec. 2.2 and Figure 1 caption state the information flow among the four models (see our reply to Reviewer AVaL). The setting in brief:
W2. Why these scenarios are meaningful
Both tasks abstract control problems in which operators' preferences are hard to write as rewards and communication may be tampered with. Networked LQ models coupled subsystems with an open-loop unstable plant, as in grid frequency control or traffic networks, and lets us detect instability of a learned controller exactly. VMAS flocking is a public benchmark (Bettini et al., 2022) for formation control, where each vehicle's operator judges the formation from that vehicle's own position. Plants are generated at random with seeds fixed in advance, all runs of each design are reported, and each attack targets one defense: clipping, retention, or distance-based removal.
Q. Practical applications
Three settings combine stakeholders with different priorities, communication that may be tampered with, and an existing conservative controller that is a candidate safe policy: