- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This work proposes a multi-policy-assisted backdoor attack framework for reinforcement learning. It leverages multiple independently trained clean peer policies to select poisoning states via victim-side salience and peer consensus, identifying states where the target action is compatible with benign behavior. It then applies triggers and performs reward-preserving target induction with peer regularization. Experiments show strong performance across multiple environments.
Critical strengths
- This paper identifies a new surface for backdoor attacks by using independently trained clean peer policies to guide the selection of backdoor injection locations. This perspective differs from existing approaches.
- This paper is clearly structured, with a relatively complete theoretical analysis and extensive experiments across multiple heterogeneous reinforcement learning environments.
Critical weaknesses
- PRISM selects states in which the target action is compatible with the peer policies’ normal behavior. If the target action itself is already a normal action in such states, the reported ASR may systematically overestimate the actual effectiveness of the backdoor. The paper does not report the difference in the probability of selecting the target action between triggered states and clean states, so this concern has not yet been ruled out.
- This method requires the additional training of multiple peer policies, which introduces extra computational and training costs compared with single-policy backdoor attacks. Also, the impact of the number of peer policies on attack effectiveness is not clearly investigated. The proposed method therefore introduces an additional hyperparameter, which may further increase the cost of achieving desirable attack performance.
Questions for authors
- To address the concern that the reported ASR may be inflated, could you report the probability of selecting the target action under clean states and triggered states, respectively, so as to demonstrate the actual attack gain introduced by the trigger?
- Could you report the practical cost required to achieve the current results, including the cost of training peer policies and hyperparameter tuning, and compare it with the baseline methods? This would facilitate an accurate assessment of the value of this method.