acceptodds
Under review as a conference paper at ICLR 2027

Who Gets Blamed? Attribution-Evasive Adversarial Attacks in Cooperative Multi-Agent Reinforcement Learning

Abstract

Existing adversarial attacks in cooperative multi-agent reinforcement learning (c-MARL) mainly focus on degrading team performance, treating a lower return or win rate as the primary measure of attack success. However, after a team failure, post-hoc attribution analysis may assign responsibility among agents by estimating their relative contributions to the observed failure trajectory. An attack that successfully disrupts the team but makes the compromised agent highly attributable therefore leaves a strong diagnostic footprint. This motivates a distinct adversarial objective: a compromised agent should not only induce team failure, but also avoid becoming the most attributable cause of that failure. We study attribution-evasive adversarial attacks in cooperative multi-agent reinforcement learning and propose AE-APL (Attribution-Evasive Adversarial Policy Learning). The proposed framework augments a standard adversarial policy objective with an exposure-aware penalty, using a counterfactual Joint-Q attribution critic to estimate a local value-based surrogate of each agent’s harmful contribution. The resulting attribution scores are converted into a relative blame exposure signal, which penalizes the compromised agent when it becomes disproportionately responsible under attribution while preserving a controllable attack-stealth trade-off. Experiments on MaMuJoCo and SMAC show that AE-APL improves attribution stealth over strong attack baselines while maintaining competitive attack effectiveness.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
MARL;Adversarial attack

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_m4UnOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a new adversarial objective for cooperative MARL: an attribution-evasive attack. A compromised agent should not only cause team failure but also avoid becoming the most attributable agent in post-hoc attribution analysis. The authors observe that existing attacks optimize only disruption, whereas auditors localize the responsible agent by counterfactual attribution, so a 'high-disruption + high-attribution' attack leaves a diagnostic footprint. Therefore, this paper proposes the AE-APL method, which augments the adversarial PPO objective with an exposure-aware penalty. A centralized Joint-Q critic constructs feasible counterfactual references for each agent, computes the local harmful contribution, normalizes it into attribution shares, and builds share-excess and pairwise soft-rank exposure terms. An attack-capability gate ensures that the penalty is applied only when the attack is strong enough. Experiments in MaMuJoCo tasks and three SMAC scenarios show that AE-APL improves attribution stealth over strong attack baselines while maintaining competitive attack effectiveness.

Critical strengths

(1) The new problem formulation captures the key comparison constraint. Treating attribution exposure as an adversarial objective and insisting on comparison at matched attack strength, avoiding the trivial solution of 'attack less to be less exposed'. This makes the paper more than another attack and it exposes a systematic blind spot in attribution/forensics pipelines and is directly useful for designing more robust auditors. (2) The independence and multi-view nature of the evaluation in this paper is seldom seen in similar works. Seven post-training auditors span counterfactual-value, Shapley/coalition, gradient families, auditor training data which is disjoint from attack training/evaluation, the benign policy which is unavailable during attack training, and simulator replay provides environment-level counterfactual validation. This substantially reduces the possibility of overfitting the training critic.

Critical weaknesses

(1) There is a conceptual gap between 'reduced attribution exposure' and 'reduced true responsibility', and the metrics can be strategically exploited. Normalized rank/Top-1 only measures the compromised agent's position in an ordering. An attacker can 'spread the blame' or exploit the locality of one-step counterfactual attribution to stay inconspicuous under current auditors while remaining the true cause of coordinated failure. Appendix Table 12 shows that attacker raw harm drops from 3.144 to 1.063, but the benign raw-harm sum also drops from 20.711 to 18.608 and the maximum benign raw harm from 9.744 to 8.978, suggesting that the failure is not explained by 'some benign agent becoming more harmful', but that the attacker may have learned to distribute harm through coordination dynamics that existing attribution functions fail to capture. The authors should state explicitly that this is 'evading current attribution methods' rather than 'being less responsible'. (2) The comparison with the strongest baselines is not Pareto-based, and an external control optimized for the same objective is missing. AMI/GMA/ICLR/ICML/USENIX do not optimize attribution, and their hyperparameters are not tuned for this trade-off and AE-APL winning on its own objective is expected. What is missing is an external baseline or simple control that also aims to reduce exposure, for example, regularization toward the benign policy, a sparse attack that intervenes only in critical states, or a matched-exposure baseline that directly weakens the attack. In addition, the capability floor is calibrated from the ICLR baseline (i.e., J_floor), so matched strength depends on the strength of one particular baseline. A hard-threshold gate may also cause 'attack weakens -> gate closes -> attack strengthens' oscillations during training, yet the paper only reports an offline sweep over ρ and does not analyze training stability or sensitivity of the final policy to J_floor. Without these controls, it is difficult to determine whether AE-APL's Pareto frontier is better than simply attacking less or behaving more like the benign policy.

Questions for authors

Q1. Does the paper claim that reducing attribution exposure is equal to reducing true responsibility? If not, can the conclusion be stated explicitly as 'evading a family of post-hoc attribution methods', with a discussion of how an auditor aware of this attack (e.g., multi-step counterfactual, causal responsibility, or an auditor designed to detect blame-spreading) would fare? Q2. The continuous-action counterfactual uses batch-mean interpolation, which is not semantically a feasible benign action. Could this systematically over- or under-estimate c_i? Can you use the benign policy (available only at training time) as an oracle reference to bound the surrogate error? Q3. J_floor depends on the ICLR baseline. If a stronger or weaker reference attack were used, would the conclusions be stable? Does the hard-threshold gate cause training oscillations? Q4. In Table 12, both attacker and benign raw harm decrease while the team still fails. Please provide a mechanistic explanation and validate that 'true responsibility' indeed decreases using a non-Q-based causal/environment-level attribution.

Reviewer_gH6tOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes AE-APL, an attribution-evasive adversarial policy learning method for cooperative multi-agent reinforcement learning that aims to degrade team performance while reducing the likelihood of the compromised agent being identified through post-hoc attribution analysis.

Critical strengths

The motivation for considering attack stealthiness is well justified and interesting.

Critical weaknesses
  1. The experimental results require fairer comparisons.

  2. The writing and presentation of the paper need further improvement.

  3. The motivation for attribution-based evasion is insufficient. The authors should clarify which existing defense mechanisms rely on attribution analysis and demonstrate that the proposed attack can effectively evade such detection methods.

Please refer to the questions section for details.

Questions for authors
  1. The authors should clarify how the trade-off between attack stealthiness and attack effectiveness is balanced. Are there experimental metrics that can quantify this trade-off? The current results seem to rely primarily on tuning the hyperparameter β\beta.

  2. For the experimental results, it would be preferable to use the names of the proposed methods as baseline labels rather than the names of the conferences where the papers were published.

  3. The authors should include the following relevant work as an additional baseline: Robust Multi-Agent Coordination via Evolutionary Generation of Auxiliary Adversarial Attackers.

  4. The paper's readability needs improvement. For example, in Section 4, Tables 1–3 and Figure 1 are located far from their corresponding citations in the text. The authors could also add an overview diagram of the proposed framework to help readers better understand the overall approach.

  5. Do policy-based attacks necessarily need to evade detection during post-hoc auditing? Since an attacker could target different agents in different episodes rather than consistently compromising a single agent, such a dynamic strategy might be more generalizable. The authors should clarify the motivation for requiring attribution evasion in the proposed setting.

Reviewer_aiaTOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes AE-APL, a deployment-time attack on MARL. The attacker controls one victim agent with a policy trained to degrade the team performance while minimizing its detectability through counterfactual attribution. To achieve this, the adversary's objective function incorporates a penalty term based on the victim agent's relative attribution exposure. This term is computed by estimating the counterfactual contributions of all agents and measuring the victim's contribution relative to those of the other agents. The authors conduct experiments in both discrete and continuous action spaces to show the effectiveness of the proposed attack and its ability to evade detection by existing attribution methods.

Critical strengths

The evaluations include both discrete and continuous action space scenarios, and the results show improved stealthiness (but only with respect to the evaluated attribution-based detection schemes).

Critical weaknesses

1- The paper claims there is no requirement for access to the benign policies for the adversary, and highlights it as one of the main properties of the proposed attack. However, the training procedure of the attack is not consistent with this claim. In particular, computing counterfactual contributions requires estimating the Q-function, which itself depends on the agents' benign policies. In the pseudocode of the method (Algorithm 1) the benign policies are explicitly mentioned as required inputs and are used to obtain the rollouts.

2- The contribution of the paper appears incremental. The idea of incorporating a penalty term into the adversary's objective function to reduce its identifiability is not particularly novel (e.g., see [1]). The main contribution of the paper is to define and compute this penalty based on counterfactual blame. However, the paper does not provide any insight or evidence to explain what advantage this specific choice brings compared to other possible notions of undetectability. Also, the paper's evaluations do not establish whether evading the considered attribution-based detection methods also implies undetectability under other detection criteria (like the behavioral abnormality used in [1])

[1] "Decentralized Anomaly Detection in Cooperative Multi-Agent Reinforcement Learning", IJCAI, 2023

3- Even among the attribution blame-based detection methods, the evaluated approaches are not sufficiently strong. The evaluated counterfactual auditors score responsibility using changes to actions at a single timestep. This does not establish how responsibility would be assigned when considering an agent’s behavior across multiple decisions. Two of the cited works (Triantafyllou et al, 2021 and 2022) examine changes to entire policies or combinations of actions across time. Neither approach is evaluated, and the paper’s single-timestep Shapley and replay tests do not address this issue.

Questions for authors

How the Q-function and counterfactual scores are obtained if the benign policies are unknown to the adversary?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.