acceptodds
Under review as a conference paper at ICLR 2027

MatBak: Targeted-Behavior Adversarial Attacks on Multiple Reinforcement Agents via Preference-Based Intention Policies

Abstract

Adversarial attacks provide a crucial lens for evaluating the robustness of multi-agent reinforcement learning (MARL) systems, revealing worst-case failures and security risks in multi-agent systems (MAS) driven by deep reinforcement learning methods. However, existing attacks on MARL systems primarily aim to degrade cumulative returns, capturing generic performance failures while overlooking attacker-specified collective behaviors that are critical in safety-sensitive MAS. In this work, we study targeted-behavior adversarial attacks against MARL systems, where an attacker seeks to induce preferred team-level behaviors via bounded observation perturbations. Unlike standard reward-degradation attacks, targeted-behavior attacks require translating high-level attacker intentions into coordinated behavioral changes among interacting agents. We propose Multi-Agent Targeted-Behavior Adversarial Attack (MatBak), a preference-based framework for eliciting attacker-preferred collective behaviors in MARL systems. MatBak first learns an intention policy from attacker preferences. It employs a victim-observation encoder to evaluate attacker utility, then optimizes an observation-perturbation adversary via a dual-weighted bi-level mechanism measuring both agent-level engagement and state-level occupancy, enabling adaptive victim selection, targeted credit assignment, and fine-grained behavior manipulation. Across SMAC and MetaDrive tasks with MAT, MAPPO, and QMIX agents, MatBak achieves the highest attacker return in 26 of 27 settings; when used for adversarial training, it reduces average attacker return by 22.1% and improves average environment return by 11.4% over the strongest non-MatBak baselines.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Adversarial Attacks, Multi-Agent Systems, Targeted Behavior, Preference-Based Reinforcement Learning, Multi-Agent Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_J3tPOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies targeted-behavior adversarial attacks on cooperative MARL. Instead of just lowering team reward, the attacker tries to induce a specific collective behavior, such as dispersed formations in SMAC or traffic standstill in MetaDrive, by perturbing the observations of a subset of agents. MatBak extends RAT to the multi-agent setting by proposing 1. an attention-based victim-observation encoder selects the top-k agents to perturb, and 2. an adversary is trained to match the intention policy through a dually weighted KL loss. The authors report the highest attacker return in 26 of 27 settings and improved robustness when MatBak is used for adversarial training.

Critical strengths
  • Problem framing. Targeted-behavior attacks on multi-agent systems are well motivated and safety-relevant.
  • Agent selection. Adaptive victim selection is a natural multi-agent contribution and could be valuable on its own if properly validated.
  • Eval. Experiments cover two benchmarks and three victim algorithms (MAT, MAPPO, QMIX), and report both attacker return and attack success rate.
  • Paper is well written and relatively easy to follow
Critical weaknesses
  • Further validation of the agent-selection mechanism would strengthen the core contribution. Attention-based victim selection is arguably the paper’s most distinctive multi-agent contribution, so it would help to show clearly that it outperforms simpler alternatives. Natural comparisons include:

    • attacking a fixed set of k agents throughout the episode;
    • a RAT variant with consistent rather than random targets;
    • gradient-saliency selection, which could be computed cheaply on the attacker’s own encoder. The observation that last-k outperforms random-k is interesting, and it suggests target consistency may account for part of the gain. Disentangling these effects would make the case for attention-based selection much more convincing.
  • The role of each component could be clarified. The framework combines several components, and the current ablations remove the state and individual weightings together, which makes it hard to see what each contributes. Ablating them separately, and comparing the bi-level outer loop against a simpler heuristic weighting such as advantage-based weights, would help justify the added complexity.

  • The relationship between attacker return and attack success could be discussed further. In the MAT-SCSC setting, MatBak achieves the best attacker return (Table 7) but a comparatively low success rate (Table 8, and Table 9 under noise). Since success rates are used to argue against reward hacking, an explanation of this case would be valuable. One possibility is that frequent changes in the selected agents prevent vehicles from being brought to a full stop. Per-episode analysis could shed light on this.

  • The statistical analysis could be strengthened. Some settings show high variance relative to the mean (e.g., QMIX-8m at 585.99 ± 229.58) with five seeds, and several differences from RAT fall within the reported standard deviations. Confidence intervals, significance tests, or per-seed results would help establish the reliability of the improvements. A discussion of why MatBak’s variance tends to exceed RAT’s would also help.

Questions for authors
  • How does top-k attention selection compare to a fixed agent set, to RAT with consistent targets, and to gradient-based selection?
  • How often does the selected set change within an episode? Does this churn correlate with success rate, especially on SCSC?
  • What explains the return/ASR mismatch on MAT-SCSC? Can you report per-episode stopped-vehicle counts?
  • What does each weighting contribute separately? How does the bi-level loop compare to a heuristic weighting, such as advantage-based weights?
  • What are the training compute and victim query costs? Does the outer and inner loop cause high training computation? is it realistic in real world setting?
4 comments

We appreciate your time and effort in reviewing our work.

W1 & Q1: Agent-selection

As suggested, we add following comparisons:

① MatBak-fixed-k / RAT-fixed-k: attacking k agents which are randomly initialized and then fixed throughout the episode;

② MatBak-grad-k: attacking k agents selected by the top-k gradient norms of the encoder-fitted value function (Eq 4) w.r.t. local observations.

MapMatBakMatBak-fixed-kMatBak-grad-kRAT-fixed-k
MAT-MMM2684.06±128.33456.89±116.47500.95±71.79570.55±151.87
QMIX-3s5z624.93±318.92459.79±78.19414.69±37.55352.83±67.83

MatBak outperforms all baselines; MatBak-fixed-k underperforms MatBak, so the effect of target consistency is isolated.

W2 & Q4: Component effect

(1) As suggested, we ablate the two weighting components separately. Please see fAkQ (W2).

(2) We also replace the bi-level outer loop with heuristics: state weights set by the normalized joint advantage (adv-weight) and agent weights by the normalized local value (val-weight).

MapMatBakw/ adv-weightw/ val-weight
MAT-3s5z688.28±87.61670.03±46.09676.07±35.51
MAT-MMM2684.06±128.33436.67±23.65476.41±87.96
QMIX-3s5z624.93±318.92391.36±54.42459.84±79.02
QMIX-MMM2665.52±85.57271.01±30.62279.94±25.00

MatBak outperforms all variants, confirming each component’s isolated effect.

W3 & Q2 & Q3: Metric mismatch

(1) As the reviewer hypothesizes, ASR is a team-level threshold metric counting vehicles that reach a full stop, while attacker return is a dense signal crediting any speed reduction. Episodes that slow many vehicles without stopping them therefore earn high return but modest ASR, which explains the divergence in a few scenarios (e.g., SCSC).

(2) As suggested, we run a per-episode analysis on MAT-SCSC, reporting average vehicle counts over 60 test episodes:

Algoavg stoppedavg slowed-downavg total affectedavg speed reduction rate (affected)
PAAD4.8001.1335.93370.01%
MatBak-fixed-k2.4171.0333.45072.24%
MatBak1.4673.3334.80085.04%

① While PA-AD stops more vehicles, MatBak slows more vehicles and achieves a higher speed-reduction rate, which explains the metric mismatch.

② MatBak-fixed-k stops more vehicles but produces fewer slowed-down ones and a smaller speed reduction, so changes in the victim set may indeed prevent vehicles from reaching a full stop, as the reviewer hypothesized.

③ MatBak-fixed-k still stops fewer vehicles than PA-AD, so the mismatch mainly stems from the differing measures of attack effectiveness.

W4: Statistical reliability

(1) As suggested, we strengthen the statistical analysis by constructing one-sided 95% bootstrap percentile confidence intervals for the mean-attacker-return difference (Δ) between MatBak and RAT, and report the lower bounds (L95) on SMAC. Results:

ScenarioMatBakRATΔL95
MAPPO–3s5z630.85441.64189.20135.56
MAPPO–8m553.38359.52193.8696.64
MAPPO–MMM565.38438.40126.9842.79
MAPPO–MMM2530.75463.4767.282.02
MAT–3s5z742.93718.3024.62-34.26
MAT–8m379.60284.6894.9238.46
MAT–MMM517.20488.3528.85-27.12
MAT–MMM2570.43465.10105.3440.90
QMIX–3s5z731.40541.72189.6924.09
QMIX–8m585.99114.07471.92313.86
QMIX–MMM623.80489.10134.70-61.54
QMIX–MMM2518.14343.96174.1817.40

Δ is statistically positive (L95 >0) in 75% of scenarios, supporting the reliability of our advantage.

(2) Why the variance of MatBak tends to exceed RAT has been discussed in the original paper (lines 376-379).

Q5: Training cost

(1) As suggested, we report the average policy update time (ms) and training RAM usage (GiB) on two SMAC maps, comparing MatBak with competitve baselines:

MethodMAT-3s5z UpdateMAT-3s5z RAMMAT-MMM2 UpdateMAT-MMM2 RAM
SARL123.40 ± 3.731.52132.53 ± 9.241.52
AMI574.89 ± 27.501.67568.13 ± 17.641.72
RAT24.26 ± 1.022.8435.59 ± 3.293.82
MatBak33.88 ± 1.03 (1.40×)2.77 (0.98×)35.72 ± 1.03 (1.00×)3.55 (0.93×)

Attacker return performance:

MethodMAT-3s5zMAT-MMM2
MatBak639.23426.28
RAT571.90414.86
SA-RL219.96151.75
AMI203.35145.87

We note that the cost metric alone is not comparable: random noise injection is cheapest yet rarely effective. While SARL and AMI cost less RAM, their return is evidently lower (<40%) and they incur $>3.53.5\times$ update latency. The training cost of MatBak grows moderately over RAT (<1.50×). We report victim query cost at matched performance to baselines:

AlgoMAT-3s5z UpdateMAT-3s5z RAMMAT-MMM2 UpdateMAT-MMM2 RAM
RAT31.78 ± 1.322.8448.23 ± 4.993.82
MatBak40.20 ± 1.35 (1.26×)2.77 (0.98×)47.02 ± 1.74 (0.97×)3.56 (0.93×)

Victim query modestly increases update time (<1.32×) while RAM usage remains comparable.

(2) Real-world feasibility also see 4taG (Q1).

Dear Reviewer J3tP,

Thank you for your thoughtful assessment and for acknowledging our strengths in problem framing, method design, experimental evaluaiton, and paper writing.

We have made every effort to address each of your concerns point by point. We believe these additions substantially strengthen the paper. Should these additions resolve the issues raised in your initial assessment, we would be very grateful if you could consider updating your score. If any concern remains, please let us know; we would welcome the opportunity to discuss it further and provide any clarification you need.

Dear Reviewer J3tP,

We hope this message finds you well. As the discussion period is nearing its end, we would be grateful if you could let us know whether our responses have addressed your concerns. If any concern remains, please let us know, and we would welcome the opportunity to discuss it further and provide any clarification you may need. Your comments have been very helpful to us, and we are eager to resolve any remaining issues to improve our work.

Thank you again for the time and effort you have devoted to reviewing our paper.

Best regards,

The Authors

Reviewer_J3tPOpenReview ↗

Reply

Thank you for the reply. I am maintaining my current score. There are still concerns with the additional results. For instance, The fixed target attack assume random subset, however, the proper way is to select the optimal subset and keep it fix through out the episode.

Reviewer_4taGOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a test-time observation attack method to induce targeted behavior in multi-agent RL. MATBAK learns a reward function from the attacker's preferences, which is then used to train an intention policy. Then, an attention mechanism is employed to encode agents' observations and select which agents to attack. Finally, a bilevel optimization framework is used to learn the attacker's perturbation policy. Several experiments have been done to illustrate the effectiveness of the attack, as well as its applicability to enhance the robustness of the system when used in adversarial training.

Critical strengths

The proposed attack has the potential to improve the robustness of MARL systems when used to generate perturbations during adversarial training. However, this is a secondary contribution of the paper.

Critical weaknesses

1- The proposed attack relies on very strong assumptions about the attacker's capabilities. More specifically, in practical deployment scenarios, agents typically operate on separate devices, which is one of the main factors distinguishing multi-agent systems from single-agent. However, the paper assumes that the attacker not only has access to the observations of all agents but can also select and attack any agent at any time step. Such a setting might be feasible during training, but they are very strong and unrealistic assumptions when applied to the deployment time.

2- The novelty of the paper with respect to its main motivation is limited. More precisely, the main motivation of the paper is applying targeted attacks to MARL. However, those parts of the method that are related to producing a targeted behavior (preference learning, intention policy, and adversarial policy matching) are already present in RAT for the single-agent case. MATBAK’s main additions are the attention mechanism for victim selection and individual weighting. These are general mechanisms that could also be applied to untargeted attacks, for example, by using the negative team reward as the learning signal. More importantly, these additions are relevant only under the strong attacker capability assumptions discussed in Weakness 1.

Questions for authors

1- Could the authors elaborate on what deployment architectures or compromise mechanisms would allow an attacker to access the joint observations of all agents and adaptively perturb the observations of selected agents across separate devices at deployment time?

2- The paper states that the attacker does not require access to the victims' internal information. Does this also include the victims' model weights? If the attacker does not have access to these weights, could you clarify how the gradients with respect to the victims' policies in (7)-(10) are computed?

4 comments

We appreciate your time and effort in reviewing our work.

W1: Assumption plausibility

(1) Studying the observation vulnerability of RL agents under the assumption that the attacker can access and perturb their observation at each time step is widely adopted by literature in this line of research (see Sec. 2). The significance of this assumption lies in enabling researchers to analyze how observation perturbations affect RL agents and to develop corresponding defenses. Notably, the plausibility of our problem framing has been supported by fAkQ (Strength 1) and J3tP (Strength 1).

(2) This assumption does not compromise deployment feasibility: the attacker requires no access to the victims' internal params, gradients, or rewards, and manipulates only their observation signals, which are externally exposed and can be manipulated in practice. Examples supported by literature:

① laser spoofing that injects fake points into the LiDAR measurements of production-grade autonomous driving systems [1]

② adversarial perturbations to camera and radar inputs for vehicle perception [2]

③ spoofed observation messages injected into cooperative UAV swarms [3]

(3) The distributed regime has been validated by our experiments on decentralized-execution victims (lines 369-371).

W2: Novelty scope

(1) Our method provides a general framework for targeted attacks; reducing a targeted attack to an untargeted one via the negative team reward is a special case with a weaker objective. That our mechanisms also cover this case does not diminish their contribution to the targeted setting we study.

(2) The components inherited from RAT are insufficient in MARL settings, where the attacker must also decide which agents to perturb and how much each perturbation contributes to collective target behavior. These multi-agent-specific challenges are what this work addresses, recognized by fAkQ (Strength 2) and J3tP (Strength 2). We also contribute:

  • bi-level formulation with convergence guarantee

  • adverarial-training variants that enhance robustness

  • the first systematic study of targeted attacks across diverse types of MARL systems

(3) Our contributions are not task-agnostic. The victim-observation encoder is trained by the preference-driven reward model and fits intent-aligned value functions. The resulting per-agent value utilities drive intent-aligned victim selection, and the encoder-fitted value estimates are fed directly into the outer-loop for intent-aligned dual-weighting. Selection and weighting are thus co-designed with the targeted objective rather than generic heuristics.

Q1: Deployment feasibility

(1) Consider autonomous driving systems (validated by experiments on MetaDrive). In Vehicle-to-Everything (V2X) networks, each node receives the observations broadcast by all others, so compromising a single node already exposes the joint observations of all agents. Software backdoor attack modifies the observations before they are fed to the RL models in such systems.

(2) Equipping a batch of vehicles with injection-capable sensing front-ends also lets the attacker adaptively perturb the observations of selected agents hosted on separate devices at deployment. For example, a delivery or service-maintained fleet whose auxiliary perturbation module is installed during manufacturing or via after-sales access and controlled by the attacker.

(3) Vehicle-level sensor spoofing is already feasible (see W1). Extending it to several independently addressable devices is primarily an engineering effort in scale rather than a fundamentally new attack primitive.

Q2: Access to victim information

As stated in original paper, victim model weights are not required.

(1) We clarify a possible misunderstanding: in Eqs 7-10, the differentiation variable is not the victim policy, and the gradient is taken w.r.t. the adversary's parameters α (Eq 10) and computed on the attacker side. The one-step approximation simply propagates the dependence of the adversary update on the dual weighting mechanism.

(2) The victim policy enters only in Eq 7, where the objective uses the action distribution of the deployed, fixed victim model under the perturbed observation. This quantity can be estimated by black-box probing methods (see lines 145--149) like victim query (validated in J3tP, Q5), a well-established technique in black-box model extraction [4] [5].

[1] Cao Y, et al. Adversarial sensor attack on lidar-based perception in autonomous driving. ACM CCS, 2019.

[2] Zhou C, et al. Adversarial robustness of camera-radar sensor fusion for autonomous driving. arXiv:2610.16012, 2026.

[3] Lv Z, et al. Safe multi-agent reinforcement learning for wireless applications against adversarial communications. IEEE TIFS, 2024.

[4] Tramèr F, et al. Stealing machine learning models via prediction APIs. USENIX Security, 2016.

[5] Papernot N, et al. Practical black-box attacks against machine learning. ACM AsiaCCS, 2017.

Dear Reviewer 4taG,

Thank you for your time and for your critical reading of our paper. We believe that the two weaknesses you raised in your initial assessment mainly stem from an ambiguity in our presentation of the threat model and of Eqs 7-10, rather than from limitations of the method itself.

We have provided direct clarifications point-by-point in our rebuttal. Should these clarifications resolve the issues raised in your initial assessment, we would be very grateful if you could consider updating your score. If any concern remains, please let us know; we would welcome the opportunity to discuss it further and provide any clarification you need.

Dear Reviewer 4taG,

We hope this message finds you well. As the discussion period is nearing its end, we would be grateful if you could let us know whether our responses have addressed your concerns. If any concern remains, please let us know, and we would welcome the opportunity to discuss it further and provide any clarification you may need. Your comments have been very helpful to us, and we are eager to resolve any remaining issues to improve our work.

Thank you again for the time and effort you have devoted to reviewing our paper.

Best regards,

The Authors

I appreciate the effort the authors have put into addressing my concerns. However, my main concerns largely remains.

W1, Q1- On the threat model assumption, the paper assumes that the attacker can (1) read the joint observations of all agents, and (2) perturb the observation of "any selected target at any time-step". The second part itself means that the attacker should compromise all deployed agents (normally in distinct devices) together. This combination grant the adversary extensive capabilities that, in my opinion, are not sufficiently justified. The argument and evidence the authors provided in rebuttal (W1), supports that an attacker can compromise "one target" and and continuously spoofing its observations. I have no objection to this. However, assuming that an attacker can compromise all agents involved in a task and freely choose which one to attack at each time step is a much stronger assumption. In my view, this does not represent an inherent threat for multi-agent deployments.

The examples the authors provide in Q1 are also not very convincing to me. The V2X example might satisfy (1) I mentioned above, but not (2). The fleet-wide injection-module example also seems a synthetic example that already preserves the extensive compromise assumption. It does not show that ordinary external sensor spoofing provides equivalent capabilities. Even if we accept the plausibility of such examples, they restrict the applicability of the proposed attack to a narrow set of deployment scenarios. In more common MARL deployment scenarios, accessing synchronized observations of all agents together and compromising all at will seems overly generous to the attacker.

W2- On novelty, I am not claiming that MaBak does not have additional components compared to RAT, but those components are mostly relevant if we assume that the concept of dynamic agent selection at deployment time is a meaningful attack capability. As explained above, I remain unconvinced about that.

Q2- As the authors confirmed the victim policies enter (7), thus according to the chain rule, the gradient of victim policy with respect to the perturbed observations should be involved in the gradient calculation (keeping the victim’s weights fixed does not eliminate this). The authors' response does not clearly specify how they handled that. Querying action probabilities allows evaluation of the loss, but it does not automatically provide this derivative.

Reviewer_fAkQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper is about multi-agent safety: whether it can get a whole team of agents to behave the expected way just by changing what a few of them see. The environment adopted includes disrupting a few vehicles’ observations and end up blocking traffic across an intersection. The proposed method works in two steps. First, it learns what the target behavior should look like. A scripted teacher compares pairs of simulated trajectories and says which one it prefers. Those preferences are used to train a reward model, and then an intention policy that shows what the attacker is going for. Second, it uses attention to pick out the agents that matter most, and learns small tweaks to their observations that nudge their actions toward that intention policy. Along the way, it also learns how much weight to give different agents and situations. They test the approach in two simulators, with scripted teach preferences.

Critical strengths
  1. The paper focuses on an interesting problem. Inducing a specific team behavior, such as blocking traffic or breaking up a combat formation, is useful to study multi-agent vulnerabilities. The focus on how limited interventions spread through agent interactions makes the problem interesting.

  2. The method design is reasonable. The intention policy defines the desired behavior, attention helps choose whom to attack, and the weighting mechanisms focus training on important agents and situations.

  3. Performance drops substantially when the intention policy is removed and the attacker is trained directly with reward-model feedback. This supports the value of explicitly learning a target policy.

Critical weaknesses
  1. The paper motivates the approach as learning from human expert preferences, and mentioned in the method design in section 4.1 that "human experts first label preferences over the victim trajectory segments consisting of observation-action pairs", but then "employ a scripted teacher here for simplicity". In the experiments the preference labels actually come from scripts. The experiments also only cover just two predefined goals: breaking up combat formation and breaking up traffic. This leaves open how well the method handles noisy, inconsistent, or ambiguous human feedback. The results look promising in simulators, but they only show that the method works with scripted preferences in simulation. Whether it holds up with real human feedback or in the real world is still an open question.

  2. The ablations need more claritification. Figure 6 removes the state and agent weights together, so it is hard to tell how much each one helps. This matters because RAT already includes an intention policy and state-occupancy adjustment.

  3. The attention scores may need more validation to choose attack targets. An agent that matters for predicting team value may not be the best agent to perturb. The top-k strategy beats random-k and last-k, which is useful evidence, but it would be more convincing to check the rankings, for example, whether the attention ranking matches the attack gains from individual agents.

Questions for authors

Why do you choose to attack 40% of agents? How well does the method work with a smaller attack budget? Will MatBak still have a clear advantage when it can target only 10%, or when the perturbation magnitude is reduced? I am wondering how did you choose the attack hyperparameters and how sensitive are the results to these choices.

4 comments

We appreciate your time and effort in reviewing our work.

W1: Scripted preferences

(1) Human preference is a motivating instantiation of this work: the attacker can recognize the desired target behavior (through 0-1 labels) but cannot express it as an explicit reward. Our contribution focuses on how to attack and defend under such an implicitly specified intention, a situation that commonly arises in practice.

(2) Under this problem setup, and following common practice in both PbRL (PEBBLE; Liu et al., 2022) and targeted-behavior attacks (RAT; Bai et al., 2025), we use a scripted teacher that generates 0-1 preference labels in place of annotations by discerning human annotators. Notably, preference annotation by scripted teachers effectively supports the reproducibility of our experimental results, which is a key reason why this practice is widely adopted in the literature.

(3) We note that the scripted preference labels can be readily replaced by human preference labels without changing the algorithmic procedure. To verify this, we collect human preference labels on two SMAC maps, retrain the reward model, and then compare MatBak against all baselines under the PbRL setting. The experimental results are as follows:

methodMAT-3s5zMAT-MMM2
Adv-ZS205.34 ± 5.11136.55 ± 2.72
Adv-TP220.55 ± 5.28142.64 ± 15.07
Adv-xAI207.83 ± 7.31134.08 ± 2.80
AMI206.32 ± 7.09133.20 ± 2.95
SA-RL211.18 ± 1.77150.24 ± 10.10
PA-AD271.41 ± 12.10135.56 ± 10.18
RAT372.84 ± 19.09335.36 ± 100.69
MatBak439.51 ± 35.86359.65 ± 74.63

Results show that MatBak consistently outperforms all baseline methods, validating its effectiveness and justifying the use of scripted preferences as a proxy for human preferences in our setting. We also discuss why MatBak tend to exhibit relatively higher variance compared to baselines on SMAC in lines 376-379 of the original paper.

W2: Separate weighting ablation

As suggested, we ablate the state weighting and the individual weighting separately:

MapMatBakw/o state-weightw/o individual-weight
MAT-3s5z688.28±87.61683.11±70.03624.59±17.76
MAT-MMM2684.06±128.33432.86±32.07420.53±79.40
QMIX-3s5z624.93±318.92376.69±57.36432.83±93.21
QMIX-MMM2665.52±85.57287.36±34.34286.17±32.91

Results show that:

① the two weighting components yield complementary gains;

② each of them (especially the individual weighting, which RAT lacks) accounts for a substantial part of the improvement.

W3: Attention-rank validation

As suggested, we validate the correlation between attention ranking and leave-one-out gains on MetaDrive:

① with a fixed attacker and budget, we attack the top-kk agents (40%) and, for each agent ii, measure the drop Δi\Delta_i in attacker return when ii is excluded.

② since all 10 agents are attacked over 12 test episodes, we average attention and Δi\Delta_i per agent and rank both descending (1 = highest), giving matched ranks 1, ... ,10.

MapNum of agentsΔi\Delta_i rank by attention index 1-10Σd²Spearman ρ
MAT–SCSC104,1,2,5,9,7,6,3,10,8600.6364
MAPPO–intersection101,6,3,4,9,2,5,8,10,7620.6242
QMIX–roundabout101,4,8,3,2,6,10,5,9,7660.6000

The average Spearman ρ\rho is 0.620.62, which shows positive correlation. This result complements our ablation (top-kk vs. random-kk and last-kk) and confirms that high-attention agents are effective targets.

Q1: Attacker-budget sensitivity

(1) We choose 40% as a moderate, protocol-matched budget: it leaves the majority of victims (>50%) unperturbed, keeping the attack cost modest relative to a full-victim attack. All methods share this setting for a fair comparison.

(2) As suggested, we conduct experiments on two SMAC maps, lowering the victim-set fraction down to 10% and report attacker return:

MapAMIRATMatBak
MAT-MMM2138.59±1.05531.92±40.20555.24±97.94
QMIX-3s5z248.82±2.46321.88±45.66455.33±45.46

MatBak consistently outperforms competitive baselines AMI and RAT as the budget shrinks, confirming that its advantage is insensitive to the attacker budget.

Dear Reviewer fAkQ,

Thank you for your thoughtful assessment and for acknowledging our strengths in problem framing, method design, and experimental evaluaiton.

We have made every effort to address each of your concerns point by point. We believe these additions substantially strengthen the paper. Should our rebuttal resolve the issues raised in your initial assessment, we would be very grateful if you could consider updating your score. If any concern remains, please let us know; we would welcome the opportunity to discuss it further and provide any clarification you need.

Dear Reviewer fAkQ,

We hope this message finds you well. As the discussion period is nearing its end, we would be grateful if you could let us know whether our responses have addressed your concerns. If any concern remains, please let us know, and we would welcome the opportunity to discuss it further and provide any clarification you may need. Your comments have been very helpful to us, and we are eager to resolve any remaining issues to improve our work.

Thank you again for the time and effort you have devoted to reviewing our paper.

Best regards,

The Authors

Thank you for the comments and for conducting the additional experiments. I would like to maintain my original score.

Comments on the paper

1 comment

We are grateful to the reviewers for their rigorous review comments, which have been instrumental in strengthening our manuscript.

(1) We are pleased that the reviewers have reached a clear consensus on the merits of our work and summarize the key strengths below:

  • Problem framing. Two of the three reviewers agreed our problem framing "is useful to study multi-agent vulnerabilities" (fAkQ), "well motivated and safety-relevant" (J3tP). We note that this consensus rebuts the core weakness raised by 4taG (overly strong assumption).

  • Method design. Two of the three reviewers agreed our method design is "reasonable" (fAkQ) and is "a natural multi-agent contribution and could be valuable on its own" (J3tP). We note that this consensus rebuts the other weakness raised by 4taG (limited contribution w.r.t. motivation).

  • Experimental evaluaiton. The reviewer acknowledged that our experimental evaluation “supports the value of learning taregt” (fAkQ) and “is extensive to cover two benchmarks and three victim algorithms, reporting both attacker return and attack success rate” (J3tP).

  • Paper writing. Our paper was prased as "well-writen" and "easy to follow". ( J3tP)

(2) We also want to acknowledge the initial concerns, as addressing them directly has improved our paper:

  • Scripted preference (fAkQ). We have clarified that the use of scripted preference in this research is supported by literature and facilitates reproducibility in this line of research. We have also provided experimental results with human preference labels, which validate the effecitveness of our method and further justifies the use of scripted preferences.

  • Detailed ablation (fAkQ, J3tP). As suggested, we have ablated the state weighting and the individual weighting separately and compared them against a simpler heuristic weighting (advantage-based weights). Results have validated that each of them accounts for a substantial part of the improvement.

  • Agent selection (fAkQ, J3tP). As suggested, we have further validated our agent selection by examining the attention rankings and comparing our scheme against simpler alternatives, namely fixed-kk selection and gradient-saliency selection. Together with the random-kk and last-kk comparisons in the original paper, these results adequately confirm the effectiveness of our attention-based selection scheme.

  • Assumption plausibility & deployment feasibility (4taG, W1&Q1).

    ① We have clarified that our assumption is widely adopted by literature in this line of research (see Sec. 2). This assumption allows researchers to analyze how observation perturbations affect RL agents and to develop corresponding defenses.

    ② We have clarified that our assumption does not compromise deployment feasibility, since the attacker requires no access to the victims' internal params, gradients, or rewards, and manipulates only their observation signals, which are externally exposed and can be manipulated in practice.

    ③ The assumption plausibility & deployment feasibility of our work is supported by reviewers fAkQ and J3tP, and we have also provided substantial literature evidence in both the original paper and in our rebuttal.

  • Gradients with respect to the victims' policies in Eqs 7-10 (4taG, Q2). We have clarified a possible misunderstanding here in this question: in Eqs 7-10, the differentiation variable is not the victim policy, and the gradient is taken w.r.t. the adversary's parameters α (Eq 10) and computed on the attacker side. Therefore, "gradients with respect to the victims' policies" are not required or computed in Eqs 7-10.

To enhance quality, the final manuscript will incorporate all improrenens. We will integrate all new results and provide more precise clarifications on our threat model.

(3) Finally, we are excited by the reviewers' recognition of our work's potential. fAkQ highlights that our paper "focuses on an interesting problem", and that "the focus on how limited interventions spread through agent interactions makes the problem interesting". J3tP highlights that our research is "well motivated and safety-relevant" and the proposed mechanism "is a natural multi-agent contribution and could be valuable on its own". These postive feedbacks affirm the value of this work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.