acceptodds
Under review as a conference paper at ICLR 2027

MAStrike: Shapley-Guided Collusive Red-Teaming on Multi-Agent Systems

Abstract

Hierarchical multi-agent systems (MAS) are rapidly deployed in high-stakes workflows across various domains such as finance and software engineering. In these systems, safety and security are inherently distributed across role-specialized sub-agents, significantly expanding the attack surface, especially for risk involving coordination, such as privilege escalation and cross-agent collusion. Existing red-teaming approaches against MAS remain limited: they rely on heuristic selection of target agents and perturb isolated message streams, leaving critical questions unanswered as which agents are most responsible for system safety, and how compromised agents can coordinate to bypass defenses. To bridge these gaps, we propose MAStrike, a closed-loop framework for collusive red-teaming in hierarchical MAS. We propose the first agent-level Shapley value analysis for MAS, quantifying each agent's marginal contribution to system robustness under task-specific distributions. Building on this principled attribution, we design an autonomous red-teaming agent guided by Shapley values to identify vulnerable coalitions and generate coordinated, role-aware adversarial manipulations. These attacks are iteratively refined through structured failure diagnosis, identifying blocking conditions and refining the injections accordingly. In addition, we construct a comprehensive MAS red-teaming benchmark, MABench, spanning diverse high-stakes domains, including finance, software engineering, and CRM, and further show that MAStrike transfers to independently built MAS with different orchestration topologies. We construct controllable MAS environments to perform risk assessment in MAS with agent collusion optimization. Extensive experiments on hierarchical MAS with different frontier models demonstrate that MAStrike significantly outperforms existing heuristic baselines, achieving 61.8% ASR against Claude Opus 4.7 and 55.6% against GPT-5.5. Our analysis on real-world MAS further uncovers non-trivial Shapley distributions and complex agent interactions, revealing critical vulnerabilities overlooked by prior single-agent or template-based methods.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Multi-Agent, Red-Teaming

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_9EwJOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MAStrike, a Shapley-guided framework for collusive red-teaming in hierarchical multi-agent systems. It uses agent-level Shapley values and pairwise interaction indices to identify influential coalitions, followed by coordinated, role-aware adversarial manipulation and iterative refinement. The paper also introduces MABench, covering finance, software engineering, and CRM. Experiments across three frontier models demonstrate substantial improvements in attack success rates, supported by ablation studies and evaluation on TAMAS.

Critical strengths

Strong empirical evaluation. MAStrike achieves substantial ASR improvements across three domains and three frontier models. The authors also provide attribution validation, estimation reliability analysis, component ablations, and evaluation on an independently constructed benchmark.

Critical weaknesses
  1. Restricted threat model and evaluation scope. The reported attacks modify compromised agents' system prompts, assuming substantial attacker control. Moreover, MABench consists of constructed hierarchical MAS with distributed safety gating. Although these settings are appropriate for studying collusion, the practical applicability of the findings to less privileged attacks and broader MAS deployments remains uncertain.

  2. Incomplete assessment of Shapley estimation efficiency and transfer. While the paper evaluates estimation reliability and per-case attack costs, the total offline coalition-evaluation overhead is not fully reported. Furthermore, the TAMAS evaluation uses exact per-instance Shapley computation rather than the offline task-similarity-based transfer procedure used in MABench. This limits the evidence for the efficiency and generalizability of the complete selection pipeline under consistent evaluation conditions.

Questions for authors
  1. What is the total computational overhead of offline Shapley estimation, and how does it affect the amortized cost of red-teaming compared with the baselines?

  2. Can the authors evaluate TAMAS using the same offline Shapley estimation and task-transfer procedure adopted in MABench, rather than exact per-instance coalition enumeration?

1 comment

We thank the reviewer for the positive assessment and the two questions.

Q1. Total computational overhead of offline Shapley estimation, and its effect on the amortized cost of red-teaming vs baselines.

The selection cost is paid once, offline, on synthetic reference tasks, and reused for every benchmark task, every target model, and every kk. The baselines pay their generation cost on every target case with no reuse.

  • Offline. Each synthetic task probes up to 64 coalitions, and each probe is a single MAS run under the cheap attribution channel with no red-team loop, so the total is at most 64×64 \times # synthetic tasks ×nest\times n_{\text{est}} MAS calls, computed once.
  • Online. The measured red-team cost is 2.95 rewrite calls per case, half of AiTM's 5.89, plus at most 3 MAS runs.
  • Amortization. Because the offline cost is one-time and reused across all tasks and models, the amortized selection overhead approaches 0 as more targets are attacked. We will add this accounting in the revision.

Q2. Can TAMAS be evaluated with the same offline Shapley estimation and task-transfer procedure used in MABench, rather than exact per-instance enumeration?

Thank you for the suggestion. We will add the Shapley estimation for TAMAS in the revision as well.

Reviewer_MFTSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MAStrike, a red-teaming framework for hierarchical LLM multi-agent systems in which a coalition of compromised agents colludes to trigger a sensitive tool. Agent-level Shapley values and pairwise interaction indices are estimated from coalition attack success rates, obtained by overriding the outputs of compromised agents on synthetic workflows. These are transferred to new tasks by embedding similarity and used to select a synergy-aware coalition of size kk. A red-teaming agent then jointly rewrites the system prompts of the selected agents and refines them through failure diagnosis.

Critical strengths
  1. Holding the generator fixed and varying only the coalition (Shapley 62% vs. heuristic 40% vs. random 28%, exact McNemar tests), the minimal-generation ablation, and the transfer to the independently built TAMAS harness (72% vs. 13% with a heuristic coalition, 83% oracle) show where the gains come from. This is more careful than most red-teaming papers.

  2. MABench implements three hierarchical MAS with separation-of-duties gating, role-scoped MCP tools mirroring real services, deterministic resets, and both benign and malicious suites with execution-based judges. It is a useful testbed for distributed safety in agentic systems, independent of the attack method.

Critical weaknesses
  1. The attack replaces the system prompts of kk agents, which is close to full insider control of those agents, and the user request itself is malicious. The paper does not discuss how an attacker would obtain write access to system prompts, or how results change under weaker channels such as tool or environment injection only. Under this threat model, the findings mainly show that the MAS lacks defense in depth, which the benchmark is co-designed to expose.

  2. The baseline comparison is not like-for-like. TAMAS and GCA use a suffix channel while MAStrike overrides prompts, and the baselines were not designed for this benchmark or its coalitions. The strengthened baselines and component ablations are run only on finance with Claude Opus 4.7. The red-teaming backbone is also unclear: Table 2 says attacks are optimized on Claude Opus 4.7, while Appendix A states the red-team agent uses Gemini 3.1 Pro by default. Near-0% baseline ASR therefore partly reflects the setup rather than the method.

  3. The need for Shapley values is not well established. The coalition value comes from an output-override channel rather than the actual attack, and agreement with the real attack is checked only on finance (agent-level r=0.88r = 0.88, coalition-level r=0.71r = 0.71). The selection score is an acknowledged heuristic that double-counts pairwise dividends. Appendix D notes that every pair is already evaluated in the coalition tables, so for k=2k = 2 one could select the highest-value pair directly. The paper does not show what the Shapley machinery adds over that simpler rule.

  4. Each workflow has 20 tasks, and most ablations use 50 cases in one domain. Benign success is near 0% in some CRM categories (e.g., Deal & Opportunity for GPT-5.5 and Claude), and the claim that ASR is not confounded by incapability is checked only for Claude. The guardrail case study uses proprietary Salesforce guardrails and reports no quantitative results. No defense is proposed, and the paper has no reproducibility or ethics statement or commitment to release code and the benchmark.

Questions for authors
  1. How would an attacker obtain write access to agent system prompts in practice? What ASR does MAStrike reach using only tool-output or environment injection?

  2. For k=2k = 2, how does selecting the highest-value pair from the coalition table compare with Shapley-based selection? What is the cost and benefit of Shapley at larger kk?

  3. Can all baselines be run with the same prompt-override channel, red-team backbone and case information, across all three domains and backbones?

  4. How should ASR be interpreted in categories where benign success is near 0%, and what are the refusal rates of GPT-5.5 and Gemini there?

  5. Will MABench and the code be released? Can the guardrail case study report quantitative detection and false-positive rates?

1 comment

We appreciate the detailed review. We answer the questions in detail below.

W1 / Q1. The attack replaces kk agents' system prompts (near insider control) and the request is malicious. How would an attacker obtain prompt write access? What ASR under tool/environment injection only?

See C1. We separate the attribution channel (override, used only for scoring) from the attack channel (every reported ASR), and measure MAStrike across different channels.

MethodASR
MAStrike (system-prompt override)70.0%
MAStrike w/ tool-response injection86.7%
MAStrike w/ system-prompt suffix20.0%
AutoTransform (system-prompt override)1.0%
Other baselines0.0%

MAStrike exceeds every baseline on every channel. The system-prompt override models the realistic worst case of a compromised sub-agent, such as a sub-agent pulled from an agent registry, a poisoned agent card, or a malicious MCP server, all documented MAS supply-chain threats in which the adversary controls that agent's instructions. We will state the threat model and its deployment scenarios explicitly, and add a Limitations section in the revision.

W2 / Q3. Baselines not like-for-like; strengthened baselines only on finance with Opus-4.7; red-team backbone unclear (Table 2 says Opus-4.7, Appendix A says Gemini-3.1-Pro).

Like-for-like. We ran the full 2×22{\times}2 factorial that gives the baselines MAStrike's channel and MAStrike's coalition.

selection and channelTAMAS suffixTAMAS overrideGCA suffixGCA override
heuristic (identity+policy)0/200/200/200/20
Shapley (identity+device)0/200/200/200/20

All 8 cells are 0.0%, which shows the gap comes from the injection generation process, not from channel or coalition access.

Red-team backbone. Thank you for catching this inconsistency. The red-team generator is Gemini-3.1-Pro (Appendix A); the "Opus-4.7" in Table 2 is the target MAS being attacked, not the generator. We will rename the Table 2 caption to "target MAS".

Finance concentration. We will extend the override and Shapley factorial to engineering and CRM. The stability and interaction-order ablations already cover both domains.

W3 / Q2. Is Shapley needed? For k=2k{=}2 one could pick the highest-value pair directly from the coalition table; the selection score double-counts pairwise dividends.

"Pick the highest-value pair directly" is the oracle: it requires having already attacked every pair on the target task. On a held-out task we never see its coalition table. The role of ϕ\phi and SII is to be estimated once on synthetic reference tasks and transferred by embedding similarity (Eq. 7), so we select without enumerating the target's pairs.

  • Transfer. On TAMAS (4 agents, exact), our transferred selection recovers the oracle pair on 90% of instances vs 42% for the heuristic, and the transferred attack reaches 72% against an 83% oracle.
  • Interaction matters. Synergy (ϕ+SII\phi+\text{SII}) beats marginal ϕ\phi-top-2 on the tasks where they differ (68% vs 48%). Dropping the pairwise term changes the coalition on 54% of engineering tasks and lowers ASR from 39% to 22%; adding the third-order term changes only 15% of tasks and does not help (95% vs 95%), so order-2 is the right truncation.

W3 (fidelity). Agreement of the override-channel importance with the real attack is checked only on finance.

On finance we check this exactly: we recompute a second Shapley vector using the realized attack outcomes as the value function over all 26=642^6{=}64 coalitions. The two vectors give Pearson r=0.71r{=}0.71 (p<10−5p{<}10^{-5}), 87% of agent pairs identically ordered, and agent-level r=0.88r{=}0.88. The attribution channel is used only to rank, and the ranking matches the real attack. We will add this and a similar version on engineering and CRM.

W4 / Q4. Small nn; benign BSR near 0% in some CRM categories; ASR-not-confounded checked only for Claude; refusal rates of GPT-5.5/Gemini.

  • Multi-seed. Re-running MAStrike 5 times gives a mean of 60.0% ± 4.0, Clopper-Pearson [53.6, 66.1] over 250 attacks. We will report intervals in the main tables.
  • 0% BSR is over-denial, not a broken task. These cases execute correctly and the MAS denies a legitimate action; it never takes a wrong one (e.g. the CRM agents refuse to book a six-figure deal the requester legitimately owns). Over-denial cannot inflate ASR, because ASR requires the MAS to proceed on a malicious request. We will relax the benign limits, report the updated BSR, and report GPT-5.5 and Gemini refusal rates on the affected categories.

W5 / Q5. No defense proposed; no reproducibility/ethics statement or release; guardrail case study has no quantitative results.

We will add discussions on defenses, code release, and case study in the revision.

We hope our response addresses the concerns, and we would be grateful if the reviewer would reconsider the score.

Reviewer_pcmMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposa a red-teaming framework for hierarchical multi-agent systems (MAS), where several agents share the job of blocking unsafe actions. It treats attack success rate (ASR) as the value of a coalition of compromised agents, estimates Shapley values and pairwise interaction scores on synthetic workflows, and transfers these scores to new tasks through task similarity. A red-team agent then writes role-aware prompts for the selected agents and refines them using failure feedback. This paper also build MABENCH, a sandboxed benchmark covering finance, software engineering, and CRM. At k=2, MASTRIKE reports an average ASR of 61.8% on Claude Opus 4.7, 55.6% on GPT-5.5, and 51.0% on Gemini 3.1 Pro, while no baseline average exceeds 19.4%.

Critical strengths

They proposa a principled agent selection. The paper replaces role-based guessing with a measurable score for each agent and agent pair. Figures 5 and 6 show that important agents change across tasks, which fixed role-based rules cannot capture.

The ablation study part is clear and easy to follow. The authors fix the generator and vary the coalition (62%, 40%, 28%), then fix the coalition and vary generation (62% vs. 12%). This design shows which componet brings the most improvement.

Critical weaknesses

The comparison with baselines seems not fair. MASTRIKE changes more of an agent's instructions than two of the other baselines. Table 4 shows that MASTRIKE replaces an agent’s full system prompt, while TAMAS and GCA can only add a suffix to the original prompt. So when Table 3 reports 62.0% vs. 0.0%, we cannot tell whether MASTRIKE has a better method or simply more access.

The paper does not show that attacks need collusion. The authors say a successful attack requires agents to collude, but they do not report ASR with zero or one compromised agent. The main results start at two compromised agents, so I cannot tell whether two agents are actually necessary. The paper calls joint prompt generation critical, but it never compares this with writing each agent's prompt separately.

Questions for authors

What ASR do TAMAS and GCA reach if they can also replace the full system prompt and attack the same agents as MASTRIKE?

What share of malicious tasks succeeds only when at least two agents are compromised?

How much does ASR change when MASTRIKE writes each agent’s prompt separately instead of jointly, using the same agents and the same number of optimization rounds?

1 comment

We thank the reviewer for the suggestions. We answer the questions below.

Q1. What ASR do TAMAS and GCA reach if they can also replace the full system prompt and attack the same agents as MAStrike?

We ran the full 2×22{\times}2 factorial that removes both axes, so the baselines get exactly MAStrike's channel and MAStrike's coalition.

selection and channelTAMAS suffixTAMAS overrideGCA suffixGCA override
heuristic (identity+policy)0/200/200/200/20
Shapley (identity+device)0/200/200/200/20

All 8 cells are 0.0%. Even with MAStrike's channel and coalition, the baselines do not improve, so the gap comes from the injection generation process rather than from channel access.

Q2. What share of malicious tasks succeeds only when at least two agents are compromised?

We report ASR at every coalition size:

kkASR
0 (uncompromised MAS)0.0%
128.0%
267.4%
368.2%
479.1%

When k=0k{=}0 and k=1k{=}1, the system has low ASR, and at k=0k{=}0 it refuses every malicious request. When k≥2k{\ge}2, the ASR improves significantly. On the five account-takeover tasks, k=1k{=}1 never succeeds, so those tasks require at least two compromised agents.

Q3. How much does ASR change when MAStrike writes each agent's prompt separately instead of jointly, same agents and same number of optimization rounds?

We ran joint vs separate at matched agents and matched rounds. We run it at k=3k{=}3, where the members' outputs must stay mutually consistent, so inconsistency can be caught.

GenerationASR
Joint (coordinated co-generation)84.0% (42/50)
Separate (per-agent)72.0% (36/50)

Joint optimization improves the ASR by 12%. We will add the results and discussion in the revision.

We hope that removing the access constraint (Q1) and the collusion-necessity gap (Q2), plus the requested joint-vs-separate ablation (Q3), addresses the core of the weak-reject, and we would be grateful if the reviewer would reconsider the score.

Reviewer_GzbfOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents MAStrike, a Shapley-guided red-teaming framework for hierarchical multi-agent systems (MAS). The proposed approach estimates agent-level Shapley values and pairwise interaction indices to identify influential agents and vulnerable coalitions. Based on these attribution signals, an autonomous red-teaming agent generates coordinated, role-aware adversarial manipulations and iteratively refines them through execution feedback and structured failure diagnosis.

The authors further introduce MABench, a benchmark covering finance, software engineering, and CRM workflows, to evaluate collusive attacks against distributed safety mechanisms. Experimental results across three frontier language models demonstrate substantial improvements over existing red-teaming baselines. Additional analyses investigate coalition selection, attack optimization, Shapley estimation reliability, and generalization to independently constructed multi-agent systems.

Critical strengths
  1. Meaningful integration of agent-level attribution and coordinated red-teaming.

The paper addresses an important security problem in multi-agent systems by considering both individual agent influence and cross-agent interactions. The integration of Shapley-based coalition selection with coordinated, role-aware attack generation provides a well-motivated framework for studying distributed vulnerabilities. The coalition-selection ablation in Figure 3 is particularly convincing, showing an ASR of 62.0% compared with 40.0% for heuristic selection and 28.0% for random selection.

  1. Extensive experimental evaluation with useful component-level analyses.

The evaluation covers three application domains and multiple frontier language models. The proposed method substantially outperforms the reported baselines. The supplementary analyses strengthen the empirical contribution by examining individual components, Shapley estimation reliability, attack optimization budgets, and transfer to independently constructed systems. These experiments provide useful evidence that coalition selection and coordinated attack generation both contribute to the observed improvements.

Critical weaknesses
  1. The attacker capability assumptions limit the practical applicability of the findings.

Although the threat model considers several attack channels, Appendix C clarifies that the reported attacks are conducted by modifying the compromised agents' system prompts. This represents a relatively powerful attacker capability compared with realistic indirect prompt-injection scenarios, where adversaries may control only external documents, tool outputs, or other lower-trust inputs. Consequently, the reported attack success rates do not necessarily demonstrate comparable effectiveness under more restricted attacker access. The paper would benefit from a clearer distinction between privileged agent compromise and lower-trust prompt injection, together with a more explicit discussion of the corresponding deployment scenarios.

  1. The evaluation setting and baseline differences limit the generalizability of the results.

MABench is deliberately constructed with distributed safety gates that require multiple compromised agents to succeed. While this is appropriate for studying collusion, the effectiveness of the proposed approach may depend on these structural assumptions. The external evaluation on TAMAS provides useful additional evidence, but it primarily covers relatively small systems. Moreover, the compared baselines differ in their injection mechanisms and optimization procedures. Although the authors provide strengthened baselines and controlled ablations, these evaluations are concentrated in the finance domain. Further evidence would help establish whether the improvements remain consistent under more diverse system architectures and matched attack configurations.

Questions for authors
  1. Attacker capabilities and practical deployment.

Appendix C indicates that the actual attacks modify agent system prompts. How would MAStrike perform when the attacker can manipulate only lower-trust inputs, such as retrieved documents, tool responses, or shared-memory content, without modifying any system prompts? This distinction would help clarify the practical threat scenarios addressed by the framework.

  1. Generalizability and experimental fairness.

Can the authors further clarify the extent to which MAStrike's effectiveness depends on the distributed safety-gating structure of MABench? In addition, would the performance advantage remain comparable across engineering and CRM when baseline methods are evaluated under more closely matched attack-generation and optimization configurations?

  1. Stability of Shapley-based coalition selection.

For the larger engineering and CRM systems, Shapley values and interaction indices are approximated using sampled coalitions. How stable are the selected coalitions across independent sampling runs, particularly when multiple agents have similar attribution scores? Quantifying this uncertainty would help assess the reliability of the proposed coalition-selection strategy.

3 comments

We thank the reviewer for the positive assessment and the three focused questions.

Q1. How would MAStrike perform when the attacker can manipulate only lower-trust inputs?

See C1. We measured MAStrike on additional injection channels, including lower-trust ones, holding the coalition, red-team model, and iteration budget fixed and varying only where the adversarial content enters.

MethodASR
MAStrike (system-prompt override)70.0%
MAStrike w/ tool-response injection86.7%
MAStrike w/ system-prompt suffix20.0%
AutoTransform (system-prompt override)1.0%
Other baselines0.0%

MAStrike with tool-response injection achieves the highest ASR. Even on its weakest channel, the system-prompt suffix, MAStrike still exceeds every baseline, whether the baseline uses the override channel or the suffix channel. The Shapley attribution and coalition selection are channel-agnostic: they identify which agents to compromise given whatever channel the attacker has.

Q2. To what extent does MAStrike's effectiveness depend on MABench's distributed safety-gating structure? Does the advantage hold in engineering/CRM under closely matched attack-generation and optimization configs?

Independently-built system (TAMAS). On TAMAS's released harness (their MAS, agents, tools, tasks, and judge, none of which we designed or tuned), MAStrike's Shapley selection reaches 72% vs a role-name heuristic coalition 13% and TAMAS-as-published 9%, close to the 83% oracle, pooled over 5 scenarios × 2 models. Our selection recovers the hand-designed oracle colluding pair on 90% (43/48) of instances vs 42% for the LLM heuristic. The advantage survives on a system built by others.

On the design itself: distributed separation-of-duties is the standard enterprise maker/checker pattern. A benchmark where one compromised agent suffices would model a system with no defense-in-depth and make the collusion question trivial. The gating is what makes the attribution question meaningful, not what inflates our numbers.

Q3. How stable are the selected coalitions across independent sampling runs, especially under near-ties?

Three results:

  • Finance, exact reference. With 26=642^6{=}64 the reported ϕ\phi is exact. Offline subsampling (uniform draws, R=200R{=}200/budget) recovers the exact top-2 set on 94% at m=40m{=}40, 98% at m=48m{=}48, 100% at m=56m{=}56, with Spearman rank correlation to the exact vector rising 0.80→0.96→1.00. The paper's sampler prioritizes small coalitions and converges at least as fast, so this uniform number is a conservative lower bound.
  • Engineering / CRM, varying budget. Sweeping the sampling budget 32→96, the selected coalition is identical in engineering at every budget; in CRM it alternates between exactly two coalitions, and this is the near-tie. The downstream ASR is 1.00 at every budget in both domains: when the identity flips, it flips between agents of near-identical marginal contribution, so the attack is equally effective either way.
  • Takeaway. Near-ties are the only source of instability, and by construction a near-tie means a near-identical attack. We will add the Spearman/top-set table and this near-tie discussion in the revision.

Thank you for the detailed rebuttal. Regarding Q2, I still have concerns about experimental fairness. Would MAStrike's performance advantage still hold in the Engineering and CRM domains under more closely matched injection channels, attack-generation settings, and optimization budgets? The existing ablation experiments are useful, but they are mainly conducted in the Finance domain. I think further clarification is needed to determine whether the reported improvements come from the proposed framework or differences in experimental settings. If this concern is not adequately addressed, I may consider lowering my current rating.

We ran the controlled ablation in both domains. Each baseline receives MAStrike's exact attack configuration: the system-prompt override channel and the Shapley-selected coalition. All runs are k=2k{=}2 on Opus-4.7. The full selection-by-channel ablation for the baselines (same format as the Finance domain) is below.

CRM:

selection \ channelTAMAS suffixTAMAS overrideGCA suffixGCA override
heuristic0%0%0%0%
Shapley0%4%0%25%

Engineering:

selection \ channelTAMAS suffixTAMAS overrideGCA suffixGCA override
heuristic0%20%12%10%
Shapley10%22%7%28%

Comparing the strongest setting with MAStrike:

Method (override channel, Shapley coalition)CRM ASREngineering ASR
TAMAS3.6%22.0%
GCA25.0%28.0%
MAStrike33.3%71.0%

MAStrike remains strongest in both domains. In CRM, the baselines stay at zero everywhere except the Shapley-and-override cell. In engineering the override channel is the dominant lever and the baselines reach the low scores in several cells, but Shapley-and-override is still the strongest baseline cell and every cell remains far below MAStrike (71%). We will add the full per-task matched ablation for all three domains in the revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.