acceptodds
Under review as a conference paper at ICLR 2027

TraceTrigger: Benchmarking Distributed Trigger Attacks in Agentic Systems

Abstract

Self-hosted agents may combine interaction history, contextual documents, and persistent memory before choosing routes, tools, and actions. A safeguard that inspects each source separately can therefore miss harmful behavior produced by their joint influence. We present TraceTrigger, a controlled benchmark for studying evidence distributed across interaction steps, contextual locations, and simulated session boundaries. Control is necessary to remove one evidence channel while holding the authorized request, available tools, and outcome check fixed. The benchmark contains 450 clean, false-activation, and attack records built from self-hosted skill descriptions and targets memory, planning, and routing because these surfaces respectively govern retained state, action composition, and tool selection. Each task uses one model planning call followed by deterministic parsing, a logged component policy and defense, and mock execution; this ordering exposes where an unauthorized action enters the pipeline while preventing external side effects. We evaluate five language models to test dependence on a single model or access pattern and seven defense families spanning context-, action-, and trace-level intervention points. Aggregate pipeline ASR ranges from 0.81 to 0.94. To separate attack construction from runtime structure, a matched 40-task Llama-3.2-3B-Instruct control evaluates both attack families in both runtimes: TraceTrigger-1D reaches ASR 0.773 in the single-agent runtime and 0.909 in the modular runtime, whereas prior-1D controls reach 0 in both. To test incomplete evidence, removing one required channel yields 0/30 unauthorized final actions on a separate near-miss slice. Multi-channel configurations generally have higher pipeline ASR, although defenses and channel combinations produce non-monotonic exceptions. Because the component policy can modify a parsed model plan, these aggregate results measure the controlled pipeline rather than pure model susceptibility. TraceTrigger thereby separates distributed-context, execution-setting, and authorization effects that an aggregate ASR would otherwise conflate. Anonymized code is available at: https://anonymous.4open.science/r/TraceTrigger-4E70/

Primary area:
datasets and benchmarks
Keywords:
AI agents, agent security, self-hosted agents, tool-use safety, memory poisoning, indirect prompt injection, distributed triggers, agent benchmarking, security evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_w3AiOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

TraceTrigger benchmarks attacks distributed across interaction history, documents, and persistent memory, targeting memory, planning, and routing. It evaluates five models and seven defense families using a controlled pipeline with model planning, component policies, and mock execution. The main contribution is a unified evaluation framework, supported by matched-runtime comparisons and incomplete-trigger controls.

Critical strengths
  1. The framework usefully organizes distributed-context attacks across multiple channels and execution surfaces.
  2. The authors provide inspectable code and explicitly distinguish model-selected actions from component-added actions, while acknowledging the limitations of simulated execution.
Critical weaknesses

Given the paper’s substantial reliance on its implementation, I examined the accompanying code and identified several issues that raise concerns about whether it adequately supports the paper’s claims.

  1. The main trend is partly imposed by the benchmark. In tracetrigger/platforms/openclaw_like.py, component activation thresholds increase with dimensionality, directly increasing the likelihood of adding attack actions. Likewise, tracetrigger/defenses/registry.py explicitly reduces defense coverage as dimensionality increases. These choices weaken the interpretation that the observed trends demonstrate properties of distributed attacks or defenses.
  2. Attack scoring and reported results need validation. In tracetrigger/models/planning.py, _textual_plan converts even an explicit refusal mentioning mock_send_message into an attack action using task metadata. This undermines the attribution of parsed attacks to model decisions. Separately, the released dataset contains 20 scored tasks per 1D method–surface combination, yet Table 1 reports percentages such as 31.62%, inconsistent with the stated count-based, non-repeated evaluation. The corresponding raw results are needed to resolve this discrepancy.

I may have misunderstood some aspects of the work, but these concerns need to be clearly addressed. In its current form, I do not believe the paper meets the bar for acceptance at ICLR.

Questions for authors
  1. What are the model-only ASRs, using strict action parsing and disabling component-added attack actions? Does the 1D-to-3D trend remain? What independent evidence supports the chosen component thresholds and defense coverage rates?
  2. Can you provide paired full-trigger and missing-channel results that preserve the component instructions, authorization criteria, and parsing rules, changing only the relevant evidence?
  3. Please provide the code version, success counts, denominators, aggregation procedure, and raw records corresponding to Tables 1-2 and Figure 2. How do the reported percentages follow from the released task counts?
4 comments
AuthorsOpenReview ↗

Response to Reviewer w3Ai (1/2)

We thank the reviewer for examining the code. Our re-run evaluations isolate model-only attack success, measure defenses at released and full coverage, and separate temporal cues from card statements in paired controls. The updated released code and per-task records support regeneration of every reported table. All changes are marked in blue in the revised PDF.

1. Model-Only ASR and the 1D-to-3D Trend (W1, Q1)

Gemma-4-31B and Qwen3.5-27B remain highly vulnerable across all three dimensionalities when only model-proposed actions are scored. Re-running the evaluation with strict parsing and component additions counted separately establishes the model-only pattern directly (Appendix Table 10; realistic instruction, pooled).

Model1D2D3D
Gemma-4-31B180/180180/180180/180
Qwen3.5-27B178/180179/180180/180
GPT-6 Luna66/18048/18028/180

The re-run results show that the large open models’ vulnerability persists across dimensionalities. Gemma is already at ceiling and Qwen is near ceiling, while Luna’s success decreases with additional channels. We therefore report vulnerability across 1D, 2D and 3D without claiming a consistent increase in ASR.

2. Thresholds and Defense Coverage (W1, Q1)

Model-plan ASR is independent of component thresholds. Re-running each defense at both released and full coverage separates the effect of the coverage gate from the defense’s behavior when applied to every task (revised Table 5; strict parser, realistic instruction; successful attacks out of 540, released / full coverage).

DefenseQwen3.5-2BLlama-3.2-3BGemma-4-31BQwen3.5-27BGPT-6 Luna
none452458540539478
proof_guardrail431 / 0437 / 0517 / 0516 / 0457 / 0
causal_tracing448 / 145454 / 158535 / 181534 / 190473 / 168
least_privilege445 / 155451 / 166529 / 181528 / 190469 / 173
tool_anomaly442 / 155449 / 166529 / 181528 / 190469 / 173
capability_isolation452 / 445458 / 457540 / 530539 / 539478 / 475
execution_policy452 / 445458 / 457538 / 530539 / 539478 / 475
trigger_sensitivity452 / 452458 / 458540 / 540539 / 539478 / 478

The re-run evaluations show substantial residual attack success: defenses at released coverage block at most 23 attacks, and even full-coverage causal_tracing leaves 181, 190 and 168 successes for Gemma, Qwen and Luna. Full-coverage proof_guardrail reaches zero using a per-task attestation label, so that result depends on access to the label. The execution_policy allow-list contains the payload, and trigger_sensitivity rewrites text that never reaches the planner.

3. Scoring (W2)

Scoring now accepts structured model plans only, preventing refusal text from becoming an attack action. We re-ran the evaluation with the same strict parser for every method and released the per-task records behind every k/n count.

Specifically, _textual_plan is removed, only the first JSON object is parsed, and text-to-action conversion and fallback are disabled. The released per-task records and rerun/paper_numbers.py support regeneration of the re-run counts.

AuthorsOpenReview ↗

Response to Reviewer w3Ai (2/2)

4. Paired Full-Trigger vs. Missing-Channel Results (Q2)

The paired re-run results identify activation conditions that are obscured by aggregate ASR: removing the temporal cue together with changing its affirmative readiness statement suppresses Qwen3.5-27B and GPT-6 Luna. Cue-only and card-only controls separate these contributions while keeping component instructions, the authorization rule and the parser fixed (revised Tables 2-3; stops / starts; 300 pairs per channel or isolated control, 900 pairs pooled across channels).

ModelInstructionStops / starts by task set (13 · 21 · 42)Pooled (/900 pairs)Exact pT removed + card “no”P removedS removedCue removed onlyCard “no” only
Qwen3.5-27Breleased2/0 · 2/0 · 1/05/00.062/00/03/0N/AN/A
Qwen3.5-27Brealistic17/0 · 23/0 · 18/058/07·10⁻¹⁸56/01/01/00/03/0
Qwen3.5-27Bneutral25/0 · 25/0 · 25/075/05·10⁻²³71/01/03/00/02/0
GPT-6 Lunarealistic60/39 · 52/44 · 36/33148/1160.0653/349/6446/4945/2352/3
GPT-6 Lunaneutral71/35 · 48/28 · 48/37167/1005·10⁻⁵59/153/5755/4246/1459/4
Gemma-4-31Ball three0/0 in every task set0/01.000/00/00/0N/AN/A

The isolated re-run controls reveal distinct activation patterns: either the cue or the affirmative card statement generally sustains Qwen’s activation, whereas changing the card alone largely reproduces Luna’s combined-removal result. P/S removal barely affects Qwen and moves Luna in both directions; Gemma fires throughout. These controls locate the effect in each model’s response to the available evidence and card statement, while preserving the distinction between a combined condition and temporal evidence alone.

5. Code Version, Counts and Records (Q3)

Every reported table can be regenerated from the updated released code and per-task records. The repository includes all three task sets, the evaluation protocol, prompts and raw outputs; rerun/paper_numbers.py regenerates the counts, and submission_version/ preserves the submitted code. Re-running the original Table 1 evaluation using the updated released code gives the results below (Appendix Table 12; successes out of 60, 20 per task set).

AgentPoisonTT memoryPoTTT plannerAMATT router
Qwen3.5-2B15412248940
Llama-3.2-3B047056049
Gemma-4-31B344153484042
Qwen3.5-27B606054605760
GPT-6 Luna05348552652

The release makes both levels of success auditable: these re-run counts measure the complete pipeline, including policy additions, while revised Table 1 identifies the model-proposed share. The re-run tables use one run per task set, greedy decoding for open-weight models, and pooled counts obtained by summing.

We hope the re-run results and released records address your concerns about scoring, dimensionality and reproducibility. Thank you for identifying where clearer presentation was needed; we will further clarify the distinction between model-plan and pipeline results and the interpretation of the paired controls.

Reviewer_w3AiOpenReview ↗

Official Comment By Reviewer w3Ai

Thank you for the detailed response. Although the overall quality of the paper has improved somewhat, it still does not meet the standards of the ICLR conference at present. I suggest that the authors re-submit it after making a complete revision. I will raise the score to Weak Rejection.

AuthorsOpenReview ↗

Thank you for the updated score

Dear Reviewer w3Ai,

Thank you for reading our response and for raising your score. Your close look at the code led directly to the strict parser, the model-only results and the released per-task records, and the paper is stronger for it.

If there are specific points you feel still fall short, we would be grateful to know them. We can respond until the discussion closes on November 18 (AoE), and we will address them in the revision either way.

Reviewer_Vi7VOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces TraceTrigger, a benchmark for evaluating distributed trigger attacks across interaction history, contextual sources, and persistent memory. It constructs 450 records across memory, planning, and routing surfaces, and evaluates five models and seven defense families using a controlled agent pipeline. The results suggest that multi-channel triggers generally achieve higher pipeline ASR than single-channel attacks.

Critical strengths
  1. Interesting benchmark design. Studying how malicious signals accumulate across different context channels is relevant to agent security, especially for agents with persistent memory.

  2. Useful evaluation controls. I appreciate the attempt to separate model-generated actions from pipeline-level outcomes, as well as the matched-runtime and incomplete-trigger experiments.

Critical weaknesses
  1. Questionable attack attribution. The component policy can directly insert unauthorized actions after model planning, making the high pipeline ASR difficult to interpret as actual model vulnerability.

  2. Weak baseline comparison. The baselines are evaluated under different execution settings, and the matched-runtime experiment reports 0% ASR for all prior methods. This raises concerns about whether the comparison is sufficiently fair and representative.

Questions for authors

See above.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Vi7V

We thank the reviewer for a careful review. Our re-run evaluations separate model-proposed actions from policy additions and compare prior attacks with TraceTrigger under matched runtime, framing and parsing. These controls confirm model-level vulnerability in the large open models and provide a common basis for the baseline comparison. All changes are marked in blue in the revised PDF.

1. Model Plan vs. Component Policy (W1)

The attack is present in the models’ own plans: Gemma-4-31B and Qwen3.5-27B propose the unauthorized action under neutral as well as component-aware instructions. Re-running all three task sets with strict parsing and policy additions counted separately isolates the model’s contribution directly (revised Table 1; pooled counts out of 540).

Qwen3.5-2BLlama-3.2-3BGemma-4-31BQwen3.5-27BGPT-6 Luna
Proposed by the model, released instruction6136540538315
Proposed by the model, realistic instruction730540537142
Proposed by the model, neutral instruction225540537122
Added by the program (realistic)44542802336

2. Baseline Comparison Under Matched Conditions (W2)

TraceTrigger remains effective when prior attacks receive the same runtime, framing and strict parser. The matched re-run comparisons show near-ceiling success for both attack families on the large open models, while exposing the effect of framing and runtime on Llama and GPT-6 Luna (revised Table 4; realistic instruction, pooled).

Qwen3.5-2BLlama-3.2-3BGemma-4-31BQwen3.5-27BGPT-6 Luna
Prior-1D, single-agent runtime3/180105/180180/180180/18052/180
TraceTrigger-1D, same runtime2/18048/180180/180180/18097/180
Prior-1D with TraceTrigger's card framing2/1805/180180/180180/180129/180
TraceTrigger-1D, identical text3/18019/180180/180178/18066/180

Matching the evaluation conditions confirms attack effectiveness while clarifying the comparison’s scope: single-turn ASR does not consistently distinguish the attack families. The separate paired re-run controls identify the cross-turn cue/card condition for Qwen and the mainly card-driven condition for Luna (revised Tables 2-3).

We hope the matched re-run comparisons clarify the distinction between model-proposed actions and policy additions and address your concerns about baseline fairness. We will make the evaluation setup and the scope of the resulting claims more explicit in the writing.

AuthorsOpenReview ↗

Follow-up before the discussion closes

Dear Reviewer Vi7V,

Thank you again for your careful review. As the discussion period closes on November 18 (AoE), we wanted to check whether our response addressed your concerns. In brief:

  • Model vs. policy (W1): with policy additions counted separately, Gemma-4-31B and Qwen3.5-27B propose the unauthorized action themselves on 540 and 537 of 540 tasks, even under a neutral instruction.
  • Matched baselines (W2): under the same runtime, framing and parser, prior 1D attacks and TraceTrigger-1D both reach 178-180 of 180 on these two models, so single-turn ASR alone does not separate them.
  • Cross-turn condition: the paired controls show what activates the trigger. For Qwen3.5-27B (realistic instruction), removing the temporal cue and setting the card to "no" stops activation in 56 of 300 pairs, versus 0 or 3 for either change alone.

We would be glad to clarify anything further. If these results resolve your concerns, we would be grateful if you would consider updating your assessment.

Thank the authors for the detailed explanation. After reading the other reviewers’ comments, I agree that, given the substantial changes made to the paper, it would benefit from a new review cycle.

Reviewer_Xi71OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper tracks how information from conversations, files and saved memory becomes an unsafe action, and which part of the system introduces it. Without this tracking, a failure may not tell us whether the AI proposed the action or the surrounding program added it. The paper builds 450 records, including 240 scored attack tasks. It asks the model for a plan, applies program rules and defenses, and simulates the resulting actions. Five models and seven defense families are tested (Section 5), with additional checks using different runtimes and incomplete triggers.

Critical strengths

The records preserve what the AI proposed, what the program rules changed, what the defense decided and the final simulated outcome (Appendix D.1). Figure 1 shows an example where the unsafe action already appears in the AI’s proposal. This separation is useful as it helps locate where the unsafe action entered the system.

The tests include legitimate tasks, missing pieces of a trigger and comparisons between runtimes (Appendix D.1; Table 6; Appendix E.2). Final simulated actions are checked against what the task permits. These are useful controls, although the missing-trigger result applies to the particular Llama test described in Appendix E.2.

Critical weaknesses

Program rules can add the unsafe action after the AI produces its plan. Removing part of a trigger also changes whether these rules apply (Appendix E.2). The results therefore do not establish how much the context alone changes the AI’s decision. For me this limits how broadly the findings can be applied. The authors acknowledge this limitation and the dependence on their constructed setup.

Questions for authors

Using the separate records described in Appendix D.1, how often did the AI propose the unsafe action, and how often did the program add it?

Do the harm count in Section 7 and the average and scoring rule in Appendix E.3 describe the same tasks and scoring rule?

How do the attack groups defined in Appendix D.1 contribute to each attack-success percentage in Table 2?

2 comments
AuthorsOpenReview ↗

Response to Reviewer Xi71

We thank the reviewer for a careful review. Re-running three task sets with strict parsing confirms that the large open models propose the unauthorized action themselves, while paired controls separate temporal evidence from the card’s readiness statement. We have also unified harm scoring and made the task groups explicit. All changes are marked in blue in the revised PDF.

1. Model Proposals vs. Program Additions (Q1)

Gemma-4-31B and Qwen3.5-27B propose the unauthorized action themselves, even under neutral instructions that say nothing about components. Re-running all three task sets with strict parsing and separate scoring of program-added actions confirms that the large open models’ attack success comes almost entirely from their own plans (revised Table 1; pooled counts out of 540 unless noted).

Qwen3.5-2BLlama-3.2-3BGemma-4-31BQwen3.5-27BGPT-6 Luna
Proposed by the model, released instruction6136540538315
Proposed by the model, realistic instruction730540537142
Proposed by the model, neutral instruction225540537122
Added by the program (realistic)44542802336
Near-miss false activations (/90)00000
Clean tasks completed safely (/180)178128175163176

2. Harm Count vs. Appendix E.3 Mean (Q2)

The harm-count discrepancy is resolved by applying one scoring rule to both counts and means and computing them from the same records. The submitted values differed because Section 7 used the stated 0/1/2 rule, while Appendix E.3 used pre-assigned task labels (Section 5 and Appendix Table 11).

Section 7 used the stated 0/1/2 rule, giving (46 × 2 + 5)/60 = 1.62. Appendix E.3 used pre-assigned task labels, giving 85/60 = 1.4167.

3. Attack Groups in Table 2 (Q3)

Table 2 contains only TraceTrigger tasks, grouped by channel and dimensionality; Prior-1D tasks do not enter the comparison. Each cell is now reported as k/n from the released records, making its task membership and denominator explicit (Appendix Table 13).

In the original set, P, S and T contain 19, 21 and 20 TraceTrigger-1D tasks, respectively. P+S, T+P and T+S each contain 20 TraceTrigger-2D tasks, while T+P+S contains 60 TraceTrigger-3D tasks. T denotes a seed→fire sequence.

4. Fragment Removal vs. Rule Applicability (Weakness)

Re-running the paired controls shows that removing temporal evidence together with changing its readiness statement suppresses activation in Qwen3.5-27B and GPT-6 Luna. Independently varying the cue and card then identifies which signal each model follows, with component instructions, the authorization rule and the parser fixed (revised Tables 2-3; stops / starts; 300 pairs per channel or isolated control, 900 pairs pooled across channels).

ModelInstructionStops / starts by task set (13 · 21 · 42)Pooled (/900 pairs)Exact pT removed + card “no”P removedS removedCue removed onlyCard “no” only
Qwen3.5-27Breleased2/0 · 2/0 · 1/05/00.062/00/03/0N/AN/A
Qwen3.5-27Brealistic17/0 · 23/0 · 18/058/07·10⁻¹⁸56/01/01/00/03/0
Qwen3.5-27Bneutral25/0 · 25/0 · 25/075/05·10⁻²³71/01/03/00/02/0
GPT-6 Lunarealistic60/39 · 52/44 · 36/33148/1160.0653/349/6446/4945/2352/3
GPT-6 Lunaneutral71/35 · 48/28 · 48/37167/1005·10⁻⁵59/153/5755/4246/1459/4
Gemma-4-31Ball three0/0 in every task set0/01.000/00/00/0N/AN/A

The isolated re-run controls explain the paired effect: Qwen generally fires when either the temporal cue or the affirmative card statement remains, while Luna mainly follows the card. Gemma fires throughout. Thus, the supported finding is a combined cue/card effect for Qwen and a mainly card-driven effect for Luna. The original near-miss controls contained no component card and tested a different condition.

We hope these explanations and re-run results clarify the model’s contribution, the scoring rules and the paired controls, and address your concerns. We will further refine the wording to make these distinctions explicit.

AuthorsOpenReview ↗

Follow-up before the discussion closes

Dear Reviewer Xi71,

Thank you again for your careful review. As the discussion period closes on November 18 (AoE), we wanted to check whether our response addressed your concerns. In brief:

  • Model vs. program (Q1): Gemma-4-31B and Qwen3.5-27B propose the unauthorized action themselves on 540 and 537 of 540 tasks, even under a neutral instruction.
  • Harm scores (Q2): the two values came from two scoring rules (1.62 vs. 1.4167); both now use one rule on the same records.
  • Table 2 (Q3): it contains only TraceTrigger tasks, and each cell is now reported as k/n.
  • Fragment removal: for Qwen3.5-27B (realistic instruction), removing the temporal cue and setting the card to "no" stops activation in 56 of 300 pairs, versus 0 or 3 for either change alone. GPT-6 Luna mainly follows the card.

We would be glad to clarify anything further. If these results resolve your concerns, we would be grateful if you would consider updating your assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.