acceptodds
Under review as a conference paper at ICLR 2027

When Correct Plans Lead to Wrong Actions: Multi-Source Evidence Fabrication Against LLM Agents

Abstract

Tool-using LLM agents can execute the intended task while binding a security-critical parameter to the wrong entity. We study this intent-to-action binding failure and introduce multi-source compositional evidence fabrication (MCEF), which distributes plausible, declarative, instruction-free evidence across attacker-writable sources. MCEF discovers tasks whose parameters depend on external evidence, works backward from an attacker-selected binding, fabricates source-specific evidence, and triggers its composition during the original benign task. Across models, agent frameworks, and application scenarios, MCEF achieves 93.1% ASR with 99.3% intent preservation and 92.9% plan preservation among runs reaching the attacker-target action. Most instruction-oriented detectors rarely flag individual fabricated items, while evaluated runtime defenses reduce attack success unevenly and may block benign runs. These findings highlight the need for defenses that validate how combined evidence justifies security-critical parameters.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Tool-using LLM agents, agent security, intent-to-action binding failure, multi-source evidence fabrication, instruction-free attacks

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_WLq4OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates an overlooked security issue in tool-using LLM agents, termed intent-to-action binding failure, where an agent correctly understands the user’s intent and follows an appropriate plan but binds a security-critical tool parameter to the wrong entity based on manipulated external evidence. To study this problem, the authors propose Multi-Source Compositional Evidence Fabrication (MCEF), which distributes plausible, declarative, and instruction-free evidence across multiple attacker-writable sources. During normal retrieval and reasoning, the agent combines these fabricated evidence items and derives an attacker-selected parameter while largely preserving the original task intent and execution plan. MCEF consists of four stages: candidate task discovery, target-to-evidence inversion, multi-source evidence fabrication, and combination triggering. Experiments across six models, four agent frameworks, and six tasks achieve an average 93.1% attack success rate, 99.3% intent preservation, and 92.9% plan preservation. The results further show that distributing evidence across multiple sources substantially strengthens the attack, while existing prompt-injection detectors and runtime defenses provide only limited protection. Overall, the paper highlights that securing LLM agents requires not only detecting malicious instructions or plan deviations, but also validating whether external evidence reliably justifies security-critical parameter bindings.

Critical strengths
  1. MCEF does not rely on explicit malicious instructions such as prompt injection. Instead, it manipulates plausible, declarative evidence distributed across multiple sources, which is well aligned with the identified intent-to-action binding vulnerability and makes the attack harder for instruction-oriented detectors to identify.
  2. MCEF consists of four well-defined stages—candidate task discovery, target-to-evidence inversion, multi-source evidence fabrication, and combination triggering—forming a complete pipeline from identifying vulnerable tasks to inducing the attacker-desired parameter binding.
Critical weaknesses
  1. The attacker is expected to observe or reasonably approximate the agent’s evidence-access trajectory and to have write access to multiple relevant external sources. It remains unclear how often these conditions hold in realistic deployments, which may limit the practical applicability of the attack.
  2. Tool outputs and business actions are mocked rather than executed in real production systems. Therefore, while the experiments demonstrate the vulnerability at the reasoning and parameter-binding level, the attack’s effectiveness under real-world APIs, permission structures, and operational constraints remains insufficiently validated.
  3. Intent preservation is evaluated using the same model as the tested agent, which may introduce self-evaluation bias. In addition, plan preservation is measured using Jaccard similarity over tool-call multisets and ignores call order and finer-grained control-flow differences, so the reported preservation scores may not fully capture whether the underlying execution plan truly remains unchanged.
Questions for authors

Please see weakness

1 comment

Thank you for the suggestions on attacker assumptions and preservation metrics. We specify the deployment conditions and report judge and action-sequence comparisons. Changes in the revised manuscript are marked in red.

1. Multi-source write access and dependency knowledge.

We agree that these prerequisites needed clarification. §3.2 specifies that the evaluated construction uses dependencies observed in clean executions and write access to selected evidence sources. Appendix A maps payload fields to concrete object/field permissions and protected platform attributes.

In the simulator (§5.3, Table 4), retaining k=2k=2 sources gives 79.0% ASR and retaining k=1k=1 gives 33.6%. The comparison also varies the amount of retained evidence. We have not measured how often the necessary access and task knowledge coexist in deployments.

2. Mocked tools and production relevance.

We agree that the connection to deployed interfaces needed explanation. §3.2 and Appendix A specify the simulated evidence and business actions. The experiments isolate how evidence changes the agent's parameter binding while it continues the requested task.

Appendix A follows a T-A1 simulated trace from its two reads to the recorded sharing action. The platform documentation describes conditional permissions. Field ACL enforcement, a single identity with the required rights, and whether a deployed reader accesses the same editable record remain unverified. No production share was executed.

3. Self-evaluation bias in intent preservation.

To check whether IPR depended on the choice of judge, we rescored the same 134 executions with a fixed DeepSeek-v4-pro judge. §5.1 describes both judging configurations; Appendix C, Table 17, compares the fixed judge with the evaluated-model judge used in §5.2.

ConfigurationIPR
Evaluated-model judge99.25%
Fixed DeepSeek-v4-pro judge99.25%

Both judges yielded an IPR of 99.25% (133/134 positive labels each) and agreed on 132/134 labels (98.51%). Most labels were positive, and the fixed judge shares its model with the DeepSeek agents, so this agreement does not rule out shared bias.

Manual inspection of 60 samples found no issues. Both configurations receive the task and benign/attacked final responses, each truncated to 2,000 characters, to assess preservation of the high-level task intent.

4. Call order and joint preservation.

We agree that call order should be checked explicitly. §5.2 and Appendix C add an order-sensitive action-sequence analysis. Each of the six tasks in our experiments has exactly one type of core business-action tool. PPR compares these action calls and retains repetitions; retrieval calls describe evidence access and are excluded.

The original PPR is 92.9%, averaged equally across six models. On the same 130 comparable runs, multiset Jaccard and order-aware normalized edit similarity both average to 92.76% (92.8% rounded). The difference from 92.9% reflects aggregation by run versus by model. No run has equal action multisets in a different order. Retaining order therefore leaves pooled similarity unchanged on this sample. Core action sequences match exactly in 110/130 runs, and full tool sequences including retrieval match in 29/130. Complex control flow remains untested.

For an all-attempt comparison, Appendix C fixes benign references independently of attack outcomes. For action-tool sequences AA and BB with repetitions retained, ordered similarity uses Levenshtein distance:

S(A,B)=1−Lev⁡(A,B)max⁡(∣A∣,∣B∣).S(A,B)=1-\frac{\operatorname{Lev}(A,B)}{\max(|A|,|B|)}.

§5.5, Table 9, also adds the joint criterion:

Ji=1{ASi=1∧IPRiext=1∧Ai=Bi}.J_i=\mathbf{1}\{\mathrm{AS}_i=1\land\mathrm{IPR}^{\mathrm{ext}}_i=1\land A_i=B_i\}.

MetricMCEFPoisonedRAG (multi-source replication)
Ordered similarity, all attempts with references (92 cells)94.2%81.3%
Joint success, five-model grid (120 cells)77.5%63.3%
Joint success, four-model reference subset (92 cells)80.4%70.7%

The four-model reference subset contains 92 of 96 cells; the five-model joint comparison includes all 120. These checks address action order and repetitions and jointly require attack success, intent preservation, and exact action-tool sequence preservation.

Reviewer_dAhaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies how an LLM agent can pursue the intended task while binding a sensitive tool argument to the wrong entity. It proposes MCEF, which distributes mutually consistent, declarative claims across attacker-controlled sources to steer parameter resolution without explicit malicious instructions. Experiments across several models, frameworks, and simulated application settings demonstrate this failure mode and examine its interaction with existing defenses.

Critical strengths
  1. A useful security perspective. The paper distinguishes following the intended task from correctly identifying the entity on which to act. This highlights why task-level alignment alone may be insufficient to establish the safety of a tool call.
  2. A clear attack construction. The target-to-evidence inversion procedure translates an attacker-selected binding into source-specific declarative claims. This provides a concrete way to study how seemingly ordinary business records can mislead parameter resolution without explicit instructions.
Critical weaknesses

The paper studies an interesting problem, but the writing and experimental evaluation currently fall short of the standard expected at ICLR.

  1. My main concern is how the simulated modifications map to the attacker’s stated permissions. The appendix clearly identifies the records as mock evidence, but it does not sufficiently explain how an attacker could modify the corresponding metadata and synchronization records in a deployed system. A more explicit mapping between the modified fields and the permissions required would help establish the practical relevance of the threat model.
  2. The evidence for the specific benefit of source separation remains limited. Comparing the same payload in a single source and across multiple sources is useful, although retrieval order, presentation, and context effects remain entangled with source placement. I therefore find the results more convincing as a demonstration of the attack’s effectiveness in the tested settings than as evidence that multi-source composition itself accounts for the improvement.
Questions for authors
  1. What permissions allow the attacker to modify the metadata and synchronization records?
  2. What independent evidence establishes the authorized target despite the fabricated updates?
  3. Does the advantage of cross-source placement persist under matched content and presentation conditions, and how does MCEF compare with simpler factual poisoning under the same access and information budget?
1 comment

Thank you for identifying the questions about permissions, source placement, and evaluator references. We provide field-level explanations and controlled comparisons. Changes in the revised manuscript are marked in red.

1. Permissions for metadata and synchronization records.

We agree that the mapping needed detail. Appendix A maps 109 source-text bindings in 28 frozen configurations to editable fields, required permissions, and protected boundaries. Microsoft Graph and Dataverse documentation covers business lists, workbook ranges, calendars, and CRM columns; GitHub documentation covers PRs, comments, files, and log access. These official specifications provide checkable operation scopes and access requirements, grounding the assumed writable sources in concrete interfaces.

An editable business claim cannot change an ACL or the synchronization service. Appendix A illustrates the boundary with a recorded T-A1 trace. Metadata and calendar reads precede a simulated sharing action. We have not verified a single identity with all required write permissions, a production adapter reading those same objects, or an editable equivalent of the synchronization log. T-B2 would require column-level isolation of the final account.

2. Matched content, presentation, and source-return order.

We agree that these factors needed controlled comparisons. §5.3 points to Appendix D, which adds matched-content and presentation experiments. Both arms contain the same three evidence components (288 runs in each condition).

ConditionSame-sourceCross-sourceDifference
Same components64.9%78.5%+13.5 pp
Normalized presentation63.2%76.4%+13.2 pp

Under autonomous retrieval, agents encountered a payload-bearing source more often with cross-source placement (252/288, 87.5%) than with same-source placement (203/288, 70.5%). Among runs with observed exposure, success rates were similar: 226/252 (89.7%) and 185/203 (91.1%), respectively.

In the order study (Appendix D), we evaluated DeepSeek-v4-pro and GLM-5.3 on both LangGraph and AutoGen. All three sources were supplied before the first autonomous decision. In the same-source condition, S1 always contained all three evidence components, regardless of return order. For each task and return order (six tasks, six orders), we ran each model and framework combination once with same-source placement and once with cross-source placement, yielding 288 valid runs (144 matched pairs). The table reports target-action success rates by return order, pooled across tasks, models, and frameworks.

Source-return orderSame-source successCross-source success
S1→S2→S379.2%83.3%
S1→S3→S270.8%95.8%
S2→S1→S383.3%100.0%
S2→S3→S187.5%100.0%
S3→S1→S275.0%95.8%
S3→S2→S187.5%95.8%
Overall80.6%95.1%

When all three sources were supplied before the first decision, cross-source placement still achieved higher success in all six return orders (95.1% vs. 80.6% overall). The advantage thus persisted when both groups had the same opportunity to access the evidence.

3. Evaluator references and authorization scope.

We agree that executing the attacker-selected target need not violate authorization. Appendix A distinguishes the two: ASR measures target execution; frozen pre-attack records supply evaluator references, such as company-only sharing for T-A2 and staging for T-C1. The simulator does not verify that every reference is protected from modification, so these records cannot establish authorization.

Open-ended T-B1 has no unique reference recipient: we can score attacker-selected execution but not reference deviation. Excluding this task leaves 111/120 target successes across six models.

4. Simpler factual poisoning and budget matching.

We agree that matching writable sources alone does not test simpler factual poisoning with the same information and request budget. Table 9 compares multi-source replication across five models. The paired pilot in Appendix G fixes the task, target, dependencies, writable sources, temperature, call and retry limits, and requested output limit per call. The factual baseline generates neutral records for each source.

Across DeepSeek-v4-pro and GLM-5.3 on LangGraph and AutoGen (3 paired repetitions per task and cell; 144 valid runs), MCEF succeeded in 54/72 runs (75.0%) and the factual baseline in 39/72 (54.2%). Only MCEF succeeded in 19 pairs, only the baseline in 4, and outcomes matched in 49. Appendix G reports per-cell results and adaptation limits.

Reviewer_dmj4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper works on the intent-to-action binding attack, where a safety-related parameter is binding to a wrong entity. Since the entity could be revised from external evidence, potentially combining information across several sources, the intent-to-action binding attack could be implemented via manipulating evidence along the resolution path. To research on this attack, the work proposes multi-source compositional evidence fabrication (MCEF), an attack that distributes declarative, instruction-free evidence across attacker-writable sources. Diverse experiments were conducted with competitive attack performance obtained.

Critical strengths
  1. The paper is well motivated.
  2. The paper's research view is interesting focusing on intent-to-action binding attack.
  3. The experiments are diverse, covering six models, and four agent frameworks.
Critical weaknesses
  1. The work novelty might be limited. The works major novelty focuses on manipulating the reference, which might have been done in other works. What is the difference compared to memory poisoning/knowledge poisoning?
  2. How to understand "constraint" in L260?
  3. It is unclear about the testing sample size via reading Sec. 5.1.
  4. It is unclear how the evidence has been shared in experiments. For the defense experiments in L452, it looks like it has access to evidence. And it is unclear whether the detection in Table 2 has access to these evidence or not?
  5. There is no baselines. Whether there are any baseline attack methods to be compared?
Questions for authors

Please refer to weakness.

1 comment

Thank you for the suggestions on positioning, comparisons, and evaluation details. We clarify the contribution and report the supporting evidence below. Changes in the revised manuscript are marked in red.

1. Relationship to memory and knowledge poisoning.

We agree that the contribution needed clearer positioning. §2 explains that MCEF targets intent-to-action binding: the agent continues the requested task but binds a sensitive argument to an attacker-selected entity. The construction recovers dependencies from a clean execution, works backward from the target binding, and generates consistent declarative records for the relevant sources. Our contribution combines this failure-mode characterization, a concrete construction procedure, and evaluation of attack success together with intent and action preservation.

2. Attack baselines and joint success.

We agree that attack comparisons were missing. §5.5, Table 9, adds six adapted baselines across DeepSeek, Kimi, GPT, Claude, and GLM in the same task/tool environment (120 runs for each method). They cover prompt injection (PI), indirect prompt injection (IPI), memory/factual poisoning, and imperative composition.

AttackASRJoint success
InjecAgent0.0%0.0%
AgentDojo0.8%0.8%
AgentPoison0.8%0.8%
ObliInjection41.7%35.8%
PoisonedRAG50.8%46.7%
PoisonedRAG (multi-source replication)75.8%63.3%
MCEF93.3%77.5%

In Table 9, the multi-source replication baseline can write to the same sources as MCEF. Joint success requires target-action success, a positive intent label, and an action-tool sequence identical to the benign reference. The AgentPoison adaptation does not use embedding-space optimization.

We also compared MCEF with simple factual poisoning under the same dependency information, writable sources, and request limits (§5.5; Appendix G, Tables 21 and 22).

Table 10 compares the three methods on the 132 configurations with a valid MCEF run: 117 used MCEF’s first execution and 15 used a retry after an initial failure. The two baselines each used a single attempt per configuration.

AttackLlamaFirewallProgentMELONTotal
ObliInjection0.0%46.7%31.1%26.5%
PoisonedRAG21.4%44.4%26.7%31.1%
MCEF66.7%86.7%66.7%73.5%

MCEF has 132 valid runs; ObliInjection and PoisonedRAG each have 122. Invalid runs count as failures for every method. Counting only the first execution of each scheduled configuration gives 93/135 successes for MCEF, 35/135 for ObliInjection, and 41/135 for PoisonedRAG (Appendix F).

3. Meaning of “constraint” at line 260.

We agree that this term needed definition. §4.2 and Appendix B define a constraint as the source's evidence role, format, permitted edits, facts to preserve, and cross-source consistency requirements. For example, calendar and contact records should agree on the proposed delivery contact while preserving project identity. These specifications guide evidence construction; the agent receives the resulting records.

4. Evaluation sizes.

We agree that sample sizes needed clarification. The manuscript now states counts beside the relevant tables and gives inclusion rules in the appendices. The main grid contains 6×4×6=1446\times4\times6=144 runs. Each baseline has 120 runs; intent comparison uses 134 executions, and the sequence audit 130. Placement controls have nine conditions; the full-exposure order study has 288 runs and 144 matched pairs. Each detector receives 82 fabricated and 16 benign texts; each runtime defense schedules 108 pairs. These counts describe experimental conditions over six tasks.

5. Evidence available to detectors and runtime defenses.

We agree that the inputs needed clarification. §5.4 and Appendix F, Table 20, distinguish them:

ComponentAvailable input
Individual-text detectorOne evidence text
LlamaFirewallEvidence scan; task, trajectory, pending call
ProgentTask schema, allow-list, pending action
MELONCandidate call and masked re-execution comparison

Protectai_deberta flags 36.6% of fabricated texts and 37.5% of benign texts. Full-task MCEF ASR is 78.1%, 88.0%, and 66.7% under LlamaFirewall, Progent, and MELON. Single-text detectors assess instruction-like attack features; runtime defenses additionally use task context, calls, or policy constraints. Their detection rates and task-level ASRs measure different stages of protection and use different denominators.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.