- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3(+1)
- Soundness
- 2
Read review
Paper summary
In this paper, the authors identify task construction as a novel attack surface in proactive large language model agents. They demonstrate that adversary-controlled environmental contexts can induce agents to produce malicious target behaviors while preserving the user’s original underlying intent.
The authors propose ProAtk, a red-teaming framework. Integrating task anchoring, target-compatible task grafting, Pareto optimization of task fidelity and target attachment, and knowledge distillation, this framework constructs lightweight attack programs.
Experiments conducted across three types of victim agents and four task scenarios show that this method achieves the highest average joint attack success rate while largely maintaining the normal benign functionality of the model.
Critical strengths
-
This paper presents a novel and well-argued security threat model.
- It identifies task construction as a previously understudied attack surface in proactive agents, shifting the research focus from manipulating the execution of predefined tasks to influencing how agents infer user intent from environmental context.
- This distinction reveals a fundamental class of vulnerabilities unique to proactive systems: adversary-controlled evidence can shape seemingly legitimate, user-permitted tasks without issuing explicit malicious instructions.
- The threat model features clear formal definitions and accounts for real-world constraints, including partial observability and non-authoritative environmental interventions.
-
The paper features solid and comprehensive experimental results.
- ProAtk achieves significant improvements over existing attack baselines while largely preserving benign utility.
- The evaluation covers three victim agents and four task categories, accompanied by ablation studies, defense robustness analysis, cross-dataset experiments, and human evaluation of proposal acceptability.
Critical weaknesses
-
Approval is evaluated at an interface that hides consequential actions, so the central security claim is largely built into the evaluation design. Both the LLM approval judge (Appendix F.5) and the 80-context human study (Section 5.5) see only the natural-language proposal and the original context, not the function calls, arguments, or recipients (Appendix B). A user can thus approve a “readiness check” without seeing that it transfers recent activity data to a third-party service (Appendix E.2). Because the threat model also requires targets to be plausible task-serving steps (Section 3), high ASR-u may reflect insufficient disclosure of tool consequences rather than successful manipulation of task formulation. An approval evaluation that discloses data transfers and recipients is needed to support the broader claim that user approval offers insufficient protection.
-
Attack success rates are computed on method-specific subsets, so the headline comparison is not interpretable. Equation (14) includes only contexts that actually received an attack, and ASR-u/ASR-j further restrict to contexts with an approved clean proposal. ProAtk’s learned gate and payload filtering therefore determine which contexts enter its denominator. Intervention coverage and per-method denominators are not reported, and aggregate SR/FTR cannot reveal abstention on difficult eligible cases. The reported 89.2% ASR-j and the 45.6-point advantage over AgentBait thus do not establish superiority on a common population. Results on a fixed clean-approved cohort that count skipped or invalid attacks as failures, reported together with coverage and conditional rates, would make the central comparison interpretable.
Questions for authors
-
Do the target-function descriptions shown to victims explicitly disclose external data transmission and its recipient?
-
For each method, how many eligible contexts are excluded because of gating or invalid payloads?
-
What is ASR-j on a shared clean-approved cohort when these exclusions count as unsuccessful attacks?
-
Were the target configurations in Appendix D.2 unseen during attacker training, and was the same frozen attacker used throughout?
5 comments
Response to Reviewer C8em (2/2)
W2, Q2 & Q3. Attack success metrics and selective intervention.
Thank you for your comment. In proactive settings, an attacker must decide when to intervene. ProAtk therefore uses a gate to avoid intervening when the user does not need assistance, following the selective-intervention objective in Sec. 3. This motivates distinguishing when an attack is applied from how effective it is once applied. Eqs. (14)–(15) measure the latter through conditional ASR. To clarify, the gate is trained to predict assistance needs, with no decision rule based on attack difficulty. The comparison thus measures conditional attack effectiveness under each method’s intervention policy.
As noted above, gating is the main reason the evaluation subsets differ across methods. To provide a direct comparison, we disable the gate and use all test instances only requiring assistance as the denominator for ASR-a. ProAtk retains its advantage over the baselines across metrics:
| Method | ASR-a (%) | ASR-u (%) | ASR-j (%) |
|---|---|---|---|
| TopicAttack | 69.48 | 13.74 | 4.95 |
| ChatInject | 43.19 | 28.02 | 4.40 |
| AgentBait | 74.65 | 36.26 | 23.63 |
| ProAtk | 93.43 | 88.46 | 87.91 |
We further assess how well each method’s intervention decisions align with assistance needs, treating contexts requiring assistance as positive and those requiring no assistance as negative. ProAtk achieves an F1 score of 80.32%, exceeding all three baselines (64.12%, 70.13%, and 71.12%). With the gate enabled, we further compute ASR-j on a fixed clean-approved cohort shared by all methods, counting skipped and invalid attacks as failures. Under this calculation, ProAtk still achieves 70.33%, compared with 23.63% for AgentBait. This rate reflects beyond the conditional attack effectiveness, since gated cases contribute failures. The paper’s conditional ASR-j specifically measures how often an attack succeeds once it is actually applied.
Q4. Transfer to unseen target configurations.
In Appendix D.2, 8 of the 9 target configurations were totally unseen during attacker training. The only exception was the default setting , , also used in the main experiments. We used the same frozen attacker across all nine configurations. These results support the attacker’s transferability to unseen target configurations with varying levels of relevance and harmfulness.
Looking Forward to Your Feedback
Dear Reviewer C8em,
Thank you for your thoughtful review and helpful suggestions. We briefly summarize our responses below.
- ProAtk retains high approval rates when complete function calls, arguments, and data recipients are disclosed.
- ProAtk maintains a substantial advantage on a shared clean-approved cohort, even when skipped and invalid attacks count as failures.
- We confirmed transfer to unseen target configurations using the same frozen attacker.
As the discussion period is ending soon, we would appreciate your feedback. We hope our clarifications and additional results address your concerns, and would be grateful if you could consider adjusting your assessment accordingly. We would be happy to answer any further questions.
Response to Reviewer C8em (1/2)
We thank you for the time and effort you have devoted to reviewing our work. We have carefully considered your feedback and used it to improve our study. Our responses to your comments follow below.
W1 & Q1. Approval evaluation.Thank you for raising this point. Our claim concerns task formulation and proposals “whose natural-language description the user would accept” (Sec. 1), with approval assessed “at the task-description level” (Sec. 6). Accordingly, approval based on reviewing individual actions and their arguments falls outside the scope of this approval-related claim. Among our metrics, ASR-a specifically measures target-action inclusion in proposed calls, and ASR-j combines this with task preservation and proposal approval. These measures align with our stated claim. To further clarify our setting, we elaborate on the following two points.
1) Plausibility constraint
The plausibility constraint in Sec. 3 defines which actions could serve as auxiliary steps in useful assistance. This is necessary to exclude inherently infeasible cases where no semantically coherent path can incorporate. The substantial ASR-j gap between ProAtk and the baselines under this shared constraint further shows that semantic feasibility alone is insufficient for attack success.
2) Tool disclosure
Our implanted target functions explicitly require recent user activity records as inputs. The evaluation simulates tool use without involving actual data transfers or real recipients. In this setting, the tool descriptions do not specify transmission or receiver. This is consistent with adversarial methods like MalTool and Les Dissonances, which model tools that can transmit email content [1] or location data [2] to attacker-controlled servers without disclosing these transfers in their descriptions.
We have also conducted additional experiments examining how different disclosure levels affect approval. We define three levels of increasing information disclosure. L0 follows the paper’s setting. L1 adds action-level information including the names and descriptions of the functions that would be called. L2 further provides all raw function calls and complete arguments, explicitly identifying the data recipient, i.e., 'ThirdParty Data Service', and the parameters it would receive.
We use GLM-5.3 Flash as the victim model, keeping the proposals and approval judge fixed across conditions. We conduct experiments on Content and Productivity and report micro-average ASR-u.
Although greater disclosure reduces ASR-u across all methods, ProAtk retains high approval rates under action-level review, even when complete calls, arguments, and data recipients are disclosed.
[1] Hu, Yuepeng, et al. “MalTool: Malicious Tool Attacks on LLM Agents,” arXiv:2602.12194, 2026.
[2] Li, Zichuan, et al. “Les Dissonances: Cross-Tool Harvesting and Polluting in Pool-of-Tools Empowered LLM Agents,” NDSS 2026.