acceptodds
Under review as a conference paper at ICLR 2027

When Context Shapes the Task: Red-Teaming Proactive Agents at Task Formulation

12 views

Abstract

Proactive agents infer what assistance to offer from context without an explicit user task. In the absence of such a task, the observations they consume determine not only how to act, but also what task to propose. This shifts a security boundary upstream: attacker-controlled context can influence task formulation before planning or execution begins. Existing agent red-teaming largely assumes that the task is fixed independently of such context and targets how that task is executed. We instead study whether an attacker with partial observations can manipulate task formulation to insert a target action into the completion path while preserving user acceptance of the resulting task description. To expose this vulnerability, we introduce ProAtk, a red-teaming framework based on task grafting. ProAtk infers the underlying user task from attacker-visible context and constructs a target-compatible requirement into its completion path. It uses Pareto search to preserve the underlying task while making the target action part of its completion, then distills successful grafts into a search-free attacker. Across three victim agents and four task categories, ProAtk consistently achieves the strongest joint attack performance while largely preserving benign utility of the victim agent. Human evaluation further corroborates that its target-bearing proposals remain useful and acceptable. Code is available at https://anonymous.4open.science/r/ProAtk-ED87.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Agents, Proactive Agents, Agent Security, Red Teaming

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_C8emOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3(+1)
Soundness
2
Read review
Paper summary

In this paper, the authors identify task construction as a novel attack surface in proactive large language model agents. They demonstrate that adversary-controlled environmental contexts can induce agents to produce malicious target behaviors while preserving the user’s original underlying intent.

The authors propose ProAtk, a red-teaming framework. Integrating task anchoring, target-compatible task grafting, Pareto optimization of task fidelity and target attachment, and knowledge distillation, this framework constructs lightweight attack programs.

Experiments conducted across three types of victim agents and four task scenarios show that this method achieves the highest average joint attack success rate while largely maintaining the normal benign functionality of the model.

Critical strengths
  • This paper presents a novel and well-argued security threat model.

    • It identifies task construction as a previously understudied attack surface in proactive agents, shifting the research focus from manipulating the execution of predefined tasks to influencing how agents infer user intent from environmental context.
    • This distinction reveals a fundamental class of vulnerabilities unique to proactive systems: adversary-controlled evidence can shape seemingly legitimate, user-permitted tasks without issuing explicit malicious instructions.
    • The threat model features clear formal definitions and accounts for real-world constraints, including partial observability and non-authoritative environmental interventions.
  • The paper features solid and comprehensive experimental results.

    • ProAtk achieves significant improvements over existing attack baselines while largely preserving benign utility.
    • The evaluation covers three victim agents and four task categories, accompanied by ablation studies, defense robustness analysis, cross-dataset experiments, and human evaluation of proposal acceptability.
Critical weaknesses
  • Approval is evaluated at an interface that hides consequential actions, so the central security claim is largely built into the evaluation design. Both the LLM approval judge (Appendix F.5) and the 80-context human study (Section 5.5) see only the natural-language proposal and the original context, not the function calls, arguments, or recipients (Appendix B). A user can thus approve a “readiness check” without seeing that it transfers recent activity data to a third-party service (Appendix E.2). Because the threat model also requires targets to be plausible task-serving steps (Section 3), high ASR-u may reflect insufficient disclosure of tool consequences rather than successful manipulation of task formulation. An approval evaluation that discloses data transfers and recipients is needed to support the broader claim that user approval offers insufficient protection.

  • Attack success rates are computed on method-specific subsets, so the headline comparison is not interpretable. Equation (14) includes only contexts that actually received an attack, and ASR-u/ASR-j further restrict to contexts with an approved clean proposal. ProAtk’s learned gate and payload filtering therefore determine which contexts enter its denominator. Intervention coverage and per-method denominators are not reported, and aggregate SR/FTR cannot reveal abstention on difficult eligible cases. The reported 89.2% ASR-j and the 45.6-point advantage over AgentBait thus do not establish superiority on a common population. Results on a fixed clean-approved cohort that count skipped or invalid attacks as failures, reported together with coverage and conditional rates, would make the central comparison interpretable.

Questions for authors
  • Do the target-function descriptions shown to victims explicitly disclose external data transmission and its recipient?

  • For each method, how many eligible contexts are excluded because of gating or invalid payloads?

  • What is ASR-j on a shared clean-approved cohort when these exclusions count as unsuccessful attacks?

  • Were the target configurations in Appendix D.2 unseen during attacker training, and was the same frozen attacker used throughout?

5 comments
AuthorsOpenReview ↗

Response to Reviewer C8em (1/2)

We thank you for the time and effort you have devoted to reviewing our work. We have carefully considered your feedback and used it to improve our study. Our responses to your comments follow below.


W1 & Q1. Approval evaluation.

Thank you for raising this point. Our claim concerns task formulation and proposals “whose natural-language description the user would accept” (Sec. 1), with approval assessed “at the task-description level” (Sec. 6). Accordingly, approval based on reviewing individual actions and their arguments falls outside the scope of this approval-related claim. Among our metrics, ASR-a specifically measures target-action inclusion in proposed calls, and ASR-j combines this with task preservation and proposal approval. These measures align with our stated claim. To further clarify our setting, we elaborate on the following two points.

1) Plausibility constraint

The plausibility constraint in Sec. 3 defines which actions could serve as auxiliary steps in useful assistance. This is necessary to exclude inherently infeasible cases where no semantically coherent path can incorporate. The substantial ASR-j gap between ProAtk and the baselines under this shared constraint further shows that semantic feasibility alone is insufficient for attack success.

2) Tool disclosure

Our implanted target functions explicitly require recent user activity records as inputs. The evaluation simulates tool use without involving actual data transfers or real recipients. In this setting, the tool descriptions do not specify transmission or receiver. This is consistent with adversarial methods like MalTool and Les Dissonances, which model tools that can transmit email content [1] or location data [2] to attacker-controlled servers without disclosing these transfers in their descriptions.

We have also conducted additional experiments examining how different disclosure levels affect approval. We define three levels of increasing information disclosure. L0 follows the paper’s setting. L1 adds action-level information including the names and descriptions of the functions that would be called. L2 further provides all raw function calls and complete arguments, explicitly identifying the data recipient, i.e., 'ThirdParty Data Service', and the parameters it would receive.

We use GLM-5.3 Flash as the victim model, keeping the proposals and approval judge fixed across conditions. We conduct experiments on Content and Productivity and report micro-average ASR-u.

MethodL0 (%)L1 (%)L2 (%)
TopicAttack23.9122.8319.57
ChatInject41.9432.2624.73
AgentBait32.2631.1823.66
ProAtk94.0388.0674.63

Although greater disclosure reduces ASR-u across all methods, ProAtk retains high approval rates under action-level review, even when complete calls, arguments, and data recipients are disclosed.

[1] Hu, Yuepeng, et al. “MalTool: Malicious Tool Attacks on LLM Agents,” arXiv:2602.12194, 2026.

[2] Li, Zichuan, et al. “Les Dissonances: Cross-Tool Harvesting and Polluting in Pool-of-Tools Empowered LLM Agents,” NDSS 2026.

AuthorsOpenReview ↗

Response to Reviewer C8em (2/2)

W2, Q2 & Q3. Attack success metrics and selective intervention.

Thank you for your comment. In proactive settings, an attacker must decide when to intervene. ProAtk therefore uses a gate to avoid intervening when the user does not need assistance, following the selective-intervention objective in Sec. 3. This motivates distinguishing when an attack is applied from how effective it is once applied. Eqs. (14)–(15) measure the latter through conditional ASR. To clarify, the gate is trained to predict assistance needs, with no decision rule based on attack difficulty. The comparison thus measures conditional attack effectiveness under each method’s intervention policy.

As noted above, gating is the main reason the evaluation subsets differ across methods. To provide a direct comparison, we disable the gate and use all test instances only requiring assistance as the denominator for ASR-a. ProAtk retains its advantage over the baselines across metrics:

MethodASR-a (%)ASR-u (%)ASR-j (%)
TopicAttack69.4813.744.95
ChatInject43.1928.024.40
AgentBait74.6536.2623.63
ProAtk93.4388.4687.91

We further assess how well each method’s intervention decisions align with assistance needs, treating contexts requiring assistance as positive and those requiring no assistance as negative. ProAtk achieves an F1 score of 80.32%, exceeding all three baselines (64.12%, 70.13%, and 71.12%). With the gate enabled, we further compute ASR-j on a fixed clean-approved cohort shared by all methods, counting skipped and invalid attacks as failures. Under this calculation, ProAtk still achieves 70.33%, compared with 23.63% for AgentBait. This rate reflects beyond the conditional attack effectiveness, since gated cases contribute failures. The paper’s conditional ASR-j specifically measures how often an attack succeeds once it is actually applied.

Q4. Transfer to unseen target configurations.

In Appendix D.2, 8 of the 9 target configurations were totally unseen during attacker training. The only exception was the default setting I=0.5I=0.5, H=0.5H=0.5, also used in the main experiments. We used the same frozen attacker across all nine configurations. These results support the attacker’s transferability to unseen target configurations with varying levels of relevance and harmfulness.

AuthorsOpenReview ↗

Looking Forward to Your Feedback

Dear Reviewer C8em,

Thank you for your thoughtful review and helpful suggestions. We briefly summarize our responses below.

  • ProAtk retains high approval rates when complete function calls, arguments, and data recipients are disclosed.
  • ProAtk maintains a substantial advantage on a shared clean-approved cohort, even when skipped and invalid attacks count as failures.
  • We confirmed transfer to unseen target configurations using the same frozen attacker.

As the discussion period is ending soon, we would appreciate your feedback. We hope our clarifications and additional results address your concerns, and would be grateful if you could consider adjusting your assessment accordingly. We would be happy to answer any further questions.

Reviewer_PT43OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates a security vulnerability in proactive LLM agents, where attacker-controlled environmental context can manipulate task formulation before planning and execution. Unlike conventional agent attacks that target the execution of a predefined task, the proposed attack aims to induce an attacker-specified action while preserving the underlying user task and the acceptability of the agent's proposal. To address this problem, the authors introduce ProAtk, a red-teaming framework based on task grafting. ProAtk first infers the underlying task from partially observable context, constructs a target-compatible intermediate requirement, and generates contextual evidence to induce the desired action. It then employs Pareto search to jointly optimize Task Fidelity and Target Attachment, and distills successful attacks into an open-weight model for search-free deployment. The authors evaluate ProAtk across three victim agents and four task categories, comparing it against existing attack baselines. The experiments report substantial improvements in joint attack success while largely preserving benign utility. Additional ablation studies, defense evaluations, cross-dataset experiments, and human assessments further support the effectiveness of the proposed approach.

Critical strengths

S1. A meaningful and well-motivated security problem. The paper identifies an important attack surface arising from the defining characteristic of proactive agents: tasks are inferred from environmental observations rather than explicitly specified by users. The distinction between task-formulation manipulation and execution-time prompt injection is conceptually useful. In particular, the proposed task-grafting mechanism illustrates how non-authoritative contextual evidence may cause an agent to incorporate attacker-desired actions into an otherwise plausible task. This extends the discussion of agent security beyond conventional instruction-following attacks.

S2. A reasonably comprehensive empirical evaluation supporting the proposed mechanism. The paper evaluates ProAtk across multiple victim models and task categories, and includes comparisons with representative attack baselines, defense evaluations, component ablations, additional datasets, and human validation. The distinction between target attachment, preservation of useful assistance, and their joint success is particularly valuable. The ablations provide evidence that task anchoring, explicit graft construction, and search play complementary roles. The human evaluation further strengthens the claim that the attack's advantage is not solely an artifact of an LLM-based approval judge.

Critical weaknesses

W1. Potential instability of Pareto search due to limited rollout-based estimation. The Pareto search relies on only four surrogate rollouts per candidate to estimate Task Fidelity and Target Attachment probabilities. Such limited sampling may introduce substantial estimation noise, particularly for the conditional probability qA=P(TA=1∣TF=1)q_A = P(\mathrm{TA}=1 \mid \mathrm{TF}=1). For instance, when only one rollout satisfies TF, the empirical estimate of qAq_A can only be 0 or 1. Consequently, the estimated Pareto dominance relationships may be sensitive to sampling randomness, potentially affecting candidate selection and subsequent search trajectories. Although the paper demonstrates improvements over alternative search strategies, it does not sufficiently examine the stability of these estimates or the resulting search outcomes. A sensitivity analysis with different rollout budgets (n∈4,8,16n \in {4,8,16}) and results across multiple random seeds would help substantiate the claimed effectiveness of Pareto search.

Questions for authors

Q1. Limitations and generalizability. The paper does not include a dedicated discussion of its limitations. Could the authors clarify the main limitations of ProAtk, particularly regarding its reliance on text-based simulated environments, partial observability, and proposal-level rather than execution-level evaluation? How might these factors affect the generalizability of the reported attack effectiveness to real-world proactive agents?

Q2. Inconsistent hyperparameter descriptions. Appendix C.4 contains two apparent inconsistencies: it states that two successful rollouts suffice when n=4n=4 and τs=3/4\tau_s=3/4, and later describes the experimental configuration as K=n=3K=n=3, contradicting Table 4. The configuration reported in the repository's hyperparameter.txt is consistent with Table 4, specifying K=3K=3, n=4n=4, and three required joint TF–TA successes. These appear to be documentation errors, and the manuscript should be corrected accordingly.

Q3. Limited reproducibility of the core offline search pipeline. The provided repository includes implementations and instructions for student distillation, inference, and victim evaluation. However, I could not locate an executable implementation of the offline teacher pipeline described in Algorithm 1, including task anchoring, graft construction, surrogate rollouts, TF/TA scoring, and Pareto-frontier updates. The documented training workflow instead assumes that accepted task-anchor and graft records are already available. As this search pipeline is a central methodological contribution, the available artifacts do not currently allow independent verification of its implementation or reproduction of the distillation data. I recognize that releasing attack-generation code may raise security concerns. Could the authors clarify whether these components were intentionally withheld and, if so, provide an appropriate alternative for verification?

2 comments
AuthorsOpenReview ↗

Response to Reviewer PT43

We appreciate your time and constructive suggestions. Your feedback has helped us strengthen the evidence for search stability and clarify the experimental settings. We hope that these additions and our responses address your concerns and provide a stronger basis for a favorable assessment.


W1 & Q2. Pareto search hyperparameters and search stability.

We apologize for the typos in Appendix C.4 and thank you for pointing them out. We have corrected them accordingly. The correct setting is K=3K=3, n=4n=4, and τs=3/4\tau_s=3/4, requiring three joint TF–TA successes, in correspondence with Table 4 and the configuration provided in our code repository.

We agree that four rollouts may yield potential noisy estimates. The rollout budget balances estimation precision against search cost. In your example, a candidate with only one faithful rollout can have q^A=1\hat q_A=1, but its joint success rate is at most 1/41/4, below the 3/43/4 acceptance threshold. It may therefore influence frontier selection, but cannot enter the distillation data on that evidence alone.

To assess this sensitivity, we repeat the search with n∈{4,8,16}n\in\{4,8,16\} across three seeds {2025, 2026, 2027}, fixing K=3K=3, rmax⁡=3r_{\max}=3, and τs=3/4\tau_s=3/4. We report construction pass rates and mean LLM calls of the teacher and referee. All results are mean ± SD over three seeds.

RolloutsPass rate (r=0)Pass rate (r=1)Pass rate (r=2)Pass rate (r=3)Mean LLM calls
474.95 ± 0.9884.78 ± 0.4889.99 ± 0.3592.92 ± 0.2117.24 ± 0.18
869.79 ± 1.3280.91 ± 0.7789.11 ± 1.9693.11 ± 1.2530.79 ± 1.37
1666.29 ± 0.3580.40 ± 1.4989.11 ± 1.0193.25 ± 0.9056.38 ± 1.82

Final construction pass rates remain consistently high across rollout budgets, with limited variation across seeds. Larger budgets offer only marginal gains at substantially higher cost, supporting our default setting as a stable and efficient choice.

Q1: Limitations discussion.

Thank you for this suggestion. For richer contexts beyond text, ProAtk could use multimodal models for task anchoring and graft construction based on attacker-accessible observations. Search could retain the TF/TA objectives because they assess task preservation and target inclusion in the resulting proposals. In practical deployments, partial observability may also involve access restrictions that vary across information sources and over time. Assessing proactive-agent safety in these settings therefore calls for studying how attacks adapt across repeated interactions as the available evidence changes. For proposal- versus execution-level evaluation, please see our response to cZEP’s Q1 and Appendix B, which discuss the rationale for task-level approval and the complementary role of action-level review.

Q3: Additional artifact release.

Our repository already includes the prompts, hyperparameters, preprocessing scripts, and student training and evaluation code. We have now added an executable implementation of the offline teacher pipeline, enabling independent verification and regeneration of the distillation data. Given the potential misuse risks of the generated attack data and checkpoints, we will provide these artifacts through controlled research access to support authorized security research, consistent with the principle of minimizing harm in the ICLR Code of Ethics.

AuthorsOpenReview ↗

Looking Forward to Your Feedback

Dear Reviewer PT43,

Thank you for your thoughtful review and helpful suggestions. We briefly summarize our responses below.

  • Additional experiments support stable search outcomes across rollout budgets and random seeds.
  • We clarified the method’s limitations and corrected the hyperparameter descriptions.
  • We released an executable offline teacher pipeline to support independent verification and reproduction.

As the discussion period is ending soon, we would appreciate your feedback. We hope our clarifications and additional results address your concerns, and would be grateful if you could consider adjusting your assessment accordingly. We would be happy to answer any further questions.

Reviewer_cZEPOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces ProAtk, a red-teaming framework that attacks proactive LLM agents at the task formulation stage — before planning or execution begins. Unlike prior work that assumes a fixed task and attacks execution, ProAtk shows that an attacker with partial observations can manipulate environmental evidence to insert a target action into the agent's proposed task while preserving user acceptability. The method uses task-anchored grafting, bi-objective Pareto search and distillation into a search-free attacker.

Critical strengths

Principled bi-objective Pareto optimization with significant‌ empirical results

Critical weaknesses

Evaluation may overestimate real-world ASR due to LLM judges and simplified approval

Questions for authors

1, What fraction of successful proposals would be caught under action-level review? 2, Does using a different judge model change ASR estimates?

2 comments
AuthorsOpenReview ↗

Response to Reviewer cZEP

We sincerely thank you for your valuable feedback, which, together with the other reviewers’ comments, has helped us strengthen the paper. We hope that our revisions and responses address your concerns and merit a more favorable assessment.


W1 & Q1. Action-level review.

  1. We would like to first clarify the rationale for proposal-level approval. Please see Appendix B for a more detailed discussion.

    Practical relevance: Task-level confirmation is used in everyday assistants. For example, Siri Suggestions asks whether to add an invitation received in Mail to Calendar, while Gemini in Gmail offers an “Add to calendar” action when it detects event information in an email. Our setting examines approval at this user-facing task level. This is also consistent with prior studies, e.g., ProactiveAgent [1] evaluates acceptance of inferred tasks, and PARE [2] obtains approval through assistant messages before execution.

    Focus on task formulation: Our goal is to determine whether adversarial evidence can change what assistance the user agrees to delegate while keeping it useful and acceptable. Proposal-level approval directly evaluates this decision. Including detailed calls would make the outcome depend additionally on the user's ability to detect risky operations.

  2. We also agree that disclosing additional action information provides a fuller understanding of proactive-agent security. We therefore define three levels of increasing information disclosure. L0 follows the paper’s setting. L1 adds action-level information including the names and descriptions of the functions that would be called. L2 further provides all raw function calls and complete arguments, explicitly identifying the data recipient, i.e., 'ThirdParty Data Service', and the parameters it would receive.

    We use GLM-5.3 Flash as the victim model, keeping the proposals and approval judge fixed across conditions. We conduct experiments on Content and Productivity and report micro-average ASR-u.

MethodL0 (%)L1 (%)L2 (%)
TopicAttack23.9122.8319.57
ChatInject41.9432.2624.73
AgentBait32.2631.1823.66
ProAtk94.0388.0674.63

Although greater disclosure reduces ASR-u across all methods, ProAtk retains high approval rates under action-level review, even when complete calls, arguments, and data recipients are disclosed.

W1 & Q2. Different judge model.

ProAtk maintains a substantial advantage over the baseline across ddifferent judge models. We measure judge sensitivity by rescoring the same agent proposals with GPT-6.1 Sol, DeepSeek-V4.1 Flash, and Kimi-K3.1 Pro, alongside the original GLM-5.3 Max judge. We adopt GLM-5.3 Flash as the victim model and conduct experiments on Content and Productivity, and micro-average of ASR-u and ASR-j are reported.

Judge ModelMethodASR-u (%)ASR-j (%)
GLM-5.3 MaxAgentBait32.2613.98
ProAtk94.0392.54
GPT-6.1 SolAgentBait30.1112.90
ProAtk97.0195.52
DeepSeek-V4.1 FlashAgentBait27.9611.83
ProAtk89.5589.55
Kimi-K3.1 ProAgentBait37.6318.28
ProAtk95.5294.03

We can observe that across judge models, ProAtk maintains a substantial advantage compared with the baseline. Both its ASR-u and ASR-j remain high even under the most conservative judge.

[1] Lu, Yaxi, et al. “Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance,” ICLR 2025.

[2] Nathani, Deepak, et al. “Proactive Agent Research Environment: Simulating Active Users to Evaluate Proactive Assistants,” arXiv:2604.00842, 2026.

AuthorsOpenReview ↗

Looking Forward to Your Feedback

Dear Reviewer cZEP,

Thank you for your thoughtful review and helpful suggestions. We briefly summarize our responses below.

  • ProAtk retains high approval rates under action-level review.
  • ProAtk’s substantial advantage over baseline persists across different judge models.

As the discussion period is ending soon, we would appreciate your feedback. We hope our clarifications and additional results address your concerns, and would be grateful if you could consider adjusting your assessment accordingly. We would be happy to answer any further questions.

Reviewer_mLASOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies adversarial attacks on proactive agents. They propose a method (ProAtk) which injects information into the environment that makes an otherwise unnecessary action appear useful (e.g., while shopping for a graphics card, the agent is tricked into proposing to leak recent purchase activities and private messages to a third party). They test these planted facts using a surrogate model and see if it proposes the harmful action as part of the otherwise benign task. They then fine-tune another model on successful attacks to generate new attacks more efficiently(without searching). Across three victim agents and four task categories, ProAtk achieves strong attack success rates while preserving usefulness of the agent.

Critical strengths

The attack idea is interesting and increasingly relevant with the emergence of proactive agents (e.g., Muse).

Critical weaknesses

The setup of the baselines appears to be underspecified (e.g., models used to generate and how the baselines were adapted to proactive agents). It is therefore unclear if the performance improvement is a result of the method or weak adaptations.

Questions for authors

The attack relies on having the agents decide the next task themselves (i.e., they’re proactive). Did you test whether this attack still works if the agent is explicitly asked to do something (i.e., it’s not being proactive)?

2 comments
AuthorsOpenReview ↗

Response to Reviewer mLAS

We appreciate your thoughtful comments and insightful questions. They have prompted us to clarify experimental details and to better understand the applicability of ProAtk. Below, we address your comments point by point.


W1. Baseline implementation details.

Thank you for this suggestion. We would like to point out that the high ASR-a scores already show that the baselines remain highly effective. In Tables 1 and 3, AgentBait achieves 100% ASR-a on GPT–Travel, and ChatInject exceeds ProAtk on Qwen–Productivity (91.80% vs. 84.44%). The high ASR-a scores confirm that the baselines remain effective after adaptation. Their lower joint success arises because these methods struggle to maintain attack effectiveness when user approval is required.

All methods receive the same attacker-visible context and prescribed target function, and share the tool pool and evaluation protocals. AgentBait and TopicAttack use Qwen3.8-Max for payload generation, the same model used for ProAtk’s offline task anchoring and graft construction.

  • AgentBait expresses its five inducement families as cues tied to the observed activity, presenting the target action as a useful step in providing assistance.

  • TopicAttack constructs a compact fabricated conversation that transitions from the observed activity to the target request, retaining the fabricated assistant completion and attention reminder.

  • ChatInject uses its fixed single-request variant to present the target action as apparent user intent through Qwen ChatML role markers. We cap the request text according to the length limit before inserting it into the template. This implementation requires no generation model.

To ensure a fair comparison while respecting each baseline’s design and implementation, we apply the semantic constraints only to ProAtk and do not impose them on the baselines. We have expanded Appendix C.1 to clarify how each baseline is implemented and adapted to proactive settings.

Q1. Applicability to non-proactive settings.

Yes, ProAtk remains effective under explicit user instructions. Although non-proactive settings fall outside the scope of our study, we agree that evaluating ProAtk under explicit user instructions would provide complementary insights into its applicability. To this end, we additionally adapt ProactiveMobile to a non-proactive setting and compare ProAtk with AgentBait. Specifically, for each instance requiring assistance, we rewrite the first reference task as an explicit user request. The agent then plans how to fulfill this request using the context and tool pool. We reuse frozen ProAtk payloads generated under the proactive setting, and report average SR and ASR-a/u/j on Content and Productivity. The intervention gate is disabled.

Non-proactive results (%):

SRASR-aASR-uASR-j
Clean92.44———
AgentBait80.6759.3843.0111.83
ProAtk87.3977.6392.5471.64

For convenient comparison, we also provide corresponding proactive results (%):

SRASR-aASR-uASR-j
Clean47.06———
AgentBait23.5382.8132.2613.98
ProAtk49.5898.6894.0392.54

The results show an increase in SR under the non-proactive setting, as explicit user instructions provide agents with a clear task objective and a direct basis for assistance. Meanwhile, the explicit task specification leaves less room for adversarial context to steer the agent's planning toward the target action. Although ProAtk exhibits some performance degradation, it retains considerable effectiveness, suggesting that it remains applicable even under explicit user instructions.

AuthorsOpenReview ↗

Looking Forward to Your Feedback

Dear Reviewer mLAS,

Thank you for your thoughtful review and helpful suggestions. We briefly summarize our responses below.

  • We added detailed baseline implementations and adaptations to the revised manuscript.
  • Additional experiments confirm that ProAtk remains effective in non-proactive settings and substantially outperforms the baseline.

As the discussion period is ending soon, we would appreciate your feedback. We hope our clarifications and additional results address your concerns, and would be grateful if you could consider adjusting your assessment accordingly. We would be happy to answer any further questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank all reviewers for your valuable time and constructive feedback. We are particularly grateful for your recognition of the following aspects of our work:

  • Novel problem formulation. Our work identifies task construction as a “previously understudied attack surface in proactive agents” (Reviewer C8em). The distinction between task-formulation manipulation and execution-time prompt injection is “conceptually useful” (Reviewer PT43). We formalize this setting through a “novel and well-argued security threat model” with “clear formal definitions” (Reviewer C8em).
  • Practical relevance. The attack idea is “interesting and increasingly relevant with the emergence of proactive agents” (Reviewer mLAS). Our work addresses a “meaningful and well-motivated security problem” (Reviewer PT43), and the threat model “accounts for real-world constraints” (Reviewer C8em).
  • Sound methodology. ProAtk’s optimization is recognized as “Principled bi-objective Pareto optimization” (Reviewer cZEP). The distinction between target attachment, preservation of useful assistance, and their joint success is “particularly valuable” (Reviewer PT43).
  • Comprehensive experimental evaluation. Our evaluation provides “solid and comprehensive experimental results” (Reviewer C8em), with “significant empirical results” also recognized by Reviewer cZEP. ProAtk achieves “significant improvements over existing attack baselines while largely preserving benign utility” (Reviewer C8em).

We also appreciate the concerns and suggestions raised by the reviewers. In response, we conducted the following additional experiments during rebuttal:

  • We evaluated ProAtk under explicit user instructions, showing that it remains effective in the non-proactive setting.
  • We re-evaluated the same proposals using three additional LLM judges, confirming that ProAtk maintains an advantage across all judges.
  • We evaluated approval under increasing disclosure of action details, showing that ProAtk retains high approval rates under action-level review.
  • We tested different rollout budgets across multiple random seeds, showing that ProAtk’s search outcomes remain stable.
  • We added comparisons on a shared clean-approved cohort, confirming that ProAtk retains its advantage.

In light of the additional experiments and clarifications, we believe our rebuttal provides substantive responses to the reviewers’ concerns, with the responding reviewer acknowledging that most of their concerns have been addressed. As follow-up feedback from some reviewers is still pending, we respectfully ask that our responses and additional evidence be considered in the final assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.