acceptodds
Under review as a conference paper at ICLR 2027

When Context Shapes the Task: Red-Teaming Proactive Agents at Task Formulation

Abstract

Proactive agents infer what assistance to offer from context without an explicit user task. In the absence of such a task, the observations they consume determine not only how to act, but also what task to propose. This shifts a security boundary upstream: attacker-controlled context can influence task formulation before planning or execution begins. Existing agent red-teaming largely assumes that the task is fixed independently of such context and targets how that task is executed. We instead study whether an attacker with partial observations can manipulate task formulation to insert a target action into the completion path while preserving user acceptance of the resulting task description. To expose this vulnerability, we introduce ProAtk, a red-teaming framework based on task grafting. ProAtk infers the underlying user task from attacker-visible context and constructs a target-compatible requirement into its completion path. It uses Pareto search to preserve the underlying task while making the target action part of its completion, then distills successful grafts into a search-free attacker. Across three victim agents and four task categories, ProAtk consistently achieves the strongest joint attack performance while largely preserving benign utility of the victim agent. Human evaluation further corroborates that its target-bearing proposals remain useful and acceptable. Code is available at https://anonymous.4open.science/r/ProAtk-ED87.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.