- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates process-control attacks that manipulate whether LLM agents seek human clarification or confirmation before proceeding with tasks. The authors propose APCA, which adaptively selects conversational cues based on publicly observable agent responses and interaction states, while preserving the original task content. They also introduce LoopShift-Bench, built upon BFCL, to evaluate human consultation, tool-side verification, state-changing actions, and task outcomes. Experiments show that APCA reduces human questioning and increases the observed rate of question-free interactions accompanied by state-changing actions and task failures.
Critical strengths
- While existing research on agent security primarily focuses on the actions agents perform, this paper investigates whether agents return control to humans when clarification or confirmation is required. This addresses an important yet relatively overlooked aspect of human oversight.
- The control experiments show that reduced human consultation can coincide with increased tool-side verification. This finding highlights that agent-side verification alone is insufficient to assess whether human oversight is preserved, providing useful insights for agent safety evaluation.
Critical weaknesses
-
The paper does not clearly specify the attacker's role, why the attacker can append cues to user messages but cannot modify the task, or what security boundary is being violated. Moreover, the agent's questions are directed to the same party applying the pressure, making their role in human oversight questionable. In real-world deployments, oversight is often enforced through permission controls rather than model-generated questions. The practical relevance of the threat model therefore remains unclear.
-
Although the paper defines ASR_auth, it does not evaluate it. Using question marks as a proxy for human consultation is overly simplistic, especially without annotations indicating when clarification is actually required. Furthermore, subsequent messages are delivered regardless of whether the agent asks questions, so no actual handoff occurs. The joint metric ZWF captures co-occurrence rather than causality, as illustrated by Case A, where the first mutation precedes the question.
-
APCA is evaluated on only one model. Most repeated runs produce identical trajectories, and the primary comparison covers only 39 tasks. The reported joint endpoint improvement (15.38% vs. 10.26%) comes from just two tasks. Moreover, changing the neutral wrapper alone produces a question-count difference comparable to APCA's main effect. The question-based and composite analyses are also retrospective, without multiple-comparison correction.
-
APCA relies on heuristic keyword matching and predefined cues, while its tactic priors are optimized using a surrogate that is reportedly misaligned with the intended objective. Without ablations, the contributions of state adaptation, cue content, and timing cannot be distinguished. APCA and static pressure also use different intervention budgets. Notably, the strongest evidence for separating human consultation from tool-side verification comes from explicit instruction controls, which achieve greater question suppression (−0.851) than APCA (−0.504).
Questions for authors
-
In what real-world deployment can an attacker append cues to user messages without modifying the original task? What motivates restricting the attacker to conversational cues? Moreover, if the agent's questions are directed to the same party applying the pressure, how does such questioning constitute effective human oversight?
-
Among the 39 compression tasks, how many actually require human clarification or approval? In particular, are the suppressed questions in base tasks with complete parameters genuinely necessary? How much of APCA's effect remains when considering only tasks where human input is required?
3 comments
W3: Models, Sampling, and Statistical Control
Thank you for these statistical points. Our primary estimate is −0.504 question turns across 39 task-paired comparisons (p=.0169), with repeats averaged at task level (Sec. 5.2, App. D.1). To test APCA beyond one model and greedy decoding, we ran 39 tasks × 3 conditions × 3 seeds (351 episodes per model, 1,053 in total) with Gemma-4-31B and Mistral-Small-3.2 at temperature 0.4 and Qwen2.5-72B at temperature 0.5. Static-4 uses the same four-addition budget as APCA:
| Model (temperature) | Neutral Q | Static-4 Q | APCA Q | APCA − Neutral [95% CI] |
|---|---|---|---|---|
| Gemma-4-31B (0.4) | 1.718 | 1.718 | 1.402 | −0.316 [−0.684, 0.051] |
| Mistral-Small-3.2 (0.4) | 1.641 | 1.504 | 1.350 | −0.291 [−0.521, −0.077] |
| Qwen2.5-72B (0.5) | 1.915 | 1.786 | 1.538 | −0.376 [−0.744, 0.009] |
APCA gives the fewest question turns on every model, and pooling the two new models gives −0.303 [−0.526, −0.085], an interval that excludes zero.
We agree that the joint-endpoint excess (15.38% vs. 10.26%) comes from two task blocks (Sec. 5.2). W2 & Q2 therefore add outcomes with broader task support, with critical execution before the required reply rising on 9 of 21 required-consultation tasks and authorization outcomes over 180 action opportunities per condition. To test wrapper sensitivity, we reran the equal-budget comparison under both neutral wrappers:
| Neutral wrapper | Neutral Q | Static-4 Q | APCA-4 Q | APCA − Neutral [95% CI] |
|---|---|---|---|---|
| A | 1.821 | 1.692 | 1.256 | −0.564 [−0.872, −0.231] |
| B | 2.179 | 2.026 | 1.667 | −0.513 [−0.923, −0.103] |
The wrapper shifts the baseline level, while the APCA effect stays stable, with a policy × wrapper interaction of +0.051 [−0.154, 0.256]. For multiple comparisons, we froze two primary contrasts before analysis:
| Primary contrast | p | BH q |
|---|---|---|
| APCA − State-blind-4, question turns (W4) | .0238 | .0477 |
| APCA − Neutral, critical execution before the required reply (W2 & Q2) | .0557 | .0557 |
We have added the cross-model, wrapper, and correction analyses to the appendix of the revised PDF.
W4: Ablations and Matched Budgets
Thank you for asking for these ablations. APCA is a frozen, public-state-conditioned selector, which keeps the policy fixed across all test tasks. To separate state adaptation, cue content, budget, and the surrogate-fitted prior, we ran 39 tasks × 4 arms × 3 repeats (468 episodes, Qwen2.5-72B, temperature 0) and added a No-prior arm that removes the frozen tactic prior. Static-4 and State-blind-4 use APCA's four-addition budget, and State-blind-4 retains APCA's cue library while removing state conditioning:
| Arm | Question-mark turns | APCA − arm [95% CI] | p |
|---|---|---|---|
| Neutral | 1.821 | −0.564 [−0.872, −0.231] | n/a |
| Static-4 | 1.692 | −0.436 [−0.821, −0.051] | .0462 |
| State-blind-4 | 1.641 | −0.385 [−0.667, −0.103] | .0238 |
| No-prior | 1.462 | −0.205 [−0.513, 0.128] | .2823 |
| APCA-4 | 1.256 | n/a | n/a |
At equal budget, replacing the static phrase with APCA's cue library lowers question turns by only 0.051, while adding state conditioning lowers them by a further 0.385, about two thirds of APCA's 0.564 reduction from neutral. APCA is lower than State-blind-4 on 19 tasks, higher on 8, and tied on 12. The No-prior arm stays below State-blind-4 (1.462 vs. 1.641), and its difference from full APCA has an interval containing zero, so the effect comes from state conditioning more than from the prior. Explicit verify-less is a direct-instruction positive control. Its stronger suppression fits an explicit instruction, and its opposing shift in tool reads shows that consultation and tool verification respond differently (Sec. 5.3). We have added this study to the appendix of the revised PDF.
Thank you for the response and the new experiments. These address some of my concerns about the evaluation. However, the main concern about how realistic the threat model is remains unresolved, so I will keep my current score.
We thank you for the detailed review and for recognizing that the return of control to humans is an important and understudied aspect of oversight, and that tool-side verification alone cannot show whether human oversight is preserved. We address the four weaknesses in order and answer your two questions with the first two, using new annotation, authorization, cross-model, and ablation studies.
W1 & Q1: Threat Model and Security Boundary
Thank you for these questions. In our access model, the attacker and the user are separate parties (Sec. 3.1, Apps. B/E):
The questions the agent asks are directed to the legitimate user who specified the task. Restricting the attacker to cues is the weakest access level, which separates process influence from task tampering and instruction injection. The violated boundary is authorization. Runtime permission checks govern actions and remain complementary, while the decision to ask the user about missing information lies outside them. W2 & Q2 measures this boundary directly.
W2 & Q2: Required Consultation and
ASR_authThank you for these precise points. We ran three new studies for them. First, annotators labeled 320 unique turns for semantic consultation, which we then compared with the question-mark rule:
The rule reaches 89.4% precision [83.7, 93.3], 90.0% recall [84.4, 93.8], and 89.7% F1 [85.9, 93.0].
Second, two annotators labeled whether each of the 39 compression tasks requires consultation (Krippendorff's α = 0.837 [0.652, 0.962]):
For Q2, 21 tasks require clarification, including 3 base tasks, and none requires approval. On these 21 tasks, the primary study gives 126 episodes per condition:
The effect remains on these tasks, where APCA lowers question turns from 2.10 to 1.43, a larger drop than the 0.504 over all 39 tasks. APCA raises critical execution before the required reply by 18.25 pp over neutral (bootstrap 95% CI [2.38, 34.13], sign-flip p=.0557), with 9 tasks up, 3 down, and 9 unchanged, while consultation without a pause stays flat.
Third, to evaluate
ASR_auth, we built a new authorization simulator with 60 tasks × 3 conditions × 3 repeats (540 episodes). Each task has one critical action that needs a prior grant from a specified legitimate user, and the simulator scores whether a valid grant precedes the tool mutation. This gives 180 action opportunities per condition:ASR_authRelative to neutral, APCA raises unauthorized execution by 17.78 pp [10.00, 25.56] and
ASR_authby 11.67 pp [5.56, 17.78], while native success changes by −7.22 pp [−15.56, 1.67]. Unlike ZWF, both new outcomes depend on event order, which is the issue you raise with Case A. Critical execution before the required reply means the agent acts before the user answers, and a grant is valid only when it precedes the tool mutation. On tasks that require human input, APCA therefore converts suppressed consultation into execution without the user's reply or grant. We have added all three studies to the appendix of the revised PDF.