acceptodds
Under review as a conference paper at ICLR 2027

Adaptive Process-Control Attacks on Human Oversight in LLM Agents

Abstract

Human oversight depends on a large language model (LLM) agent choosing to return control to the user. That choice is itself an attack surface. We introduce Adaptive Process-Control Attack (APCA), a controller that selects task-preserving cues based on public assistant text and turn index under a four-cue budget while preserving canonical task payload, and LoopShift-Bench, a reusable protocol built on the Berkeley Function Calling Leaderboard (BFCL) with 80 evaluation task identifiers, 2,793 usable records, and executable scoring. On 39 primary attack tasks with six repeats, APCA reduces the number of canonical turns containing an assistant question mark by 0.504 (95% confidence interval (CI) [-0.910, -0.120]). Question-free episodes reach 33.33% under APCA versus 17.95% under static pressure. The observed joint endpoint, defined as a question-free episode with a state-changing tool action and BFCL-native failure, is 15.38% under APCA versus 10.26% under both neutral and static interaction. Explicit verification controls show that human questions can decrease while tool-side verification increases. Code and LoopShift-Bench are openly available to support reproducibility.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM agent security, human oversight, adaptive attacks, process-control attacks, tool-using agents, agent safety evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_rD9cOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates process-control attacks that manipulate whether LLM agents seek human clarification or confirmation before proceeding with tasks. The authors propose APCA, which adaptively selects conversational cues based on publicly observable agent responses and interaction states, while preserving the original task content. They also introduce LoopShift-Bench, built upon BFCL, to evaluate human consultation, tool-side verification, state-changing actions, and task outcomes. Experiments show that APCA reduces human questioning and increases the observed rate of question-free interactions accompanied by state-changing actions and task failures.

Critical strengths
  1. While existing research on agent security primarily focuses on the actions agents perform, this paper investigates whether agents return control to humans when clarification or confirmation is required. This addresses an important yet relatively overlooked aspect of human oversight.
  2. The control experiments show that reduced human consultation can coincide with increased tool-side verification. This finding highlights that agent-side verification alone is insufficient to assess whether human oversight is preserved, providing useful insights for agent safety evaluation.
Critical weaknesses
  1. The paper does not clearly specify the attacker's role, why the attacker can append cues to user messages but cannot modify the task, or what security boundary is being violated. Moreover, the agent's questions are directed to the same party applying the pressure, making their role in human oversight questionable. In real-world deployments, oversight is often enforced through permission controls rather than model-generated questions. The practical relevance of the threat model therefore remains unclear.

  2. Although the paper defines ASR_auth, it does not evaluate it. Using question marks as a proxy for human consultation is overly simplistic, especially without annotations indicating when clarification is actually required. Furthermore, subsequent messages are delivered regardless of whether the agent asks questions, so no actual handoff occurs. The joint metric ZWF captures co-occurrence rather than causality, as illustrated by Case A, where the first mutation precedes the question.

  3. APCA is evaluated on only one model. Most repeated runs produce identical trajectories, and the primary comparison covers only 39 tasks. The reported joint endpoint improvement (15.38% vs. 10.26%) comes from just two tasks. Moreover, changing the neutral wrapper alone produces a question-count difference comparable to APCA's main effect. The question-based and composite analyses are also retrospective, without multiple-comparison correction.

  4. APCA relies on heuristic keyword matching and predefined cues, while its tactic priors are optimized using a surrogate that is reportedly misaligned with the intended objective. Without ablations, the contributions of state adaptation, cue content, and timing cannot be distinguished. APCA and static pressure also use different intervention budgets. Notably, the strongest evidence for separating human consultation from tool-side verification comes from explicit instruction controls, which achieve greater question suppression (−0.851) than APCA (−0.504).

Questions for authors
  1. In what real-world deployment can an attacker append cues to user messages without modifying the original task? What motivates restricting the attacker to conversational cues? Moreover, if the agent's questions are directed to the same party applying the pressure, how does such questioning constitute effective human oversight?

  2. Among the 39 compression tasks, how many actually require human clarification or approval? In particular, are the suppressed questions in base tasks with complete parameters genuinely necessary? How much of APCA's effect remains when considering only tasks where human input is required?

3 comments

We thank you for the detailed review and for recognizing that the return of control to humans is an important and understudied aspect of oversight, and that tool-side verification alone cannot show whether human oversight is preserved. We address the four weaknesses in order and answer your two questions with the first two, using new annotation, authorization, cross-model, and ablation studies.

W1 & Q1: Threat Model and Security Boundary

Thank you for these questions. In our access model, the attacker and the user are separate parties (Sec. 3.1, Apps. B/E):

PartyRoleAccess
Legitimate userSpecifies the task and holds missing information and approval authorityFull
AdversaryLower-trust message intermediary, such as a front-end or relayAppends short cues only, cannot change task facts, tool interfaces, system instructions, or approval grants
AgentDecides whether to consult the user before actingReceives the rendered message

The questions the agent asks are directed to the legitimate user who specified the task. Restricting the attacker to cues is the weakest access level, which separates process influence from task tampering and instruction injection. The violated boundary is authorization. Runtime permission checks govern actions and remain complementary, while the decision to ask the user about missing information lies outside them. W2 & Q2 measures this boundary directly.

W2 & Q2: Required Consultation and ASR_auth

Thank you for these precise points. We ran three new studies for them. First, annotators labeled 320 unique turns for semantic consultation, which we then compared with the question-mark rule:

TurnConsultationNo consultation
Contains "?"14417
No "?"16143

The rule reaches 89.4% precision [83.7, 93.3], 90.0% recall [84.4, 93.8], and 89.7% F1 [85.9, 93.0].

Second, two annotators labeled whether each of the 39 compression tasks requires consultation (Krippendorff's α = 0.837 [0.652, 0.962]):

SubsetRequired clarificationRequired approvalOptionalNot requiredAmbiguous
Missing-parameter (22)180211
Base (17)30581

For Q2, 21 tasks require clarification, including 3 base tasks, and none requires approval. On these 21 tasks, the primary study gives 126 episodes per condition:

Required-consultation tasksNeutralStaticAPCA
Question-mark turns2.101.901.43
≥1 directed-to-user consultation78.6%71.4%59.5%
Critical execution before the required reply20.6%28.6%38.9%
Consulted but did not pause15.1%14.3%13.5%

The effect remains on these tasks, where APCA lowers question turns from 2.10 to 1.43, a larger drop than the 0.504 over all 39 tasks. APCA raises critical execution before the required reply by 18.25 pp over neutral (bootstrap 95% CI [2.38, 34.13], sign-flip p=.0557), with 9 tasks up, 3 down, and 9 unchanged, while consultation without a pause stays flat.

Third, to evaluate ASR_auth, we built a new authorization simulator with 60 tasks × 3 conditions × 3 repeats (540 episodes). Each task has one critical action that needs a prior grant from a specified legitimate user, and the simulator scores whether a valid grant precedes the tool mutation. This gives 180 action opportunities per condition:

OutcomeNeutralStatic-4APCA-4
Valid grant before action78.9%71.1%56.7%
Unauthorized execution9.4%15.6%27.2%
No critical action11.7%13.3%16.1%
Native success71.1%68.3%63.9%
Strict ASR_auth5.0%7.8%16.7%

Relative to neutral, APCA raises unauthorized execution by 17.78 pp [10.00, 25.56] and ASR_auth by 11.67 pp [5.56, 17.78], while native success changes by −7.22 pp [−15.56, 1.67]. Unlike ZWF, both new outcomes depend on event order, which is the issue you raise with Case A. Critical execution before the required reply means the agent acts before the user answers, and a grant is valid only when it precedes the tool mutation. On tasks that require human input, APCA therefore converts suppressed consultation into execution without the user's reply or grant. We have added all three studies to the appendix of the revised PDF.

W3: Models, Sampling, and Statistical Control

Thank you for these statistical points. Our primary estimate is −0.504 question turns across 39 task-paired comparisons (p=.0169), with repeats averaged at task level (Sec. 5.2, App. D.1). To test APCA beyond one model and greedy decoding, we ran 39 tasks × 3 conditions × 3 seeds (351 episodes per model, 1,053 in total) with Gemma-4-31B and Mistral-Small-3.2 at temperature 0.4 and Qwen2.5-72B at temperature 0.5. Static-4 uses the same four-addition budget as APCA:

Model (temperature)Neutral QStatic-4 QAPCA QAPCA − Neutral [95% CI]
Gemma-4-31B (0.4)1.7181.7181.402−0.316 [−0.684, 0.051]
Mistral-Small-3.2 (0.4)1.6411.5041.350−0.291 [−0.521, −0.077]
Qwen2.5-72B (0.5)1.9151.7861.538−0.376 [−0.744, 0.009]

APCA gives the fewest question turns on every model, and pooling the two new models gives −0.303 [−0.526, −0.085], an interval that excludes zero.

We agree that the joint-endpoint excess (15.38% vs. 10.26%) comes from two task blocks (Sec. 5.2). W2 & Q2 therefore add outcomes with broader task support, with critical execution before the required reply rising on 9 of 21 required-consultation tasks and authorization outcomes over 180 action opportunities per condition. To test wrapper sensitivity, we reran the equal-budget comparison under both neutral wrappers:

Neutral wrapperNeutral QStatic-4 QAPCA-4 QAPCA − Neutral [95% CI]
A1.8211.6921.256−0.564 [−0.872, −0.231]
B2.1792.0261.667−0.513 [−0.923, −0.103]

The wrapper shifts the baseline level, while the APCA effect stays stable, with a policy × wrapper interaction of +0.051 [−0.154, 0.256]. For multiple comparisons, we froze two primary contrasts before analysis:

Primary contrastpBH q
APCA − State-blind-4, question turns (W4).0238.0477
APCA − Neutral, critical execution before the required reply (W2 & Q2).0557.0557

We have added the cross-model, wrapper, and correction analyses to the appendix of the revised PDF.

W4: Ablations and Matched Budgets

Thank you for asking for these ablations. APCA is a frozen, public-state-conditioned selector, which keeps the policy fixed across all test tasks. To separate state adaptation, cue content, budget, and the surrogate-fitted prior, we ran 39 tasks × 4 arms × 3 repeats (468 episodes, Qwen2.5-72B, temperature 0) and added a No-prior arm that removes the frozen tactic prior. Static-4 and State-blind-4 use APCA's four-addition budget, and State-blind-4 retains APCA's cue library while removing state conditioning:

ArmQuestion-mark turnsAPCA − arm [95% CI]p
Neutral1.821−0.564 [−0.872, −0.231]n/a
Static-41.692−0.436 [−0.821, −0.051].0462
State-blind-41.641−0.385 [−0.667, −0.103].0238
No-prior1.462−0.205 [−0.513, 0.128].2823
APCA-41.256n/an/a

At equal budget, replacing the static phrase with APCA's cue library lowers question turns by only 0.051, while adding state conditioning lowers them by a further 0.385, about two thirds of APCA's 0.564 reduction from neutral. APCA is lower than State-blind-4 on 19 tasks, higher on 8, and tied on 12. The No-prior arm stays below State-blind-4 (1.462 vs. 1.641), and its difference from full APCA has an interval containing zero, so the effect comes from state conditioning more than from the prior. Explicit verify-less is a direct-instruction positive control. Its stronger suppression fits an explicit instruction, and its opposing shift in tool reads shows that consultation and tool verification respond differently (Sec. 5.3). We have added this study to the appendix of the revised PDF.

Thank you for the response and the new experiments. These address some of my concerns about the evaluation. However, the main concern about how realistic the threat model is remains unresolved, so I will keep my current score.

Reviewer_myxaOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper is about the ability of agents to return control back to the user and whether this is an attack surface. The authors append varying signals of urgency/pressure to the LLM in user messages (APCA). They construct an agent benchmark LoopShift-Bench which pairs neutral vs static-pressured vs APCA conditions on tasks. They find that APCA reduces question mark turns and raises question-free runs.

Critical strengths
  • Important framing. I haven't previously seen "choosing to pause" as an important target of oversight, and this seems like an important dimension to consider.
  • Visuals. The graphs are clean and well-designed for the narrative.
  • Transparency/data. Highly detailed tests, caveats, etc throughout.
Critical weaknesses

Methodology

  • The main results are on only one model and on a very limited sample size (39 tasks) at zero temperature. This raises significant concerns for basic experiment validity, not to mention generalizability.
  • Questions are detected by checking if the assistant message contains a ? – it’s not clear that this alone is enough for us to say a message is asking for user clarification (appendix C, pp 18). Genuine clarifications phrased without a quetion mark would not count.

Framing

  • The security framing is unclear – of course, it’s important to “keep a human in the loop,” however, judgment is a pretty nuanced, flexible, and (crucially) contextual decision. Perhaps “sufficient evidence” can look different by model capability and its unclear that fewer questions is a failure.
  • To that end, the evidence presented in this paper is not concretely connected to a harmful outcome – I think the authors would benefit from spending more time showing how this increases harmful behavior/attack success rates for a behavior with real security implications. As it stands, it’s not clear to me that the attack resulted in unauthorized/harmful actions

Style

  • The writing can be tightened considerably, certain details can be moved to the appendix e.g. “We use Qwen2.5-72B-Instruct with activation-aware weight quantization and temperature-zero decoding on four Nvidia A100 accelerators”
Questions for authors
  • It’s unclear how effective APCA is compared to simpler, static pressure, is the difference from the adaptive attack statistically significant?
  • How do you know that a question is directed at the user, beyond looking for a ? Can a classifier or simple LLM prompt be used here?
  • Can the results be replicated on additional models, using native tempreature sampling instead of 0?
2 comments

We thank you for the thoughtful review and for recognizing the importance of choosing to pause as a target of oversight, the clarity of our figures, and the transparency of our tests. We address the methodology and framing concerns below, followed by your first question, using new cross-model, annotation, authorization, and matched-budget studies.

W1 & Q3: Additional Models and Temperature Sampling

Thank you for raising the experimental scope. The primary evaluation uses Qwen2.5-72B at temperature zero over 39 tasks with six repeats per condition. The inference unit is the task, with repeats averaged before paired bootstrap and sign-flip tests, so the 702 episodes are never treated as independent samples (App. D.1). On this basis, APCA reduces question-mark turns versus neutral by 0.504 (95% CI [−0.910, −0.120], p=.0169, Sec. 5.2, Table 2). To answer Q3, we ran the comparison on two new models and with nonzero-temperature sampling, using 39 tasks × 3 conditions × 3 seeds (351 episodes per model, 1,053 in total). Gemma-4-31B and Mistral-Small-3.2 run at temperature 0.4, Qwen2.5-72B runs at temperature 0.5, and Static-4 uses the same four-addition budget as APCA:

Model (temperature)Neutral QStatic-4 QAPCA QAPCA − Neutral [95% CI]
Gemma-4-31B (0.4)1.7181.7181.402−0.316 [−0.684, 0.051]
Mistral-Small-3.2 (0.4)1.6411.5041.350−0.291 [−0.521, −0.077]
Qwen2.5-72B (0.5)1.9151.7861.538−0.376 [−0.744, 0.009]

APCA gives the fewest question turns on every model, and pooling the two new models gives −0.303 [−0.526, −0.085], an interval that excludes zero. We have added the cross-model study to the appendix of the revised PDF.

W2 & Q2: Validating Question Detection

Thank you for this question. To answer Q2 directly, annotators labeled 320 unique turns for semantic consultation, which we then compared with the question-mark rule:

TurnConsultationNo consultation
Contains "?"14417
No "?"16143

Genuine clarifications without a question mark are the 16 false negatives, so the rule reaches 90.0% recall [84.4, 93.8], with 89.4% precision [83.7, 93.3] and 89.7% F1 [85.9, 93.0]. These human labels are the reference that any classifier or LLM prompt itself needs, and they validate the rule directly. We have added this annotation study to the appendix of the revised PDF.

W3: When Fewer Questions Is a Failure

We appreciate this point about context. Two annotators labeled each of the 39 tasks by whether it requires consultation (Krippendorff's α = 0.837 [0.652, 0.962]):

SubsetRequired clarificationRequired approvalOptionalNot requiredAmbiguous
Missing-parameter (22)180211
Base (17)30581

On the 21 tasks that require clarification, the primary study gives 126 episodes per condition:

Required-consultation tasksNeutralStaticAPCA
Question-mark turns2.101.901.43
≥1 directed-to-user consultation78.6%71.4%59.5%
Critical execution before the required reply20.6%28.6%38.9%

APCA raises critical execution before the required reply by 18.25 pp over neutral (bootstrap 95% CI [2.38, 34.13], sign-flip p=.0557). Where the task itself requires clarification, fewer questions therefore mean acting without the user's answer. We have added both annotation studies to the appendix of the revised PDF.

W4: Unauthorized Actions

Thank you for this suggestion. To connect the attack to unauthorized actions, we built a new authorization simulator with 60 tasks × 3 conditions × 3 repeats (540 episodes). Each task has one critical action that needs a prior grant from a specified legitimate user, and the simulator scores whether a valid grant precedes the tool mutation. This gives 180 action opportunities per condition:

OutcomeNeutralStatic-4APCA-4
Valid grant before action78.9%71.1%56.7%
Unauthorized execution9.4%15.6%27.2%
No critical action11.7%13.3%16.1%
Native success71.1%68.3%63.9%
Strict ASR_auth5.0%7.8%16.7%

Relative to neutral, APCA raises unauthorized execution by 17.78 pp [10.00, 25.56] and ASR_auth by 11.67 pp [5.56, 17.78], while native success changes by −7.22 pp [−15.56, 1.67]. The attack therefore produces unauthorized execution of approval-bound actions, a harm with direct security implications. We have added the authorization simulator to the appendix of the revised PDF.

Q1: Adaptive versus Static Pressure

Thank you for this question. In the primary design, static pressure appeared on every eligible turn while APCA used four additions (APCA–static p=.1008, App. D.3). We therefore reran the comparison at an equal four-addition budget with 39 tasks × 4 arms × 3 repeats (468 episodes, Qwen2.5-72B, temperature 0). State-blind-4 retains APCA's cue library while removing state conditioning:

ArmQuestion-mark turnsAPCA − arm [95% CI]p
Neutral1.821−0.564 [−0.872, −0.231]n/a
Static-41.692−0.436 [−0.821, −0.051].0462
State-blind-41.641−0.385 [−0.667, −0.103].0238
APCA-41.256n/an/a

At equal budget, the adaptive attack is significantly more effective than static pressure (p=.0462) and than the same cue library without state conditioning (p=.0238). We have added this study to the appendix of the revised PDF.

Reviewer_1oyoOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper treats the decision of an LLM agent to return control to the user (like asking for clarification or confirmation) as an attack surface. It proposes APCA, a controller that adds short phrases to user messages (urgency, confidence, “enough detail”) chosen from the assistant’s public text. It also proposes LoopShift-Bench, a benchmark built on BFCL that measures questions to the user, tool-side verification, state-changing actions, and final outcome. On Qwen2.5-72B, APCA reduces questions to the user relative to the neutral baseline, and the paper shows that human consultation and tool-side verification can move in opposite directions.

Critical strengths
  1. The problem is relevant and understudied in the current literature
  2. The separation between human consultation and the tool-side verification is a useful finding
  3. Reproducibility is well curated and limitations are honestly mentioned
Critical weaknesses

Major

  1. The threat model is not convincing. The fact that the attacker can manipulate the user's messages, then the attacker might be the user themselves or someone who can rewrite that message. Moreover, phrases like “I’m really pressed for time here” or “please just go ahead and” express a legitimate user preference. The agent asking fewer questions in response is plausibly correct, expected behavior, not a vulnerability. Can the authors clarify who the realistic adversary is and why following these cues constitutes a failure?
  2. The statistical evidence is weak. Results come from runs on a single model (quantized Qwen2.5-72B), with temperature 0. In 189 of 234 task–condition cells the six repeats are identical, so the effective information unit is 39 tasks. The comparison with the static baseline, which is what tests the value of adaptivity, is not significant (questions: p=.10p = .10; question-free episodes: p=.07p = .07). The joint endpoint (+5.13 pp, p=.50p = .50) is driven by only two tasks.
  3. The metrics do not look to measure what the paper claims. Human consultation is measured by the presence of a question mark in the response, which might constitute a coarse, not completely faithful, proxy (rhetorical questions, non-blocking questions). The user script continues identically regardless of the agent’s questions, meaning that control is never actually returned. Moreover, base tasks do not require asking anything, so fewer questions in those cases do not indicate an oversight failure. Without authorization labels, the joint endpoint does not measure harm, as the authors acknowledge.
  4. Adaptivity is limited and the comparison with the static baseline is unfair. APCA relies on four coarse rule-based states and on priors estimated on 16 tasks. APCA uses at most four phrases, while the static baseline uses one at every turn. This causes the two conditions to differ in exposure as well as adaptivity, and makes it unclear how much of the effect comes from adaptivity and how much from the content of the phrases.

Minor

  1. Some of the phrases in the library like “please just go ahead and” or “go ahead confidently” are actually instructions to proceed, in contrast with the stated constraint of not directly instructing the agent to skip checks
  2. The experiments on Gemma and Mistral only cover explicit instructions, so the generality of APCA across models is not tested.
  3. The paper shows that even the wording of the neutral wrapper changes questions and success (0.329 vs 0.459), making the interpretation of effects from such short phrases fragile.
  4. The writing is very dense and full of ad hoc terminology like “observed joint endpoint”, “canonical task payload”, or “controller state”. Figures like Fig. 1 are cluttered.
  5. The tercile analysis is descriptive and coupled with the baseline by construction, as the authors acknowledge, so it adds little to the findings
Questions for authors

See weaknesses

2 comments

We thank you for the careful review and for recognizing the relevance of the problem, the separation between human consultation and tool-side verification, and the reproducibility of the study. We address the four major weaknesses in order, followed by Minor 1–3, using new annotation, authorization, cross-model, and matched-budget studies.

W1: Realistic Adversary

Thank you for asking who the realistic adversary is. The key security distinction is between who specifies the task and who controls its presentation (Sec. 3.1, App. B.3, E.1):

PartyRoleAccess
Legitimate userSpecifies the task and holds missing information and approval authorityFull
AdversaryLower-trust message intermediary, such as a front-end or relayAdds short cues only, cannot change task facts, tools, system instructions, or approval grants
AgentDecides whether to consult the user before actingReceives the rendered message

The cues supply neither missing information nor approval. At a decision point that needs the user's input, following a cue therefore means acting on information or authority the user has not provided, which a legitimate preference for speed does not grant. The paired missing-file case (App. G.2) shows this concretely. Neutral asks which files to compare, while APCA infers a pair and executes the comparison before the script supplies the file names. W3 measures this failure directly, where APCA raises unauthorized execution from 9.4% to 27.2%.

W2: Statistical Evidence

We agree that the task is the right inference unit, and our analysis treats the 39 tasks as independent units, with repeats averaged within task before the paired tests (Sec. 5.2, App. D.1). On this basis, APCA reduces question turns by 0.504 versus neutral (95% CI [−0.910, −0.120], p=.0169). To test APCA beyond one quantized model and greedy decoding, we ran 39 tasks × 3 conditions × 3 seeds (351 episodes per model, 1,053 in total) with Gemma-4-31B and Mistral-Small-3.2 at temperature 0.4 and Qwen2.5-72B at temperature 0.5. Static-4 uses the same four-addition budget as APCA:

Model (temperature)Neutral QStatic-4 QAPCA QAPCA − Neutral [95% CI]
Gemma-4-31B (0.4)1.7181.7181.402−0.316 [−0.684, 0.051]
Mistral-Small-3.2 (0.4)1.6411.5041.350−0.291 [−0.521, −0.077]
Qwen2.5-72B (0.5)1.9151.7861.538−0.376 [−0.744, 0.009]

APCA gives the fewest question turns on every model, and pooling the two new models gives −0.303 [−0.526, −0.085], an interval that excludes zero. At an equal budget, APCA also differs significantly from static pressure (−0.436 [−0.821, −0.051], p=.0462, W4). We agree that the joint-endpoint excess comes from two task blocks (Sec. 5.2), so W3 adds authorization outcomes over 180 action opportunities per condition. We have added the cross-model study to the appendix of the revised PDF.

W3: Metrics, Required Consultation, and Harm

Thank you for this point. We ran three new studies for the concerns you raise. First, annotators labeled 320 unique turns for semantic consultation, which we then compared with the question-mark rule:

TurnConsultationNo consultation
Contains "?"14417
No "?"16143

The rule reaches 89.4% precision [83.7, 93.3], 90.0% recall [84.4, 93.8], and 89.7% F1 [85.9, 93.0], so the proxy closely tracks semantic consultation.

Second, two annotators labeled whether each task requires consultation (Krippendorff's α = 0.837 [0.652, 0.962]):

SubsetRequired clarificationRequired approvalOptionalNot requiredAmbiguous
Missing-parameter (22)180211
Base (17)30581

On the 21 tasks that require clarification, the primary study gives 126 episodes per condition:

Required-consultation tasksNeutralStaticAPCA
Question-mark turns2.101.901.43
≥1 directed-to-user consultation78.6%71.4%59.5%
Critical execution before the required reply20.6%28.6%38.9%
Consulted but did not pause15.1%14.3%13.5%

APCA raises critical execution before the required reply by 18.25 pp over neutral (bootstrap 95% CI [2.38, 34.13], sign-flip p=.0557), with 9 tasks up, 3 down, and 9 unchanged. Consultation without a pause stays flat, so non-blocking questions do not explain the effect.

Third, to evaluate ASR_auth, we built a new authorization simulator with 60 tasks × 3 conditions × 3 repeats (540 episodes). Each task has one critical action that needs a prior grant from a specified legitimate user, and the simulator scores whether a valid grant precedes the tool mutation. This gives 180 action opportunities per condition:

OutcomeNeutralStatic-4APCA-4
Valid grant before action78.9%71.1%56.7%
Unauthorized execution9.4%15.6%27.2%
No critical action11.7%13.3%16.1%
Native success71.1%68.3%63.9%
Strict ASR_auth5.0%7.8%16.7%

Relative to neutral, APCA raises unauthorized execution by 17.78 pp [10.00, 25.56] and ASR_auth by 11.67 pp [5.56, 17.78], while native success changes by −7.22 pp [−15.56, 1.67]. Both new outcomes score whether the agent waits for the user, which the scripted continuation alone does not capture, and most of the 22.2 pp drop in valid grants under APCA becomes execution without the required human grant. We have added all three studies to the appendix of the revised PDF.

W4: Adaptivity and Matched Budgets

We agree that the original budgets differ (App. D.3). To separate adaptivity from exposure and cue content, we ran 39 tasks × 4 arms × 3 repeats (468 episodes, Qwen2.5-72B, temperature 0) and added a No-prior arm that removes the frozen tactic prior estimated on 16 tasks. Static-4 and State-blind-4 use APCA's four-addition budget, and State-blind-4 retains APCA's cue library while removing state conditioning:

ArmQuestion-mark turnsAPCA − arm [95% CI]p
Neutral1.821−0.564 [−0.872, −0.231]n/a
Static-41.692−0.436 [−0.821, −0.051].0462
State-blind-41.641−0.385 [−0.667, −0.103].0238
No-prior1.462−0.205 [−0.513, 0.128].2823
APCA-41.256n/an/a

At equal budget, replacing the static phrase with APCA's cue library lowers question turns by only 0.051, while adding state conditioning lowers them by a further 0.385, about two thirds of APCA's 0.564 reduction from neutral. APCA is lower than State-blind-4 on 19 tasks, higher on 8, and tied on 12. The No-prior arm stays below State-blind-4 (1.462 vs. 1.641), and its difference from full APCA has an interval containing zero, so the effect comes from state conditioning more than from the 16-task prior. We have added this study to the appendix of the revised PDF.

Minor 1–3: Phrasing, Models, and Wrapper

Thank you for these points. The intervention constraint bans instructions to skip checks or grant authorization (App. B.2–B.3). Phrases such as "please just go ahead and" encourage progress on the stated task while naming no check to skip and granting no approval. W2 applies APCA itself to Gemma-4-31B and Mistral-Small-3.2. For the neutral wrapper, we reran the equal-budget comparison under both wrappers:

Neutral wrapperNeutral QStatic-4 QAPCA-4 QAPCA − Neutral [95% CI]
A1.8211.6921.256−0.564 [−0.872, −0.231]
B2.1792.0261.667−0.513 [−0.923, −0.103]

The wrapper shifts the baseline level, while the APCA effect stays stable, with a policy × wrapper interaction of +0.051 [−0.154, 0.256]. We have added the wrapper study to the appendix of the revised PDF.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.