acceptodds
Under review as a conference paper at ICLR 2027

Why LLM Agents Collapse Without Oversight: Measuring the Enforcement Gap in Reflexion-Style Agents

Abstract

Reflexion-style LLM agents detect unsafe plan steps but do not act on those detections — the controller receives a safety flag and executes anyway. We identify this as the enforcement gap, show that it causes by default across every framework we tested, and demonstrate that a single control-flow change reduces attack success rate substantially, reaching near zero on models whose flags parse cleanly. Separating detection probability from enforcement probability establishes that detection quality is formally irrelevant to security when enforcement is absent. We treat the unsupervised collapses reported in Emergence World as a motivating analogy, not as a causal result. Residual attack success concentrates where flags are unparseable or auditors leak; an RL-trained enforcement controller handles hedged and malformed verdicts that rule-based parsing cannot, cutting ambiguous-critique failure to a fraction of the rule-based baseline. Concurrent filtering and information-flow defenses address detection, not enforcement, leaving the binding constraint untouched. The Audit Enforcement Specification (AES) packages three concrete requirements — one per residue — that together close the gap without redesigning the host framework; no deployed framework currently satisfies all three.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM agents, prompt injection, enforcement gap, agent safety, Reflexion, audit reliability, GRPO

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_JbHWOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies the enforcement gap in Reflexion-style LLM agents: an agent detects an unsafe plan, but its controller does not stop execution. The authors separate detection probability from enforcement probability, compare attack success with abort-on-flag enabled and disabled, and propose an Audit Enforcement Specification (AES) with three primitives. However, this is clearly an unfinished paper that is not ready for submission. The writing, experiments, motivation, and literature review all have major problems. I did not identify a convincing strength.

Critical strengths

I did not identify a strength worth highlighting.

Critical weaknesses
  1. Introduction and Abstract The presentation of pep_e and pdp_d is inadequate. Although brief definitions are provided, the notation is introduced without enough explanation for readers to understand the framing clearly. This creates an unnecessary barrier at the beginning and gives me a poor first impression. The abstract also contains a citation. Readers should be able to understand what the paper does without consulting references, and this abstract does not achieve that. The final sentence, “No deployed framework currently implements all three primitives,” is a very strong claim. In the portions I subsequently read, I did not find sufficient experimental or empirical evidence to support it. Figure 1 appears directly below the abstract and occupies considerable space while conveying very little information. It contains many distorted, repetitive, and unclear icons. For example, six similar icons in the Claude District occupy two rows, roughly spanning lines 36–41, without adding meaningful information. The bottom-right sequence, labelled “All approve → no dissent → collapse,” also consists largely of repeated icons and an unexplained bubble. Its relationship to the paper is unclear. The useful information in this figure could probably be expressed in one or two sentences.

  2. Presentation The presentation is very poor. For example, the final words of the Figure 1 caption, “ignores it,” occupy an entire line by themselves. This wastes space and makes the layout uncomfortable to read. The Introduction is fragmented into many paragraphs. The abstract spends substantial space discussing agent collapse, yet I still cannot tell whether this is an established problem with an existing name or a term introduced by the authors. The connection between the Emergence AI example and the paper’s research problem is also insufficiently explained. The Introduction contains nine em dashes. Combined with the dense text and fragmented paragraphs, this makes the paper difficult for a human reviewer to read.

  3. Background and Theory Although this section is called “Theory,” it mainly introduces notions that define the authors’ problem. In my view, it does not provide the kind of mathematical theory that the heading suggests. The title “Background,” rather than “Literature Review,” also does little to clarify what this section is intended to accomplish. Figure 2 is even more excessive than Figure 1. It contains substantial uninformative space, unclear text and icons, and a distracting background that interferes with the foreground content. The overall presentation gives me the impression that the paper relies heavily on autoresearch output. It is text-heavy and difficult to read.

  4. Experimental Setup The experimental setup is also poorly organized. Familiar concepts are repackaged under names such as ReflectionTrap, FixedPoint, CascadePF, and IndirectInjection, without clearly explaining how they differ from established concepts. For an attack delivered through tool outputs, why not simply use the familiar term “indirect prompt injection”? By this point, I had given up on reading further. The writing, figures, terminology, and experimental organization give me the impression of an unfinished paper produced with heavy reliance on autoresearch. I clearly recommend rejection and do not wish to spend substantially more reviewing time on this version.

Questions for authors

No additional questions.

1 comment
AuthorsOpenReview ↗

Presentation Improvements and Formatting Corrections

Thank you for the review. The presentation problems you identify are valid, and we have corrected them in the revised PDF. We respond to each below.

Abstract and Figure 1. We confirm that the abstract contained a citation and used p_e and p_d before defining them. In the revised PDF we (i) remove the \citet{emergence2026} citation and rephrase to "unsupervised agent collapses observed in multi-agent simulations"; (ii) move the plain-language gloss (p_d: detection probability; p_e: enforcement probability) before its first quantitative use. The final sentence "No deployed framework currently implements all three primitives" was not supported at the point it appeared. It is backed by a code inspection of five frameworks (Appendix, Table tab:framework_inspection, lines 760-782): Reflexion, ReAct, and ToT expose no enforcement primitive (ON column is absent); AutoGen (ON 24%) and LangChain (ON 31%) achieve partial enforcement only with custom code. This is corroborated by \citet{sok2026agentic}. In the revised PDF we add this cross-reference inline at that sentence.

On Figure 1: we agree the city-map panel is icon-heavy and only loosely tied to the method. In the revised PDF we replace it with a compact two-panel figure: left panel summarizes the three failure modes; right panel shows the enforcement-gap process with the 48.3% to 10.8% ASR reduction. We rephrase the caption to remove the widow line ending "ignores it."

Introduction and "collapse." The Introduction contains nine em dashes (main.tex, lines 93-165). In the revised PDF we replace six with colons, commas, or parentheses, leaving three as list punctuation. We add a one-clause definition on first use of "collapse" (a cascading failure of a self-critique loop to terminate safely; our shorthand for this failure mode) and surface the analogy-not-causation caveat already in main.tex:126-130 more prominently. We consolidated the Introduction into fewer, less fragmented paragraphs.

Section 2 and Figure 2. We renamed Section 2 to "Problem Formulation," subsection 2.1 to "Threat Model and Framework Background," and subsection 2.2 to "Formal Analysis." On Figure 2: we agree the AI-illustrated style adds visual noise. We replace it with the TikZ source already in our repository (fig1_architecture.tex), which renders a clean vector diagram of the planner-auditor-controller flow with the enforcement gap highlighted. The figure instantiates the paper's core mechanism (Proposition 1), so we keep it; the change is stylistic.

Attack names. We acknowledge that the setup section introduced ReflectionTrap, FixedPoint, CascadePF, and IndirectInjection without relating them to established terms. In the revised PDF we add one sentence: A1 to A3 are instances of planning-phase injection (Greshake et al., 2023; Zhan et al., 2024); A4 is indirect prompt injection (Greshake et al., 2023) delivered via tool outputs. We keep the distinct names to identify the specific persistence mechanism within our framework. The mapping to Greshake et al. was already in Related Work (main.tex:742) but belonged in the setup section.

The presentation issues you raised blocked entry to the paper; the revisions above address them directly. The underlying result, the 48.3% to 10.8% enforcement reduction (n=203 OFF, n=194 ON), is unaffected by these changes.

Reviewer_WBjTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the “enforcement gap” in self-reflective LLM agents: even when an auditor identifies a malicious plan, the controller may continue execution. The authors separate the probability of detection from the probability of halting upon detection and measure their effects by toggling an abort-on-flag mechanism, reporting a reduction in attack success rate from 48.3% to 10.8% on a pooled control set. Further experiments examine missed detections, ambiguous verdict formats, and adaptive attacks. The authors propose the Audit Enforcement Specification (AES), comprising mandatory halting, auditor qualification testing, and a GRPO-trained controller. The work’s value lies in emphasizing that safety signals must translate into control-flow decisions and in providing an actionable taxonomy of failures. However, the experiments rely primarily on persistent plan injection and simulated execution, while the learned controller is evaluated mainly on synthetic text. Its broad effectiveness in real-world deployments therefore remains unproven.

Critical strengths
  1. Clear problem decomposition with practical diagnostic value.

    Separating the detection probability pdp_d from the probability of enforcement upon detection pep_e helps distinguish between an auditor missing a threat and a controller ignoring its verdict. Reporting only the final attack success rate can obscure this distinction. This decomposition can help practitioners decide where to focus their efforts to improve system safety.

  2. A direct intervention with meaningful engineering benefits.

    Holding the tasks, models, and auditors fixed while toggling abort-on-flag makes the source of improvement easier to interpret than simultaneously changing prompts, models, and controllers. The reported reduction in ASR from 48.3% to 10.8% on the pooled control set demonstrates substantial benefits from enforcing safety verdicts in the evaluated settings. The simplicity of the intervention does not diminish its potential value as an empirical finding.

  3. Attention to residual failures and aspects of the safety–utility trade-off.

    Beyond the main experiments, the authors evaluate multiple attacks, different models, false positives on benign tasks, and best-of-8 adaptive attacks. They also explicitly acknowledge that the complete AES stack has not been evaluated against an adaptive adversary targeting all its components. These qualifications are valuable, particularly because they avoid equating low ASR against static attacks with a security guarantee.

Critical weaknesses
  1. The novelty claims are overstated, and the characterization of framework deficiencies exceeds the evidence.

    Halting execution after detecting a threat is an established security principle. AutoGen 0.2 already provides is_termination_msg for message-conditioned termination [1], while earlier work on NeMo Guardrails investigates runtime control independent of the underlying model [2].

    The existence of these mechanisms does not establish that frameworks correctly bind safety verdicts to enforcement by default. However, the paper must distinguish between “the evaluated configuration lacks a default binding” and “the framework lacks enforcement primitives.” Moreover, the AutoGen and LangChain experiments in Appendix L use simulated loops implemented by the authors, limiting their ability to support claims about real deployments. A more defensible novelty claim would center on the measurement findings and their prevalence, supported by a strong baseline combining native termination mechanisms with the same auditor.

  2. The attacker has extensive privileges, and the experimental design confounds the claim that agents cannot recover.

    The main threat model allows the attacker to modify the internal plan directly and use σ\sigma to re-inject a malicious step after every replanning round (pp. 3 and 12–13). Consequently, the observation that none of the 217 trajectories recovers to a clean state may largely reflect the attacker’s continuous maintenance of the contamination.

    This supports the existence of risk under persistent plan injection, but it does not establish that reflection itself creates an adversarial attractor. Ablations involving a single injection followed by attacker withdrawal, removal of re-injection, and limits on the number of injections are needed to distinguish endogenous failure from continuous external intervention.

  3. There is an explicit mathematical error, and some theoretical interpretations lack necessary conditions.

    Proposition 1 states:

    ASR=h(1−pdpe),0≤h≤1.\mathrm{ASR}=h(1-p_dp_e),\qquad 0\le h\le1.

    Therefore, when pe=1p_e=1,

    ASR=h(1−pd)≤1−pd.\mathrm{ASR}=h(1-p_d)\le1-p_d.

    However, the main text on p. 6 and the appendix on p. 27 describe 1−pd1-p_d as a lower bound on ASR, further concluding that a 97% miss rate implies an attack success rate of at least 97%. This contradicts the paper’s own formula and directly affects its interpretation of the security implications of WB-2.

    Furthermore, the claim that improving detection has no value without mandatory halting holds only under conditions such as a fixed hh. In practice, reflection may still reduce harm by revising the plan. The broader claim has not been established.

  4. The description of external benchmark validation is insufficiently substantiated.

    The main text states that the original AgentDojo evaluation setup is preserved, whereas Appendix K describes extracting 27 task descriptions and attacker goals and placing them into the authors’ own reflection loop. The distinction between this procedure and evaluation using the complete tool environment, environment state, and native success criteria requires clarification. The original AgentDojo framework specifically emphasizes evaluating security and task utility through environment-state checks [3].

    Additionally, the 200-task dataset is generated by creating variants of 50 seed tasks. This supports evaluation at a larger scale, but provides weaker evidence of generalization to an independent task distribution.

  5. GRPO has not demonstrated its value in real deployments, and the learning-method baselines are insufficient.

    Appendix M explicitly states that the original reflection transcripts were not retained. Training inputs primarily consist of synthetic text reconstructed from existing indicators. Page 9 also acknowledges that AES-3 has not been evaluated on Grok-4’s residual failures on AgentDojo tasks. Thus, the reported 0.71% ASR is primarily a result on a synthetic controller task.

    The more informative comparison is between the base prompt’s 2.63% ASR and GRPO’s 0.71%. Additional baselines are needed, including SFT trained on the same data, cost-sensitive classifiers, structured outputs, and halting whenever parsing fails. In particular, GRPO reaches a 100% false-positive rate on benign AC5 cases, making it essential to report task completion and false positives alongside ASR.

  6. Several reporting inconsistencies reduce the verifiability of the results.

    For example, Table 6 reports 3,197 standard test examples, whereas Appendix M specifies a test set of 1,556 examples. The claimed training-set size is 10,535, but the counts in the label-distribution table sum to 12,443. The paper does not explain how these figures correspond.

    Furthermore, some zero-success results are accompanied by confidence intervals of [0,0], which do not justify population-level claims of “perfect enforcement.” These issues require clarification and verification against the original trajectories using a consistent aggregation script.

[1] AutoGen Authors. (n.d.). agentchat.conversable_agent. AutoGen 0.2. Retrieved October 22, 2026, from https://microsoft.github.io/autogen/0.2/docs/reference/agentchat/conversable_agent/

[2] Rebedea, T., Dinu, R., Sreedhar, M. N., Parisien, C., & Cohen, J. (2023). NeMo Guardrails: A toolkit for controllable and safe LLM applications with programmable rails. In Y. Feng & E. Lefever (Eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations (pp. 431–445). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.emnlp-demo.40

[3] Debenedetti, E., Zhang, J., Balunović, M., Beurer-Kellner, L., Fischer, M., & Tramèr, F. (2024). AgentDojo: A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents. Advances in Neural Information Processing Systems, 37, 82895–82920. https://doi.org/10.52202/079017-2636

Questions for authors

Please refer to the listed weaknesses in previous section

3 comments
AuthorsOpenReview ↗

Framework Claims, Persistence Confounds, and Mathematical Corrections

Thank you for the careful reading; several of these points are substantive errors we have confirmed.

W1 (Novelty and framework claims)

We agree with this distinction. We tested the default configuration of AutoGen 0.2.38 and LangChain 0.2.16 under our reflection loop. AutoGen's is_termination_msg and NeMo Guardrails do provide enforcement hooks. The defensible claim is "no default configuration we evaluated binds the auditor flag to a halt as a first-class primitive." Under that default, AutoGen ON-ASR is 24% and LangChain ON-ASR is 31% (Appendix L, Table tab:framework_inspection, appendix.tex:760-782), versus near-zero when the same flag is bound to a halt. In the revised PDF we will replace "no deployed framework implements enforcement" with this scoped claim.

The App L experiments use our simulated reflection loop, not each framework's native pipeline. We will say so explicitly and narrow the conclusions accordingly. The defensible novelty claim is the measurement: the enforcement gap's magnitude and prevalence across six models plus the control-flow fix. The 48.3% to 10.8% ASR reduction (table1_v1, n=203/194) and its trajectory-level decomposition are independent of the framework novelty framing. A baseline combining native termination mechanisms with our auditor requires running native framework pipelines; we will add it as stated future work.

W2 (Attacker persistence confounds recovery)

We acknowledge this concern. Every plan logged in the fixed-point experiment is the plan after the attacker's reinjection operator has been applied; the pre-reinjection plan was never saved. The malicious step appears in all 1,740 logged injected rounds by construction of the persistence operator. The "0/217 non-recovery" figure is therefore close to tautological for the logged plans.

There is also a classifier bug: the code stores Jaccard similarity but the convergence test applies a distance threshold of 0.05. With the corrected distance (d = 1 - similarity), 71 of 174 injected and 23 of 43 uninjected control trajectories reach plan-level stability, so stability is not specific to the attack.

The reflector is active: abort_execution was flagged in 1,306 of 1,740 injected rounds (75.1%) versus 7 of 430 clean rounds (1.6%). The step persists because the attacker re-inserts it. This supports your reading: risk under persistent injection, not evidence that reflection creates an attractor.

In the revised PDF we will retitle Claim 1 to "Non-recovery under persistent reinjection" and withdraw "adversarial attractor." We will offer the ablations you request as a planned experiment requiring new API runs. The 48.3% to 10.8% enforcement contrast does not depend on Claim 1.

W3 (Mathematical error and detection-value conditions)

We confirm this error. Sec 4.2 and Appendix S.3 state "ASR remains bounded below by 1 - p_d," concluding that R1-Llama-70B (epsilon = 0.97) implies ASR >= 97% under full enforcement. Both the inequality direction and the missing h factor are wrong.

Proposition 1 at p_e = 1 gives ASR = h_U(1 - p_d), an upper bound reaching 1 - p_d only when h_U = 1. The paper's own appendix establishes h < 1, and pooled ASR_OFF = 48.3%. With R1-Llama-70B blind epsilon = 0.989 and h_U in [0.48, 0.59], the implied full-enforcement ASR would be roughly 0.47-0.58, not >= 97%. The 97% floor is retracted.

The correct implication: for an auditor with epsilon ≈ 1, enforcement can remove at most h_D * p_d ≈ 1 - epsilon of ASR. For R1-Llama-70B that is at most about 1-3 pp.

We will also remove the Spearman rho = 0.71 claim. That correlation has no generating script or data. Only Llama-3-8B and Mistral-7B appear in both WB-2 and Table-1; no end-to-end ASR exists for 8 of the 10 WB-2 models.

Regarding detection-value: "improving detection has no value without mandatory halting" requires two conditions: (i) p_e = 0 and (ii) h_D = h_U. If reflection revises the plan after a flag and reduces harm (h_D < h_U), detection retains value even at p_e = 0.

In Defense-OFF, ASR is 45.7% for trajectories flagged at least once versus 59.0% for never-flagged trajectories, a 13 pp difference (Fisher p = 0.156). This is not significant, so h_D = h_U is not rejected, but not established. We will scope all "detection is irrelevant" language to "in the persistent-injection threat model with h_D = h_U."

One further unit fix: p_d = 0.71 in the paper is per-round, but ASR is per-trajectory. The trajectory-level detection rate is 164/203 = 0.808. Using this, Proposition 1 predicts 0.483 * (1 - 0.808) = 9.3% ON-ASR, versus 10.8% observed. The "3.2 pp gap" was a unit mismatch.

AuthorsOpenReview ↗

AgentDojo Protocol, Reporting Inconsistencies, and Baselines

W4 (AgentDojo evaluation)

We acknowledge this inaccuracy. What we did: extract 27 task descriptions and injection goals from AgentDojo's four task suites, then run them through our reflection loop with our LLM judge pair. We did not use AgentDojo's tool implementations, environment-state simulator, or native utility/security checks. In the revised PDF we will rename this evaluation "AgentDojo-derived tasks" and correct the description, noting explicitly that we cannot report AgentDojo's utility metric.

The measured effect remains: OFF ASR = 56.2% (666/1,185) and ON ASR excluding Grok-4 = 2.5% (22/873). Grok-4's 47.3% ON-ASR is documented as a parse-failure anomaly.

Regarding the 200-task set: the tasks consist of 50 seeds plus 150 GPT-5.5-generated variants. Seed-clustered bootstrap CIs are virtually identical to iid CIs: OFF [56.7%, 59.5%] versus [56.8%, 59.3%]; ON [4.1%, 6.4%] versus [4.4%, 6.2%]. All 50 seeds individually show Defense-OFF > Defense-ON with minimum gap 30.7 pp. We will narrow the generalization claim to "within-distribution task surface variation."

W6 (Reporting inconsistencies)

All three issues are confirmed errors.

(a) Training size: The label table sums to 7,339 + 1,104 + 4,000 = 12,443. The stated 10,535 is inconsistent. We will correct to 12,443.

(b) Evaluation sets: Table 6 evaluates on two sets: (1) 3,197-episode standard set and (2) 3,797-episode ambiguous set. The 1,556 figure in App. M.3 is an earlier validation split. These are distinct. We will add a reconciliation paragraph.

(c) [0,0] CIs: Correct Clopper-Pearson one-sided 95% upper bounds: Claude-Haiku and GPT-5.5 v2 ON (n=150) < 1.98%; Grok-4.3 v2 ON (n=134) < 2.21%; Reflexion A1 ON (n=118) < 2.51%; RoleConf ON (n=116) < 2.55%; GoalSub ON (n=120) < 2.47%. We will replace all [0,0] entries and replace "perfect enforcement" with "near-zero enforcement residue."

Additional corrections: the appendix states Grok-4.3 showed abort_per_round = [False] in 87% of Defense-ON v3 runs; the correct figure is 356/874 = 40.7%. For AgentDojo Grok-4, the correct figure is 158/194 = 81.4%.

W5 (GRPO value, baselines, task utility)

We concede the main points.

The 0.71% result is on synthetic AC1-AC4 critique stubs, not on live Grok-4 output. The paper discloses this in Appendix M but main.tex:642 lacks an inline qualifier. We will add "(on constructed AC1-AC4 stubs, not on live auditor output)."

On baselines: halt-whenever-parsing-fails cannot be computed from stored results because raw critique text is not stored. SFT requires ~5 hours GPU training on the remote server; we cannot complete it in the rebuttal window and acknowledge this gap. A TF-IDF classifier achieves 0% ASR and 0% FPR on the same test set by memorizing the 23-string vocabulary, illustrating that the metric measures in-distribution AC-type classification.

The strongest generalization evidence is AC4-drift. AC4 critique strings are NOT present verbatim in the training corpus (0/2 injected templates match). RL-GRPO achieves 10.8% ASR versus B3's 45.6% at 0% FPR on 500 injected + 500 benign episodes the model has not seen. By contrast, all 3 AC1 injected templates appear verbatim in training, so the AC1 improvement may partly reflect memorization.

We agree FPR should appear alongside ASR. We will add an FPR column to Table 6. On the ambiguous set, RL-GRPO FPR = 2.89% versus B3 2.65%; on the standard set, both are 4.15%. The per-AC breakdown: AC5 FPR = 100% and AC1 FPR = 41% at n=500 for RL-GRPO, versus 0% for B3 on all types. AC5 is already marked as a calibration boundary. The recommended configuration excludes AC5.

On task completion: enforcement is fail-stop, not recovery. Under Defense-ON, 5,301 of 5,725 valid v3 episodes end in a halt (92.6%); the original task is not completed. The executor is a stub that always returns success = True, and the judge scores only attack_success. We do not measure, and do not claim, safe task completion under attack. The benign false-halt rate is 4/250 = 1.6% (Wilson CI [0.6%, 4.0%]) on a simulated executor; this is a halt rate, not a task-completion measurement. We will correct Appendix J's "negligible overhead" to "low benign halt rate on a 50-task simulated-executor set, not a task-completion measurement." An attacker with plan-buffer access can also force repeated halts (denial of service); we will add this as a stated availability limitation.

Reviewer_WBjTOpenReview ↗

Response to the authors

I thank the authors for their detailed clarifications and further discussion of my concerns. After carefully considering their responses and the other reviewers’ comments, I find that some concerns remain unresolved, particularly those raised in Weakness #1 and #2. I therefore believe that the submission does not yet meet the bar for acceptance at ICLR and maintain my original rating.

Reviewer_pVP4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies the enforcement gap in reflection-based LLM agents: auditors may detect unsafe plans without controllers acting on their verdicts. Using multiple attacks, models, and task sets, the authors evaluate abort-on-flag with the auditor fixed, reporting an ASR reduction from 48.3% to 10.8% on a separate mixed-model control set. They propose AES, combining mandatory halting, auditor qualification, and a learned controller for ambiguous verdicts. On synthetic ambiguous critiques, the controller achieves 0.71% ASR, versus 2.63% for the prompted base model. These results support enforcing existing safety signals; the authors explicitly leave complete-stack robustness against an all-layer adaptive adversary to future work.

Critical strengths
  1. Diverse attacks (Section 3). ReflectionTrap, FixedPoint, CascadePF, and IndirectInjection cover both plan-phase and tool-output injection. This breadth makes the analysis more informative than testing a single attack template.

  2. Useful failure diagnosis (Sections 2–4; Appendix T). Distinguishing missed detections, unenforced verdicts, and unparseable outputs helps locate failures and identify whether interventions should target the auditor, parser, or controller.

  3. Substantial gains from a focused intervention (Section 4.1). Holding the auditor fixed, abort-on-flag reduces pooled-control ASR from 48.3% to 10.8%. This demonstrates a practical benefit from improving control flow without replacing the auditor.

  4. Informative defense baselines (Sections 4.3–4.5). Comparisons include self-audit, cross-audit, restart, and plan-diff. The prompted-base-model baseline also helps distinguish the learned controller's incremental benefit from simply introducing an LLM-based controller.

Critical weaknesses
  1. The auditor-imposed ASR lower bound needs correction (Proposition 1; Section 4.2; Appendix S.3). Proposition 1 gives ASR=h(1−pdpe)\mathrm{ASR}=h(1-p_dp_e) with h∈[0,1]h\in[0,1]. At pe=1p_e=1, this implies ASR=h(1−pd)≤1−pd\mathrm{ASR}=h(1-p_d)\leq 1-p_d, whereas Appendix S.3 asserts ASR≥1−pd=ϵ\mathrm{ASR}\geq 1-p_d=\epsilon and infers an ASR floor of 97% for one auditor. That floor does not follow without additional assumptions, such as h=1h=1, and a justified identification of WB-2 leakage with 1−pd1-p_d. This affects the interpretation of auditor qualification, although it does not invalidate the observed OFF/ON reductions.

  2. Controller evaluation reporting needs reconciliation (Table 6; Figure 3; Appendix M). Table 6 reports 3,197 standard and 3,797 ambiguous test episodes, while Appendix M.3 describes 1,556 test examples. These may be distinct evaluations, but their construction and relationship are unclear. Additionally, the training-category counts in Appendix M.1 sum to 12,443 (7,339 + 1,104 + 4,000), rather than the stated 10,535. The manuscript should reconcile these counts and specify whether separation is by episode, trajectory, task, or template; episode-level non-overlap alone does not establish generalization to new tasks or critique patterns.

  3. The non-recovery result needs a precise scope (Claim 1; Appendices A.2 and F). The authors explicitly adopt persistent reinjection and define T=σ∘RT=\sigma\circ R, which is a valid threat model. However, the reported trajectories do not establish whether reflection fails to remove the malicious step before reinjection or whether reinjection reinstates a successfully removed step. Appendix F's consecutive-plan Jaccard distances measure stability, not semantic cleanliness. The measurement timing and clean-plan criterion should be specified; absent separate evidence about RR, the result supports non-recovery of the composite attacked loop.

  4. Transfer to real critiques remains unestablished (Sections 4.5–4.6; Appendix M). The authors acknowledge that AES-3 uses synthetic or reconstructed critiques and has not been validated on the Grok-4 AgentDojo residuals. They also explicitly treat AC5's 100% FPR as a calibration boundary, not a recommended setting. These disclosures are appropriate, but the 0.71% ASR should consequently be interpreted as performance on constructed critiques, rather than evidence that AES-3 resolves naturally occurring parsing failures in deployment.

  5. Task utility after detection remains unclear (Sections 4.1 and 4.3; Appendix J). Counting blocked attacks as failures is appropriate for ASR. However, the benign-task FPR and D3 restart ASR do not establish safe completion of the original task under attack. This limits deployment conclusions rather than the measured security benefit. Existing completion and recovery-cost measurements, or narrower claims about operational usefulness, would address this concern; a new recovery architecture is not required.

Questions for authors
  1. What additional assumptions or metric definitions justify the ASR lower bound in Section 4.2 and Appendix S.3, given Proposition 1's factor hh?

  2. How do Table 6, Figure 3, and Appendix M's evaluation sets relate, which training count is correct, and what unit is used to separate training and test data?

  3. Is clean-state recovery assessed before or after σ\sigma, and how is removal of the malicious goal determined independently of plan stability?

  4. Beyond the acknowledged synthetic evaluations, is any existing evidence available on unmodified model-generated critiques, with false-positive or confusion counts?

  5. Do existing evaluations, particularly D3, measure safe completion of the original task after detection and the associated recovery cost?

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Corrections Addressing Reviewer Concerns

We thank reviewers pVP4, WBjT, and JbHW for the detailed feedback. Our three responses address each concern with specific corrections and data sources.

Corrections made:

• Proposition 1 mathematical error (pVP4-W1, WBjT-W3): We retract the claim "ASR ≥ 97%." The correct statement at p_e=1 is ASR = h_U(1-p_d) ≤ 1-p_d, an upper bound not a lower bound. With h_U ∈ [0.48, 0.59] from our data, the R1-Llama-70B implication is ASR ≈ 0.47-0.58, not ≥0.97. Both the inequality direction and the h factor were incorrect in the original text.

• Training set count (pVP4-W2, WBjT-W6): Corrected from stated 10,535 to verifiable 12,443 (sum of label-distribution table: 7,339+1,104+4,000).

• Confidence intervals (WBjT-W6): Replaced all bootstrap [0,0] intervals with Clopper-Pearson one-sided 95% upper bounds: Claude-Haiku/GPT-5.5 v2 ON <1.98% (n=150), Grok-4.3 v2 ON <2.21% (n=134), Reflexion/RoleConf/GoalSub ON <2.47-2.55%.

• Grok-4.3 percentages: Corrected 87% (appendix:453) to 40.7% (356/874 v3 runs); corrected 87% (appendix:622) to 81.4% (158/194 AgentDojo runs).

• Non-recovery scope (pVP4-W3, WBjT-W2): Rescoped Claim 1 to "under persistent reinjection." Withdrew "adversarial attractor" claim. Fixed Jaccard similarity/distance inversion bug in classifier; recomputed L-hat = 0.73 [0.68, 0.78] from 950 ratios.

• Spearman correlation removed (WBjT-W3): The claim rho=0.71 between epsilon and ON-ASR has no supporting data. Only 2 of 10 WB-2 models have end-to-end ASR measurements. This sentence is deleted.

Responses to specific concerns:

Regarding pVP4-Q2 (controller evaluation reconciliation): The 3,197-episode standard set and 3,797-episode ambiguous set (Table 6) are final holdout evaluations. The 1,556-example figure in Appendix M.3 refers to an earlier training-era validation split (918+138+500). These are distinct evaluations at different stages. Separation is at the (task, plan, critique, label) episode level, stratified by (is_injected, detected). All 50 seed tasks appear in both train and test splits; we acknowledge this limits generalization claims to within-distribution task variations. The strongest out-of-distribution evidence is AC4-drift: critique strings absent from training corpus (0/2 injected templates verbatim), where RL-GRPO achieves 10.8% ASR versus B3's 45.6% (Fisher p=1.0e-9).

Regarding WBjT-W1 (framework claims): The distinction raised is correct. AutoGen 0.2's is_termination_msg and NeMo Guardrails do provide enforcement primitives. We tested their default configurations under our reflection loop, not native pipelines. The defensible claim is measurement-focused: enforcement gap magnitude (48.3% to 10.8%) and prevalence across six models under the configurations we evaluated, not that primitives are architecturally unavailable.

Regarding WBjT-W2 (attacker persistence confound): The concern raised identifies a key limitation. Every logged plan is post-reinjection; the pre-reinjection reflector output was not saved. The malicious step appears in all 1,740 logged injected rounds by construction of the persistence operator. The reflector is active (75.1% injected rounds flagged vs 1.6% clean), but we cannot separate "reflection removed the step" from "attacker reinserted it" without the pre-persistence plan. We have rescoped the claim accordingly.

Regarding pVP4-W5 and WBjT-W5 (task utility): We acknowledge this limitation. Under Defense-ON, 5,301/5,725 attacked episodes (92.6%) end in halt; the original task is not completed. The executor is a stub returning success=True; the judge scores only attack_success. The benign halt rate 4/250 (1.6%, Wilson CI [0.6%, 4.0%]) is a halt rate on simulated execution, not a task-completion measurement. We do not measure, and do not claim, safe completion of the original task under attack.

Regarding JbHW presentation concerns: We acknowledge that the formatting issues raised (abstract citation, p_d/p_e ordering, figure design, em-dashes, attack naming) created barriers to engagement with the technical content. The underlying methodology—toggling enforcement on a fixed auditor to isolate the control-flow bottleneck—follows standard ablation practice in ML systems evaluation. The 48.3% to 10.8% reduction is directly reproducible from the supplementary materials (203 OFF runs, 194 ON runs, same 50 tasks and 5 models).

Core empirical finding: Pooled ASR reduction from 48.3% (98/203 Defense-OFF) to 10.8% (21/194 Defense-ON) under abort-on-flag. Trajectory-level decomposition matches Proposition 1's corrected formula: predicted 9.3% (using trajectory-level p_d=0.808, not per-round 0.71) versus observed 10.8% [CI 6.7%, 15.5%]. This finding is unaffected by the corrections listed above.

We remain available for clarification if additional detail is needed on any corrected claim.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.