- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies the enforcement gap in Reflexion-style LLM agents: an agent detects an unsafe plan, but its controller does not stop execution. The authors separate detection probability from enforcement probability, compare attack success with abort-on-flag enabled and disabled, and propose an Audit Enforcement Specification (AES) with three primitives. However, this is clearly an unfinished paper that is not ready for submission. The writing, experiments, motivation, and literature review all have major problems. I did not identify a convincing strength.
Critical strengths
I did not identify a strength worth highlighting.
Critical weaknesses
-
Introduction and Abstract The presentation of and is inadequate. Although brief definitions are provided, the notation is introduced without enough explanation for readers to understand the framing clearly. This creates an unnecessary barrier at the beginning and gives me a poor first impression. The abstract also contains a citation. Readers should be able to understand what the paper does without consulting references, and this abstract does not achieve that. The final sentence, “No deployed framework currently implements all three primitives,” is a very strong claim. In the portions I subsequently read, I did not find sufficient experimental or empirical evidence to support it. Figure 1 appears directly below the abstract and occupies considerable space while conveying very little information. It contains many distorted, repetitive, and unclear icons. For example, six similar icons in the Claude District occupy two rows, roughly spanning lines 36–41, without adding meaningful information. The bottom-right sequence, labelled “All approve → no dissent → collapse,” also consists largely of repeated icons and an unexplained bubble. Its relationship to the paper is unclear. The useful information in this figure could probably be expressed in one or two sentences.
-
Presentation The presentation is very poor. For example, the final words of the Figure 1 caption, “ignores it,” occupy an entire line by themselves. This wastes space and makes the layout uncomfortable to read. The Introduction is fragmented into many paragraphs. The abstract spends substantial space discussing agent collapse, yet I still cannot tell whether this is an established problem with an existing name or a term introduced by the authors. The connection between the Emergence AI example and the paper’s research problem is also insufficiently explained. The Introduction contains nine em dashes. Combined with the dense text and fragmented paragraphs, this makes the paper difficult for a human reviewer to read.
-
Background and Theory Although this section is called “Theory,” it mainly introduces notions that define the authors’ problem. In my view, it does not provide the kind of mathematical theory that the heading suggests. The title “Background,” rather than “Literature Review,” also does little to clarify what this section is intended to accomplish. Figure 2 is even more excessive than Figure 1. It contains substantial uninformative space, unclear text and icons, and a distracting background that interferes with the foreground content. The overall presentation gives me the impression that the paper relies heavily on autoresearch output. It is text-heavy and difficult to read.
-
Experimental Setup The experimental setup is also poorly organized. Familiar concepts are repackaged under names such as ReflectionTrap, FixedPoint, CascadePF, and IndirectInjection, without clearly explaining how they differ from established concepts. For an attack delivered through tool outputs, why not simply use the familiar term “indirect prompt injection”? By this point, I had given up on reading further. The writing, figures, terminology, and experimental organization give me the impression of an unfinished paper produced with heavy reliance on autoresearch. I clearly recommend rejection and do not wish to spend substantially more reviewing time on this version.
Questions for authors
No additional questions.
Presentation Improvements and Formatting Corrections
Thank you for the review. The presentation problems you identify are valid, and we have corrected them in the revised PDF. We respond to each below.
Abstract and Figure 1. We confirm that the abstract contained a citation and used p_e and p_d before defining them. In the revised PDF we (i) remove the \citet{emergence2026} citation and rephrase to "unsupervised agent collapses observed in multi-agent simulations"; (ii) move the plain-language gloss (p_d: detection probability; p_e: enforcement probability) before its first quantitative use. The final sentence "No deployed framework currently implements all three primitives" was not supported at the point it appeared. It is backed by a code inspection of five frameworks (Appendix, Table tab:framework_inspection, lines 760-782): Reflexion, ReAct, and ToT expose no enforcement primitive (ON column is absent); AutoGen (ON 24%) and LangChain (ON 31%) achieve partial enforcement only with custom code. This is corroborated by \citet{sok2026agentic}. In the revised PDF we add this cross-reference inline at that sentence.
On Figure 1: we agree the city-map panel is icon-heavy and only loosely tied to the method. In the revised PDF we replace it with a compact two-panel figure: left panel summarizes the three failure modes; right panel shows the enforcement-gap process with the 48.3% to 10.8% ASR reduction. We rephrase the caption to remove the widow line ending "ignores it."
Introduction and "collapse." The Introduction contains nine em dashes (main.tex, lines 93-165). In the revised PDF we replace six with colons, commas, or parentheses, leaving three as list punctuation. We add a one-clause definition on first use of "collapse" (a cascading failure of a self-critique loop to terminate safely; our shorthand for this failure mode) and surface the analogy-not-causation caveat already in main.tex:126-130 more prominently. We consolidated the Introduction into fewer, less fragmented paragraphs.
Section 2 and Figure 2. We renamed Section 2 to "Problem Formulation," subsection 2.1 to "Threat Model and Framework Background," and subsection 2.2 to "Formal Analysis." On Figure 2: we agree the AI-illustrated style adds visual noise. We replace it with the TikZ source already in our repository (fig1_architecture.tex), which renders a clean vector diagram of the planner-auditor-controller flow with the enforcement gap highlighted. The figure instantiates the paper's core mechanism (Proposition 1), so we keep it; the change is stylistic.
Attack names. We acknowledge that the setup section introduced ReflectionTrap, FixedPoint, CascadePF, and IndirectInjection without relating them to established terms. In the revised PDF we add one sentence: A1 to A3 are instances of planning-phase injection (Greshake et al., 2023; Zhan et al., 2024); A4 is indirect prompt injection (Greshake et al., 2023) delivered via tool outputs. We keep the distinct names to identify the specific persistence mechanism within our framework. The mapping to Greshake et al. was already in Related Work (main.tex:742) but belonged in the setup section.
The presentation issues you raised blocked entry to the paper; the revisions above address them directly. The underlying result, the 48.3% to 10.8% enforcement reduction (n=203 OFF, n=194 ON), is unaffected by these changes.