- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors study modular LLM agents in which a planner, an executor, and a verifier pass generated task state to each other. They argue that injected text can enter this state, for example when the verifier writes an attacker target into the carry forward goal after the current action was already correct. They introduce StateLure, a probe that asks the agent to write an attacker value into state, and they measure string matches in four state fields across twelve open weight models, two synthetic environments, and three scaffolds. Adding a verifier raises exposure from 0.300 to 0.682 while the immediate attack success rate stays at 0.165, and a paired audit on five models shows that standard injection also reaches these fields. In 20 selected Mistral-7B episodes, the contaminated state causes wrong target actions in 11 of 20 cases, but only when the carry forward summary replaces the user goal as the task statement. The authors also present TrustState Runtime, a fixed target check that removes attacker targets from state using the known correct value.
Critical strengths
The paper points to a clear and practical blind spot in how injection attacks are scored. Most benchmarks only check the current action, but the authors show that a verifier can write the attacker target into state after the action, so the immediate success rate misses it. This matters to my assessment because it suggests that current agent security evaluations may undercount risk in systems with several modules.
The evaluation design is careful and fair. Conditions are paired by task, seed, model, scaffold, and environment, scoring is deterministic without an LLM judge, and the matched IPI baseline is selected on a separate pilot split. This makes me trust the exposure numbers as measured, even though I question what they imply.
Critical weaknesses
The evidence that exposed state causes later harm is thin, and this later harm is the central claim of the paper. The only downstream test uses 20 selected episodes from Mistral-7B in Office-60, and wrong actions appear only when the summary replaces the user goal, so the result is close to expected by design. Also, C and P count any string match, including quoted or rejected mentions, and the StateLure payload directly asks the verifier to rewrite the state, so high exposure is partly built into the probe.
The defense adds little beyond the measurement. TrustState Runtime receives the correct target and the exact attacker value from the benchmark oracle, so its zero success rate is close to guaranteed. The authors’ own checks show that it keeps stale targets in 7 of 16 revision cases and lets crafted alternative targets pass in all 100 schemas, so I do not see a usable defense here.
Questions for authors
How often does verifier written target state lead to wrong actions in unselected multi step rollouts, across more models and both environments?
In a manual check of a random sample of C and P hits, what fraction adopt the attacker target as task guidance, and what fraction only quote, reject, or deny it?
Do common agent frameworks such as LangGraph or AutoGen use a carry forward summary as the next task statement by default, or does the original user goal usually stay in context?