acceptodds
Under review as a conference paper at ICLR 2027

TEMPO An Injection-Independent Check–Use Race Condition in Tool-Augmented LLM Agents

Abstract

Tool-augmented large language model agents are typically secured via a check- then-use paradigm: agents verify a shared entity (recipient address, file contents, inbox event) before executing downstream actions. We show that the inherent la- tency between check and use creates an exploitable race window: an adversary that mutates the underlying shared state within this window causes the agent to act on a value it never verified. We formalize this vulnerability class as TEMPO, and show it is mechanistically orthogonal to prompt injection: while injection ma- nipulates the content/semantic layer, TEMPO exploits the state/temporal layer. A content-robust system prompt that reduces prompt injection success from 91.7% to 12.5% leaves TEMPO attack success unchanged at 100%. Across eight vic- tim models, two adversary types, and three race primitives, TEMPO achieves 100% attack success against reconnaissance-equipped attackers. The vulnerability transfers seamlessly to a disjoint payment approval workflow and reproduces on a production-grade LangGraph+SQLite stack with an out-of-process adversary. Benchmarking standard defense primitives alongside their latency costs reveals highly non-portable guard coverage: the same version check blocks 98.7% of attacks on one victim but only 7.1% on another. Finally, an adaptive re-arming at- tacker is strictly dominated by the simpler open-loop baseline in all configurations where the open-loop attack succeeds.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM agent security、prompt injection、TOCTOU / check–use race、tool-using agents、agent robustness

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_soSvOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TEMPO studies injection-independent time-of-check-to-time-of-use (TOCTOU) attacks against tool-using LLM agents. An adversary modifies shared state after validation but before execution, causing subsequent actions to use different data. The paper introduces three attack primitives, evaluates multiple victim models and defenses, and reproduces attacks with an independent-process adversary in LangGraph. Its central finding is that content-level safeguards cannot ensure state consistency, while execution-level defenses depend on object binding and protection scope.

Critical strengths

The paper distinguishes content safety from state consistency. Matched experiments show that content-hardening measures reduce prompt-injection success but provide little protection against the corresponding TOCTOU attacks. This offers useful evidence that agent security requires safeguards at the execution layer.

Critical weaknesses
  1. The causal interpretation of the defense results is insufficiently isolated. Section 4.4 attributes cross-model variation primarily to execution timing, but the appendices document additional mechanisms, including missing-snapshot allow paths, snapshot re-baselining, and independent refusal by use tools. Some successful attacks under CAS involve retries that adopt attacker-modified state after CAS correctly rejects the initial attempt. These outcomes also depend on snapshot management and authorization policies. Although the paper discloses these mechanisms, their contributions are not sufficiently separated to support the timing-based explanation.

  2. Key statistics supporting the defense conclusions are inconsistent. Table 6 reports a locking ASR of 66.7%, whereas the text describes 66.7% protection and attributes all residual failures to delayed-context attacks. These statements do not align under the balanced three-primitive evaluation. Table 16 also leaves unresolved whether the guard-trigger count is 300 or 450. Since defense analysis is a central contribution, consistent accounting is necessary to assess the conclusions reliably.

Questions for authors
  1. Could you provide a complete successful attack trace under CAS, identifying each state modification, validation, snapshot update, and final use? Did the final action use a state different from the one checked during the last successful validation?

  2. Could you reconcile the locking results and guard-trigger counts using consistent per-run records, and separately report genuine guard interventions, missing-snapshot allow paths, snapshot re-baselining, and independent tool refusals?

2 comments
AuthorsOpenReview ↗

Response to Reviewer so5V: timing controls, CAS trace, and locking accounting

We thank the reviewer; both weaknesses are correct.

W1. The weakness is correct: §4.4 attributed cross-model variation to execution timing while the appendices separately documented missing-snapshot paths, snapshot re-baselining, and independent refusal by the use tool. A controlled experiment separates them. Same task, same mutation, same initial state; only the write time varies — pre_check, check_use (after the check returns, before the guard), post_use (after the action record is written) — crossed with four defenses, 30 runs per cell, 360 runs total, zero crashes and zero censoring:

armfirst check reads attackerfirst use blocked by guardfirst action to attackerlater action to attacker
pre_check (all four defenses)30/300/3030/300/30
check_use / none0/300/3030/3030/30
check_use / version, recheck, CAS0/3030/3030/3030/30
post_use (all four defenses)0/300/300/3030/30

All 12 cells reach ASR 100%. The data support the causal statement: "The variance is not in the rule logic; it is in what happens after a rejection: whether the victim retries, whether the retry re-baselines the snapshot, and whether the surrounding tool independently authorizes the action." Timing is necessary for the window to exist; it does not decide the outcome.

W2. The weakness is correct: the locking figure was described as a protection rate while the table reports ASR, and the guard-trigger count was left ambiguous. The reconciliation, the per-family split and the category accounting are under Q2.

Q1. Complete trace, check_use/cas, trial 0:

  1. Validation. verify_recipient reads contacts.Bob at version 0, source seed, email [email protected].
  2. State modification. The attacker writes [email protected] at version 1, source attacker, after the check returns and before the guard.
  3. CAS rejects the first use. The guard compares version 1 against the snapshot version 0, raises cas-version-changed, and the use returns {"status": "blocked", "reason": "cas-version-changed"}.
  4. Snapshot update. The agent re-checks; verify_recipient now returns version 1, source attacker, and the defense stores the new snapshot.
  5. Final use. The retry compares version 1 against version 1, CAS allows it, and send_email returns {"status": "sent", "email": "[email protected]", "version_read": 1, "source_read": "attacker"}.

Direct answer. The final action used version 1, the state of the last successful validation — so it is not different from it; it differs from the first validation (version 0). CAS correctly rejected the first attempt; the retry re-validated on the attacker's state, so the comparison that mattered was v1 == v1. The failure is snapshot re-baselining, not a missing atomicity check.

Q2. Locking results and accounting.

Here H = 16 genuine holds; Rj = 300 runs with at least one rejection; H/Rj = 16/300 = 5.33% is a post-selection diagnostic. NRP = BP × (1 − ASR), with BP the benign-task pass rate.

locking armnASRprotection = 1 − ASRNRP
raw payload9060/90 = 66.7% [56.4, 75.5]33.3%0.333
covert payload9030/90 = 33.3% [24.5, 43.6]66.7%—

Residual failures are not in delayed-context:

race familyexploited / runs
delayed-context0/30
shared-state30/30
event-injection30/30

Guard trigger: the guard fires in all 450 runs; 300 is the number of runs with at least one rejection. Appendix B's "300 of those 450" therefore mislabels its denominator, and H/Rj is conditioned on a post-treatment event — a diagnostic, never compared across levels.

Category accounting, per run:

categorycount
genuine guard interventions16
missing-snapshot allow paths0
re-baselined → later exploited268
re-baselined → later refused by the tool16
never rejected → exploited150
total450

The 16 tool refusals are the refused subset of the 284 re-baselined runs (268 + 16), not an additional category on top of the 284. The guard rejects at least one use in 66.7% of runs (300/450). Of those 300 rejected runs, 16 (5.33%) end in a genuine hold and another 16 (5.33%) are refused by the use tool; 268 are exploited after a retry. That conversion — 66.7% rejection coverage → 5.33% genuine holds — is the quantitative form of "the guard fires but does not help".

Thank you for the detailed clarifications and additional results. My concerns have been largely addressed, and I am inclined to raise my overall rating to 3.

Reviewer_miVfOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper shows that the latency between check and use in tool-augmented LLM agents creates a race window in which an adversary can mutate shared state so that the agent acts on an unverified value. It formalizes this vulnerability as TEMPO and shows that it is mechanistically orthogonal to prompt injection, since content hardening substantially reduces injection success without affecting TEMPO. Across diverse victim models, adversary types and race primitives, TEMPO achieves consistently high attack success and also transfers to a payment workflow and a LangGraph deployment. The authors benchmark standard defenses and find highly non-portable coverage, while adaptive re-arming attackers do not outperform the simpler open-loop baseline.

Critical strengths
  • This paper identifies a novel threat model.. It clearly distinguishes check-use races from prompt injection and formalizes the necessary architectural conditions and a falsifiable orthogonality criterion. This exposes an important gap in content-focused agent defenses and motivates state-level security analysis.
  • The empirical study connects the vulnerability to actionable defense design. The evaluation covers multiple attacker capabilities and defense families and includes a real LangGraph/SQLite deployment. Its negative results show that re-checking and conventional locking often operate at the wrong granularity, while identity binding across the full check-use span is more principled and actionable.
Critical weaknesses
  • Limited and highly repeated scenario data. The main experiments use only synthetic task families, with each family containing a small number of distinct entities that are repeatedly run. The resulting run counts overstate the amount of independent evidence, and no standard agent-security benchmark is used. The core claims may not generalize to realistic workflows.
  • Incomplete coverage of frontier models. The model grid is broad but uneven, and the defense evaluation covers only a small subset that does not systematically include current frontier agent systems or reasoning modes.
Questions for authors

See Weaknesses

2 comments
AuthorsOpenReview ↗

Response to miVf: instances and victim coverage

We thank the reviewer; both weaknesses are correct.

W1. The main experiments used three synthetic families with few entities, each executed 20 times, so run counts overstated the independent evidence and no standard benchmark was used. The matrix is now 33 distinct instances over five structures. The five structures are variants of the same check→use race, not five unrelated attacks:

shapewhat changestype
shared_stateone existing shared value (e.g., a contact email) is replacedsubstitution
shared_state_multiseveral fields or keys are replaced togethersubstitution, multi-field
shared_state_addthe key is absent at check and is created laterinsertion
delayed_contextcontent is read, then used after a delay; the payload is replaced in betweensubstitution, delayed
event_injectionan event/queue payload is rewritten after a status checkevent substitution

event_injection is not insertion; insertion is the shared_state_add probe and a new boundary probe, not one of the paper's original three primitives.

The resulting per-shape matrix is:

shapeentitiesinstancesreps
shared_state883
shared_state_multi883
shared_state_add883
delayed_context883
event_injection113
total8333

event_injection cannot be entity-varied — its plan names no person, so a substitution would produce byte-identical tasks — and contributes one instance rather than eight. Entity variation is not renaming: each entity enters the contact table with its own address, each instance targets a different one, and variants are verified pairwise distinct by content hash in code. The effect reproduces on the 33 instances (n = 99 per arm, v4-flash-0731):

conditionASR (in-window)ASR_u
no defense88.9% (88/99)100% (99/99)

Standard benchmark. On deepseek-v4.1-flash, we evaluate the full AgentDojo Workspace suite: 40 user tasks for clean completion, and all 17 tasks with a check→use pair for attack—not a hand-picked subset:

BenchmarkTasksVuln.Attack runsTask successASR_vulnClean
AgentDojo Workspace40175111/17 (64.7%)22/51 (43.1%)29/40 (72.5%)

Vuln. counts tasks with a check→use pair; only those are attacked. Task success is vulnerable tasks with at least one exploited run of three; ASR_vuln is exploited runs over 3×Vuln.; Clean is the no-attacker pass rate. This is one full suite, not four-suite coverage; we do not claim 66/56 or benchmark-wide coverage.

W2. The defense evaluation now covers five victims, two of them non-DeepSeek (gpt-5.1 and claude-3-5-sonnet); gpt-5.1 is reasoning-capable. In-window ASR per shape and victim:

victimshapeno defensePBVfusion
v4-flash-0731 (24/shape)shared_state96%0%0%
shared_state_multi88%0%0%
shared_state_add96%100%0%†
delayed_context75%0%0%
event_injection100%0%0%
v4.1-flash (8/shape)shared_state75%0%0%
shared_state_multi88%0%0%
shared_state_add100%88%0%†
delayed_context62%0%0%
event_injection100%0%0%
v4-pro (8/shape)shared_state100%0%0%
shared_state_multi75%0%0%
shared_state_add100%88%0%†
delayed_context100%0%0%
event_injection100%0%0%
gpt-5.1 (8/shape)shared_state100%0%0%
shared_state_multi100%0%0%
shared_state_add88%100%0%†
delayed_context100%0%0%
event_injection100%0%0%
claude-3-5-sonnet (8/shape)shared_state100%0%0%
shared_state_multi88%0%0%
shared_state_add100%88%0%†
delayed_context100%0%0%
event_injection100%0%0%

Blank victim cells continue the victim above. n = runs per shape. Values are in-window ASR. † Fusion has no window on insertion; the 0% is in-window ASR, but ASR_u insertion is 100% (harmful in every insertion run). Two conclusions hold on all five: PBV never fails on substitution and fails on 53/56 insertion runs (24/24, 7/8, 7/8, 8/8, 7/8); fusion never leaves a window open yet harms on every insertion run (24/24, 8/8, 8/8, 8/8, 8/8). Reasoning-capable model. gpt-5.1 is reasoning-capable, and its defense results match the non-reasoning victims on both invariants.

Thank you for the additional experiments and clarifications. The AgentDojo evaluation and defense experiments across more models meaningfully strengthen the paper and alleviate my original concerns. Although evidence of generalization across realistic workflows remains limited, demonstrating a security vulnerability in simple workflows using synthetic data is itself valuable. My overall assessment falls between 2 and 3, leaning toward 3, and I am therefore raising my score to 3.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Response to Reviewer rt7A — Part 1/2: W1/Q1, Locking, and Q2

We thank the reviewer for the helpful comments and questions.

W1/Q1. Correct: §4.4 credited the version guard with the 7.1% suppression and described locking as protection; Table 6's 66.7% is ASR, not protection (33.3%). The correction is one run‑level ledger: qwen2.5:14b / version, n = 450.

Main notations:

NotationValueMeaning
n450total runs
exploited418harmful runs
prevented32not harmful
ASR418/450 = 92.89%attack success rate
H16 (3.56%)genuine guard holds
T16 (3.56%)tool refusals
Rj300 (66.67%)runs with at least one rejection
H/Rj16/300 = 5.33%holds among rejected runs; post‑selection
H/n16/450 = 3.56%overall hold rate
ASR (in‑window)—mutated‑in‑gap exploited/all runs
ASR_u—harmful outcomes/all runs regardless of timing
Missing‑snapshot—no snapshot, allowed without comparison
re‑baselined—retry stores attacker state as the new snapshot
NRP—BP × (1 − ASR) (BP = benign pass rate)
Substitution—key exists at check and is replaced
insertion—key absent at check and created later
Legit‑update completion—benign runs whose final action reflects the update
pre_check / check_use / post_use—mutation before check / after check before guard / after guarded action
blind—random‑time mutation
checkpoint‑timed—when check completes

The identity is exploited + H + T = n. The 7.1% suppression includes both guard blocking and tool refusal: prevented = 7.11% = H 3.56% + T 3.56%. Rj = 300 = H 16 + 284 re‑baselined; 284 = 268 re‑baselined→exploited + 16 re‑baselined→tool‑refused, and the 150 never rejected → exploited runs are not in Rj. Five categories: 16 genuine, 0 missing‑snapshot, 268 re‑baselined→exploited, 16 re‑baselined→tool‑refused, 150 never rejected; sum 450. H/Rj = 5.33% is post‑selection; H/n = 3.56% is overall. Retries = 284; crashes = 0; censored = 0.

Locking: 66.7% ASR, not protection. Separate 90‑run experiment: each arm n = 90 = 3 race families × 30.

Locking armnASRProtection = 1 − ASRNRP
raw payload9060/90 = 66.7%33.3%0.333
covert payload9030/90 = 33.3%66.7%—

raw: 60/90 = 66.7% ASR, protection 33.3%, NRP 0.333 (BP = 1.0). covert: 30/90 = 33.3% ASR, protection 66.7%, no NRP because no comparable BP. 66.7% is ASR, not protection. Guard fires 450/450 (Q1), but only 300 runs reject; 150 never reject. H/Rj = 16/300 = 5.33% is post‑selection; H/n = 16/450 = 3.56% is overall; not compared across levels.

Q2. 360 runs, same task/mutation/state, only write time varies; 30/cell, zero crashes/censoring.

armfirst check reads attackerfirst use blocked by guardfirst action to attackerlater action to attacker
pre_check (all four defenses)30/300/3030/300/30
check_use / none0/300/3030/3030/30
check_use / version, recheck, CAS0/3030/3030/3030/30
post_use (all four defenses)0/300/300/3030/30

All 12 cells: ASR 100%. (i) Pre‑check poisons the check—no verifier rejects unseen state; later action 0/30 because the first action is already bad. (ii) First‑use rejection ≠ protection: 30/30 blocked, but retry re‑baselines; first/later actions 30/30 (corrupted recheck). (iii) Post‑use: first action clean 0/30; later action 30/30, no recheck—freshness failure. First‑action‑only defense is insufficient. Later action means any subsequent action in the same run—a retry or a follow‑up—not an action by the attacker.

RowLater actionWhat happened
pre_check0/30The first action already went to the attacker; there is no clean→bad transition to observe later.
check_use / none30/30No defense; both the first and the later action went to the attacker.
check_use / defenses30/30The first use was blocked, but the retry re‑baselined onto attacker state; the later action went to the attacker (corrupted recheck).
post_use30/30The first action was clean; the attacker changed state after the record; the later action was not rechecked (freshness failure).

Part 2 covers W2/Q3.

AuthorsOpenReview ↗

Response to Reviewer rt7A — Part 2/2: W2/Q3

We thank the reviewer for the helpful comments and questions. This is Part 2/2; Part 1 covers W1/Q1, Locking, and Q2.

W2/Q3. We do not claim priority over Lilienthal & Hong (2025). The check→use gap is present in 99/99 undefended runs in this harness; its width tracks latency (median 4.68s; 6.5–12.3s on AgentDojo) and collapses to 250–330µs under fusion. Exploitation depends on timing (blind 23.3%; checkpoint‑timed 88.9–100%), so we report both bounds, not a deployment rate. External validity is bounded by real LangGraph+SQLite and AgentDojo Workspace (40 tasks;17 vulnerable;11/17 task success); all attack cells satisfy A2 (reference‑based tools), so this is a conditional vulnerability, not a deployment‑wide prevalence estimate. TEMPO is the attack, not a defense. We compare PBV/fusion:

Incremental contribution.We agree that the mechanism controls and implementation analysis, not the discovery of TOCTOU, are the contribution. We do not claim to discover TOCTOU in agents or to introduce a new vulnerability class; prior work identified the pattern and proposed tool fusion. Our contribution is narrower: (i) an executable, content‑independent adversary on a common reference‑based harness; (ii) mechanism controls (missing‑snapshot, snapshot re‑baselining, and independent tool refusal) that explain why rule‑based defenses fail; (iii) a shape boundary (insertion) and availability cost that prior TOCTOU mechanisms and mitigations do not cover.

All ASR values are conditional on A1–A2 and on this harness; the blind 23.3% is a lower bound under a fixed timing policy, not a prevalence estimate for ordinary deployments.

Three arms; full matrix:75 substitution+24 insertion; benign comparison:60 runs; all arms use the same 33 instances (75 substitution + 24 insertion), with AgentDojo Workspace 40 reported separately as a standard‑benchmark check, not as a PBV/fusion cell:

Insertion is a new boundary probe, not one of the paper's three primitives; PBV=0 is on the original substitution matrix.

conditionASR_u substitution (75)ASR_u insertion (24)ASR_u full (99)legit completion (60 benign)
no defense100% (75/75)100% (24/24)100% (99/99)100% (60/60)
pass‑by‑value binding0% (0/75)100% (24/24)24.2% (24/99)0% (0/60)
tool fusion0% (0/75)100% (24/24)24.2% (24/99)0% (0/60)

Which result demonstrates the advantage? The insertion boundary. On the original substitution matrix, both PBV and fusion achieve 0%, the result already covered by prior TOCTOU work; on the new insertion probe both achieve 100% (24/24), so their unconditional full‑matrix ASR is 24.2% (24/99), while legitimate completion is 0/60 for both. This is the result beyond prior TOCTOU mechanisms: the strongest existing mitigations are substitution‑specific, do not cover insertion, and their apparent safety comes at the cost of all legitimate updates. We do not claim a new defense or a new vulnerability class; the advantage is a boundary/availability measurement that prior work did not report under this common matrix.

Note: v4-flash-0731, 33 instances/99 runs; substitution 75, insertion 24. ASR_u = harmful/all; legit completion from 60 benign runs (no legit insertion; substitution 100% insertion, 24.2% full). In-window ASR on the 60-run matched subset: 93.3% / 0% / 0%. Update window: binding 7.94–8.29s; fusion 0.3ms. Updater 1078–1141µs; updates land, agent never re-reads. Beyond prior work: prior work measured prevalence/gaps without executing an adversary. We add real execution with an out-of-process adversary, insertion boundary (4/4 victims), availability cost (100%→0% on three victims), and a re-baselining mechanism.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.