- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates trajectory-level KV-cache overload in language-model agents, where adversarially induced reasoning is retained in the agent's interaction history and repeatedly reintroduced into subsequent prompts, leading to compounding inference-resource consumption. To investigate this vulnerability, the authors propose KOVER (KV-cache OVErload Red teaming), a two-stage red-teaming framework. Stage I optimizes an observation-channel suffix to steer the agent toward attacker-controlled content, while Stage II searches over structured reasoning programs to induce persistent reasoning and maximize peak logical KV-cache occupancy and KV occupancy-time. Experiments on three LLMs across four interactive-agent benchmarks demonstrate substantial KV-cache amplification. Additionally, a small-scale multi-tenant vLLM experiment shows that the increased KV-cache demand can lead to reduced concurrent serving capacity.
Critical strengths
- An underexplored trajectory-level resource vulnerability. This paper identifies and systematically investigates trajectory-level KV-cache overload in LM agents. In particular, it reveals how reasoning-history compounding can amplify inference-resource consumption across multiple interaction steps, highlighting a resource-safety threat overlooked by conventional single-turn analyses.
- Practical validation in a multi-tenant serving environment. Beyond benchmark-level evaluations, the authors conduct a multi-tenant vLLM experiment demonstrating that increased KV-cache demand can substantially reduce concurrent serving capacity. This provides system-level evidence of the practical impact of the identified vulnerability.
Critical weaknesses
- Lack of task performance evaluation. Although the authors repeatedly emphasize that KOVER amplifies resource consumption while preserving well-formed agent execution, the experiments do not report task success rates or evaluate the impact on task-solving performance. Preserving valid, executable actions does not necessarily imply successful task completion, making it difficult to assess whether KOVER achieves resource amplification without compromising the agent's original task.
- Limited comparison with competitive baselines. The Stage II evaluation primarily compares KOVER against simple length-padding and off-task reasoning baselines. Although these baselines account for input length and repeated injection, they do not incorporate comparable reasoning-length and persistence requirements. A stronger baseline using an unstructured prompt with matched reasoning-length and persistence requirements would help isolate the contribution of KOVER's structured payload optimization. Additionally, a comparison with the closely related OTora[1] framework would further strengthen the evaluation.
[1] Li X, Mu R, Li L, et al. OTora: A Unified Red Teaming Framework for Reasoning-Level Denial-of-Service in LLM Agents[J]. arXiv preprint arXiv:2605.08876, 2026.
Questions for authors
- Stronger baseline comparisons. Could the authors compare KOVER with an unstructured reasoning prompt that matches its reasoning-length and persistence requirements, as well as an adapted version of the closely related OTora framework under the same evaluation protocol?
- End-to-end attack effectiveness. While Stage I entry success and Stage II KV-cache amplification are evaluated separately, could the authors report end-to-end resource amplification over the full task set, including unsuccessful entry attempts?
- Analysis of Stage II payload optimization. Could the authors clarify the rationale behind the predefined payload templates and provide a case study illustrating the optimization process from C0 to C1 and C2, along with an ablation study quantifying the incremental improvements contributed by each stage?
- Robustness to memory compaction. Could the authors evaluate whether KOVER remains effective under practical agent memory-management strategies, such as history summarization or context truncation, beyond the existing history-removal ablations?
- Cross-model transferability under limited attacker access. Given that the proposed optimization relies on repeated interactions with the victim agent, could the authors evaluate whether adversarial suffixes and reasoning payloads optimized on a surrogate model can effectively transfer to different victim models?
3 comments
Response to Reviewer SBUQ (2/3)
Q3. Analysis of Stage II payload optimization
"Could the authors clarify the rationale behind the predefined payload templates and provide a case study illustrating the optimization process from C0 to C1 and C2, along with an ablation study quantifying the incremental improvements contributed by each stage?"
Thank you for this comment. Our Stage II templates are task-agnostic in form: they direct the agent to ground its reasoning in the current task state without requiring task-specific payload content. Designed for trajectory-level KV-cache overload, they expose reasoning organization, grounding, persistence, evidence refresh, and action closure as search variables to induce task-grounded reasoning that can accumulate through history retention and replay. A concrete C1-to-C2 refinement on Llama-3.1-8B-Instruct/MiniWoB++ illustrates the search. We evaluate a selected C1 EvidenceChain parent and its C2 child on the same development tasks, using identical clean references and evaluation settings. The child changes only the evidence-refresh instruction to new_current_page_evidence, keeping the reasoning family, grounding, persistence, and closing rule fixed.
| Candidate | ||
|---|---|---|
| Selected C1 EvidenceChain parent | 4.14× | 96.1× |
C2 child: parent + new_current_page_evidence | 4.46× | 114× |
The evidence-refresh edit increases peak amplification by 7.7% and occupancy-time amplification by 18.6%, providing a concrete example of the contribution of refinement beyond the persistence stage.
We further evaluate the initial, persistence-stage, and final selected candidates on WebArena:
| Model | C0: / | C1: / | Final: / |
|---|---|---|---|
| Qwen3-8B-Instruct | 1.11× / 7.62× | 1.21× / 8.40× | 1.24× / 10.63× |
| Llama-3.1-8B-Instruct | 2.24× / 32.51× | 2.37× / 38.72× | 2.59× / 57.94× |
From C0 to the final candidate, occupancy-time amplification increases by 39.5% for Qwen and 78.2% for Llama, alongside increases in peak amplification. These stage-candidate comparisons quantify the gains achieved by the search beyond the initial reasoning-family templates.
Response to Reviewer SBUQ (3/3)
Q4. Robustness to memory compaction
"Could the authors evaluate whether KOVER remains effective under practical agent memory-management strategies, such as history summarization or context truncation, beyond the existing history-removal ablations?"
Thank you for this suggestion. We evaluate summarization, a recent-history window, and context truncation under a common paired protocol. The experiments use Llama-3.1-8B-Instruct on the same new set of 20 paired MiniWoB++ task–seed instances. Each policy is applied to both Clean and KOVER, with the frozen payload, current-step output budget, and prefix-caching configuration held fixed. Ratios are computed per task within each policy and then averaged.
Reasoning windows and summaries preserve action history. Summary-only incrementally summarizes past reasoning; summary plus recent history retains the latest three reasoning steps verbatim and summarizes older reasoning. The summarizer uses the same Llama model, temperature 0, seed 0, a 256-token output cap, and a 4,096-token input cap. The 8,192-token truncation policy removes the oldest complete history records until the input fits the budget. and are computed over agent calls, following the original metric definitions. Summarizer calls are excluded from the reported resource metrics; summary text retained in subsequent agent prompts is included in the input-length accounting.
| History-management policy | ||
|---|---|---|
| Full history | 2.50× | 24.29× |
| Most recent three reasoning steps | 2.19× | 22.41× |
| Summary only | 2.05× | 22.98× |
| Summary + recent three reasoning steps | 2.30× | 22.57× |
| 8,192-token context truncation | 1.98× | 28.06× |
All four management strategies reduce peak amplification, yet the attack retains 1.98–2.30× peak amplification and 22.41–28.06× occupancy-time amplification. These policies affect the two resource dimensions differently. Context truncation enforces an input-length budget and yields the smallest peak ratio, whereas occupancy-time continues to accumulate invocation duration across the trajectory. Truncation’s 1.98× peak ratio alongside its 28.06× occupancy-time ratio illustrates why peak-memory control and cumulative resource control require separate assessment. Summarization and recent-history retention likewise produce distinct tradeoffs between the two metrics. We present these results in the new history-management evaluation (Section 4.6, Table 6).
Q5. Cross-model transferability under limited attacker access
"..., could the authors evaluate whether adversarial suffixes and reasoning payloads optimized on a surrogate model can effectively transfer to different victim models?"
Thank you for this comment. We evaluate suffix and payload transfer separately from Llama-3.1-8B-Instruct to Qwen3-8B-Instruct on WorkArena L1. The source-optimized artifacts are applied to the target model without target-model reoptimization:
| Transferred component | Metric | Source: Llama-3.1-8B-Instruct | Target: Qwen3-8B-Instruct |
|---|---|---|---|
| Stage I suffix | Entry success | 94% | 14% |
| Stage II payload | amplification | 2.24× | 1.34× |
| Stage II payload | amplification | 32.5× | 9.33× |
The two artifacts optimize different behaviors. The suffix targets a specific model–task action trigger, whereas the payload expresses a semantic reasoning procedure that the target agent can instantiate on its current observations. The observed transfer pattern is consistent with this distinction: suffix entry success is model-sensitive, while the transferred payload still produces 1.34× peak amplification and 9.33× occupancy-time amplification on the target.
Response to Reviewer SBUQ (1/3)
Dear Reviewer SBUQ,
We sincerely appreciate your positive assessment of the trajectory-level vulnerability and the measured serving-capacity impact. We provide point-by-point responses below.
Q1. Stronger baseline comparisons
Thank you for this comment. We add both requested comparators and report task completion alongside resource amplification. The comparison uses Llama-3.1-8B-Instruct on 50 MiniWoB++ task–seed instances, HTML and AXTree observations, reasoning-history retention and prefix caching enabled, a 10-step horizon, temperature 0.1, and a 3,072-token per-call output cap.
The unstructured persistent prompt requests at least 600 words before every action, matching KOVER's four blocks of at least 150 words, under the same injection protocol. It is frozen before evaluation and uses continuous prose without KOVER's named reasoning families or explicit grounding, refresh, and decision-closing clauses. Adapted OTora uses ICL-style free-text search with the same resource objective, Jcore.
KOVER achieves the largest resource amplification in this comparison. Relative to the unstructured persistent prompt, it produces 15.6% higher peak amplification and 2.94× the occupancy-time amplification, while completing 42% versus 36% of tasks.
Task completion and resource availability are separate evaluation dimensions. OTora's original formulation emphasizes slowdown with preserved correctness. KOVER optimizes trajectory-level KV demand, and reports task success as a separate outcome. Its payloads organize reasoning around the current task and conclude with an agent action; the optimization objective places its weight on resource amplification. Both KOVER and adapted OTora use the same resource objective Jcore. The joint evaluation characterizes their observed resource–utility tradeoff: adapted OTora completes 48% of tasks with 8.51× occupancy-time amplification, while KOVER completes 42% with 92.2× amplification.
Q2. End-to-end attack effectiveness
Thank you for this comment. We now report full-set resource amplification over all 50 attack attempts, including the three unsuccessful entries. For a paired episode-level resource ratio ri, the full-set mean is
rˉall=N1∑i=1Nri=p⋅rˉentry+(1−p)⋅rˉno entry.
where p is the observed entry rate and the two conditional means are computed on the corresponding subsets of the same task set. This decomposition accounts for dependence between entry and resource consumption through the measured conditional means. For this evaluation, we report peak prefill KV amplification on WorkArena L1 using Llama-3.1-8B-Instruct, which measures the maximum prompt-side KV requirement along each trajectory:
=\frac{\max_t\kappa L_{i,t}^{\mathrm{attack}}} {\max_t\kappa L_{i,t}^{\mathrm{clean}}}.$$ | Evaluation population | Episodes | Mean peak prefill KV amplification | | --------------------- | -------: | ---------------------------------: | | Successful entry | 47 | 2.21× | | Unsuccessful entry | 3 | 1.33× | | All attempts | 50 | 2.16× | Failed entries contribute their measured resource consumption. Thus, the attack retains 2.16× mean peak prefill amplification after all entry failures are included.