- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates security vulnerabilities in reflective LLM agents arising from incorrect credit assignment during experience reuse. The authors introduce Reflection-Coupled Credit Assignment Poisoning (R-CAP), which induces agents to generate persistent, overgeneralized tool-use guidance from successful trajectories without directly modifying their memory or system prompts. Experiments on AgentDojo across three LLM backbones demonstrate high reflection implantation and cross-session attack success rates while largely preserving task utility. The findings highlight an important security risk in self-authored agent memory and reflection mechanisms.
Critical strengths
-
The paper identifies a meaningful vulnerability in reflection-based credit assignment, showing how legitimate successful trajectories can produce persistent unsafe guidance without explicit memory manipulation.
-
The experiments cover multiple models and application domains, with comparisons against direct memory injection and several defenses. The preservation of task utility further highlights the attack's practical significance.
Critical weaknesses
-
The evaluation assumes automatically loaded shared reflection memory, which may limit generalizability to agent systems with selective retrieval or stronger memory isolation.
-
The evaluation primarily measures overbroad tool usage rather than actual information exposure. Additional security-oriented measurements could strengthen the practical implications.
Questions for authors
-
Could the authors clarify whether R-CAP remains effective when reflections are selectively retrieved rather than automatically loaded, particularly across different task identities?
-
How would R-CAP compare against a more closely matched direct-memory baseline with equivalent wording and tool specificity? This could help isolate the contribution of self-authored reflections.
-
Could the authors further evaluate the proposed counterfactual reflection auditing mechanism, particularly its effectiveness and computational overhead?
-
The related work discussion could be strengthened by incorporating relevant studies on agent security, memory poisoning, and efficiency, including MemPoison, Agent Security Bench (ASB), Sponge Tool Attack, SynChain, and a related survey titled Efficiency in LLM Multi-Agent Systems: A Survey. These works provide complementary perspectives on persistent memory vulnerabilities, agent-generated attacks, security evaluation, and efficiency trade-offs.
8 comments
Response to Reviewer VDHq: Q3, Q4
Q3. Counterfactual reflection auditing (CRA): effectiveness and overhead
We implemented CRA and ran it on one R-CAP reflection, from a calendar attack on Qwen3.5-9B.
- Effectiveness. The auditor extracted the recommended tool from the reflection automatically and replayed the source task three times with this tool suppressed. All three replays completed the benchmark goal, so CRA blocked the reflection.
- Overhead. The audit took 10 model generations, 42,137 tokens and 83.8 seconds on an RTX A6000, excluding model loading. The cost is paid once, when the reflection is written; later sessions pay nothing.
We will add this case and its cost to Appendix L.
Q4. Related work
Thank you for the pointers. We will add them to §2 and Appendix D.
| Work | What it does | Difference from R-CAP |
|---|---|---|
| MemPoison (Wang et al., CCS 2026); already cited as Wang et al. (2026a), reference to be updated | Plants a trigger-activated backdoor in agent memory through ordinary conversation | R-CAP has no trigger and injects no payload text; the agent's reflection step writes the rule from a successful tool trajectory |
| MemPoison benchmark (Gao et al., 2026) | Benchmark of persistent memory threats with 1,227 cases in three levels; write-time defenses handle the direct level and miss the others | Consistent with our Table 3A, where two content gates leave TOSR at 0.91; in R-CAP the record is written by the agent's own reflection writer |
| Agent Security Bench (Zhang et al., ICLR 2025) | Benchmark of attacks and defenses for LLM agents, including memory poisoning, over 10 scenarios and 13 backbones | R-CAP is one route within this space: memory formed through the reflection writer |
| Sponge Tool Attack (Li and Wang, 2026) | Rewrites the input prompt so that the agent follows longer tool trajectories; targets cost within one session | R-CAP leaves the victim's prompt untouched, acts across sessions through memory, and targets access scope |
| SynChain (Zhang et al., 2026) | A model compromised by fine-tuning writes poisoned skills, configuration and memory that are reused later | R-CAP needs no model compromise; the attacker is an ordinary user of an unmodified model |
| Efficiency in LLM Multi-Agent Systems: A Survey | Survey of efficiency in multi-agent systems | We will cite it for the cost side: R-CAP raises the average tool count from 1.25–1.33 to 1.89–2.12 (Table 1) |
Thanks to the authors for the detailed response. ICLR allows authors to modify the PDF file. I recommend that authors update the changes and discussions made into the PDF file, so that the further review can be conducted more conveniently.
After reading the rebuttal, I still have a few doubts: The authors say that R-CAP performs at least as well as direct memory injection. However, the reported TOSR is conditional on successful induction, so this qualification should be made explicit. They also report an average of 21 records and 9,286 characters for broad Email calls. However, these figures cover only 'get_received_emails', excluding 'get_unread_emails', which is also listed as a broad tool in Table 6. The response should name the specific tool or report the average across both.
Also, would the authors provide the exact cost of the API calls for different experiments? For each experiment setup, how many API quotas are consumed respectively?
Response to Reviewer VDHq: W1 / Q1, W2, Q2
We thank the reviewer for the careful reading and the constructive comments. We answer each point below.
W1 / Q1. Always-loaded shared memory, selective retrieval, isolation, and different task IDs
Different task IDs. We ran a new cross-ID evaluation on Haiku 4.5: 17 randomly drawn tasks, 66 source → victim pairs with different task IDs and the same target behavior, three runs per pair, with the reflection loaded automatically.
R-CAP stays effective when the victim's task differs from the source task.
Stronger isolation. If the reflection is not loaded for the victim, the attack fails: TOSR is 0.00 with utility 1.00 (Table 2B, Table 3A). Isolation also removes the reuse that shared reflection memory is meant to provide (§5.4).
Selective retrieval. The paper studies the credit-assignment step: what the agent writes into its reflection after a successful run, and how that reflection changes later tool use (§2). To isolate this step from retrieval, we fixed the loading policy to always-on memory, where lessons are prepended to later contexts as standing guidance (§3.1). For this reason we did not include a retrieval experiment.
We will add the cross-ID table to the paper.
W2. Tool usage versus information exposure
TOSR is measured at the tool level. Two results show what the broad calls return.
Returned volume (Appendix G). Each recorded call is replayed against the task's initial state:
Out-of-scope records (new audit). We audited 60 successful transfers from the runs behind Table 1, 20 randomly drawn tasks per backbone. In 46 of the 60 (77%; 15, 16 and 15 of 20 on GPT-4o, Haiku 4.5 and Sonnet 4.6), the broad call returned at least one record outside the scope of the victim's task. In the other 14 the broad and the narrow call returned the same records: in 11 the task's data holds a single record, and in 3 the agent itself called the narrow file or email search with an empty keyword, for which AgentDojo's search tools return the full list.
So in about three quarters of the audited transfers, the widened tool use also brings out-of-scope records into the victim's context. We will add the audit to the paper.
Q2. A direct-memory baseline matched in wording and tool specificity
A hand-written rule with equivalent wording and tool specificity would show how much of the gap between the two rows below is due to wording. It cannot show an effect of authorship: the victim sees only the text of a memory entry, and a generated reflection and a hand-written rule arrive in the same memory file under the same framing (§5.2). With identical text, the two conditions are the same input. We did not run this baseline. What self-authorship changes is what the attacker needs:
Without write access to memory, the attacker reaches through the agent's own reflection step a TOSR at least as high as direct insertion of a hand-written rule.
Our contribution concerns how that content enters memory: R-CAP induces the agent to generate and store the rule without granting the attacker direct memory-write access.