- Rating
- 3(+1)
- Confidence
- 3(+1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses security risks in persistent memory for LLM agents, including cross-session memory poisoning and malicious knowledge propagation across agents. The authors propose PerMIT, a framework with three main components: 1.a single-user memory protection mechanism that prevents untrusted content from influencing protected agents without authorization; 2.a federated trust mechanism that aggregates endorsements to enable secure memory sharing while reducing verification overhead; 3.an autonomous knowledge construction framework that allows agents to generate, verify, and reuse shared knowledge. The authors evaluate PerMIT on Trojan Hippo, community trust simulations, and τ²-bench, reporting resistance to tested memory-poisoning attacks, reduced verification effort, and improved downstream task performance.
Critical strengths
1.A well-structured memory protection framework with formal security guarantees. PerMIT isolates untrusted memory through a protected agent, a quarantined reader, and a confirmation gate. Its formal analysis provides integrity guarantees under explicit trust assumptions, offering a principled approach to persistent-memory security.
- A unified approach to secure memory sharing and autonomous knowledge construction. The federated trust mechanism enables reusable community knowledge while aiming to reduce poisoning risks and verification overhead.
Critical weaknesses
1.Insufficient end-to-end validation of cross-agent memory poisoning defenses.The final multi-agent evaluation only tests 12 direct-injection payloads and measures whether they reach the Community-Confirmed tier. It does not establish whether malicious memories can influence individual agents before promotion or induce harmful downstream behavior.
2.Unclear benefits of the federated trust mechanism.Community simulations rely on assumed reviewer accuracy, while the multi-agent experiments do not directly validate the claimed safety improvements with increasing community size. This makes it difficult to isolate the security and utility benefits attributable to the proposed federated trust mechanism.
Questions for authors
1.Could the authors evaluate whether malicious memories are accepted or used before promotion and whether they can induce harmful behavior in downstream agents? This would better validate the claimed protection against cross-agent memory poisoning.
2.Could the authors provide matched security–utility comparisons against simpler sharing or voting mechanisms? Additionally, how well do the assumed reviewer accuracy and error correlations reflect the actual autonomous agent communities? These results would help isolate the benefits of the proposed trust mechanism.
4 comments
Response to Reviewer r4AZ (2/3)
W2/Q2. Community simulations rely on assumed reviewer accuracy, while the multi-agent experiments do not directly validate the claimed safety improvements with increasing community size. [...] Could the authors provide matched security–utility comparisons against simpler sharing or voting mechanisms? Additionally, how well do the assumed reviewer accuracy and error correlations reflect the actual autonomous agent communities?
Concern (W2, Q2): how well "the assumed reviewer accuracy and error correlations reflect the actual autonomous agent communities".
Thanks for the insightful question! We measured reviewer accuracy live, and our simulations assume reviewers who are much worse than the live ones (App. F.9, F.10):
| Honest reviewers | Live communities (measured) | Simulations (assumed) |
|---|---|---|
| Planted entries accepted | 0 of 1,381 decisions | 40% |
| Legitimate entries accepted | 92–96% | 90% |
| Correlation between reviewers' errors | no errors to correlate | 0–0.1 (0.3 in the stress test of Eq. (1)) |
The table above shows that live reviewers rejected every planted entry in all 1,381 decisions, across campaigns A and B and the paper's banking runs (catch rate q ≥ 0.995 at 95% confidence). No planted entry was promoted or steered a later task (W1 above). Our simulations instead assume reviewers who miss 40% of poisons (q = 0.6) and reject 10% of legitimate entries. Lower accuracy only raises the risk, so the simulations err on the side of caution. At the live rate, Eq. (1) gives ≈4×10⁻¹⁹ for independent reviewers.
Because live reviewers made no errors on the planted entries, we evaluate correlation as a stress test. Correlation matters only when reviewers err, because it makes a poison that fools one reviewer more likely to fool the others. Each live community uses a single backbone, where errors are likely to be correlated, yet its reviewers accepted no planted entry. Even at ρ = 0.3, Eq. (1) gives ≈10⁻³ at the live catch rate. Moreover, correlation, not size, is what raises risk (Fig. 2(b)), and PerMIT's identity pools address it, since promotion requires endorsements from more than one pool. With three reviewer families, the chance that the dispute panel keeps a poison falls to 8.6×10⁻⁴ at 10⁵ users. Aligning the pools with the families also cuts an attack that targets the weakest family from 34% to 4.8% success (App. F.10).
Finally, the analysis and the simulations agree. Across five billion simulated promotion attempts, the observed promotion rate stays below the bound of Eq. (1) for every number of honest users shown a poisoned entry, and none of the nine adaptive attack strategies we simulated exceeds it (§4.2, App. A.1).
Concern (Q2): "matched security–utility comparisons against simpler sharing or voting mechanisms".
Because live reviewers rejected every planted entry, we compare PerMIT with simpler voting rules in a matched simulation where reviewers err and adversaries reject falsely, on identical candidate streams, reviewer draws, and adversaries (1,000 users, q = 0.6, App. F.10):
| Mechanism | Poison promoted | Benign wrongly evicted, no attack | Benign wrongly evicted, false-rejection attack |
|---|---|---|---|
| Majority of 10 | 0.42 | 0.001 | 0.04 |
| Quorum of 10, rejections ignored | 1.00 | 0 | 0 |
| Quorum of 10 + veto | 0.001 | 0.71 | 1.00 |
| PerMIT | 0.001 | 0 | 0 |
With correlated reviewers (ρ = 0.1), PerMIT promotes 3.7% of poisons, against 43% for majority voting. On utility, PerMIT serves 40% of benign requests from the shared store without a new confirmation (77% with a juror panel), and in both live campaigns its store kept task success within 0.05 of unfiltered sharing on every backbone. On identical live verdicts, unfiltered sharing admits every planted entry and PerMIT admits none.
Concern (W2): the experiments "do not directly validate the claimed safety improvements with increasing community size".
Safety versus community size concerns rare events, so we establish it analytically and in simulation. A live demonstration that promotion falls from 10⁻³ to 10⁻⁴ would need 10,652 poison trials (666 communities) per community size, i.e., USD 0.88M–6.5M in construction alone at our measured costs. At the deployed thresholds, PerMIT's observed promotion stays at 0–5×10⁻⁴ from 10² to 10⁵ users (ρ = 0), whereas majority voting stays at 38–46% at every size.
Response to Reviewer r4AZ (3/3)
Summary: what PerMIT contributes, and what it can do that existing defenses cannot.
In addition to answering the questions above and focusing on the multi-agent experiments, we close with a short summary of the paper as a whole. PerMIT makes five main contributions:
- Machine-checked guarantees for persistent memory. Untrusted memory reaches the agent only as opaque handles, and a value reaches its reasoning or a sensitive tool only after the gate shows it and it is approved. These guarantees are machine-checked in Lean 4 for arbitrarily long multi-session interactions (App. E). CaMeL leaves formal verification as future work, and neither CaMeL nor FIDES covers memory that persists across sessions.
- Safe and useful on persistent memory, where existing defenses are not. On the same agent, tasks, and backbones, PerMIT sends no message to the attacker, completes 51/60 attacked tasks, and follows no planted instruction, while every baseline fails at least one of these tests (table below).
- The first integrity guarantees for shared memory, to our knowledge. CaMeL, FIDES, and TMA-NM protect a single user. PerMIT turns each approval into an auditable endorsement that others can reuse, and it bounds both the promotion of poison and the wrongful eviction of benign entries. In live 90-agent communities, none of 84 planted entries was promoted (W1 above), and in simulation, reusing endorsements saves each user 38% of approvals (§4.2).
- Shared memory that agents build themselves, and that helps. On τ²-bench telecom, a fresh agent using the memory built by 90 agents performs on par with the benchmark's expert-written manual on Claude-Sonnet-4.6 and GLM-5 (0.80 vs. 0.79 and 0.97 vs. 0.98) and exceeds it on Kimi-K2.5 (0.89 vs. 0.74) (§5).
- A recursive language model (RLM) as the memory reader, the first to our knowledge. It searches memory with small programs, found the right value in all 48 lookups despite look-alike distractors, and needed at most 22-fold more input for a 3,600-fold larger memory (App. F.6).
| Test (App. F.8, F.9) | CaMeL | FIDES | TMA-NM | PerMIT |
|---|---|---|---|---|
| Persistent memory: user's message sent to the attacker (of 60) | 51 | 0 | 0 | 0 |
| Persistent memory: attacked task completed correctly (of 60) | 8 | 9 | 15 / 32‡ | 51 |
| Persistent memory: planted instruction followed in the answer (of 60) | 4 | 12 | 39 | 0 |
| Single session: utility as designed (Claude / GLM / Kimi) | 0 / 0 / 0 | 46 / 45 / 0 | 62–78 / 76–84 / 54–60§ | 90 / 87 / 89 |
| Shared memory across users | single user only | single user only | single user only | 0/84 planted entries promoted |
‡With an informed user / with an AI verifier.
§Each range covers TMA-NM's two runs on that backbone. Utility is the harmonic mean over six task suites, so it is 0 whenever one suite fails completely. PerMIT's utility is from Table 1.
The table above shows that each baseline fails at least one test: CaMeL sends the user's message to the attacker, FIDES and TMA-NM complete at most 32 of 60 attacked tasks, TMA-NM often follows planted instructions, and none of them supports shared memory. PerMIT is the only design that does well on every row.
The updated supplementary material contains the description and statistics of every new experiment, together with the updated Lean 4 files, so that the reviewer can check the latest Lean 4 results directly.
Again, we thank the reviewer for the valuable feedback. We hope that the live results address the reviewer's concerns, and please let us know if there are any other questions or suggestions.
Best,
Authors
Thank you for the detailed response and the additional experiments. These additions have addressed my main concerns. I have no further questions at this stage and will raise my rating.
Response to Reviewer r4AZ (1/3)
We thank the reviewer for the valuable feedback on improving this paper, and in particular for focusing on end-to-end validation of the federated tier! Following the reviewer's suggestions, we ran two new live campaigns, each with three 90-agent communities, and measured reviewer accuracy from their logs (revised PDF: §5, App. A.7, F.9–F.10, with changes in blue). Please find below our response.
Concern (W1): the evaluation "only tests 12 direct-injection payloads".
Following the reviewer's suggestion, we ran two new live campaigns (App. F.9). Each campaign runs three new 90-agent communities, one per backbone (Claude-Sonnet-4.6, GLM-5, Kimi-K2.5), in which two malicious identities write entries directly into shared memory before any honest agent runs:
Next to each planted entry, we add a legitimate entry on the same topic. The attackers support half of the planted entries with fake identities (Sybil endorsements) and try to remove half of the legitimate entries with false rejections.
Live tests at larger scale are costly, because agents run one after another on the shared memory. At our measured rates, one 90-agent community costs USD 13–97 to build, depending on the backbone (App. F.6). The two campaigns above thus cost at least USD 264. Scaling both the number of communities and their size by 10 would multiply this by 100, to about USD 26,000 and 1,700 hours of agent runs. Measuring the rare events that matter at scale is costlier still: showing live that promotion falls from 10⁻³ to 10⁻⁴ needs about 6 million agent episodes and USD 0.88M–6.5M per backbone (W2 below). We therefore run live tests at the largest scale that is practical, and extend the evaluation to larger communities through our analysis (Eq. (1)) and simulation. The simulations agree with the analysis, showing that PerMIT's risk does not grow with community size (W2 below).
Concern (Q1): "Could the authors evaluate whether malicious memories are accepted or used before promotion and whether they can induce harmful behavior in downstream agents?"
We would like to clarify that PerMIT's design rules this out. Until an entry is promoted, it is untrusted for every agent. Untrusted content does not enter an agent's prompt on its own: the agent sees only an opaque handle, and the content reaches its reasoning or actions only after the gate displays it and the user or a verifier approves it. These guarantees are machine-checked in Lean (App. D, E), and our experiments support them: