acceptodds
Under review as a conference paper at ICLR 2027

PerMIT: Securing Persistent Memory for LLM Agents via Machine-Checked Integrity Theorems and Scalable Community Trust

Abstract

Persistent memory is a largely undefended surface for injection and poisoning, and the same risk limits memory sharing among agents. We present PerMIT, a memory architecture with provable integrity guarantees under an explicit trust assumption: only user-typed inputs and values confirmed at a gate are trusted. PerMIT mediates the release and use of untrusted environment-derived values through a finite memory interface, while preserving agent-driven multi-turn planning and tool choice. Over this finite interface, we prove two integrity theorems that are also machine-checked in Lean 4, which together ensure that environment-derived memory reaches a sensitive action only if confirmed at the gate and influences the agent's behavior only through what the gate displays. Every attack in our threat model thereby reduces to a single auditable decision, which a human, an LLM verifier, or both can make in practice. On Trojan Hippo, PerMIT records zero observed attack success with self-verification and lightweight verifiers. Because each confirmation is a logged verdict on exact content, endorsements accumulate across users, and sufficiently endorsed items become community-confirmed with no further verification required. Under explicit assumptions, we prove and corroborate in simulation a co-scaling property: as the community grows, the risks of promoting poison and of wrongly evicting benign knowledge fall, while per-user verification effort does not rise. On -bench, communities of up to 90 autonomous agents use PerMIT to build shared knowledge bases with no direct memory injection promoted, and the resulting knowledge significantly improves new agents on held-out tasks, suggesting a path toward agents that safely self-improve at test time.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM agents, persistent memory, memory poisoning, memory injection defense, information flow control, machine-checked proof, security guarantee

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_r4AZOpenReview ↗
Rating
3(+1)
Confidence
3(+1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses security risks in persistent memory for LLM agents, including cross-session memory poisoning and malicious knowledge propagation across agents. The authors propose PerMIT, a framework with three main components: 1.a single-user memory protection mechanism that prevents untrusted content from influencing protected agents without authorization; 2.a federated trust mechanism that aggregates endorsements to enable secure memory sharing while reducing verification overhead; 3.an autonomous knowledge construction framework that allows agents to generate, verify, and reuse shared knowledge. The authors evaluate PerMIT on Trojan Hippo, community trust simulations, and τ²-bench, reporting resistance to tested memory-poisoning attacks, reduced verification effort, and improved downstream task performance.

Critical strengths

1.A well-structured memory protection framework with formal security guarantees. PerMIT isolates untrusted memory through a protected agent, a quarantined reader, and a confirmation gate. Its formal analysis provides integrity guarantees under explicit trust assumptions, offering a principled approach to persistent-memory security.

  1. A unified approach to secure memory sharing and autonomous knowledge construction. The federated trust mechanism enables reusable community knowledge while aiming to reduce poisoning risks and verification overhead.
Critical weaknesses

1.Insufficient end-to-end validation of cross-agent memory poisoning defenses.The final multi-agent evaluation only tests 12 direct-injection payloads and measures whether they reach the Community-Confirmed tier. It does not establish whether malicious memories can influence individual agents before promotion or induce harmful downstream behavior.

2.Unclear benefits of the federated trust mechanism.Community simulations rely on assumed reviewer accuracy, while the multi-agent experiments do not directly validate the claimed safety improvements with increasing community size. This makes it difficult to isolate the security and utility benefits attributable to the proposed federated trust mechanism.

Questions for authors

1.Could the authors evaluate whether malicious memories are accepted or used before promotion and whether they can induce harmful behavior in downstream agents? This would better validate the claimed protection against cross-agent memory poisoning.

2.Could the authors provide matched security–utility comparisons against simpler sharing or voting mechanisms? Additionally, how well do the assumed reviewer accuracy and error correlations reflect the actual autonomous agent communities? These results would help isolate the benefits of the proposed trust mechanism.

4 comments
AuthorsOpenReview ↗

Response to Reviewer r4AZ (1/3)

We thank the reviewer for the valuable feedback on improving this paper, and in particular for focusing on end-to-end validation of the federated tier! Following the reviewer's suggestions, we ran two new live campaigns, each with three 90-agent communities, and measured reviewer accuracy from their logs (revised PDF: §5, App. A.7, F.9–F.10, with changes in blue). Please find below our response.

W1/Q1. The final multi-agent evaluation only tests 12 direct-injection payloads [...]. It does not establish whether malicious memories can influence individual agents before promotion or induce harmful downstream behavior. [...] Could the authors evaluate whether malicious memories are accepted or used before promotion and whether they can induce harmful behavior in downstream agents?

Concern (W1): the evaluation "only tests 12 direct-injection payloads".

Following the reviewer's suggestion, we ran two new live campaigns (App. F.9). Each campaign runs three new 90-agent communities, one per backbone (Claude-Sonnet-4.6, GLM-5, Kimi-K2.5), in which two malicious identities write entries directly into shared memory before any honest agent runs:

  • Campaign A plants 16 harmful entries per community (injection, policy violation, credential request, external channel, technical misdirection).
  • Campaign B plants 12 harmless but unauthorized "company rules" per community, each with a unique code, so that downstream influence can be traced.

Next to each planted entry, we add a legitimate entry on the same topic. The attackers support half of the planted entries with fake identities (Sybil endorsements) and try to remove half of the legitimate entries with false rejections.

Totals over the three communitiesCampaign A (harmful entries)Campaign B (planted rules)
Planted entries promoted to trusted shared memory0/480/36
Honest agents who accepted a planted entry before promotion0 (13–18 reviews per entry)0 (13–16 reviews per entry)
Legitimate entries kept despite false rejections46/4836/36
Later tasks steered by a planted entry, unfiltered sharing0/144144/144
Later tasks steered by a planted entry, PerMIT0/1440/144

Live tests at larger scale are costly, because agents run one after another on the shared memory. At our measured rates, one 90-agent community costs USD 13–97 to build, depending on the backbone (App. F.6). The two campaigns above thus cost at least USD 264. Scaling both the number of communities and their size by 10 would multiply this by 100, to about USD 26,000 and 1,700 hours of agent runs. Measuring the rare events that matter at scale is costlier still: showing live that promotion falls from 10⁻³ to 10⁻⁴ needs about 6 million agent episodes and USD 0.88M–6.5M per backbone (W2 below). We therefore run live tests at the largest scale that is practical, and extend the evaluation to larger communities through our analysis (Eq. (1)) and simulation. The simulations agree with the analysis, showing that PerMIT's risk does not grow with community size (W2 below).

Concern (Q1): "Could the authors evaluate whether malicious memories are accepted or used before promotion and whether they can induce harmful behavior in downstream agents?"

We would like to clarify that PerMIT's design rules this out. Until an entry is promoted, it is untrusted for every agent. Untrusted content does not enter an agent's prompt on its own: the agent sees only an opaque handle, and the content reaches its reasoning or actions only after the gate displays it and the user or a verifier approves it. These guarantees are machine-checked in Lean (App. D, E), and our experiments support them:

  1. Not accepted before promotion. The first honest reviewer rejected every planted entry, which froze it, and the dispute panel then evicted it, even with Sybil support: 0 of 84 promoted. In the paper's original banking runs, honest reviewers also saw the direct-injection payload 127 times and accepted it 0 times.
  2. Not used through the gate. When the poison is handed to the gate on purpose, no verifier we tested lets it through (0/1,800), and adaptive attacks placed 9,302 attacker values on the gate with 0 accepted (App. F.2, F.4).
  3. No harm downstream. A later task counts as steered if it takes an action that only a harmful entry asks for (A) or emits a planted code (B) Planted rules steered 144/144 later tasks with unfiltered sharing but 0/144 with PerMIT, also for a weaker Claude-Haiku-4.5 agent. Campaign A's harmful entries contradict the agents' policy, so agents ignored them even without filtering.
AuthorsOpenReview ↗

Response to Reviewer r4AZ (2/3)

W2/Q2. Community simulations rely on assumed reviewer accuracy, while the multi-agent experiments do not directly validate the claimed safety improvements with increasing community size. [...] Could the authors provide matched security–utility comparisons against simpler sharing or voting mechanisms? Additionally, how well do the assumed reviewer accuracy and error correlations reflect the actual autonomous agent communities?

Concern (W2, Q2): how well "the assumed reviewer accuracy and error correlations reflect the actual autonomous agent communities".

Thanks for the insightful question! We measured reviewer accuracy live, and our simulations assume reviewers who are much worse than the live ones (App. F.9, F.10):

Honest reviewersLive communities (measured)Simulations (assumed)
Planted entries accepted0 of 1,381 decisions40%
Legitimate entries accepted92–96%90%
Correlation between reviewers' errorsno errors to correlate0–0.1 (0.3 in the stress test of Eq. (1))

The table above shows that live reviewers rejected every planted entry in all 1,381 decisions, across campaigns A and B and the paper's banking runs (catch rate q ≥ 0.995 at 95% confidence). No planted entry was promoted or steered a later task (W1 above). Our simulations instead assume reviewers who miss 40% of poisons (q = 0.6) and reject 10% of legitimate entries. Lower accuracy only raises the risk, so the simulations err on the side of caution. At the live rate, Eq. (1) gives ≈4×10⁻¹⁹ for independent reviewers.

Because live reviewers made no errors on the planted entries, we evaluate correlation as a stress test. Correlation matters only when reviewers err, because it makes a poison that fools one reviewer more likely to fool the others. Each live community uses a single backbone, where errors are likely to be correlated, yet its reviewers accepted no planted entry. Even at ρ = 0.3, Eq. (1) gives ≈10⁻³ at the live catch rate. Moreover, correlation, not size, is what raises risk (Fig. 2(b)), and PerMIT's identity pools address it, since promotion requires endorsements from more than one pool. With three reviewer families, the chance that the dispute panel keeps a poison falls to 8.6×10⁻⁴ at 10⁵ users. Aligning the pools with the families also cuts an attack that targets the weakest family from 34% to 4.8% success (App. F.10).

Finally, the analysis and the simulations agree. Across five billion simulated promotion attempts, the observed promotion rate stays below the bound of Eq. (1) for every number of honest users shown a poisoned entry, and none of the nine adaptive attack strategies we simulated exceeds it (§4.2, App. A.1).

Concern (Q2): "matched security–utility comparisons against simpler sharing or voting mechanisms".

Because live reviewers rejected every planted entry, we compare PerMIT with simpler voting rules in a matched simulation where reviewers err and adversaries reject falsely, on identical candidate streams, reviewer draws, and adversaries (1,000 users, q = 0.6, App. F.10):

MechanismPoison promotedBenign wrongly evicted, no attackBenign wrongly evicted, false-rejection attack
Majority of 100.420.0010.04
Quorum of 10, rejections ignored1.0000
Quorum of 10 + veto0.0010.711.00
PerMIT0.00100

With correlated reviewers (ρ = 0.1), PerMIT promotes 3.7% of poisons, against 43% for majority voting. On utility, PerMIT serves 40% of benign requests from the shared store without a new confirmation (77% with a juror panel), and in both live campaigns its store kept task success within 0.05 of unfiltered sharing on every backbone. On identical live verdicts, unfiltered sharing admits every planted entry and PerMIT admits none.

Concern (W2): the experiments "do not directly validate the claimed safety improvements with increasing community size".

Safety versus community size concerns rare events, so we establish it analytically and in simulation. A live demonstration that promotion falls from 10⁻³ to 10⁻⁴ would need 10,652 poison trials (666 communities) per community size, i.e., USD 0.88M–6.5M in construction alone at our measured costs. At the deployed thresholds, PerMIT's observed promotion stays at 0–5×10⁻⁴ from 10² to 10⁵ users (ρ = 0), whereas majority voting stays at 38–46% at every size.

AuthorsOpenReview ↗

Response to Reviewer r4AZ (3/3)

Summary: what PerMIT contributes, and what it can do that existing defenses cannot.

In addition to answering the questions above and focusing on the multi-agent experiments, we close with a short summary of the paper as a whole. PerMIT makes five main contributions:

  1. Machine-checked guarantees for persistent memory. Untrusted memory reaches the agent only as opaque handles, and a value reaches its reasoning or a sensitive tool only after the gate shows it and it is approved. These guarantees are machine-checked in Lean 4 for arbitrarily long multi-session interactions (App. E). CaMeL leaves formal verification as future work, and neither CaMeL nor FIDES covers memory that persists across sessions.
  2. Safe and useful on persistent memory, where existing defenses are not. On the same agent, tasks, and backbones, PerMIT sends no message to the attacker, completes 51/60 attacked tasks, and follows no planted instruction, while every baseline fails at least one of these tests (table below).
  3. The first integrity guarantees for shared memory, to our knowledge. CaMeL, FIDES, and TMA-NM protect a single user. PerMIT turns each approval into an auditable endorsement that others can reuse, and it bounds both the promotion of poison and the wrongful eviction of benign entries. In live 90-agent communities, none of 84 planted entries was promoted (W1 above), and in simulation, reusing endorsements saves each user 38% of approvals (§4.2).
  4. Shared memory that agents build themselves, and that helps. On τ²-bench telecom, a fresh agent using the memory built by 90 agents performs on par with the benchmark's expert-written manual on Claude-Sonnet-4.6 and GLM-5 (0.80 vs. 0.79 and 0.97 vs. 0.98) and exceeds it on Kimi-K2.5 (0.89 vs. 0.74) (§5).
  5. A recursive language model (RLM) as the memory reader, the first to our knowledge. It searches memory with small programs, found the right value in all 48 lookups despite look-alike distractors, and needed at most 22-fold more input for a 3,600-fold larger memory (App. F.6).
Test (App. F.8, F.9)CaMeLFIDESTMA-NMPerMIT
Persistent memory: user's message sent to the attacker (of 60)51000
Persistent memory: attacked task completed correctly (of 60)8915 / 32‡51
Persistent memory: planted instruction followed in the answer (of 60)412390
Single session: utility as designed (Claude / GLM / Kimi)0 / 0 / 046 / 45 / 062–78 / 76–84 / 54–60§90 / 87 / 89
Shared memory across userssingle user onlysingle user onlysingle user only0/84 planted entries promoted

‡With an informed user / with an AI verifier.

§Each range covers TMA-NM's two runs on that backbone. Utility is the harmonic mean over six task suites, so it is 0 whenever one suite fails completely. PerMIT's utility is from Table 1.

The table above shows that each baseline fails at least one test: CaMeL sends the user's message to the attacker, FIDES and TMA-NM complete at most 32 of 60 attacked tasks, TMA-NM often follows planted instructions, and none of them supports shared memory. PerMIT is the only design that does well on every row.

The updated supplementary material contains the description and statistics of every new experiment, together with the updated Lean 4 files, so that the reviewer can check the latest Lean 4 results directly.

Again, we thank the reviewer for the valuable feedback. We hope that the live results address the reviewer's concerns, and please let us know if there are any other questions or suggestions.

Best,

Authors

Thank you for the detailed response and the additional experiments. These additions have addressed my main concerns. I have no further questions at this stage and will raise my rating.

Reviewer_xti8OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes PerMIT, a memory architecture that defends LLM agents against persistent memory poisoning by isolating the planning agent from untrusted content and requiring that such content be explicitly confirmed, for a specific value and tool, before it can be used. Over this finite memory interface, the authors prove two integrity theorems (Action Integrity and Reasoning Integrity), with the single-user core machine-checked in Lean 4. They further extend the confirmation mechanism to shared memory, where endorsements accumulate across users to promote entries to a “community-confirmed” tier. Experiments on Trojan Hippo report zero observed attack success with LLM verifiers, and experiments on τ²-bench show that autonomously constructed shared knowledge improves held-out task performance for some backbones.

Critical strengths

The core theorems are machine-checked in Lean 4, and the authors provide a reproducible build environment together with documentation of the correspondence between the mechanized model and the paper’s formalization.

The paper clearly specifies which sources of content are trusted and which are untrusted, as well as the assumptions and scope under which each guarantee holds.

Critical weaknesses

I have concerns about the paper’s technical novelty. The single-user tier primarily relies on planner–reader isolation, opaque handles, and authorization before use, all of which appear in prior agent defenses. Persistent memory allows attacks to trigger across sessions, but the paper does not clearly explain what new technical challenges this persistence poses for existing defenses. In particular, if memory is treated as untrusted input whenever it is read, what additional problems do PerMIT’s reusable authorization and memory-management mechanisms address?

The evaluation lacks direct comparisons with relevant methods such as CaMeL, FIDES, and TMA-NM. These methods are closer to PerMIT’s design than baselines based on prompt constraints or disabling tool functionality. The paper cites CaMeL’s low utility in AgentDyn, but that result comes from a different experimental setting. Comparisons under the same persistent-memory tasks, models, and attack conditions would help clarify the advantages of PerMIT’s memory-specific designs.

Shared endorsement and dispute resolution are more distinctive aspects of the paper, but the evaluation does not directly compare their security with unfiltered sharing or simple voting. In Table 10, unfiltered A1 achieves higher mean utility than PerMIT on all three models. The task improvements from shared knowledge therefore leave open why this promotion mechanism is needed. These approaches should be compared using the same candidate memories and attack conditions to show which attacks the full protocol prevents and at what utility cost.

Questions for authors

See weaknesses.

Please compare PerMIT with CaMeL, FIDES, and TMA-NM under the same models, persistent-memory tasks, and attack conditions, and explain which design choices account for its advantages.

What novel design choices specifically address the persistent-memory setting? Which challenges do they solve beyond what existing agent defenses already handle?

Under the same candidate memories and attack conditions, how much does the full shared-endorsement and dispute-resolution protocol reduce attack success compared with unfiltered sharing and simple voting? Given A1’s higher mean utility in Table 10, how should we interpret the protocol’s security benefits and utility costs?

4 comments
AuthorsOpenReview ↗

Response to Reviewer xti8 (1/4)

We thank the reviewer for the valuable feedback on improving this paper! The questions on novelty and baselines made the paper sharper. Following the reviewer's suggestions, we ran all three requested baselines with their own code on the same agent, cases, and backbones, and added tests that target persistence specifically (revised PDF: §1, §2.2, §4.1, App. F.8–F.10, with changes in blue). Please find below our response.

W1/Q3. [...] Persistent memory allows attacks to trigger across sessions, but the paper does not clearly explain what new technical challenges this persistence poses for existing defenses. In particular, if memory is treated as untrusted input whenever it is read, what additional problems do PerMIT's reusable authorization and memory-management mechanisms address?

Concern (W1): the paper's "technical novelty" compared with prior agent defenses.

We agree that PerMIT reuses building blocks from the Dual-LLM and CaMeL line (a privileged planner, a quarantined reader, and opaque references), as §1 and §2.2 state. The revision now also names the problems that persistence raises and these blocks leave open (§1). Our contributions are what PerMIT adds on top of these existing building blocks:

  1. Machine-checked guarantees for persistent memory.

CaMeL leaves formal verification as future work, and neither CaMeL nor FIDES covers memory that persists across sessions. PerMIT's integrity guarantees are machine-checked in Lean 4 for arbitrarily long multi-session interactions (App. E).

  1. Much higher utility on persistent memory.

Under attack, PerMIT completes 51/60 persistent-memory tasks, against 8/60 for CaMeL and 9/60 for FIDES on the same agent and backbones (App. F.8). In our own runs, CaMeL's utility drops to 0 when no one approves its policy violations.

  1. The first integrity guarantees for shared memory, to our knowledge.

Prior shared-memory work addresses exchange, access, and portability, not whether shared content is safe to adopt. PerMIT reduces all memory risk to one auditable endorsement, shared across users and verifiers, and bounds both the promotion of poison and the eviction of benign entries. In live 90-agent communities, none of 84 planted entries was promoted (App. F.9).

  1. Shared memory that helps on an unfamiliar task.

On τ²-bench telecom, a fresh agent using the shared memory built by 90 agents performs on par with the benchmark's expert-written manual on Claude-Sonnet-4.6 and GLM-5 (0.80 vs. 0.79 and 0.97 vs. 0.98) and exceeds it on Kimi-K2.5 (0.89 vs. 0.74) (§5).

  1. A new memory architecture with a recursive language model (RLM) as the reader, the first to our knowledge.

The RLM writes and runs small programs to search memory, and the same reader both extracts values and writes summaries. It found the right value in all 48 lookups despite look-alike distractors, and a 3,600-fold larger memory needed at most 22-fold more reader input (App. F.6).

AuthorsOpenReview ↗

Response to Reviewer xti8 (2/4)

W1/Q3 (continued).

Concern (W1, Q3): the "new technical challenges" that persistence poses, and the "novel design choices" that address them.

To answer this question directly, we ran exactly the test it describes. CaMeL, FIDES, and TMA-NM all treat memory as untrusted whenever it is read, and we ran them on the same agent, backbones, and 60 live tasks as PerMIT (App. F.8):

Persistent-memory test (out of 60 tasks)CaMeLFIDESTMA-NMPerMIT
Planted address: user's message sent to the attacker51000
Planted address: task completed correctly8915 / 32‡51
Planted instruction followed in the answer412390
Planted value appears in the answer4658563†

‡With an informed user / with an AI verifier.

†All three came after the user explicitly asked to read that e-mail, and each also gave the user's own value. When the e-mail is only stored, the count is 0/60.

Even when memory is treated as untrusted whenever it is read, four challenges remain that existing defenses cannot properly address:

  1. The user's own facts stop being trusted once saved (first two rows).
    • Test: in session 1, the user gives a contact's address. A later e-mail claims a new or look-alike address for that contact. In session 3, the user asks to e-mail the contact.
    • Problem: if all memory is untrusted, a defense cannot tell the user's saved fact from the attacker's e-mail.
    • Results:
      • CaMeL's official policy sends the user's message to the attacker in 51/60 tasks and completes only 8/60 correctly. Its strict policy stops the leak but also blocks the user's own sends unless a human approves each one.
      • FIDES and TMA-NM send nothing to the attacker (0/60), but their agents adopt the fake address, so most tasks fail: FIDES completes 9/60, and TMA-NM 15/60 with an informed user or 32/60 with an AI verifier.
      • PerMIT keeps the user's fact trusted and shows it at the gate: 0/60 messages reach the attacker, and 51/60 tasks are completed with no human in the loop.
  2. Poisoned memory can change what the assistant says, not just what it does (last two rows).
    • Test: a planted e-mail either asks the assistant to "end every answer with '... ref CK-6NNN'" or states a misleading value.
    • Problem: no tool is called, so a defense that guards only actions never sees it.
    • Results: TMA-NM, which guards only tool calls, follows the planted instruction in 39/60 answers (FIDES 12/60, CaMeL 4/60), and the three baselines repeat the planted value in 46–58 of 60 answers. PerMIT follows no planted instruction (0/60) and repeats a planted value in only 3/60 answers, all after the user asked to read that e-mail.
  3. Approving every single use does not scale over a long-lived memory.
    • Problem: if every reuse of a remembered value needs a fresh approval, approvals pile up over time.
    • Results: PerMIT approves a value for a tool once and then safely reuses that approval. On TMA-NM's own benchmark, it needs 7 approvals instead of 99–201, and over 50 simulated sessions, 24–37 instead of 169–192. With the new Lean results, reuse can also be limited to one task, with proofs (App. F.7).
  4. Shared memory lets one poisoned entry reach every user.
    • Problem: CaMeL, FIDES, and TMA-NM protect a single user, and, to our knowledge, no existing defense checks whether shared content is safe to adopt.
    • Our approach: PerMIT extends the same endorsement to shared memory. Each endorsement shows the value, its source, and the trusted reference, so other users and verifiers can audit and reuse it, and checking scales with the community.
    • Simulation: safety holds as the community grows (the chance of promoting a poisoned entry stays at most 5×10⁻⁴ from 100 to 100,000 users with independent reviewers), no benign entry is wrongly removed, and each user needs 38% fewer approvals.
    • Live tests: 90-agent communities used PerMIT to safely build a shared knowledge base that helps agents in an unfamiliar environment (§5, App. F.9–F.10).

In short, each challenge is met by a design choice specific to persistent memory: trust labels that survive storage (1), isolation of unendorsed memory from the agent's plans and answers (2), reusable exact-value endorsement (3), and auditable shared endorsement (4). Moreover, crystallized summaries stay untrusted, and confirmation copies values rather than relabeling them, so summaries cannot launder untrusted text. Action Integrity and Reasoning Integrity, which cover choices 1–3, are machine-checked over unbounded multi-session histories.

AuthorsOpenReview ↗

Response to Reviewer xti8 (3/4)

W2/Q2. The evaluation lacks direct comparisons with relevant methods such as CaMeL, FIDES, and TMA-NM. [...] Please compare PerMIT with CaMeL, FIDES, and TMA-NM under the same models, persistent-memory tasks, and attack conditions, and explain which design choices account for its advantages.

Concern (W2, Q2): comparisons "under the same models, persistent-memory tasks, and attack conditions".

Following the reviewer's suggestion, we ran all three baselines on the same Trojan Hippo agent, cases, backbones (Claude-Sonnet-4.6, GLM-5, and Kimi-K2.5 for each system), and gate occupants (App. F.8). CaMeL uses its official repository, imported unmodified. FIDES uses its public reference planner with taint tracking, and TMA-NM uses its own reference monitor (tma.py). We gave each baseline every advantage we could: clarified planner prompts, an informed user, an AI verifier, and, for TMA-NM, a verifier that sees all past user turns. On the cross-session substitution test, the designs separate only under attack:

Cross-session substitution (out of 60 tasks)CaMeLFIDESTMA-NMPerMIT
No attack: message delivered correctly52575353
Attack: user's message sent to the attacker51000
Attack: task completed correctly8915 / 32‡51

‡With an informed user / with an AI verifier.

The table above shows that without an attack, all four designs deliver most messages. Under the same attack, however, PerMIT is the only design that keeps the user's message from the attacker and still completes most tasks. The answer-integrity results are in the table under W1 above.

Concern (W2): CaMeL's low utility in AgentDyn "comes from a different experimental setting".

In the same single-session setting, all four designs stop the native attacks. Without a human in the loop, however, only PerMIT keeps its utility (utility is the harmonic mean over six task suites, so it is 0 whenever one suite fails completely):

Single-session testCaMeLFIDESTMA-NMPerMIT
Native attacks (ASR)0/2580/1200/1200 (Table 1)
Utility as designed (Claude / GLM / Kimi)0 / 0 / 046 / 45 / 062–78 / 76–84 / 54–60§90 / 87 / 89 (Table 1)

§TMA-NM requested no authorization on these suites, so the occupant does not matter, and each range covers its two runs on that backbone.

The table above shows that without a human, CaMeL's policies block its tasks, consistent with AgentDyn's report, now measured in the same setting as PerMIT. When we add a verifier or a human to answer their confirmations, CaMeL and FIDES recover comparable single-session utility (80–92 for CaMeL and 90–96 for FIDES across the three backbones, App. F.8), but they still fail on persistent memory (table under W1 above). CaMeL is also cheaper per task. Accordingly, §2.2 now presents the AgentDyn result as specific to its own setting and adds these same-setting measurements.

Concern (Q2): "explain which design choices account for its advantages".

Five design choices account for PerMIT's advantages, each addressing a limitation of the baselines:

  1. Trust labels that survive storage. The user's typed facts stay trusted in later sessions. The baselines treat them like any stored text, so CaMeL leaks in 51/60 tasks, and FIDES and TMA-NM complete only 9/60 and at most 32/60, against 0/60 leaks and 51/60 completed for PerMIT.
  2. A gate that shows the trusted reference. Without a reference, even an AI verifier cannot decide: under CaMeL's strict policy, it approved only 2 of 116 decisions, rejecting the user's own address. With the reference, PerMIT's verifier decides automatically, and no harmful action passed any verifier we tested (0/1,800).
  3. Isolation of the agent from unendorsed memory. Untrusted memory reaches the agent only as opaque handles, so a planted instruction is followed in 0/60 PerMIT answers, against 39/60 for TMA-NM, which guards only tool calls.
  4. Reusable exact-value endorsement. One approval per value and tool keeps approvals bounded: 7 instead of 99 under TMA-NM's single-use rule on its own benchmark, with task- or time-limited reuse proved in Lean (App. F.7).
  5. Auditable, shareable endorsements. Other users and verifiers can check and reuse each decision, so PerMIT also secures shared memory, which no baseline supports: none of 84 planted entries was promoted in live communities (App. F.9).

Moreover, when we replay the 1,308 recorded agent proposals of TMA-NM's own benchmark through PerMIT's rule, none of its 827 evaluable attack proposals execute, even without a trusted registry or tool allow-list (matching TMA-NM's own result).

AuthorsOpenReview ↗

Response to Reviewer xti8 (4/4)

W3/Q4. Shared endorsement and dispute resolution are more distinctive aspects of the paper, but the evaluation does not directly compare their security with unfiltered sharing or simple voting. In Table 10, unfiltered A1 achieves higher mean utility than PerMIT on all three models. [...] Given A1's higher mean utility in Table 10, how should we interpret the protocol's security benefits and utility costs?

Concern (W3, Q4): "In Table 10, unfiltered A1 achieves higher mean utility than PerMIT".

Thanks for the insightful question! We would like to clarify that Table 10 is a benign control: A1 contains no attack, so a filter can only remove items there. A paired test on the same episodes finds no significant difference on any backbone after Holm correction (App. A.7, F.9): Δ = +0.083 / +0.031 / +0.010, with p = 0.08 / 0.50 / 1.00.

Concern (W3, Q4): security against "unfiltered sharing" under "the same candidate memories and attack conditions".

The security benefit appears under attack, which we now measure on identical candidate memories in three live 90-agent communities, where two malicious identities write directly into shared memory and add Sybil endorsements and false rejections (App. F.9):

Same candidates and verdicts (3 communities)Unfiltered sharingPerMIT
Harmful entries promoted48/480/48
Planted instructions promoted36/360/36
Downstream episodes that follow a planted instruction144/1440/144
Benign twins kept84/8482/84

The downstream result is the same with a weaker Claude-Haiku-4.5 consumer.

Concern (W3, Q4): security against "simple voting".

Because live reviewers rejected every planted entry, we compare PerMIT with simpler voting rules in a matched simulation where reviewers err and adversaries reject falsely, on identical candidates, reviewer draws, and adversaries (1,000 users, 60% catch rate, best-response adversary, App. F.10):

MechanismPoison promotedNumber of users who used the poisonBenign wrongly evicted, no attackBenign wrongly evicted, false-rejection attack
Unfiltered1.0018300
Single reviewer1.001830.101.00
Majority of 100.42770.0010.04
Quorum of 10, rejections ignored1.0017200
Quorum of 10 + veto0.0010.80.711.00
PerMIT0.001900

Only PerMIT combines low promotion, bounded exposure, and immunity to false-rejection attacks.

Concern (Q4): "Given A1's higher mean utility in Table 10, how should we interpret the protocol's security benefits and utility costs?"

The security benefits are shown above: PerMIT stops every planted entry that unfiltered sharing admits. The utility cost of this protection is small, for four reasons:

  1. No significant loss on benign tasks, and almost none under attack. Table 10's gap is not significant (above), and in both live campaigns PerMIT's store kept task success within 0.05 of unfiltered sharing on every backbone, while blocking every planted entry.
  2. The highest utility among the defenses we compared. As designed, PerMIT reaches 87–90 utility, the highest of the four designs on every backbone (W2 above), and it completes 51/60 attacked persistent-memory tasks, against at most 32/60 for CaMeL, FIDES, and TMA-NM (W1 above).
  3. Sharing saves confirmations. PerMIT serves 40% of benign requests from the shared store without a new confirmation (77% with a juror panel), and reusing community-confirmed entries saves each user 38% of approvals (§4.2). Majority voting serves more requests (89%), but it also promotes 42% of poisons.
  4. Reading memory stays cheap as the shared store grows. The RLM reader searches memory with small programs, so a 3,600-fold larger memory needs at most 22-fold more reader input and under 6 s per query (table below, App. F.6).
E-mails in memoryCharactersReader input per query (median tokens)Reader time per query (median)Correct answers
14001.9K2.1 s9/9
102.8K2.7K2.1 s9/9
5012.7K6.5K3.1 s9/9
20056K9.0K3.7 s9/9
1,000287K41K5.6 s9/9
5,0001.4M18K4.7 s3/3

The updated supplementary material contains the description and statistics of every new experiment, together with the updated Lean 4 files, so that the reviewer can check the latest Lean 4 results directly.

Again, we thank the reviewer for the valuable feedback. We hope that the new comparisons address the reviewer's concerns about novelty, baselines, and the federated protocol, and please let us know if there are any other questions or suggestions.

Best,

Authors

Reviewer_85ntOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PerMIT, a memory architecture that isolates untrusted content from an LLM agent and requires confirmation of exact memory-derived values before trusted use. The authors formalize the memory interface and provide Lean-checked proofs of core action and reasoning integrity properties under explicit trust assumptions. A community endorsement protocol enables reuse of confirmed shared knowledge. Experiments evaluate single-user security and utility on Trojan Hippo, and shared-knowledge construction and utility on τ2\tau^2-bench.

Critical strengths
  1. Lean-checked core proofs establish single-user integrity guarantees under explicit trust assumptions.
  2. Community endorsements enable verification reuse and shared knowledge that can benefit new agents.
  3. Trojan Hippo experiments show zero observed attack success with automated verification while preserving useful capabilities.
Critical weaknesses
  1. The formal guarantees do not establish endorsement correctness or resolve prompt-injection risks against the LLM verifier. End-to-end security therefore still depends on a verifier whose reliability is outside the scope of the proofs.
  2. The single-user experimental setup should explicitly specify the RLM reader's model configuration and its pairing with the protected agent. This limits reproducibility and makes it difficult to separate architectural benefits from model-dependent effects.
  3. Existing ablations do not fully disentangle the contributions of the RLM reader, isolation mechanism, and verification gate. This leaves the necessity and incremental benefit of each component unclear.
Questions for authors
  1. Do the RLM reader’s root and sub-models use the same backbone as the protected agent?
  2. Beyond OpenGate, can matched ablations isolate the contributions of the RLM reader and isolation interface, including a verifier-only baseline without RLM?
  3. How much query-time latency does PerMIT add over the same reader without verification?
4 comments
AuthorsOpenReview ↗

Response to Reviewer 85nt (1/2)

We thank the reviewer for the positive assessment and the precise questions, which helped us improve this paper! Each question is now answered with new measurements in the revised PDF (App. F, with changes in blue). Please find below our response.

W1. The formal guarantees do not establish endorsement correctness or resolve prompt-injection risks against the LLM verifier. End-to-end security therefore still depends on a verifier whose reliability is outside the scope of the proofs.

Concern (W1): the guarantees "do not establish endorsement correctness".

We agree, and this is by design. No proof can guarantee that a person or an LLM always judges correctly, which is exactly why our experiments measure endorsement correctness. What the proofs establish instead is that all memory risk is reduced to one decision. Under assumptions A1–A3, and even if endorsements can be wrong, a value the user never typed can reach a sensitive tool only after the gate has shown exactly that value for that tool and it has been approved (new composition proposition, §3.2, App. D.3). An attack therefore succeeds only if a wrong value reaches the gate and is accepted there, so its probability is at most the probability that the gate accepts a wrong value it displayed.

Moreover, this single decision is auditable. The gate shows the exact value, the tool, where the value came from, and the trusted reference it should match, such as an address the user typed earlier. Each decision can therefore be logged, checked, and judged by different users and verifiers, and in the federated tier it is shared and reviewed across the community. To our knowledge, no prior memory defense reduces memory risk to a single auditable decision in this way. Below, we measure how often this decision goes wrong.

Concern (W1): "prompt-injection risks against the LLM verifier", whose reliability is "outside the scope of the proofs".

We measured this reliability directly (App. F.2, F.4):

TestGate occupantAttacker values that reached the gateAccepted
Forced wrong values (a fully compromised reader)self-verification with explicit rules, 3 backbones1,8301 (0 at the action step)
Adaptive search (the attacker knows the verifier's prompt)5 occupants9,3020 (95% upper bound 0.040%)
Targeted probe on the Table 1 casesevery verifier we tested1,800 cases0
Targeted probe on the Table 1 casesaccept-all (no verification)300 cases300

The forced set includes 120 notes aimed at the verifier itself, such as "already approved by the user", and none was accepted. In other words, the proofs reduce all memory risk to one auditable decision, and that decision's error rate is now measured.

W2/Q1. The single-user experimental setup should explicitly specify the RLM reader's model configuration and its pairing with the protected agent. [...] Do the RLM reader's root and sub-models use the same backbone as the protected agent?

Concern (W2, Q1): the reader's "model configuration and its pairing with the protected agent".

Yes. In every PerMIT cell of Table 1, the reader's root model and its sub-calls use the same protected agent's backbone (App. B.4, new paragraph). The iteration caps are 6 (fill) and 10 (crystallize), the recursion depth is 1, and each reader call starts fresh, with no agent history. Self-verification uses the same backbone with a separate prompt and no agent history, at temperature 0 (GPT-5.4, a reasoning model, at its provider default).

Concern (W2): separating "architectural benefits from model-dependent effects".

Following the reviewer's suggestion, we keep PerMIT's isolation and gate fixed and vary the model-dependent parts: the backbone, the verifier, the prompt, and the reader design (App. F.4, F.5). Each backbone row covers every verifier and prompt we tested, and the last two rows cover all three backbones. Ranges span these settings, and counts are pooled:

Backbone or settingReader returned the poisonHarmful action executed
Claude-Sonnet-4.699–100%0/600
GLM-570–82%0/600
Kimi-K2.581–92%0/600
Single-call reader instead of the RLM83–100%0/300
Independent rerun from a clean installation74–100%0/1,800

The table above shows that the model changes how often the reader returns the poison (70–100%), but not whether a harmful action follows: none did in any setting. Security therefore comes from the architecture rather than from a particular model.

AuthorsOpenReview ↗

Response to Reviewer 85nt (2/2)

W3/Q2. Beyond OpenGate, can matched ablations isolate the contributions of the RLM reader and isolation interface, including a verifier-only baseline without RLM?

Concern (Q2): "isolate the contributions of the RLM reader and isolation interface".

Following the reviewer's suggestion, we change one component at a time on the same 100 Table 1 cases per backbone, under the targeted probe that hands the reader's output to a send (App. F.5):

ReaderIsolationGate decisionHarmful action executed (Claude / GLM / Kimi)
RLMnoNo gate270/300 (100 / 82 / 88)
RLMyesAccept-all (OpenGate)300/300
RLMyesLLM verifier (PerMIT)0/300
Single LLM callyesLLM verifier (PerMIT)0/300

Concern (Q2): "a verifier-only baseline without RLM".

As requested, we place PerMIT's verifier on every send of Trojan Hippo's own agent, with no reader, no isolation, and no persistent trust labels (App. F.5, F.8):

Three backbones combinedUndefended agentVerifier onlyPerMIT
Native attacks: successful17/1200/120 (each of 4 verifier settings)0 (Table 1)
Adaptive searches: successful10/150/150/84‡
Substitution: sent to the attacker40/605/600/60
Substitution: delivered correctly15/603/6051/60
No attack: delivered correctly54/6020/6053/60

‡In PerMIT's own harness, with the same generator and budget (App. F.2).

Concern (W3): "the necessity and incremental benefit of each component".

Each component therefore has a distinct, measured role. The table below compares PerMIT with a version or system that lacks each component (App. F.5, F.8):

Component (compared with)MeasureWithout the componentPerMIT
Gate (accept-all)Harmful action executed300/3000/300
Trust labels (verifier only)Message sent to the attacker5/600/60
Trust labels (verifier only)Correct delivery without an attack20/6053/60
Isolation (TMA-NM)Planted instruction followed in the answer39/600/60
Isolation (provenance labels only)Stored misleading value in the answer57/600/60
RLM reader (single-call reader)Harmful action executed0/3000/300

The table above shows a distinct role for each component. The gate stops poison that reaches it. Trust labels let the verifier decide correctly across sessions. Isolation keeps unendorsed memory out of the agent's answers, which no tool-call verifier can reach (the forced probe above cannot show this, since it hands the poison to the gate in every row). Finally, the RLM is not what makes PerMIT secure, since a single-call reader is equally secure. Its role is scalable search: it read a 1.4M-character memory with at most 23K input tokens per query, and no query read more than 72K at any store size (App. F.6).

Q3. How much query-time latency does PerMIT add over the same reader without verification?

Concern (Q3): latency added over "the same reader without verification".

The accept-all configuration is exactly that reader without verification (App. F.6). Without verification, a task takes a median of 85–111 s on the three backbones, and verification adds one verifier call per gated operation (ranges are over the three backbones):

VerifierMedian time per gated operationGated operations per taskAdded time per taskShare of task time
Qwen-3.5-4B (local)0.55–0.58 s6.1–7.43.7–4.1 s4%
Self-verification1.04–1.50 s5.4–7.57.5–13.3 s8–12%

The table above shows that verification adds 4–12% to the task time: about 4 s per task with a local Qwen-3.5-4B and 8–13 s with self-verification.

The updated supplementary material contains the description and statistics of every new experiment, together with the updated Lean 4 files, so that the reviewer can check the latest Lean 4 results directly.

Again, we thank the reviewer for the valuable feedback. Please let us know if there are any other questions or suggestions.

Best,

Authors

Thank you for the detailed response and the additional experiments. You have addressed my questions, and I have no further concerns and wiil keep the rating.

Dear Reviewer 85nt,

We sincerely thank you for the thoughtful engagement and assessment. We are glad the response addressed all concerns, and greatly appreciate your continued support for this work.

Regards,

Authors

Reviewer_cY7QOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes PerMIT, a framework for defending LLM agents against persistent memory poisoning. It isolates untrusted memory from the protected agent and uses a confirmation gate to control information flow, providing machine-checked integrity guarantees. The framework further enables secure memory sharing through community-based endorsements. Experiments demonstrate zero observed attack success with LLM verifiers on Trojan Hippo and improved agent performance through shared knowledge on τ²-bench.

Critical strengths
  1. The paper provides two formally defined integrity properties, Action Integrity and Reasoning Integrity, with machine-checked proofs in Lean 4. This adds theoretical rigor beyond purely empirical defenses.
  2. Rather than completely blocking untrusted memory, PerMIT allows selective endorsement and reuse, preserving useful memory capabilities while controlling unauthorized information flow.
Critical weaknesses
  1. The paper does not sufficiently investigate whether the Q-LLM itself can be manipulated by prompt injection. For example, malicious memory could instruct the Q-LLM to suppress relevant information or selectively retrieve misleading content, potentially affecting the protected agent without violating the proposed integrity guarantees. Furthermore, the evaluation lacks adaptive attacks specifically targeting the confirmation gate and LLM verifiers.

  2. In Table 1, OpenGate already achieves an ASR of only 0–10% across the evaluated models. This raises questions about how much additional protection is provided by the verification mechanism. The authors should explain why attack success is already so low without meaningful gate verification and disentangle the contributions of the reader, interface, and verifier.

  3. As acknowledged in Section 3.2, endorsements are bound to exact values and tools but not to specific tasks, purposes, or time periods. Consequently, previously authorized content may be reused in unintended contexts, potentially leading to undesirable agent behavior while still satisfying Action Integrity. This limitation raises concerns about whether the formal guarantees adequately capture task-level security.

  4. Although the paper emphasizes scalable verification and reduced user burden, it does not adequately quantify the computational and operational costs of the framework. In particular, the latency, token consumption, and API costs introduced by the RLM reader and confirmation process are not systematically evaluated. These measurements are important for assessing the practicality of PerMIT in real-world agent deployments.

  5. As shown in Table 5, the automated verifiers exhibit relatively high FPRs, this raising concerns about the practicality of PerMIT in real-world applications.

Questions for authors
  1. How does PerMIT handle prompt injection targeting the Q-LLM itself, particularly attacks that manipulate retrieval behavior or suppress relevant memory entries? Have the authors evaluated adaptive attacks specifically targeting the Q-LLM or confirmation gate?

  2. In Table 1, OpenGate already achieves an ASR of 0–10% despite accepting all requests. What explains this low attack success rate? How much additional security improvement can be attributed specifically to the LLM verifier rather than the retrieval and isolation mechanisms?

  3. Since endorsements are reusable across tasks and are not bound to specific purposes or time periods, how does PerMIT prevent previously approved content from being used in unintended contexts? Would incorporating task-specific authorization change the formal guarantees?

  4. What are the average latency, token consumption, and API costs introduced by the RLM reader and confirmation gate compared with conventional memory retrieval systems? How do these costs scale with memory size?

8 comments
AuthorsOpenReview ↗

Response to Reviewer cY7Q (1/5)

We thank the reviewer for the valuable and detailed feedback on improving this paper! Every concern led to new experiments, which we report in the revised PDF (App. F, with changes in blue). Please find below our point-by-point response.

W1/Q1. The paper does not sufficiently investigate whether the Q-LLM itself can be manipulated by prompt injection. For example, malicious memory could instruct the Q-LLM to suppress relevant information or selectively retrieve misleading content [...]. Furthermore, the evaluation lacks adaptive attacks specifically targeting the confirmation gate and LLM verifiers.

Concern (W1): a manipulated Q-LLM could affect the agent "without violating the proposed integrity guarantees".

Thanks for the insightful question! We would like to clarify that our proofs already treat the Q-LLM as adversarial (App. D.1), so a manipulated reader can push in only two directions. First, it can return attacker-chosen content, such as a misleading value or summary. This content reaches the protected agent only as an opaque handle, and it can affect an action or the agent's reasoning only through a gate decision that displays exactly that content. The revision makes this precise with a composition proposition (§3.2, App. D.3), which follows from the sink clause of our Lean invariant: without assuming correct endorsements, the attack probability is at most the probability that the gate accepts a displayed wrong value. Second, the reader can return nothing, which costs availability but not integrity, as §3.2 now states. Following the reviewer's suggestion, we measure both directions, including adaptive attacks on the gate and its verifiers (App. F.2–F.3).

(i) Concern (W1): the Q-LLM could "selectively retrieve misleading content".

To emulate a fully compromised reader, we put a wrong value on the gate in every item: 610 per verifier, including look-alike domains, "my address changed" claims, misleading retrievals, and 120 notes aimed at the verifier itself, such as "already approved by the user". Each verifier uses explicit decision rules.

Gate occupantWrong values accepted at the action stepWrong values accepted at the memory-reading step
Accept-all (by construction)490/490120/120
Claude-Sonnet-4.6 self-verification0/4900/120
GLM-5 self-verification0/4901/120
Kimi-K2.5 self-verification0/4900/120

In particular, none of the 120 notes aimed at the verifier was accepted. When neither a trusted reference nor a stated rule determines the correct value, no system can decide correctness from text alone, so the explicit rules then reject and ask the user (App. D.3).

(ii) Concern (W1, Q1): "adaptive attacks specifically targeting the confirmation gate and LLM verifiers".

The table above already covers fixed attacks aimed at the verifier itself, such as notes saying "already approved by the user". To also test attacks that adapt, we ran an automated attack generator against PerMIT's real reader, display pane, and gate. In each search, the attacker knows the verifier's prompt, sees every gate decision with its reason, and uses this feedback to refine its next attacks, trying up to 131 candidate attacks.

Gate occupantSearchesAttacker values that reached the gateAccepted
Claude-Sonnet-4.6303,6210
GLM-5108760
Kimi-K2.5108980
Qwen-3.5-4B303,5380
Qwen-3.5-2B43690
Total849,3020 (95% upper bound 0.040%)

In every search, the attacker's values reached the gate, and the gate rejected all 9,302 of them. The best attacks found also failed in all 400 runs on held-out cases that the searches never saw. We continue with the baselines and the reader in the next comment.

AuthorsOpenReview ↗

Response to Reviewer cY7Q (2/5)

W1/Q1 (continued).

(iii) Concern (W1, Q1): "adaptive attacks specifically targeting ... LLM verifiers", run in the benchmark's own adaptive mode.

To confirm that this attacker is strong, we run it in the benchmark's own adaptive mode with the same budget, on the benchmark's five RAG training cases (one per topic), for each of three agents. Each cell counts successful searches, and each search tries up to 131 candidate attacks under the benchmark's own stopping rule.

DefenseKimi-K2.5 agentGLM-5 agentClaude-Sonnet-4.6 agentAll three agents
None5/54/51/510/15
A-MemGuard5/54/50/59/15
LLM-Auditor2/52/50/54/15
PerMIT's verifier on every send0/50/50/50/15

On the same 15 case–agent pairs, the generator succeeds in 10 searches without a defense and in none against the verifier (exact McNemar test, p = 0.002). It is also much stronger than the static attacks of Table 1, where A-MemGuard and LLM-Auditor reach only 70% and 0% ASR on Kimi-K2.5. Moreover, the verifier's zeros rest on every attack the searches tried and every send the agents attempted:

Verifier on every sendKimi-K2.5GLM-5Claude-Sonnet-4.6All
Candidate attacks tried3913675741,332
Send decisions2591602421
Sends approved0000

These results agree with the larger search against PerMIT's own gate in (ii), where 9,302 attacker values reached the gate and none was accepted.

(iv) Concern (W1, Q1): malicious memory could make the Q-LLM "suppress relevant information" or "suppress relevant memory entries".

We also ran the same adaptive generator against the reader itself, aiming to make it return nothing for an address the user needs, with 400 retrieval attempts under attack and 80 without an attack (App. F.3). The attack does lower the retrieval success rate by about 40 percentage points. Suppression can hide information, but it cannot make the agent act on a wrong value. First, the reader cannot see what the user typed or confirmed (App. B.4), so those facts can never be hidden. Second, an empty result from the reader looks exactly like a memory with no answer (App. D.1), so the agent proceeds as if memory held nothing. If a payload instead makes the reader return a different address, that address still faces the gate tested in (i) and (ii).

(v) Concern (W1): whether malicious memory that manipulates the Q-LLM ends up "affecting the protected agent".

To test this end to end, we run PerMIT and an undefended memory agent on three-session tasks, 20 per backbone (App. F.5, F.8). In session 1, the user states a fact, such as a contact's address. In session 2, a planted e-mail claims a different value or carries an instruction. In session 3, the user asks the agent to e-mail that contact or to answer a question.

Three backbones combinedUndefendedPerMIT
Message sent to the attacker's address40/600/60
Message sent to the correct address15/6051/60
Planted instruction followed in the answer40/600/60
Stored misleading value repeated in the answer59/600/60

The table above shows that malicious memory never made PerMIT's agent act on the attacker's address, follow a planted instruction, or repeat a stored misleading value, while PerMIT still delivered 51 of 60 messages correctly.

AuthorsOpenReview ↗

Response to Reviewer cY7Q (3/5)

W2/Q2. In Table 1, OpenGate already achieves an ASR of only 0–10% across the evaluated models. [...] The authors should explain why attack success is already so low without meaningful gate verification and disentangle the contributions of the reader, interface, and verifier.

Concern (W2, Q2): why attack success is "already so low without meaningful gate verification".

We thank the reviewer for this question. Our original description of Table 1 was unclear, so we have revised the wording (Table 1 caption, §4.1, App. A.2) and added experiments that separate PerMIT's components to show the effect of each one.

The OpenGate row shows the effect of isolation alone. OpenGate is PerMIT with a gate that accepts everything, run on the benchmark's own attacks. Even with this open gate, the poison can never reach the protected agent if the reader does not retrieve it, or if it stays isolated behind an opaque handle. The agent never reads untrusted memory directly, so a planted instruction has nothing to act on. PerMIT's design therefore keeps the benchmark's poison away from the gate in the vast majority of cases, so even an open gate rarely sees anything harmful to approve. The 0–10% ASR comes from the few cases in which the poison got that far. In contrast, the same attacks succeed in up to 80% of cases against an agent that simply retrieves memory into its prompt (Table 1).

Isolation is only the first layer, however. If the poison does reach the gate, only the gate can stop it. To test the gate itself, the verifier rows of Table 1 use a targeted probe that hands the poison to the gate on purpose. We also ran an experiment that tests each component separately, to show how the reader, the isolation interface, and the verifier each contribute (table below).

Concern (W2, Q2): "disentangle the contributions of the reader, interface, and verifier".

Under this same probe and on the same 100 cases per backbone, we change one component at a time (App. F.4–F.5). The Gate decision column shows who decides at the gate: accept-all approves everything, the informed user is a simulated careful user who rejects the poison, and an LLM verifier decides automatically.

ReaderIsolationGate decisionReader returned the poison (Claude / GLM / Kimi)Harmful action executed
RLMnoNo gate100 / 82 / 88%270/300
RLMyesAccept-all (OpenGate)100 / 79 / 90%300/300
RLMyesInformed user (simulated)100 / 80 / 87%0/300
RLMyesLLM verifier†99–100 / 70–82 / 81–92%0/1,800
Single LLM callyesLLM verifier83 / 100 / 100%0/300

†Ranges and counts cover all verifier models and prompts we tested (App. F.4).

Observe that the reader extracts rather than filters: with either design, it hands the poison to the gate in 70–100% of cases. Once the poison arrives, the gate is what takes attack success from 100% to 0. Because the probe forces the release in every row, this table isolates the reader and the gate (without isolation, the probe skips one of its two routes, which would succeed trivially, hence 270 rather than 300). The contribution of isolation appears instead in normal use, where it keeps the poison from reaching the gate (the OpenGate row), and in the agent's answers, which a tool-call verifier cannot reach: a planted instruction is followed in 0/60 PerMIT answers, against 39/60 for TMA-NM, which has persistent labels but no isolation (App. F.8).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.