acceptodds
Withdrawn from ICLR 2027

Harmless Here, Harmful There: Safety and Privacy Inversion of Agent Memory Across Model Migration

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Persistent agent memory is treated as portable state: it is exported when a backbone is upgraded, a provider changes, cost-tiered routing splits easy and hard turns, or several agents share a store. Signatures certify that the bytes survive; they certify nothing about whether the behaviour those bytes induce survives. Because instruction-tuned models differ in instruction-hierarchy compliance, literalness, refusal, inference strength and tool use, a record one model wrote as history (“when updating the account, also export a backup copy”) can be read by the next as an instruction, an authorization, or a goal. We call this migration-induced safety and privacy inversion, study it as a property of an ordered pair of models, and contribute a formal setting with three executable-outcome metrics, two structural results on what a target-agnostic transform can and cannot preserve, a typed compiler and a distribution-free selective-migration gate, and an analysis showing a pair matrix concentrates at its model count. We measure the full source–target matrix over six open-weight models twice, a pilot and a confirmatory replicate on a frozen benchmark with fresh decoding seeds. The pre-registered gate passes both times, against a same-model control of exactly zero. The structure of the result is the finding: target identity explains of the variance across pairs, and counterfactual arms show the imported content, not the reader's baseline tendency, causes the effect on ambiguous and adversarial items. The union is carried by authority escalation, a record read as permission to act, with privacy inversion near zero on this benchmark, so the measured danger is over-action rather than disclosure. Both components we propose are measured negatives once given the right control: no target-agnostic transform cuts violations by more than , and the compatibility predictor's edge over artifact text collapses from to AUROC once the baseline is told the target. One pre-registered expectation, bidirectional inversion, does not replicate and is withdrawn. What holds is refusal: the selective gate keeps a plug-in false-migration rate of against ; a gate reading only the target's own risk ties its utility but misses the guarantee. Source-side certification does not travel with a migrated record.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM agents, agent memory, memory migration, AI safety, agent security, authority escalation, privacy, conformal risk control

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_uYKxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies whether memory artifacts that are safe under one model can induce harmful, privacy-violating, or unauthorized actions under another. It evaluates six open-weight models, proposes formal results and mitigation components, and reports that the observed effect is predominantly authority escalation driven by target identity.

Critical strengths
  1. The paper highlights a practical risk when agents change backbone models: a memory artifact that passes safety checks under one model may induce unauthorized actions under another.

  2. Executable trace-based checks, fresh-seed replication, and content-replacement controls help distinguish observed violations from decoding noise and baseline action tendencies.

Critical weaknesses
  1. The main experiment provides limited evidence specific to memory migration. Records are generator-authored, and target rollouts are shared across sources; the source therefore affects the result through item selection rather than memory production. The main results establish that source-side screening does not transfer, but the distinction from cross-model susceptibility to untrusted content remains insufficiently demonstrated.
  2. The operational scope of the gate’s guarantee is narrow. The reported FMR of 0.032 concerns a five-seed threshold label, not the latent violation event, and controls joint risk rather than violation probability among accepted migrations. Unseen-target evaluation contains zero instances.
  3. The same AI assistant generated benchmark items, implemented the oracles, and ran the analysis. The authors report human item review and independent re-scoring of 254 stored rollouts, which are valuable checks. However, agreement between implementations establishes scoring consistency more directly than semantic validity. Appendix I also leaves model revisions, inference-engine versions, and decoding details to be recorded despite reporting completed experiments.
Questions for authors

Please refer to the Weaknesses.

Reviewer_yw97OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies whether a persistent agent memory that is safe under a source model becomes harmful, leaking, or authority-granting when a different target model reads it. The authors call this migration-induced inversion. They define inversion rates over a source–target matrix and prove several elementary results: a seed-budget bound, a pigeonhole conflict-rate bound, a scalar-ordering falsification, conformal risk control, and a Janson-type concentration bound. They also propose a typed memory compiler, a fingerprint-based compatibility predictor, and a conformal selective-migration gate. Experiments cover six open-weight models of at most 9B parameters on 864 synthetic items.

Critical strengths

The study uses executable oracles instead of an LLM judge, a same-model control, and counterfactual content arms that separate the effect of memory content from the reader’s baseline tendency to act. It also checks scoring with an independent re-scorer. The runtime capability-enforcement experiment (E8) gives a concrete, deployable takeaway: authority claimed inside a migrated record should be re-established at the tool boundary.

Critical weaknesses
  1. The paper frames inversion as a property of the model pair, yet target identity explains 0.92 of the variance, and the pairwise predictor gains only +0.009 AUROC once the target is known. Privacy inversion is near zero, so the safety and privacy inversion in the title is not supported.
  2. Large portions document process instead of findings. These include pilot-versus-confirmatory bookkeeping, instrumentation defects and parser fixes, redefinitions of metrics, diagnostics formulated after a pilot run, deviation policies, and post-hoc “round 3” experiments. The main text is dominated by statements of what is not claimed: the headline statistic is explicitly not the defined SIR/PIR/AER, Theorem 4 rules out general claims about models, and remarks disclaim each result. Combined with dense notation and 43 pages for a modest result, the paper is very hard to follow. The defensive framing also obscures what was actually learned.
Questions for authors

Given that target identity explains most of the variance and the source adds almost nothing, what does the pair-based framing reveal that a per-model indirect prompt-injection benchmark would not?

Reviewer_76mFOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates whether persistent agent-memory artifacts that pass safety screening under one language model can induce unsafe behavior when transferred to another model. The authors evaluate six open-weight instruction models, 30 directed source–target migration pairs, and 864 predominantly generator-authored memory records. Their evaluation includes a 60-seed source-side safety screen and five-seed target-side measurements.

The empirical results suggest that source-approved memory can cause violations when interpreted by a different target model. However, the observed violations are primarily attributable to unauthorized authority escalation, and the variation across migration pairs is largely explained by target-model identity.

The paper further investigates a memory compiler, a pairwise compatibility predictor, and a calibrated selective migration gate intended to reduce migration risks.

Critical strengths
  1. Important and practically relevant research problem.

The portability of persistent agent memory across model upgrades is an important problem for long-running agents. A memory artifact that is safe under one model may not preserve its safety properties under another model with different instruction-following behavior or safety alignment.

The paper appropriately draws attention to this trust-boundary problem.

  1. Useful experimental controls and transparent negative findings.

The authors conduct content-replacement experiments showing that imported memory text can contribute to unsafe target behavior. The paper also includes pilot and confirmatory experiments and reports several negative results rather than omitting them.

The explicit discussion of the limitations of the five-seed measurement and the distribution-marginal guarantee is appreciated.

Critical weaknesses
  1. The experiments do not convincingly isolate a migration-specific phenomenon.

The primary benchmark uses memory records generated by an external generator rather than memories actually produced by the source agent (Section 3, PDF page 4).

The source model primarily determines which records pass safety screening, whereas the target model determines how those records are interpreted.

More importantly, target-model identity explains approximately 91.5% of the pair-level variance (Section 7, PDF page 8). The paper's bidirectional inversion hypothesis also fails to replicate (PDF page 9).

These findings suggest that the observed phenomenon may be largely explained by target-model susceptibility to instruction-bearing memory rather than a distinctive interaction between specific source and target models.

The supplementary source-authored experiment includes only 180 screened-in records (Appendix G.2, PDF page 39), which is insufficient to establish generalization to longitudinal, real-world agent memory.

  1. The proposed methods fail to establish a convincing incremental contribution.

The proposed memory compiler does not meaningfully reduce violations, while the pair-conditioned compatibility predictor improves AUROC over a target-aware baseline by only 0.009 (PDF page 9).

The selective migration gate achieves stronger safety results, but the reported comparison does not include a target-only baseline equipped with the same risk-calibration procedure.

Appendix Table 11 (PDF page 38) attributes the formal guarantee to conformal calibration rather than to the proposed pairwise compatibility features.

Consequently, it is unclear whether the improvement arises from a novel migration-aware mechanism or from applying an existing calibration procedure to a different score.

  1. The operational effectiveness of the defense is not convincingly established.

Appendix Table 12 (PDF page 39) reports that among 196 confirmed attack cases, 50 are both accepted by the full defense and still result in violations.

This result does not necessarily contradict the paper's theoretical guarantee, which depends on exchangeability assumptions. However, it exposes an important limitation when the defense is deployed against deliberately selected or adaptive attacks.

Furthermore, the runtime capability gate prevents unauthorized tool actions by construction (Appendix Tables 15–16). Although this is useful engineering, it does not establish the effectiveness of the proposed memory compiler or compatibility predictor.

  1. The model suite and memory evaluation are outdated and insufficiently representative.

All six evaluated model families originate from 2024. For an ICLR 2027 submission, the absence of models released in 2025 or 2026 substantially limits the relevance of the reported findings to contemporary agents.

More importantly, the benchmark relies predominantly on synthetic memory artifacts rather than memories accumulated during actual agent execution.

Real agent memory undergoes creation, summarization, retrieval, revision, and repeated reuse. These processes can alter both the content and interpretation of memory records.

The paper does not adequately establish that its conclusions generalize to such environments.

  1. The privacy claims are substantially broader than the measured evidence.

Although the title emphasizes both safety and privacy inversion, the measured privacy violation rate is approximately 0.002.

The overwhelming majority of observed violations are associated with authority escalation.

The paper therefore provides relatively little empirical evidence for privacy inversion as a substantial independent phenomenon.

  1. The main-paper presentation is inadequate for evaluating the empirical contribution.

The nine-page main paper contains only one plotted figure, and Figure 1 contains visibly overlapping text labels.

The most important comparisons supporting the claimed contributions are distributed across appendix tables rather than presented in the main paper.

This is particularly problematic because the paper introduces several mitigation mechanisms whose effectiveness is not consistently supported by the experiments.

A stronger presentation would explicitly summarize the central empirical findings, negative results, and relevant trade-offs in the main paper.

Questions for authors
  1. How much source–target interaction remains after controlling for target-model identity and artifact content? Can the authors establish migration-specific effects beyond source-side selection and target-side susceptibility?

  2. Can the authors evaluate memory records genuinely produced and repeatedly used by agents in multi-session environments, rather than relying primarily on externally generated artifacts?

  3. What advantage does the selective migration gate retain when the target-only baseline uses the same calibration method, calibration data, risk budget, and coverage constraints?

  4. How should the proposed gate handle deliberately selected or adaptive attacks, given that 50 out of 196 confirmed attack cases remain accepted and violating under the full defense?

  5. Can the authors clarify which safety guarantees extend beyond the exchangeable evaluation distribution?

  6. Can the authors move the main defense comparisons into the main paper, explicitly present the failure cases, and correct the overlapping labels in Figure 1?

Comments on the paper

1 comment

Dear Reviewers and Area Chairs,

We sincerely thank you for your time, thoughtful evaluations, and constructive feedback throughout the review process.

Your comments have helped us identify important limitations and opportunities to strengthen our work. We appreciate the careful consideration given to our submission.

We will take your suggestions into account, further improve the manuscript, and prepare a stronger version for submission to a future conference.

Thank you once again for your valuable feedback and contributions to improving our research.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.