acceptodds
Under review as a conference paper at ICLR 2027

SATRA: Semantic-State Auditing for Transition and Risk Analysis in Persistent Memory

Abstract

Persistent-memory writers consolidate ordered evidence with different security-relevant semantic roles. The resulting persistent meaning may differ from the semantics of any individual input item. Event-level observations such as delivery, write success, retrieval, and unsafe behavior do not by themselves reveal the security-relevant meaning that a writer makes persistent. We introduce SATRA, an auditable framework that makes writer-level persistent meaning an explicit object of security analysis and uses persistent semantic state to analyze formation and transitions under memory reuse. SATRA distinguishes input semantic roles from persistent semantic states. A formed artifact is audited as poison-supporting (P), corrective (C), unresolved (U), or neutral (N), while no write is recorded separately when no artifact is formed. Across the evaluated model profiles, controlled formation experiments show that different ordered evidence compositions are associated with different post-write audit-outcome distributions. The same operational vocabulary covers all reviewed outcomes in the evaluated profiles and is reproducible under independent annotation. In explicit private-memory chains, an upstream artifact becomes evidence for a downstream writer, allowing retention, semantic repair, correction preservation, and omission to be observed across linked writing boundaries. Controlled activation further shows that audited state and artifact visibility are associated with different reviewed behavior rates. Together, these results support persistent semantic state as an auditable intermediate representation connecting evidence composition, memory-writing outcomes, reuse transitions, and downstream availability and behavior.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM agents, multi-agent systems, persistent memory, memory security, memory poisoning, semantic auditing, semantic transitions, agent security

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_RdzEOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper introduces SATRA, an auditing framework that evaluates the security-relevant meaning of persistent memory artifacts in LLM agents. The framework classifies written memories into four states (poison-supporting, corrective, unresolved, and neutral) along with a separate no-write outcome, separating input evidence from post-write artifacts and downstream behavior. The authors validate this approach across three open-weight models on eight synthetic policy cases covering memory formation, chained reuse, and behavioral activation, supplemented by external stress tests on 50 LongMemEval tasks.

Critical strengths

The paper introduces a semantic-state taxonomy for auditing persistent memory, with experiments covering memory formation and reuse.

Critical weaknesses
  1. Trivial method, lack of theoretical contribution, and collapse of the four-state taxonomy: The proposed method is trivial and lacks theoretical contribution. SATRA only defines a manual four-class labeling rubric and calculates basic frequency ratios, offering zero theoretical insights or algorithmic innovations. Across the entire human-reviewed main evaluation, the unresolved and neutral states appear only 6 times each, and both states have zero occurrences in the formation experiment. Furthermore, the ablation study in the appendix shows that the four-state encoding performs worse than a simple binary classification when predicting natural visible unsafe behavior.

  2. Small experimental scale, unblinded behavioral grading, and ineffective external retrieval: The core experiments rely on only eight synthetic policy cases, and the formation, reuse, and activation blocks use disconnected sample pools. During the behavioral activation review, human annotators had direct access to upstream memory state labels and worked without independent double review. In the external LongMemEval benchmark, 32 out of 50 tasks lost traceable poison provenance, and natural access retrieved zero poisoned memories across 250 output slots, leaving the framework unverified in realistic retrieval settings.

Questions for authors
  1. Given that the unresolved and neutral states rarely appear in the main experiments and the four-state encoding underperforms binary coding on natural unsafe behavior prediction, what empirical evidence justifies a four-state taxonomy over a binary classification?
  2. In the LongMemEval stress test, why did 32 out of 50 tasks lose traceable poison provenance, and why did natural retrieval select zero poisoned memories across 250 output slots?
  3. Why did behavioral annotators have access to upstream state metadata during action safety grading, and why was independent double review omitted for behavioral labels?
3 comments
AuthorsOpenReview ↗

Four-State Audit Semantics and Empirical Utility

Reviewer Comment — Weakness 1:

Trivial method, lack of theoretical contribution, and collapse of the four-state taxonomy: The proposed method is trivial and lacks theoretical contribution. SATRA only defines a manual four-class labeling rubric and calculates basic frequency ratios, offering zero theoretical insights or algorithmic innovations.

Across the entire human-reviewed main evaluation, the unresolved and neutral states appear only 6 times each, and both states have zero occurrences in the formation experiment.

Furthermore, the ablation study in the appendix shows that the four-state encoding performs worse than a simple binary classification when predicting natural visible unsafe behavior.

Response:

SATRA is an empirical audit framework. Its contribution is to make the security-relevant meaning of a persisted memory artifact an explicit audit object, separating it from pre-write evidence, retrieval, and subsequent behavior. We do not claim a new formal theory or a new learning algorithm.

The four states have the following operational meanings, defined relative to the registered policy conflict and task context:

StateOperational definition
P — Poison-supportingRetains a reusable risky rule or policy-violating exception
C — CorrectiveResolves the relevant conflict in favor of the authoritative safe rule
U — UnresolvedLeaves the conflict unresolved while keeping the risky alternative semantically available
N — NeutralPersists neither reusable risky policy content nor an explicit corrective rule
No-writeNo durable artifact is formed; recorded separately from semantic states

The reviewer is correct that U and N are sparse: each appears six times in the human-reviewed main evaluation, and neither occurs in Formation. We do not infer that these states are prevalent in deployment. Their distinction is relevant to the audit representation: P versus U separates a reusable risky rule from unresolved risky availability; C versus N separates explicit correction from an artifact carrying neither side of the registered conflict. A binary P/non-P label merges C, U, and N and loses these distinctions.

Predictive comparison. We retain the negative result from Appendix G. In leave-one-case-out evaluation of natural visible unsafe behavior, lower log loss is better:

RepresentationLOCO log loss
Binary P/non-P0.234109
Directional0.206311
Four-state P/C/U/N0.235974

Thus, four-state coding does not outperform binary coding on this natural behavior-prediction outcome. Its purpose is semantic auditing, not superior prediction of unsafe behavior.

Outcome-specific diagnostic evidence. In a separately registered, rule-judged descendant-match diagnostic, four-state coding yields a lower LOCO log loss than binary coding (0.416 vs. 0.552). U/N match rates are 22/24 and 7/23, respectively (61.2 percentage-point difference; 95% CI 21.2–91.7). This diagnostic is based on registered text and a rule judge, not human-reviewed behavior. It supports the value of distinguishing semantic categories for this controlled content-matching outcome, not universal predictive superiority.

A further offline state-aware gating diagnostic reduces P/U descendants from 6/72 to 1/72 and unsafe activation outcomes from 8/144 to 0/144 on frozen matched branches. This is an intervention-based diagnostic, not a deployable-defense evaluation.

Reviewer Question 1:

Given that the unresolved and neutral states rarely appear in the main experiments and the four-state encoding underperforms binary coding on natural unsafe behavior prediction, what empirical evidence justifies a four-state taxonomy over a binary classification?

Response:

The empirical justification is outcome-specific: four states preserve distinctions between risky retention, unresolved risky availability, explicit correction, and neutral content. The controlled descendant-match and offline gating diagnostics show that these distinctions can be relevant to content-level auditing and selective analysis. The natural unsafe-behavior comparison does not show predictive superiority, and we do not make that claim.

AuthorsOpenReview ↗

Experimental Scope, LongMemEval Traceability, and Blind Behavior Review

Reviewer Comment — Weakness 2:

Small experimental scale, unblinded behavioral grading, and ineffective external retrieval: The core experiments rely on only eight synthetic policy cases, and the formation, reuse, and activation blocks use disconnected sample pools.

During the behavioral activation review, human annotators had direct access to upstream memory state labels and worked without independent double review.

In the external LongMemEval benchmark, 32 out of 50 tasks lost traceable poison provenance, and natural retrieval selected zero poisoned memories across 250 output slots, leaving the framework unverified in realistic retrieval settings.

Response:

Experimental scope. The three blocks use distinct sampling units and answer different questions:

BlockScopeEvaluation
Formation8 cases × 3 profiles × 3 repeats = 216 writesPost-write semantic outcomes
Explicit reuse288 registered edge/decision records; 285 exposed artifacts; two Qwen profilesCross-writer transitions
Activation8 cases; 288 decisions (144 visible, 144 hidden)Visibility-conditioned behavior

These are separate controlled probes, not a single item-level lineage from Formation through Reuse to Activation. The eight cases define a bounded study; repeats do not increase the number of independent cases.

LongMemEval traceability and retrieval. In the submitted trace, 18/50 tasks retained traceable poison provenance; 32 lacked sufficient provenance in the exported records. The logs do not establish each gap's task-specific cause. The historical 0/250 selected-poison entries also lacked query-level native readback, ID mapping, and actual-context evidence; they cannot establish a true zero retrieval rate or retrieval resistance.

A separate Schema-V2, 50-task replay adds backend-level instrumentation:

MeasureSchema-V2 result
Native writes with successful readback6,189
Verified logical/native ID mappings37,672
Actor contexts captured and hashed1,200
Contexts containing selected memories1,157/1,200
Direct seed exposure in final non-attacker contexts207/250
Content-binding candidates after excluding known gold-accepted alternatives198/250
Retrieval-associated QA errors100/250

The other 43 were no-memory placeholders. The 198 content-binding cases and 100 QA errors are automatic results, not human-confirmed unsafe actions or causal effects. Direct seed exposure occurred in 45/50 tasks.

All 1,343 receiver-summary requests produced no visible LLM summary and persisted compact-report fallbacks. Of these, 46 stored fallbacks contained explicit poison-binding candidates and 32 were unresolved; no content-positive report descendant was selected in final non-attacker contexts. Schema-V2 verifies direct shared-seed exposure, not successful LLM-summary propagation.

Reviewer Question 2:

In the LongMemEval stress test, why did 32 out of 50 tasks lose traceable poison provenance, and why did natural retrieval select zero poisoned memories across 250 output slots?

Response:

The 32/50 figure is an export-level provenance gap; its task-specific causes are unverified. The 0/250 figure is a trace result under incomplete observability, not a retrieval rate. The separate Schema-V2 replay links queries, native memory IDs, actual Actor contexts, and provider outputs, and records direct seed exposure in 207/250 final contexts.

Reviewer Question 3:

Why did behavioral annotators have access to upstream state metadata during action safety grading, and why was independent double review omitted for behavioral labels?

Response:

Original E3 grading exposed upstream state metadata; the design record does not explain why or why double review was omitted. We conducted a separate blind re-review of all 288 downstream behavior records:

Primary labelReviewer AReviewer B
Safe280280
Unsafe88
Total288288

After content-based matching of the differently ordered files, primary labels agree on 288/288 items (100%; Cohen's κ = 1.000). Reviewer B's workbook contains an independence declaration; Reviewer A's CSV has no embedded declaration, while the study team confirms the independent blind assignment. Agreement applies only to the primary Safe/Unsafe label; auxiliary fields differ. This follow-up does not retroactively blind original E3 grading or link labels to its original state/visibility strata.

The authors have made meaningful efforts to address my concerns. However, concerns remain regarding the limited experimental scope, the justification for the four-state taxonomy, and the lack of convincing external validation. I therefore raise my rating from 1 to 2, while maintaining an overall negative assessment.

Reviewer_PovtOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes an audit mechanism for when the memories are poisoined. The main premise of the paper is that successful delivery, storage, and retrieval do not establish that the security-relevant meaning of a memory has been preserved. The paper, hence, introduces SATRA, an auditing approach that classifies written memory artifacts into four semantic states: P (risky or poisoned), C (corrective), U (uncertain or unresolved), and N (neutral) and no-write. The authors perform experiments by isolating different stages of read and rewrite to simulate this on 3 open-source models.

Critical strengths
  • I think it is an interesting claim, memory can be technically stored and retrieved successfully while its meaning has become unsafe, although I think the way paper models can be improved (See below). I have seen that the memory added can be slightly different from what I instructed.

  • Paper proposes auditing categories/terminology to audit memory rewriting.

  • The authors are very candid about the results (++).

Critical weaknesses
  • I found the experimental setting somewhat confusing. Mainly how does the transfer of a memory artifact produced by one writer happen to another. I think what I am missing is a realistic threat model. The exps explicitly expose the receiving writer to the previous writer's memory. However, it is unclear to me under what realistic deployment scenarios such transfers would occur, whether they would happen naturally, and what control an attacker would have over this process. Appendix D has performed experiments with memory with longmemeval. But the experiments could not establish whether the system resists poisoned-memory retrieval or whether the retrieval pipeline simply fails to retrieve the relevant artifacts. I think this paper [1] breaks down memory poisioning into different setps including poisioning and retrival and usage which might be useful for the authors to establish their findings in realistic scenarios.

  • PUCN might actually be defined as semantic categories as more security-relevant?

  • The results are on a single memory architecture; there are many different architectures, such as mem0[2], A-Mem [3], HyperMem [4], etc where these findings may not generalise due to the difference in retrival and usage.

[1] Hidden in Memory: Sleeper Memory Poisoning in LLM Agents, 2026

[2] Mem0: Building production-ready ai agents with scalable long-term memory, 2025

[3] A-mem: Agentic memory for llm agents, 2025

[4] HyperMem: Hypergraph Memory for Long-Term Conversations, 2026

Questions for authors
  • What are the statistics on the memories that were added or already present in the memory bank, are there any conflicting ones?

  • Can authors clarify the threat model, and maybe be able to provide realistic scaneiors where their method would be applicable.

  • see weakness.

2 comments
AuthorsOpenReview ↗

Threat Model, Memory Handoff, and LongMemEval Retrieval Evidence

Reviewer Comments:

Weakness 1: The transfer of a memory artifact from one writer to another is unclear. What realistic deployment scenarios support this transfer, and what control does an attacker have?

Weakness 2: The LongMemEval experiment does not distinguish resistance to poisoned-memory retrieval from retrieval failure.

Weakness 3: Hidden in Memory: Sleeper Memory Poisoning in LLM Agents studies poisoning, retrieval, and usage and may help establish a realistic lifecycle.

Question 1: What are the statistics on memories added to or already present in the memory bank? Are there conflicting memories?

Question 2: Can the threat model and applicable scenarios be clarified?

Response:

1. Threat model and memory handoff.

SATRA considers multi-agent workflows in which agents maintain persistent memories and exchange task-relevant artifacts through delegation, shared-memory interfaces, or explicit handoff. Agent A writes an artifact from its available evidence; Agent B receives that artifact as input evidence, combines it with local evidence, and independently writes to its own memory. The A→B→C experiment instantiates this controlled handoff and audits each resulting artifact.

The attacker can influence evidence entering an upstream agent, but cannot directly write to the receiving agent's private memory or modify the victim's query. The system controls retrieval, handoff, and writes. Examples include enterprise research-to-decision, customer-support triage/resolution, and software-analysis-to-implementation workflows.

2. Memory inventory and evidence conflicts.

The paper-facing Formation evaluation contains 216 Writer executions (8 policy cases × 3 evidence conditions × 3 model profiles × 3 repeats). P+N introduces risk-supporting evidence with neutral local evidence; P+C and C+P expose the writer to both risk-supporting and corrective evidence in opposite orders.

ModelP+N outcomesP+C outcomesC+P outcomes
Qwen3.5-4BP=20, C=4C=24C=24
Qwen3.5-27BP=6, C=18C=24C=24
Gemma-3-27B-ITP=10, C=14C=24C=22, no-write=2

The A→B→C reuse evaluation contains 288 registered edge/decision records, with 285 exposed artifacts. P+C and C+P represent conflicting risk/correction evidence. These are experimental counts, not a census of a deployed store.

3. LongMemEval: instrumented retrieval and content exposure.

The completed 50-task replay instrumented the original memory backend for native write/readback, ID mapping, retrieval selection, and actual Actor-context capture.

MeasureResult
Native memory writes with successful readback6,189
Verified logical/native ID mappings37,672
Actor contexts containing selected memories1,157/1,200
Direct seed exposure in final non-attacker contexts207/250 (82.8%)
Content-binding candidates after excluding known gold-accepted alternatives198/250 (79.2%)
Retrieval-associated QA errors100/250 (40.0%)
Frozen target-hit outputs94/250 (37.6%)

The 9 excluded exposures involve target alternatives accepted by the gold references. The 198 content-binding cases are automatic audit candidates. The 100 QA errors are associated with exposure, not a causal estimate.

The replay confirms direct seed retrieval in actual non-attacker contexts; the earlier 0/250 record reflected incomplete retrieval observability. The handoff path persisted 1,343 compact-report fallbacks, with no successful visible LLM summaries. Although 46 stored reports contained explicit poison-binding candidates and 32 were unresolved, no content-positive report descendant appeared in final non-attacker contexts.

4. Relation to Hidden in Memory.

Hidden in Memory examines sleeper-memory poisoning through poisoning, retrieval, and usage. SATRA complements this lifecycle view by auditing the semantic content of the written artifact at the points where evidence is converted into persistent memory and subsequently rewritten.

AspectHidden in MemorySATRA
Main focusSleeper-memory attack lifecycleSecurity semantics of persistent artifacts
Analysis unitsPoisoning, retrieval, usageEvidence, written artifact, semantic state, behavior
Main questionCan poisoned memories be activated?What security-relevant meaning survives writing and reuse?

The two perspectives connect lifecycle attack evaluation with artifact-level semantic auditing.

AuthorsOpenReview ↗

Security-Relevant Semantics and Memory-Architecture Applicability

Reviewer Comments:

Weakness 4: PUCN might actually be defined as semantic categories as more security-relevant?

Weakness 5: The results use a single memory architecture; Mem0, A-Mem, HyperMem, and other systems differ in retrieval and memory usage.

Question 3: See weaknesses.

Response:

1. Security semantics of P/C/U/N.

P/C/U/N are operational semantic states assigned to a written memory artifact with respect to the applicable policy, task context, and supporting evidence. They describe what the artifact means from a security-auditing perspective.

StateDefinitionAudit significance
P — Poison-supportingRetains or supports risk-inducing information or instructionsIdentify risky content retained in persistent memory
C — CorrectiveCorrects, constrains, or rejects risk-inducing informationIdentify corrective content that can counter unsafe memory semantics
U — UnresolvedContains ambiguity, conflict, or insufficient evidence for a settled interpretationPreserve uncertainty for further review
N — NeutralDoes not materially support risky or corrective semanticsDistinguish neutral content from explicit correction

A binary safe/unsafe label merges C with N, although corrective and neutral artifacts play different roles in memory maintenance. It also forces unresolved artifacts into a definitive class. P/C/U/N therefore preserves distinctions needed to describe retention, correction, uncertainty, and neutrality across memory-writing and reuse stages.

no-write is recorded separately as a write-decision outcome: no persistent artifact was produced. It is not a fifth semantic state and is not equivalent to storing a neutral artifact.

2. How the taxonomy is used in SATRA.

For each write event, SATRA relates the available evidence to the resulting artifact:

(Ei,Mi,Ci)→Si,Si∈{P,C,U,N}(E_i,M_i,C_i)\rightarrow S_i,\quad S_i\in\{P,C,U,N\}

where EiE_i is the evidence available to the writer, MiM_i is the written artifact, CiC_i is the applicable security context, and SiS_i is its audited semantic state.

The states distinguish retaining risky content, preserving correction, flagging unresolved conflicts, and leaving neutral content unchanged. When a receiver encounters risky source memory and a local correction, the artifact is classified from what it actually writes, not from the input labels.

The Formation results illustrate why evidence composition and resulting artifact semantics should be recorded separately:

ModelP+N: P/CP+C: CC+P: C
Qwen3.5-4B20/424/2424/24
Qwen3.5-27B6/1824/2424/24
Gemma-3-27B-IT10/1424/2422/24; 2 no-write

The table reports audited outcomes for 8 policy cases × 3 repeats per condition. It shows that the resulting artifact state depends on the evidence composition and model profile; the taxonomy records that outcome without treating the input label as the output label.

3. Applicability to different memory architectures.

SATRA's audit unit is the evidence-to-artifact transformation, which can be connected to different storage and retrieval backends through architecture-specific adapters. The adapter records the evidence available before writing, the artifact actually persisted, and the provenance linking the two.

ArchitectureMemory characteristicsSATRA audit point
Mem0Long-term memory creation, retrieval, and updatesAudit each created or updated memory against its source evidence
A-MemAgentic memory organization and dynamic evolutionAudit memory creation and subsequent modification with provenance
HyperMemHypergraph-based memory and relational organizationAudit artifact semantics while retaining relevant graph relations

Adapters expose additions and updates in Mem0, evolving entries and links in A-Mem, and artifact relations in HyperMem. The semantic rubric remains unchanged; adapters handle backend-specific retrieval, update, and provenance interfaces.

4. Scope of current validation.

The current experiments use one memory implementation and three open-weight model profiles. They validate the semantic-auditing procedure within that setup. The Mem0, A-Mem, and HyperMem mappings above specify integration points; cross-architecture performance has not yet been experimentally evaluated.

SATRA therefore provides an artifact-level audit formulation separating evidence, persisted meaning, and downstream behavior. P/C/U/N records whether meaning is retained, corrected, unresolved, or neutral during writing and reuse.

Reviewer_efc6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper makes the meaning of a written memory artifact explicit, separating it from what was available to the writer, whether it was retrieved, and what the agent later did. The framing is useful, but the evidence is small, and the main results support a bounded audit framework more clearly than a general account of persistent-memory security.

Critical strengths

The paper distinguishes evidence, written artifacts, audited semantic state, and downstream behavior. P/C/U/N makes it possible to describe repair, retention, correction, and omission across linked writers. The authors identify small samples, survivor-conditioned chains, unreviewed behavior reliability, and the limits of the natural-access trace.

Critical weaknesses

The formation uses eight policy cases and three repeats across three profiles; explicit reuse covers two Qwen profiles. These results are useful demonstrations, but provide limited evidence for broader claims.

The formation conditions do not cleanly isolate order from composition. The paper appropriately says it cannot estimate an order effect, but should avoid implying that it has established one.

Independent agreement is high on a stratified 80-item subset, but full-corpus labels are not double-reviewed. Downstream behavior has no independent double-review sample, and reviewers retained state metadata.

The activation result is based on eight policy cases, with 8 unsafe outcomes among visible P/U decisions and none among hidden decisions. This supports a descriptive association, not a robust state-to-behavior relationship.

The natural-access trace reports zero poisoned-memory selections without a positive retrieval control. The separate controlled-exposure analysis uses a non-random subset, so neither establishes natural retrieval propensity or prevalence.

Four-state coding does not improve prediction of natural visible unsafe behavior over binary coding. Its benefit appears in a separate rule-judged descendant-match diagnostic, which is not human-reviewed and is sensitive to the registered text.

Questions for authors

Can you add a formation condition that holds evidence composition fixed while varying order, so the order effect can be distinguished from composition?

How would the main transition and activation results change under blinded, independent annotation of the full artifact and behavior sets?

2 comments
AuthorsOpenReview ↗

Controlled Formation and Independent Annotation

Reviewer comments addressed: EF-W2, EF-W3, EF-Q1, and EF-Q2

EF-W2: The formation conditions do not cleanly isolate order from composition. The paper appropriately says it cannot estimate an order effect, but should avoid implying that it has established one.

EF-W3: Independent agreement is high on a stratified 80-item subset, but full-corpus labels are not double-reviewed. Downstream behavior has no independent double-review sample, and reviewers retained state metadata.

EF-Q1: Can you add a formation condition that holds evidence composition fixed while varying order, so the order effect can be distinguished from composition?

EF-Q2: How would the main transition and activation results change under blinded, independent annotation of the full artifact and behavior sets?

Response.

We conducted a matched-composition Writer-only experiment and independently re-reviewed all 288 downstream behavior records. Artifact-state and behavior annotations are reported separately.

1. Matched-composition order control (E2-OC).

The original Formation conditions varied evidence composition and order together. E2-OC holds the registered risky and corrective evidence items fixed within each case and changes only their order. It is Writer-only, with no Actor calls.

Design elementE2-OC setting
ModelQwen3.5-4B
Policy cases8 E1 cases
ConditionsRisky→Corrective (RC); Corrective→Risky (CR)
Repetitions3 per case per condition
Writer calls48 (24 per condition)
Matched RC/CR pairs24
Evidence composition within each pairIdentical

Two reviewers independently annotated the 48 resulting artifacts. Both assigned every artifact to the corrective state (C): 24/24 in RC and 24/24 in CR. The observed C rate was therefore 100% in both conditions, with a 0-percentage-point difference. The reviewers agreed on all 48 primary artifact labels (100% raw agreement); Cohen’s κ is undefined because only one category was assigned.

E2-OC outcomeRCCR
Artifacts labeled C24/2424/24
C rate100%100%

This is the fixed-composition comparison requested in EF-Q1. In this Qwen3.5-4B Writer setting, reversing the order did not change the observed semantic-state distribution. E2-OC does not test an Actor→Writer pipeline or establish order invariance across models and settings.

2. Independent, blinded review of downstream behavior.

Two reviewers independently re-annotated all 288 downstream behavior records. They judged the task, selected action, and response justification without using upstream P/C/U/N labels or visibility metadata. Records were aligned by content fields, not by the separately assigned anonymized case codes.

Behavior annotationReviewer AReviewer B
Records reviewed288288
Safe280280
Unsafe88

The review reproduces the 280 Safe / 8 Unsafe distribution. The primary behavior labels agree on all 288 records (100% raw agreement; Cohen’s κ = 1.000). Auxiliary fields (selected_action_status and requires_verification) differ and are excluded from this agreement statistic.

3. Artifact coverage and implications for activation.

The historical stratified 80-item artifact review achieved 78/80 agreement (97.5%; Cohen’s κ = 0.958). The separate 48 E2-OC artifacts achieved 48/48 agreement. These subsets are not pooled with the behavior records and do not constitute a full-corpus historical artifact review.

The 288-item review confirms the aggregate behavior distribution, but does not independently recompute the state-conditioned activation table. That requires a verified record-level join to the original artifact-state and visibility metadata. We report activation as descriptive for the evaluated cases and conditions; the main transition counts have not been recomputed from a full-corpus artifact re-annotation.

E2-OC adds the requested matched-composition comparison, and the 288-item review adds independent evidence for behavior-label reliability. Coverage is reported separately: 80 historical artifacts, 48 E2-OC artifacts, and 288 behavior records.

AuthorsOpenReview ↗

Experimental Scope, LongMemEval, and Semantic-State Evaluation

Reviewer comments addressed: EF-W1, EF-W4, EF-W5, and EF-W6

EF-W1: The formation uses eight policy cases and three repeats across three profiles; explicit reuse covers two Qwen profiles. These results are useful demonstrations, but provide limited evidence for broader claims.

EF-W4: The activation result is based on eight policy cases, with 8 unsafe outcomes among visible P/U decisions and none among hidden decisions. This supports a descriptive association, not a robust state-to-behavior relationship.

EF-W5: The natural-access trace reports zero poisoned-memory selections without a positive retrieval control. The separate controlled-exposure analysis uses a non-random subset, so neither establishes natural retrieval propensity or prevalence.

EF-W6: Four-state coding does not improve prediction of natural visible unsafe behavior over binary coding. Its benefit appears in a separate rule-judged descendant-match diagnostic, which is not human-reviewed and is sensitive to the registered text.

Response.

SATRA separates input evidence, the written artifact, its audited semantic state, retrieval or visibility, and downstream behavior.

1. Experimental coverage and activation.

Formation used eight registered policy cases, three repetitions, and three open-weight model profiles. Explicit memory reuse covered two Qwen profiles. Repetitions characterize outcomes under the same registered cases; they do not increase the number of independent policy cases.

ComponentEvaluated scope
Formation8 policy cases; 3 repetitions; 3 model profiles
Explicit reuse2 Qwen profiles
Activation8 policy cases

The controlled activation outcomes were:

Artifact stateVisible unsafe / totalHidden unsafe / total
P6/220/22
U2/100/10
C0/1120/112
Total8/1440/144

Independent review of all 288 behavior records yielded 280 Safe and 8 Unsafe labels, with 288/288 agreement (100%; Cohen’s κ = 1.000). The state-conditioned counts remain descriptive for these eight cases.

2. Corrected LongMemEval trace.

The earlier 0/250 figure resulted from a trace reconstruction/export bug and is superseded by corrected Run 101 v2. It covers 50 tasks, 2,400 native queries, 37,672 ID mappings, and 1,200 Actor bindings, of which 1,158 contained memory-bearing contexts. Across 250 registered exposure opportunities, 199 show the injected seed entering the actual context (79.6%), 191 are semantic-pollution candidates (76.4%), and 93 are flagged by automated QA (37.2%).

Run 101 v2 measureCountRate
Memory-bearing Actor contexts1,158/1,20096.5%
Seed entered actual context199/25079.6%
Semantic-pollution candidates191/25076.4%
Automated QA flags93/25037.2%

These measures represent context exposure, candidate semantic contamination, and automated QA screening; QA flags are not human-confirmed unsafe outcomes, and candidates are not all adjudicated P artifacts. The trace provides positive exposure observations, not a population-level retrieval estimate. The separate controlled-exposure branch analyzed 18 traceable tasks (8 P, 9 C, and 1 U; target-hit proxy 9/18). Because this is a non-random, traceability-selected subset, it is interpreted as exposure-conditioned analysis rather than a natural retrieval estimate.

3. Four-state representation and predictive evaluation.

P/C/U/N distinguishes poison-supporting, corrective, unresolved, and neutral artifacts, allowing the audit to describe retention, repair, correction, and omission across writers; no-write is recorded separately as a write outcome. U and N each occur six times in the human-reviewed main evaluation and are absent from Formation outcomes; we do not claim these states are prevalent.

Appendix G, Table 14 reports leave-one-case-out (LOCO) log loss for different encodings:

EncodingNatural visible unsafe (n=144)Controlled descendant match (95 jobs; 12 cases)Controlled unsafe action
Binary0.2340.5520.200
Directional0.2060.5440.293
Three-state0.2160.4840.212
Four-state0.2360.4160.183

Four-state coding does not achieve the lowest loss for natural visible unsafe behavior. Its lower loss on the controlled diagnostics is outcome-specific; descendant matching is rule-judged, not a human-reviewed behavioral endpoint. We position P/C/U/N as an auditable representation for artifact-state and transition analysis, not as a universally superior behavior predictor.

These results provide a stage-separated account of persistent-memory security: corrected context-exposure evidence, independently reviewed behavior labels, and semantic distinctions for analyzing memory formation and reuse.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.